modelservice

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 28, 2026 License: MIT Imports: 21 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	APIKeyProviderOpenAI    = "openai"
	APIKeyProviderAnthropic = "anthropic"
)

Providers ConnectAPIKeyProvider connects. Each is also its instance's ID and its key's auth store key.

View Source 
const APIKeyAuthMethod = "api_key"

APIKeyAuthMethod is the auth method of a stored API key.

Variables

View Source 
var ErrNoAudioInput = errors.New("does not accept audio input")

ErrNoAudioInput reports a chat model voice would transcribe with whose catalog does not declare that it takes audio.

View Source 
var ErrNoVoiceTarget = errors.New("voice target is not configured")

ErrNoVoiceTarget reports a voice request whose target is not configured.

Functions

func APIKeyProviderInstance 

func APIKeyProviderInstance(provider string) (*config.ProviderInstanceConfig, error)

APIKeyProviderInstance returns the provider instance ConnectAPIKeyProvider records for provider, authenticating with the key stored under the provider's name.

func AdoptDefaultModel 

func AdoptDefaultModel(cfg *config.Config, target string) bool

AdoptDefaultModel makes target, a chat model just verified or added to the chat shortlist, the default model when no default is set, and reports whether it did. A default already set is never replaced.

func CatalogFilePath 

func CatalogFilePath() string

CatalogFilePath returns the absolute path to the local model catalogs cache file.

func CatalogSyncSupported 

func CatalogSyncSupported(instance *config.ProviderInstanceConfig) bool

CatalogSyncSupported reports whether Compa can list instance's models: it runs a core provider for the instance.

func CheckChatTranscriptionTarget 

func CheckChatTranscriptionTarget(cfg *config.Config, store *CatalogStore, target string) error

CheckChatTranscriptionTarget reports whether target, the chat model voice transcribes with (config.VoiceConfig.STTViaChat), takes audio: the saved catalog of its instance must declare audio input for its model (see CatalogModel.AudioInput). A chat model without audio input answers a recording it never received with text of its own, which would pass for what the user said. A nil store reads the saved catalogs.

func ConnectAPIKeyProvider 

func ConnectAPIKeyProvider(ctx context.Context, cfgPath, provider, key string, deps ConnectDeps) error

ConnectAPIKeyProvider stores key as provider's API key and connects the provider in the config at cfgPath: it discovers the models the key reaches, records the provider's instance (see APIKeyProviderInstance; an existing instance keeps its headers and runtime settings), saves the catalog, and drops targets on the instance that the catalog no longer holds (see DropTargets). When the provider cannot connect, the previous key is restored, or the new one removed.

func DeleteProviderInstanceCatalog 

func DeleteProviderInstanceCatalog(instanceID string) error

DeleteProviderInstanceCatalog removes catalog models for a specific instance.

func DisconnectAPIKeyProvider 

func DisconnectAPIKeyProvider(cfgPath, provider string, deps ConnectDeps) error

DisconnectAPIKeyProvider removes provider's stored API key and disconnects the provider in the config at cfgPath: every instance authenticating with the key is disabled, the targets on those instances are dropped (see DropTargets) and their catalogs deleted. When the config cannot be updated, the key and the config are restored.

func DiscoverExtension 

func DiscoverExtension(ctx context.Context, endpoint, secret string) (extension.InfoResponse, error)

DiscoverExtension returns the providers the daemon at endpoint serves.

func DropTargets 

func DropTargets(cfg *config.Config, keep func(target config.ExactModelTarget) bool)

DropTargets removes from cfg every exact target keep rejects: from the active models, from the model routes (a route left without targets is removed), and from the model selections — the default, per-agent, image and light models — which are cleared when they name a dropped target or a removed route.

func ExtensionSurface 

func ExtensionSurface(info extension.ProviderInfo) string

ExtensionSurface returns the primary surface of a daemon provider, recorded as its instance's protocol, or "" when it lists no surface core defines. Chat Completions is preferred, since Compa's OpenAI-compatible provider streams, then Messages; a provider serving neither, such as a speech-only one, is recorded under the first surface it lists.

func ExtensionSurfaces 

func ExtensionSurfaces(info extension.ProviderInfo) []string

ExtensionSurfaces returns the llmgw-core surfaces a daemon provider lists, lower-cased, de-duplicated and in the daemon's order; surfaces core does not define are left out.

func InstanceCatalogs 

func InstanceCatalogs(cfg *config.Config, store *CatalogStore) map[string]providers.InstanceCatalog

InstanceCatalogs returns the valid catalogs in store of cfg's provider instances (see ValidInstanceCatalog), keyed by instance ID.

func LoadInstanceCatalogs 

func LoadInstanceCatalogs(cfg *config.Config) (map[string]providers.InstanceCatalog, error)

LoadInstanceCatalogs reads the saved catalogs and returns those of cfg's provider instances.

func ModelInputModalities 

func ModelInputModalities(model core.ModelInfo) []string

ModelInputModalities returns the inputs a catalog row declares its model takes, sorted: text and image from its typed input capabilities, audio from its audio-in operation (see core.ModelOperationAudioIn), and any of them from an input modality list in its legacy capabilities, such as {"input_modalities": ["text", "audio"]} or {"modalities": {"input": [...]}}, or an explicit {"input_audio": true}. A row without typed capabilities is read through core.InferCapabilities. What is not declared is not supported: the model's id is never read.

func ModelSurfaces 

func ModelSurfaces(model core.ModelInfo) []string

ModelSurfaces returns the surfaces a catalog row reports its model serves, from its supported APIs and typed capabilities, sorted. A row that reports neither has none: its surfaces are unknown.

Audio in and out on a chat model are what its chat takes and gives (input audio, spoken answers), not the transcription and speech endpoints: only a row that serves no chat gets those surfaces from its capabilities; a chat row gets them only by listing their endpoints.

func NewExtensionClient 

func NewExtensionClient(endpoint, secret string) (*extension.Client, error)

NewExtensionClient returns the client of the daemon at endpoint using secret.

func ResolveCredentialReference 

func ResolveCredentialReference(ref string) (string, error)

ResolveCredentialReference returns the secret an instance's auth_connection_ref names. The one reference kind is credential:<key>: the access token stored under key in the auth store, which must not be expired.

func SaveCatalogs 

func SaveCatalogs(store *CatalogStore) error

SaveCatalogs atomically writes the model catalogs cache to disk.

func SaveProviderInstanceCatalog 

func SaveProviderInstanceCatalog(instance *config.ProviderInstanceConfig, models []CatalogModel) error

SaveProviderInstanceCatalog saves catalog models for a specific instance.

func ServesChat 

func ServesChat(surfaces []string) bool

ServesChat reports whether a catalog model with surfaces can be selected for chat: Compa's core provider serves Chat Completions natively or through translation.Adapter, as translation.ServesChat reports. A catalog that reports no surfaces leaves them unknown, so such a model stays selectable.

func ValidInstanceCatalog 

func ValidInstanceCatalog(key string, entry *CatalogEntry, instance *config.ProviderInstanceConfig) bool

ValidInstanceCatalog reports whether entry, saved under key, is the catalog of instance: saved under the instance's ID, for that instance, from the instance's provider kind. A catalog left behind by a deleted instance or a re-created one of another kind is not.

Types

type AnonymousProviderOutcome 

type AnonymousProviderOutcome struct {
	RegistryID string   `json:"registry_id"`
	ProviderID string   `json:"provider_id"`
	Status     string   `json:"status"`
	Models     []string `json:"models,omitempty"`
	ProbeModel string   `json:"probe_model,omitempty"`
	LatencyMS  int64    `json:"latency_ms,omitempty"`
	ErrorClass string   `json:"error_class,omitempty"`
	Error      string   `json:"error,omitempty"`
	// contains filtered or unexported fields
}

AnonymousProviderOutcome reports catalog and inference evidence separately. Models are the free models the provider's catalog lists and ProbeModel the one it was tested with; Error is a short sentence, never an upstream's answer.

type AnonymousVerifyFunc 

type AnonymousVerifyFunc func(ctx context.Context, cfg *config.Config) []AnonymousProviderOutcome

AnonymousVerifyFunc checks the free providers that need no key against cfg, which it reads but never changes, and reports each one's outcome.

type AutoConnectResult 

type AutoConnectResult struct {
	OK                bool                       `json:"ok"`
	Total             int                        `json:"total"`
	CatalogDiscovered int                        `json:"catalog_discovered"`
	Verified          int                        `json:"verified"`
	Instances         []string                   `json:"instances"`
	Outcomes          []AnonymousProviderOutcome `json:"outcomes"`
	// DefaultModel is the verified model this connection made the default
	// model, there being none; "" when it set none.
	DefaultModel string `json:"default_model,omitempty"`
}

AutoConnectResult reports the results of auto-connecting the free providers that need no key.

func AutoConnectFree 

func AutoConnectFree(ctx context.Context, cfg *config.Config, verify AnonymousVerifyFunc) (*AutoConnectResult, error)

AutoConnectFree checks the free providers that need no key and, for each whose test answer came back, enrolls its instance with the free models its catalog lists and adds the model that answered to Chat. The first model added becomes the default model when none is set. An instance the owner set up otherwise is never changed.

func AutoConnectFreeAndSave 

func AutoConnectFreeAndSave(ctx context.Context, configPath string, verify AnonymousVerifyFunc) (*AutoConnectResult, error)

AutoConnectFreeAndSave connects free providers and saves the updated configuration to disk.

type CatalogEntry 

type CatalogEntry struct {
	ID         string         `json:"id"`
	InstanceID string         `json:"instance_id,omitempty"`
	Provider   string         `json:"provider"`
	APIBase    string         `json:"api_base"`
	Models     []CatalogModel `json:"models"`
	FetchedAt  string         `json:"fetched_at"`
}

CatalogEntry is a saved list of upstream models fetched for a specific provider+key combination.

type CatalogLoader 

type CatalogLoader func(cfg *config.Config) (map[string]providers.InstanceCatalog, error)

CatalogLoader returns the model catalogs of cfg's provider instances, keyed by instance ID.

type CatalogModel 

type CatalogModel struct {
	ID          string `json:"id"`
	OwnedBy     string `json:"owned_by,omitempty"`
	DisplayName string `json:"display_name,omitempty"`
	// Surfaces are the llmgw-core surfaces the catalog reports the model
	// serves, such as chat_completions, audio_transcriptions and
	// audio_speech; empty when it reports none.
	Surfaces []string `json:"surfaces,omitempty"`
	// InputModalities are the inputs the catalog declares the model takes,
	// sorted, among "audio", "image" and "text"; empty when it declares
	// none. Nothing is inferred from the model's id.
	InputModalities []string `json:"input_modalities,omitempty"`
	// AudioInput reports that the catalog declares the model takes audio:
	// a transcription model, or a chat model that accepts input audio. Only
	// such chat models transcribe voice (config.VoiceConfig.STTViaChat).
	AudioInput bool `json:"audio_input,omitempty"`
}

CatalogModel represents a single model entry in a saved catalog.

func CatalogModelsFromCore 

func CatalogModelsFromCore(models []core.ModelInfo) []CatalogModel

CatalogModelsFromCore keeps what Compa needs of a core catalog: each model's id, display name, owner, the surfaces it serves and the inputs it takes.

func SyncCatalog 

func SyncCatalog(ctx context.Context, input ProviderCatalogSyncInput) ([]CatalogModel, error)

SyncCatalog returns the models an instance reaches with its credential: the catalog its llmgw-core provider lists (see providers.NewCoreProvider), an extension instance's through its daemon.

type CatalogStore 

type CatalogStore struct {
	Entries map[string]*CatalogEntry `json:"entries"`
}

CatalogStore holds all saved model catalogs.

func LoadCatalogs 

func LoadCatalogs() (*CatalogStore, error)

LoadCatalogs reads the model catalogs cache from disk.

type CatalogSyncFunc 

type CatalogSyncFunc func(ctx context.Context, input ProviderCatalogSyncInput) ([]CatalogModel, error)

CatalogSyncFunc queries models for a provider instance.

type ConnectDeps 

type ConnectDeps struct {
	// SyncCatalog discovers the models a key reaches. Nil uses SyncCatalog.
	SyncCatalog CatalogSyncFunc
	// LoadConfig and SaveConfig read and write the config file. Nil uses
	// config.LoadConfig and config.SaveConfig.
	LoadConfig func(path string) (*config.Config, error)
	SaveConfig func(path string, cfg *config.Config) error
	// GetCredential, SetCredential and DeleteCredential reach the auth
	// store. Nil uses auth.GetCredential, auth.SetCredential and
	// auth.DeleteCredential.
	GetCredential    func(key string) (*auth.AuthCredential, error)
	SetCredential    func(key string, cred *auth.AuthCredential) error
	DeleteCredential func(key string) error
}

ConnectDeps are the effects of ConnectAPIKeyProvider and DisconnectAPIKeyProvider. A nil field uses the default; tests and callers with their own seams replace them.

type PingResult 

type PingResult struct {
	OK         bool   `json:"ok"`
	InstanceID string `json:"instance_id"`
	LatencyMS  int64  `json:"latency_ms"`
	ModelCount int    `json:"model_count,omitempty"`
	Status     string `json:"status"`
	Error      string `json:"error,omitempty"`
}

PingResult describes the outcome of probing an endpoint's reachability.

func Ping 

func Ping(ctx context.Context, instance *config.ProviderInstanceConfig, secret string, client *http.Client) PingResult

Ping probes an instance's reachability and measures round-trip latency.

func PingWithSync 

func PingWithSync(ctx context.Context, instance *config.ProviderInstanceConfig, secret string, client *http.Client, syncFunc CatalogSyncFunc) PingResult

PingWithSync probes an instance by listing its catalog with syncFunc, nil using SyncCatalog; an instance without a catalog is probed with a GET of its endpoint over client.

type ProviderCatalogSyncInput 

type ProviderCatalogSyncInput struct {
	InstanceID        string
	ProviderKind      string
	Adapter           string
	Protocol          string
	Endpoint          string
	AuthConnectionRef string
	Headers           map[string]string
	Settings          map[string]any
	Runtime           *config.ProviderInstanceRuntime
	Secret            string
}

ProviderCatalogSyncInput defines parameters for syncing models from an upstream provider.

func CatalogSyncInputFromInstance 

func CatalogSyncInputFromInstance(instance *config.ProviderInstanceConfig) ProviderCatalogSyncInput

CatalogSyncInputFromInstance converts a config.ProviderInstanceConfig to ProviderCatalogSyncInput.

func (ProviderCatalogSyncInput) Instance 

func (in ProviderCatalogSyncInput) Instance() *config.ProviderInstanceConfig

Instance returns the provider instance input describes.

type ProviderRosterItem 

type ProviderRosterItem struct {
	ID                  string   `json:"id"`
	DisplayName         string   `json:"display_name"`
	Label               string   `json:"label,omitempty"`
	Description         string   `json:"description,omitempty"`
	Categories          []string `json:"categories,omitempty"`
	Adapter             string   `json:"adapter,omitempty"`
	Protocol            string   `json:"protocol,omitempty"`
	DefaultEndpoint     string   `json:"default_endpoint,omitempty"`
	Compatibility       string   `json:"compatibility"`
	AuthMethods         []string `json:"auth_methods,omitempty"`
	RequiresAPIKey      bool     `json:"requires_api_key"`
	RequiresBaseURL     bool     `json:"requires_base_url"`
	AnonymousAutomation bool     `json:"anonymous_automation"`
	OnboardingFields    []string `json:"onboarding_fields,omitempty"`
	Configured          bool     `json:"configured"`
	InstanceCount       int      `json:"instance_count"`
	ConfiguredInstances []string `json:"configured_instances,omitempty"`
}

ProviderRosterItem describes a provider from the curated registry and its local configuration status.

func ListRoster 

func ListRoster(cfg *config.Config) []ProviderRosterItem

ListRoster returns the providers Compa offers: every llmgw-core registry entry Compa serves, with the instances configured for it, once per provider (see rosterDuplicates). Core's registry is the only source of provider identity (label, description, default endpoint, auth); Compa keeps no provider presets of its own.

type Resolver 

type Resolver struct {
	// contains filtered or unexported fields
}

Resolver resolves model selections — exact targets "instance-id/model-id" and model route names — against a config's provider instances and their saved catalogs. It is the one model resolution every consumer shares: the agent, the gateway, the web API and the CLI.

func NewResolver 

func NewResolver(opts ...ResolverOption) *Resolver

NewResolver returns a resolver that loads the saved catalogs on every call (LoadInstanceCatalogs), resolves credentials from the auth store (ResolveCredentialReference) and builds providers with providers.CreateProviderFromInstance, unless opts override them.

func (*Resolver) Check 

func (r *Resolver) Check(cfg *config.Config, selection string) error

Check reports whether selection resolves against cfg and the saved catalogs: every target's instance exists and is enabled and its catalog holds the model. It neither resolves credentials nor builds providers, so it validates a selection when it is set. An empty selection sets no model and is valid.

func (*Resolver) Resolve 

func (r *Resolver) Resolve(cfg *config.Config, selection string) (*providers.InstanceResolution, error)

Resolve resolves selection against cfg as given — the caller decides which config is current — and the catalogs as saved now, so a catalog synced a moment ago is visible. The resolution's candidates are ordered: one for an exact target, the route's targets for a route. Credentials are resolved and providers built when a candidate's provider is first requested.

func (*Resolver) ResolveVoiceTarget 

func (r *Resolver) ResolveVoiceTarget(cfg *config.Config, target string) (VoiceTarget, error)

ResolveVoiceTarget is ResolveVoiceTarget resolving the instance's secret with r's credential resolver.

type ResolverOption 

type ResolverOption func(*Resolver)

ResolverOption overrides one of a Resolver's defaults.

func WithCatalogLoader 

func WithCatalogLoader(load CatalogLoader) ResolverOption

WithCatalogLoader replaces how the resolver loads catalogs. Nil keeps LoadInstanceCatalogs.

func WithCredentialResolver 

func WithCredentialResolver(resolve providers.InstanceCredentialResolver) ResolverOption

WithCredentialResolver replaces how the resolver turns an instance's auth_connection_ref into its secret. Nil keeps ResolveCredentialReference.

func WithProviderFactory 

func WithProviderFactory(create providers.InstanceProviderFactory) ResolverOption

WithProviderFactory replaces how the resolver builds a candidate's provider. Nil keeps providers.CreateProviderFromInstance.

type VoiceTarget 

type VoiceTarget struct {
	Instance *config.ProviderInstanceConfig
	Model    string
	// Secret is what the instance's auth_connection_ref resolved to; empty
	// when it has none.
	Secret string
}

VoiceTarget is what an exact voice target "instance-id/model-id" names: an enabled provider instance, one of its models and the instance's secret.

func ResolveVoiceTarget 

func ResolveVoiceTarget(cfg *config.Config, target string) (VoiceTarget, error)

ResolveVoiceTarget resolves target to its enabled provider instance, model and secret, read from the auth store.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.