modelservice

package
v0.0.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 13, 2026 License: MIT Imports: 24 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	APIKeyProviderOpenAI    = "openai"
	APIKeyProviderAnthropic = "anthropic"
)

Providers ConnectAPIKeyProvider connects. Each is also its instance's ID and its key's auth store key.

View Source 
const APIKeyAuthMethod = "api_key"

APIKeyAuthMethod is the auth method of a stored API key.

View Source 
const RedactedProxyPassword = "****"

RedactedProxyPassword replaces the password of a proxy URL in every view. Sending the redacted URL back unchanged keeps the stored proxy.

Variables

View Source 
var (
	// ErrInstanceInvalid: the input or the config it makes does not validate.
	ErrInstanceInvalid = errors.New("invalid provider instance")
	// ErrInstanceNotFound: no instance has the id.
	ErrInstanceNotFound = errors.New("provider instance not found")
	// ErrInstanceConflict: the instance exists already, or routes name it.
	ErrInstanceConflict = errors.New("provider instance conflict")
	// ErrInstanceDisabled: the instance is disabled.
	ErrInstanceDisabled = errors.New("provider instance is disabled")
	// ErrInstanceUnsupported: the instance cannot sync its catalog.
	ErrInstanceUnsupported = errors.New("provider instance cannot do this")
	// ErrInstanceUpstream: the provider or the credential store failed.
	ErrInstanceUpstream = errors.New("provider instance upstream failure")
	// ErrInstanceNotConfigured: no catalog sync driver was given.
	ErrInstanceNotConfigured = errors.New("provider instance operation is not configured")
	// ErrInstanceInternal: reading or writing the config, the catalogs or the
	// auth store failed.
	ErrInstanceInternal = errors.New("provider instance internal failure")
)

The kinds of InstanceError. errors.Is(err, ErrInstanceNotFound) and so on tell them apart.

View Source 
var ErrNoAudioInput = errors.New("does not accept audio input")

ErrNoAudioInput reports a chat model voice would transcribe with whose catalog does not declare that it takes audio.

View Source 
var ErrNoVoiceTarget = errors.New("voice target is not configured")

ErrNoVoiceTarget reports a voice request whose target is not configured.

Functions

func APIKeyProviderInstance 

func APIKeyProviderInstance(provider string) (*config.ProviderInstanceConfig, error)

APIKeyProviderInstance returns the provider instance ConnectAPIKeyProvider records for provider, authenticating with the key stored under the provider's name.

func AdoptDefaultModel 

func AdoptDefaultModel(cfg *config.Config, target string) bool

AdoptDefaultModel makes target, a chat model just verified or added to the chat shortlist, the default model when no default is set, and reports whether it did. A default already set is never replaced.

func CatalogFilePath 

func CatalogFilePath() string

CatalogFilePath returns the absolute path to the local model catalogs cache file.

func CatalogSyncSupported 

func CatalogSyncSupported(instance *config.ProviderInstanceConfig) bool

CatalogSyncSupported reports whether Compa can list instance's models: it runs a core provider for the instance.

func CheckChatTranscriptionTarget 

func CheckChatTranscriptionTarget(cfg *config.Config, store *CatalogStore, target string) error

CheckChatTranscriptionTarget reports whether target, the chat model voice transcribes with (config.VoiceConfig.STTViaChat), takes audio: the saved catalog of its instance must declare audio input for its model (see CatalogModel.AudioInput). A chat model without audio input answers a recording it never received with text of its own, which would pass for what the user said. A nil store reads the saved catalogs.

func CloneAnyMap added in v0.0.3 

func CloneAnyMap(source map[string]any) map[string]any

CloneAnyMap returns a shallow copy of source; nil stays nil.

func CloneRuntime added in v0.0.3 

func CloneRuntime(runtime *config.ProviderInstanceRuntime) *config.ProviderInstanceRuntime

CloneRuntime returns a deep copy of runtime.

func CloneStringMap added in v0.0.3 

func CloneStringMap(source map[string]string) map[string]string

CloneStringMap returns a copy of source; nil stays nil.

func ConnectAPIKeyProvider 

func ConnectAPIKeyProvider(ctx context.Context, cfgPath, provider, key string, deps ConnectDeps) error

ConnectAPIKeyProvider stores key as provider's API key and connects the provider in the config at cfgPath: it discovers the models the key reaches, records the provider's instance (see APIKeyProviderInstance; an existing instance keeps its headers and runtime settings), saves the catalog, and drops targets on the instance that the catalog no longer holds (see DropTargets). When the provider cannot connect, the previous key is restored, or the new one removed.

func CreateInstance added in v0.0.3 

func CreateInstance(configPath string, input InstanceInput, deps InstanceDeps) (*config.ProviderInstanceConfig, error)

CreateInstance adds the instance input describes to the config at configPath, and stores its API key. A key that fails to store takes the instance out of the config again. It holds the instance lock (see InstanceLockPath) throughout, so other processes' writes are not lost.

func DeleteInstance added in v0.0.3 

func DeleteInstance(configPath, id string, deps InstanceDeps) error

DeleteInstance removes the instance id, its catalog, its models from the active models and the model selections, and the API key stored for it. An instance routes name is not removed: the error lists the routes. A failure restores the config and the catalogs. It holds the instance lock (see InstanceLockPath) throughout.

func DeleteProviderInstanceCatalog 

func DeleteProviderInstanceCatalog(instanceID string) error

DeleteProviderInstanceCatalog removes catalog models for a specific instance.

func DisconnectAPIKeyProvider 

func DisconnectAPIKeyProvider(cfgPath, provider string, deps ConnectDeps) error

DisconnectAPIKeyProvider removes provider's stored API key and disconnects the provider in the config at cfgPath: every instance authenticating with the key is disabled, the targets on those instances are dropped (see DropTargets) and their catalogs deleted. When the config cannot be updated, the key and the config are restored.

func DiscoverExtension 

func DiscoverExtension(ctx context.Context, endpoint, secret string) (extension.InfoResponse, error)

DiscoverExtension returns the providers the daemon at endpoint serves.

func DropTargets 

func DropTargets(cfg *config.Config, keep func(target config.ExactModelTarget) bool)

DropTargets removes from cfg every exact target keep rejects: from the active models, from the model routes (a route left without targets is removed), and from the model selections — the default, per-agent, image and light models — which are cleared when they name a dropped target or a removed route.

func DropsTargets added in v0.0.3 

func DropsTargets(cfg *config.Config, keep func(target config.ExactModelTarget) bool) bool

DropsTargets reports whether DropTargets(cfg, keep) changes cfg: whether keep rejects an exact target among the active models, the routes' targets and the model selections. A route, and a selection naming it, only goes when one of its targets does.

func ExtensionSurface 

func ExtensionSurface(info extension.ProviderInfo) string

ExtensionSurface returns the primary surface of a daemon provider, recorded as its instance's protocol, or "" when it lists no surface core defines. Chat Completions is preferred, since Compa's OpenAI-compatible provider streams, then Messages; a provider serving neither, such as a speech-only one, is recorded under the first surface it lists.

func ExtensionSurfaces 

func ExtensionSurfaces(info extension.ProviderInfo) []string

ExtensionSurfaces returns the llmgw-core surfaces a daemon provider lists, lower-cased, de-duplicated and in the daemon's order; surfaces core does not define are left out.

func InstanceCatalogs 

func InstanceCatalogs(cfg *config.Config, store *CatalogStore) map[string]providers.InstanceCatalog

InstanceCatalogs returns the valid catalogs in store of cfg's provider instances (see ValidInstanceCatalog), keyed by instance ID.

func InstanceCredentialKey added in v0.0.3 

func InstanceCredentialKey(id string) string

InstanceCredentialKey is the auth store key of the API key stored for the instance id.

func InstanceDisplayName added in v0.0.3 

func InstanceDisplayName(instance *config.ProviderInstanceConfig) string

InstanceDisplayName is the name to show for an instance: an extension instance's provider as the daemon names it, the core registry's label of a registry provider kind, else the instance's id.

func InstanceFromCreateInput added in v0.0.3 

func InstanceFromCreateInput(input InstanceInput) (*config.ProviderInstanceConfig, error)

InstanceFromCreateInput returns the instance input describes, validated. The API key is not applied.

func InstanceFromUpdateInput added in v0.0.3 

func InstanceFromUpdateInput(input InstanceInput, existing *config.ProviderInstanceConfig) (*config.ProviderInstanceConfig, error)

InstanceFromUpdateInput returns existing as input updates it, validated. The API key is not applied.

func InstanceIndex added in v0.0.3 

func InstanceIndex(cfg *config.Config, id string) int

InstanceIndex returns the index of the instance id in cfg, or -1.

func InstanceLockPath added in v0.0.3 

func InstanceLockPath() string

InstanceLockPath is the path the instance mutators lock, through config.WithFileLock, for their whole load-modify-save of the config, the auth store and the catalogs. The OS lock is on its sidecar InstanceLockPath()+".flock" in the Compa home, so the launcher and the compa-kernel model commands, in other processes, take turns. It is not the lock SaveConfig or the catalog store take, so they still work inside it.

func KeepRedactedProxy added in v0.0.3 

func KeepRedactedProxy(incoming, stored *config.ProviderInstanceRuntime)

KeepRedactedProxy restores the stored proxy when incoming carries exactly the redacted form of it, so a round-tripped form keeps the secret. Any other value, including a new password, replaces the stored proxy.

func LoadInstanceCatalogs 

func LoadInstanceCatalogs(cfg *config.Config) (map[string]providers.InstanceCatalog, error)

LoadInstanceCatalogs reads the saved catalogs and returns those of cfg's provider instances.

func ModelInputModalities 

func ModelInputModalities(model core.ModelInfo) []string

ModelInputModalities returns the inputs a catalog row declares its model takes, sorted: text and image from its typed input capabilities, audio from its audio-in operation (see core.ModelOperationAudioIn), and any of them from an input modality list in its legacy capabilities, such as {"input_modalities": ["text", "audio"]} or {"modalities": {"input": [...]}}, or an explicit {"input_audio": true}. A row without typed capabilities is read through core.InferCapabilities. What is not declared is not supported: the model's id is never read.

func ModelSurfaces 

func ModelSurfaces(model core.ModelInfo) []string

ModelSurfaces returns the surfaces a catalog row reports its model serves, from its supported APIs and typed capabilities, sorted. A row that reports neither has none: its surfaces are unknown.

Audio in and out on a chat model are what its chat takes and gives (input audio, spoken answers), not the transcription and speech endpoints: only a row that serves no chat gets those surfaces from its capabilities; a chat row gets them only by listing their endpoints.

func NewExtensionClient 

func NewExtensionClient(endpoint, secret string) (*extension.Client, error)

NewExtensionClient returns the client of the daemon at endpoint using secret. The secret only goes to a daemon providers.CheckExtensionEndpoint accepts.

func NormalizeRuntime added in v0.0.3 

func NormalizeRuntime(runtime *config.ProviderInstanceRuntime) *config.ProviderInstanceRuntime

NormalizeRuntime returns a copy of runtime with its text settings trimmed, or nil when every setting is the provider default.

func RedactProxyURL added in v0.0.3 

func RedactProxyURL(raw string) string

RedactProxyURL returns raw with the password of its userinfo replaced by RedactedProxyPassword. URLs without a password come back unchanged.

func RedactRuntime added in v0.0.3 

func RedactRuntime(runtime *config.ProviderInstanceRuntime) *config.ProviderInstanceRuntime

RedactRuntime returns a copy of runtime safe to show.

func ResolveCredentialReference 

func ResolveCredentialReference(ref string) (string, error)

ResolveCredentialReference returns the secret an instance's auth_connection_ref names. The one reference kind is credential:<key>: the access token stored under key in the auth store, which must not be expired.

func RoutesReferencingInstance added in v0.0.3 

func RoutesReferencingInstance(routes []*config.ModelRouteConfig, instanceID string) []string

RoutesReferencingInstance returns the names of the routes with a target of the instance, sorted; never nil.

func SaveCatalogs 

func SaveCatalogs(store *CatalogStore) error

SaveCatalogs atomically writes the model catalogs cache to disk.

func SaveProviderInstanceCatalog 

func SaveProviderInstanceCatalog(instance *config.ProviderInstanceConfig, models []CatalogModel) error

SaveProviderInstanceCatalog saves catalog models for a specific instance.

func ServesChat 

func ServesChat(surfaces []string) bool

ServesChat reports whether a catalog model with surfaces can be selected for chat: Compa's core provider serves Chat Completions natively or through translation.Adapter, as translation.ServesChat reports. A catalog that reports no surfaces leaves them unknown, so such a model stays selectable.

func SortedKeys added in v0.0.3 

func SortedKeys[V any](m map[string]V) []string

SortedKeys returns the keys of m, sorted; never nil.

func UpdateInstance added in v0.0.3 

func UpdateInstance(configPath, id string, input InstanceInput, deps InstanceDeps) (*config.ProviderInstanceConfig, error)

UpdateInstance updates the instance id as input describes. input.ID must be id. A new connection - kind, protocol, endpoint, adapter, credential, headers or settings - drops the instance's catalog and its models from the active models, the routes and the model selections, as disabling it does; disabling an instance routes name is refused. A failure restores the config, and the credential it replaced; when the stored credential the instance uses cannot be read, nothing changes. It holds the instance lock (see InstanceLockPath) throughout.

func ValidInstanceCatalog 

func ValidInstanceCatalog(key string, entry *CatalogEntry, instance *config.ProviderInstanceConfig) bool

ValidInstanceCatalog reports whether entry, saved under key, is the catalog of instance: saved under the instance's ID, for that instance, from the instance's provider kind. A catalog left behind by a deleted instance or a re-created one of another kind is not.

func ValidateInstance added in v0.0.3 

func ValidateInstance(instance *config.ProviderInstanceConfig) error

ValidateInstance checks an instance as the API accepts it: valid, with an adapter Compa runs and, when it has one, an absolute endpoint URL without userinfo or query.

Types

type AnonymousProviderOutcome 

type AnonymousProviderOutcome struct {
	RegistryID string   `json:"registry_id"`
	ProviderID string   `json:"provider_id"`
	Status     string   `json:"status"`
	Models     []string `json:"models,omitempty"`
	ProbeModel string   `json:"probe_model,omitempty"`
	LatencyMS  int64    `json:"latency_ms,omitempty"`
	ErrorClass string   `json:"error_class,omitempty"`
	Error      string   `json:"error,omitempty"`
	// contains filtered or unexported fields
}

AnonymousProviderOutcome reports catalog and inference evidence separately. Models are the free models the provider's catalog lists and ProbeModel the one it was tested with; Error is a short sentence, never an upstream's answer.

type AnonymousVerifyFunc 

type AnonymousVerifyFunc func(ctx context.Context, cfg *config.Config) []AnonymousProviderOutcome

AnonymousVerifyFunc checks the free providers that need no key against cfg, which it reads but never changes, and reports each one's outcome.

type AutoConnectResult 

type AutoConnectResult struct {
	OK                bool                       `json:"ok"`
	Total             int                        `json:"total"`
	CatalogDiscovered int                        `json:"catalog_discovered"`
	Verified          int                        `json:"verified"`
	Instances         []string                   `json:"instances"`
	Outcomes          []AnonymousProviderOutcome `json:"outcomes"`
	// DefaultModel is the verified model this connection made the default
	// model, there being none; "" when it set none.
	DefaultModel string `json:"default_model,omitempty"`
}

AutoConnectResult reports the results of auto-connecting the free providers that need no key.

func AutoConnectFree 

func AutoConnectFree(ctx context.Context, cfg *config.Config, verify AnonymousVerifyFunc) (*AutoConnectResult, error)

AutoConnectFree checks the free providers that need no key and, for each whose test answer came back, enrolls its instance with the free models its catalog lists and adds the model that answered to Chat. The first model added becomes the default model when none is set. An instance the owner set up otherwise is never changed.

func AutoConnectFreeAndSave 

func AutoConnectFreeAndSave(ctx context.Context, configPath string, verify AnonymousVerifyFunc) (*AutoConnectResult, error)

AutoConnectFreeAndSave connects free providers and saves the updated configuration to disk.

type CatalogEntry 

type CatalogEntry struct {
	ID         string         `json:"id"`
	InstanceID string         `json:"instance_id,omitempty"`
	Provider   string         `json:"provider"`
	APIBase    string         `json:"api_base"`
	Models     []CatalogModel `json:"models"`
	FetchedAt  string         `json:"fetched_at"`
}

CatalogEntry is a saved list of upstream models fetched for a specific provider+key combination.

type CatalogLoader 

type CatalogLoader func(cfg *config.Config) (map[string]providers.InstanceCatalog, error)

CatalogLoader returns the model catalogs of cfg's provider instances, keyed by instance ID.

type CatalogModel 

type CatalogModel struct {
	ID          string `json:"id"`
	OwnedBy     string `json:"owned_by,omitempty"`
	DisplayName string `json:"display_name,omitempty"`
	// Surfaces are the llmgw-core surfaces the catalog reports the model
	// serves, such as chat_completions, audio_transcriptions and
	// audio_speech; empty when it reports none.
	Surfaces []string `json:"surfaces,omitempty"`
	// InputModalities are the inputs the catalog declares the model takes,
	// sorted, among "audio", "image" and "text"; empty when it declares
	// none. Nothing is inferred from the model's id.
	InputModalities []string `json:"input_modalities,omitempty"`
	// AudioInput reports that the catalog declares the model takes audio:
	// a transcription model, or a chat model that accepts input audio. Only
	// such chat models transcribe voice (config.VoiceConfig.STTViaChat).
	AudioInput bool `json:"audio_input,omitempty"`
}

CatalogModel represents a single model entry in a saved catalog.

func CatalogModelsFromCore 

func CatalogModelsFromCore(models []core.ModelInfo) []CatalogModel

CatalogModelsFromCore keeps what Compa needs of a core catalog: each model's id, display name, owner, the surfaces it serves and the inputs it takes.

func SyncCatalog 

func SyncCatalog(ctx context.Context, input ProviderCatalogSyncInput) ([]CatalogModel, error)

SyncCatalog returns the models an instance reaches with its credential: the catalog its llmgw-core provider lists (see providers.NewCoreProvider), an extension instance's through its daemon.

type CatalogStore 

type CatalogStore struct {
	Entries map[string]*CatalogEntry `json:"entries"`
}

CatalogStore holds all saved model catalogs.

func LoadCatalogs 

func LoadCatalogs() (*CatalogStore, error)

LoadCatalogs reads the model catalogs cache from disk.

type CatalogSyncFunc 

type CatalogSyncFunc func(ctx context.Context, input ProviderCatalogSyncInput) ([]CatalogModel, error)

CatalogSyncFunc queries models for a provider instance.

type ConnectDeps 

type ConnectDeps struct {
	// SyncCatalog discovers the models a key reaches. Nil uses SyncCatalog.
	SyncCatalog CatalogSyncFunc
	// LoadConfig and SaveConfig read and write the config file. Nil uses
	// config.LoadConfig and config.SaveConfig.
	LoadConfig func(path string) (*config.Config, error)
	SaveConfig func(path string, cfg *config.Config) error
	// GetCredential, SetCredential and DeleteCredential reach the auth
	// store. Nil uses auth.GetCredential, auth.SetCredential and
	// auth.DeleteCredential.
	GetCredential    func(key string) (*auth.AuthCredential, error)
	SetCredential    func(key string, cred *auth.AuthCredential) error
	DeleteCredential func(key string) error
}

ConnectDeps are the effects of ConnectAPIKeyProvider and DisconnectAPIKeyProvider. A nil field uses the default; tests and callers with their own seams replace them.

type InstanceCatalog added in v0.0.3 

type InstanceCatalog struct {
	InstanceID   string         `json:"instance_id"`
	ProviderKind string         `json:"provider_kind"`
	Models       []CatalogModel `json:"models"`
	FetchedAt    string         `json:"fetched_at"`
}

InstanceCatalog is the saved catalog of one instance.

func ListInstanceCatalogs added in v0.0.3 

func ListInstanceCatalogs(cfg *config.Config, store *CatalogStore) []InstanceCatalog

ListInstanceCatalogs returns the saved catalogs of cfg's instances, sorted by instance and their models by id; never nil.

type InstanceDeps added in v0.0.3 

type InstanceDeps struct {
	GetCredential     func(key string) (*auth.AuthCredential, error)
	SetCredential     func(key string, cred *auth.AuthCredential) error
	DeleteCredential  func(key string) error
	ResolveCredential func(ref string) (string, error)
	SyncCatalog       CatalogSyncFunc
}

InstanceDeps are the effects of the instance operations. A nil credential function uses the auth store. SyncCatalog and ResolveCredential have no default: DefaultInstanceDeps fills them.

func DefaultInstanceDeps added in v0.0.3 

func DefaultInstanceDeps() InstanceDeps

DefaultInstanceDeps are the production effects: the auth store, ResolveCredentialReference and SyncCatalog.

type InstanceError added in v0.0.3 

type InstanceError struct {
	Kind    error
	Message string
	Routes  []string
	Err     error
}

InstanceError is why an instance operation failed. Message is the whole sentence to show, as the launcher's API answers it; Kind is one of the ErrInstance* errors; Routes are the routes that name the instance, for a conflict that they cause.

func (*InstanceError) Error added in v0.0.3 

func (e *InstanceError) Error() string

func (*InstanceError) Is added in v0.0.3 

func (e *InstanceError) Is(target error) bool

Is matches the error's Kind.

func (*InstanceError) Unwrap added in v0.0.3 

func (e *InstanceError) Unwrap() error

type InstanceInput added in v0.0.3 

type InstanceInput struct {
	ID                string
	ProviderKind      string
	Adapter           string
	Protocol          string
	Endpoint          string
	Runtime           *config.ProviderInstanceRuntime
	State             config.ProviderInstanceState
	AuthConnectionRef *string
	APIKey            *string
	Headers           *map[string]string
	Settings          *map[string]any
	ClearAuth         bool
	ClearHeaders      bool
	ClearSettings     bool
}

InstanceInput describes a provider instance to create, or what an update sets: the fields of the launcher's create and update request bodies. AuthConnectionRef, APIKey, Headers and Settings are write-only; nil leaves them as they are. A non-blank APIKey is stored in the auth store under InstanceCredentialKey(ID), and the instance authenticates with it. Runtime nil keeps an updated instance's runtime settings; {} resets them.

func InputFromInstance added in v0.0.3 

func InputFromInstance(instance *config.ProviderInstanceConfig) InstanceInput

InputFromInstance is the update input that keeps instance as it is: its public fields, with every write-only field left unchanged.

type InstanceView added in v0.0.3 

type InstanceView struct {
	ID                   string                          `json:"id"`
	DisplayName          string                          `json:"display_name"`
	ProviderKind         string                          `json:"provider_kind"`
	Adapter              string                          `json:"adapter"`
	Protocol             string                          `json:"protocol"`
	Endpoint             string                          `json:"endpoint"`
	State                config.ProviderInstanceState    `json:"state"`
	CredentialConfigured bool                            `json:"credential_configured"`
	HeaderNames          []string                        `json:"header_names"`
	SettingNames         []string                        `json:"setting_names"`
	Runtime              *config.ProviderInstanceRuntime `json:"runtime,omitempty"`
}

InstanceView is what can be shown of a provider instance: its credential reference, headers and settings are write-only, so only whether a credential is configured and the header and setting names show.

func ListInstances added in v0.0.3 

func ListInstances(cfg *config.Config) []InstanceView

ListInstances returns the views of cfg's instances, sorted by id; never nil.

func NewInstanceView added in v0.0.3 

func NewInstanceView(instance *config.ProviderInstanceConfig) InstanceView

NewInstanceView returns the view of instance.

type PingResult 

type PingResult struct {
	OK         bool   `json:"ok"`
	InstanceID string `json:"instance_id"`
	LatencyMS  int64  `json:"latency_ms"`
	ModelCount int    `json:"model_count,omitempty"`
	Status     string `json:"status"`
	Error      string `json:"error,omitempty"`
}

PingResult describes the outcome of probing an endpoint's reachability.

func Ping 

func Ping(ctx context.Context, instance *config.ProviderInstanceConfig, secret string, client *http.Client) PingResult

Ping probes an instance's reachability and measures round-trip latency.

func PingWithSync 

func PingWithSync(ctx context.Context, instance *config.ProviderInstanceConfig, secret string, client *http.Client, syncFunc CatalogSyncFunc) PingResult

PingWithSync probes an instance by listing its catalog with syncFunc, nil using SyncCatalog; an instance without a catalog is probed with a GET of its endpoint over client.

type ProviderCatalogSyncInput 

type ProviderCatalogSyncInput struct {
	InstanceID        string
	ProviderKind      string
	Adapter           string
	Protocol          string
	Endpoint          string
	AuthConnectionRef string
	Headers           map[string]string
	Settings          map[string]any
	Runtime           *config.ProviderInstanceRuntime
	Secret            string
}

ProviderCatalogSyncInput defines parameters for syncing models from an upstream provider.

func CatalogSyncInputFromInstance 

func CatalogSyncInputFromInstance(instance *config.ProviderInstanceConfig) ProviderCatalogSyncInput

CatalogSyncInputFromInstance converts a config.ProviderInstanceConfig to ProviderCatalogSyncInput.

func (ProviderCatalogSyncInput) Instance 

func (in ProviderCatalogSyncInput) Instance() *config.ProviderInstanceConfig

Instance returns the provider instance input describes.

type ProviderRosterItem 

type ProviderRosterItem struct {
	ID                  string   `json:"id"`
	DisplayName         string   `json:"display_name"`
	Label               string   `json:"label,omitempty"`
	Description         string   `json:"description,omitempty"`
	Categories          []string `json:"categories,omitempty"`
	Adapter             string   `json:"adapter,omitempty"`
	Protocol            string   `json:"protocol,omitempty"`
	DefaultEndpoint     string   `json:"default_endpoint,omitempty"`
	Compatibility       string   `json:"compatibility"`
	AuthMethods         []string `json:"auth_methods,omitempty"`
	RequiresAPIKey      bool     `json:"requires_api_key"`
	RequiresBaseURL     bool     `json:"requires_base_url"`
	AnonymousAutomation bool     `json:"anonymous_automation"`
	OnboardingFields    []string `json:"onboarding_fields,omitempty"`
	Configured          bool     `json:"configured"`
	InstanceCount       int      `json:"instance_count"`
	ConfiguredInstances []string `json:"configured_instances,omitempty"`
}

ProviderRosterItem describes a provider from the curated registry and its local configuration status.

func ListRoster 

func ListRoster(cfg *config.Config) []ProviderRosterItem

ListRoster returns the providers Compa offers: every llmgw-core registry entry Compa serves, with the instances configured for it, once per provider (see rosterDuplicates). Core's registry is the only source of provider identity (label, description, default endpoint, auth); Compa keeps no provider presets of its own.

type ProviderTarget added in v0.0.3 

type ProviderTarget struct {
	Target     string `json:"target"`
	InstanceID string `json:"instance_id"`
	ModelID    string `json:"model_id"`
	// Label is the name to show for the model: its catalog display name,
	// else its id.
	Label string `json:"label"`
	// InstanceLabel is the name to show for the target's instance (see
	// InstanceDisplayName).
	InstanceLabel string `json:"instance_label"`
	ProviderKind  string `json:"provider_kind"`
	OwnedBy       string `json:"owned_by,omitempty"`
	DisplayName   string `json:"display_name,omitempty"`
	// Surfaces are the llmgw-core surfaces the catalog reports the model
	// serves, such as audio_transcriptions and audio_speech.
	Surfaces  []string `json:"surfaces,omitempty"`
	FetchedAt string   `json:"fetched_at"`
	// AudioInput reports that the catalog declares the model takes audio
	// (see CatalogModel.AudioInput). It is not part of the JSON.
	AudioInput bool `json:"-"`
}

ProviderTarget is one model of an enabled instance's catalog.

func ChatShortlistTargets added in v0.0.3 

func ChatShortlistTargets(cfg *config.Config, all []ProviderTarget) []ProviderTarget

ChatShortlistTargets returns the targets of all that are on cfg's chat shortlist (the active models) and serve chat; never nil. Chat offers these.

func ListProviderTargets added in v0.0.3 

func ListProviderTargets(cfg *config.Config, store *CatalogStore) []ProviderTarget

ListProviderTargets returns every model of the enabled instances' catalogs, sorted by target; never nil.

type Resolver 

type Resolver struct {
	// contains filtered or unexported fields
}

Resolver resolves model selections — exact targets "instance-id/model-id" and model route names — against a config's provider instances and their saved catalogs. It is the one model resolution every consumer shares: the agent, the gateway, the web API and the CLI.

func NewResolver 

func NewResolver(opts ...ResolverOption) *Resolver

NewResolver returns a resolver that loads the saved catalogs on every call (LoadInstanceCatalogs), resolves credentials from the auth store (ResolveCredentialReference) and builds providers with providers.CreateProviderFromInstance, unless opts override them.

func (*Resolver) Check 

func (r *Resolver) Check(cfg *config.Config, selection string) error

Check reports whether selection resolves against cfg and the saved catalogs: every target's instance exists and is enabled and its catalog holds the model. It neither resolves credentials nor builds providers, so it validates a selection when it is set. An empty selection sets no model and is valid.

func (*Resolver) Resolve 

func (r *Resolver) Resolve(cfg *config.Config, selection string) (*providers.InstanceResolution, error)

Resolve resolves selection against cfg as given — the caller decides which config is current — and the catalogs as saved now, so a catalog synced a moment ago is visible. The resolution's candidates are ordered: one for an exact target, the route's targets for a route. Credentials are resolved and providers built when a candidate's provider is first requested.

func (*Resolver) ResolveVoiceTarget 

func (r *Resolver) ResolveVoiceTarget(cfg *config.Config, target string) (VoiceTarget, error)

ResolveVoiceTarget is ResolveVoiceTarget resolving the instance's secret with r's credential resolver.

type ResolverOption 

type ResolverOption func(*Resolver)

ResolverOption overrides one of a Resolver's defaults.

func WithCatalogLoader 

func WithCatalogLoader(load CatalogLoader) ResolverOption

WithCatalogLoader replaces how the resolver loads catalogs. Nil keeps LoadInstanceCatalogs.

func WithCredentialResolver 

func WithCredentialResolver(resolve providers.InstanceCredentialResolver) ResolverOption

WithCredentialResolver replaces how the resolver turns an instance's auth_connection_ref into its secret. Nil keeps ResolveCredentialReference.

func WithProviderFactory 

func WithProviderFactory(create providers.InstanceProviderFactory) ResolverOption

WithProviderFactory replaces how the resolver builds a candidate's provider. Nil keeps providers.CreateProviderFromInstance.

type SyncResult added in v0.0.3 

type SyncResult struct {
	InstanceID string         `json:"instance_id"`
	Models     []CatalogModel `json:"models"`
	Total      int            `json:"total"`
}

SyncResult is a synced catalog: the launcher's catalog sync answer.

func SyncInstanceCatalog added in v0.0.3 

func SyncInstanceCatalog(ctx context.Context, configPath, id string, deps InstanceDeps) (*SyncResult, error)

SyncInstanceCatalog fetches the instance id's models through deps.SyncCatalog and saves them as its catalog. The models the new catalog no longer lists leave the active models, the routes and the model selections. A disabled instance, an adapter that cannot list models or an instance without an endpoint is refused. The fetch runs without the instance lock; the save then takes it, loads the config again and applies the catalog to the instance as it is now. An instance removed, made ineligible or given a new connection during the fetch is refused.

type VoiceTarget 

type VoiceTarget struct {
	Instance *config.ProviderInstanceConfig
	Model    string
	// Secret is what the instance's auth_connection_ref resolved to; empty
	// when it has none.
	Secret string
}

VoiceTarget is what an exact voice target "instance-id/model-id" names: an enabled provider instance, one of its models and the instance's secret.

func ResolveVoiceTarget 

func ResolveVoiceTarget(cfg *config.Config, target string) (VoiceTarget, error)

ResolveVoiceTarget resolves target to its enabled provider instance, model and secret, read from the auth store.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.