Documentation
¶
Index ¶
- Variables
- func Chat(ctx context.Context, req Request, ...) (libmodelprovider.LLMChatClient, libmodelprovider.Provider, string, error)
- func Embed(ctx context.Context, embedReq EmbedRequest, ...) (libmodelprovider.LLMEmbedClient, libmodelprovider.Provider, string, error)
- func NormalizeModelName(modelName string) string
- func PromptExecute(ctx context.Context, req Request, ...) (libmodelprovider.LLMPromptExecClient, libmodelprovider.Provider, string, error)
- func Randomly(candidates []libmodelprovider.Provider) (libmodelprovider.Provider, string, error)
- func Stream(ctx context.Context, req Request, ...) (libmodelprovider.LLMStreamClient, libmodelprovider.Provider, string, error)
- type EmbedRequest
- type Policy
- type Request
- type Selection
Constants ¶
This section is empty.
Variables ¶
var ErrNoAudioCapableModel = fmt.Errorf("%w: no available model supports audio input", ErrNoSatisfactoryModel)
ErrNoAudioCapableModel is returned when an audio-bearing request has no audio-capable candidate; it wraps ErrNoSatisfactoryModel.
var ErrNoAvailableModels = errors.New("no models found in runtime state")
ErrNoAvailableModels is returned when no providers are available.
var ErrNoSatisfactoryModel = errors.New("no model matched the requirements")
ErrNoSatisfactoryModel is returned when providers exist but none match requirements.
var ErrNoVisionCapableModel = fmt.Errorf("%w: no available model supports image input (vision)", ErrNoSatisfactoryModel)
ErrNoVisionCapableModel is returned when an image-bearing request has no vision-capable candidate; it wraps ErrNoSatisfactoryModel.
var ErrPinnedModelLacksAudio = errors.New("explicitly requested model lacks audio capability")
ErrPinnedModelLacksAudio is returned when an audio-bearing request pins a model that cannot accept audio; it does not wrap ErrNoSatisfactoryModel.
var ErrPinnedModelLacksVision = errors.New("explicitly requested model lacks vision capability")
ErrPinnedModelLacksVision is returned when an image-bearing request pins a model that cannot accept images; it does not wrap ErrNoSatisfactoryModel.
Functions ¶
func Chat ¶
func Chat( ctx context.Context, req Request, getModels func(ctx context.Context, backendTypes ...string) ([]libmodelprovider.Provider, error), resolver func(candidates []libmodelprovider.Provider) (libmodelprovider.Provider, string, error), ) (libmodelprovider.LLMChatClient, libmodelprovider.Provider, string, error)
Chat implements the chat resolution workflow using the provided dependencies.
func Embed ¶
func Embed( ctx context.Context, embedReq EmbedRequest, getModels func(ctx context.Context, backendTypes ...string) ([]libmodelprovider.Provider, error), resolver func(candidates []libmodelprovider.Provider) (libmodelprovider.Provider, string, error), ) (libmodelprovider.LLMEmbedClient, libmodelprovider.Provider, string, error)
Embed implements the embedding resolution workflow using the provided dependencies.
func NormalizeModelName ¶
NormalizeModelName standardizes model names for comparison: lowercased, separators and organization prefix stripped, quantization suffixes and any trailing ":version" removed.
func PromptExecute ¶
func PromptExecute( ctx context.Context, req Request, getModels func(ctx context.Context, backendTypes ...string) ([]libmodelprovider.Provider, error), resolver func(candidates []libmodelprovider.Provider) (libmodelprovider.Provider, string, error), ) (libmodelprovider.LLMPromptExecClient, libmodelprovider.Provider, string, error)
PromptExecute implements the prompt execution resolution workflow using the provided dependencies.
func Randomly ¶
func Randomly(candidates []libmodelprovider.Provider) (libmodelprovider.Provider, string, error)
Randomly is a Policy that selects a random provider and random backend.
func Stream ¶
func Stream( ctx context.Context, req Request, getModels func(ctx context.Context, backendTypes ...string) ([]libmodelprovider.Provider, error), resolver func(candidates []libmodelprovider.Provider) (libmodelprovider.Provider, string, error), ) (libmodelprovider.LLMStreamClient, libmodelprovider.Provider, string, error)
Stream implements the streaming resolution workflow using the provided dependencies.
Types ¶
type EmbedRequest ¶
type EmbedRequest struct {
ModelName string
ProviderType string
Tracker libtracker.ActivityTracker
}
type Policy ¶
type Policy = func(candidates []libmodelprovider.Provider) (libmodelprovider.Provider, string, error)
Policy selects one provider and one of its backends from the candidate set.
func StickyOrRandom ¶
StickyOrRandom returns a Policy that pins a session to one provider+backend via rendezvous hashing over the candidate set; an empty key behaves exactly like Randomly.
type Request ¶
type Request struct {
ProviderTypes []string
ModelNames []string
ContextLength int
// RequiresVision restricts chat/stream resolution to vision-capable providers.
RequiresVision bool
// RequiresAudio restricts chat/stream resolution to audio-capable providers.
RequiresAudio bool
Tracker libtracker.ActivityTracker
}
type Selection ¶ added in v0.40.0
type Selection struct {
Provider libmodelprovider.Provider
Backend string
}
Selection is one resolved (provider, backend) pair; the first in an ordered set is the policy's pick, later ones are failover alternates.
func ChatSelections ¶ added in v0.40.0
func ChatSelections( ctx context.Context, req Request, getModels func(ctx context.Context, backendTypes ...string) ([]libmodelprovider.Provider, error), policy Policy, ) ([]Selection, error)
ChatSelections resolves the failover-ordered (provider, backend) pairs for a chat request; the caller excludes a backend whose refusal is terminal for it and moves to the next.
func PromptExecuteSelections ¶ added in v0.40.0
func PromptExecuteSelections( ctx context.Context, req Request, getModels func(ctx context.Context, backendTypes ...string) ([]libmodelprovider.Provider, error), policy Policy, ) ([]Selection, error)
PromptExecuteSelections resolves the failover-ordered (provider, backend) pairs for a prompt-execution request; see ChatSelections for the iteration contract.
func StreamSelections ¶ added in v0.40.0
func StreamSelections( ctx context.Context, req Request, getModels func(ctx context.Context, backendTypes ...string) ([]libmodelprovider.Provider, error), policy Policy, ) ([]Selection, error)
StreamSelections resolves the failover-ordered (provider, backend) pairs for a streaming request; see ChatSelections for the iteration contract.
Source Files