Documentation
¶
Index ¶
- Variables
- func Chat(ctx context.Context, req Request, ...) (libmodelprovider.LLMChatClient, libmodelprovider.Provider, string, error)
- func Embed(ctx context.Context, embedReq EmbedRequest, ...) (libmodelprovider.LLMEmbedClient, libmodelprovider.Provider, string, error)
- func NormalizeModelName(modelName string) string
- func PromptExecute(ctx context.Context, req Request, ...) (libmodelprovider.LLMPromptExecClient, libmodelprovider.Provider, string, error)
- func Randomly(candidates []libmodelprovider.Provider) (libmodelprovider.Provider, string, error)
- func Stream(ctx context.Context, req Request, ...) (libmodelprovider.LLMStreamClient, libmodelprovider.Provider, string, error)
- type EmbedRequest
- type Policy
- type Request
Constants ¶
This section is empty.
Variables ¶
var ErrNoAvailableModels = errors.New("no models found in runtime state")
ErrNoAvailableModels is returned when no providers are available.
var ErrNoSatisfactoryModel = errors.New("no model matched the requirements")
ErrNoSatisfactoryModel is returned when providers exist but none match requirements.
var ErrNoVisionCapableModel = fmt.Errorf("%w: no available model supports image input (vision)", ErrNoSatisfactoryModel)
ErrNoVisionCapableModel is returned when a request carries image attachments but no candidate model supports vision. It wraps ErrNoSatisfactoryModel so existing no-match handling (e.g. the resolution self-heal cycle) still fires.
var ErrPinnedModelLacksVision = errors.New("explicitly requested model lacks vision capability")
ErrPinnedModelLacksVision is returned when an image-bearing request explicitly names a model that exists but cannot accept images. It does NOT wrap ErrNoSatisfactoryModel: the refusal is deterministic, so retry / self-heal cycles must not fire for it.
Functions ¶
func Chat ¶
func Chat( ctx context.Context, req Request, getModels func(ctx context.Context, backendTypes ...string) ([]libmodelprovider.Provider, error), resolver func(candidates []libmodelprovider.Provider) (libmodelprovider.Provider, string, error), ) (libmodelprovider.LLMChatClient, libmodelprovider.Provider, string, error)
Chat implements the chat resolution workflow using the provided dependencies.
func Embed ¶
func Embed( ctx context.Context, embedReq EmbedRequest, getModels func(ctx context.Context, backendTypes ...string) ([]libmodelprovider.Provider, error), resolver func(candidates []libmodelprovider.Provider) (libmodelprovider.Provider, string, error), ) (libmodelprovider.LLMEmbedClient, libmodelprovider.Provider, string, error)
Embed implements the embedding resolution workflow using the provided dependencies.
func NormalizeModelName ¶
NormalizeModelName standardizes model names for comparison: lowercased, separators and organization prefix stripped, quantization suffixes and any trailing ":version" removed.
func PromptExecute ¶
func PromptExecute( ctx context.Context, req Request, getModels func(ctx context.Context, backendTypes ...string) ([]libmodelprovider.Provider, error), resolver func(candidates []libmodelprovider.Provider) (libmodelprovider.Provider, string, error), ) (libmodelprovider.LLMPromptExecClient, libmodelprovider.Provider, string, error)
PromptExecute implements the prompt execution resolution workflow using the provided dependencies.
func Randomly ¶
func Randomly(candidates []libmodelprovider.Provider) (libmodelprovider.Provider, string, error)
Randomly is a Policy that selects a random provider and random backend.
func Stream ¶
func Stream( ctx context.Context, req Request, getModels func(ctx context.Context, backendTypes ...string) ([]libmodelprovider.Provider, error), resolver func(candidates []libmodelprovider.Provider) (libmodelprovider.Provider, string, error), ) (libmodelprovider.LLMStreamClient, libmodelprovider.Provider, string, error)
Stream implements the streaming resolution workflow using the provided dependencies.
Types ¶
type EmbedRequest ¶
type EmbedRequest struct {
ModelName string
ProviderType string
Tracker libtracker.ActivityTracker
}
type Policy ¶
type Policy = func(candidates []libmodelprovider.Provider) (libmodelprovider.Provider, string, error)
Policy selects one provider and one of its backends from the candidate set. Randomly is the canonical Policy value; StickyOrRandom builds session-affine ones.
func StickyOrRandom ¶
StickyOrRandom returns a Policy that pins a session to one provider+backend via rendezvous (highest-random-weight) hashing over the candidate set: the same sessionKey picks the same provider+backend for as long as that pair stays in the candidate set, and when it leaves only the sessions pinned to it move. Distinct keys spread across the set. An empty key behaves exactly like Randomly.
type Request ¶
type Request struct {
ProviderTypes []string
ModelNames []string
ContextLength int
// RequiresVision restricts chat/stream resolution to vision-capable
// providers. Callers derive it from the presence of image attachments in
// the request's messages (see modelrepo.MessagesHaveImages) rather than
// setting it by hand.
RequiresVision bool
Tracker libtracker.ActivityTracker
}
Source Files