Documentation
¶
Index ¶
- Variables
- func Chat(ctx context.Context, req Request, ...) (libmodelprovider.LLMChatClient, libmodelprovider.Provider, string, error)
- func Embed(ctx context.Context, embedReq EmbedRequest, ...) (libmodelprovider.LLMEmbedClient, libmodelprovider.Provider, string, error)
- func NormalizeModelName(modelName string) string
- func PromptExecute(ctx context.Context, req Request, ...) (libmodelprovider.LLMPromptExecClient, libmodelprovider.Provider, string, error)
- func Randomly(candidates []libmodelprovider.Provider) (libmodelprovider.Provider, string, error)
- func Stream(ctx context.Context, req Request, ...) (libmodelprovider.LLMStreamClient, libmodelprovider.Provider, string, error)
- type EmbedRequest
- type Request
Constants ¶
This section is empty.
Variables ¶
View Source
var ErrNoAvailableModels = errors.New("no models found in runtime state")
ErrNoAvailableModels is returned when no providers are available.
View Source
var ErrNoSatisfactoryModel = errors.New("no model matched the requirements")
ErrNoSatisfactoryModel is returned when providers exist but none match requirements.
Functions ¶
func Chat ¶
func Chat( ctx context.Context, req Request, getModels func(ctx context.Context, backendTypes ...string) ([]libmodelprovider.Provider, error), resolver func(candidates []libmodelprovider.Provider) (libmodelprovider.Provider, string, error), ) (libmodelprovider.LLMChatClient, libmodelprovider.Provider, string, error)
Chat implements the chat resolution workflow using the provided dependencies.
func Embed ¶
func Embed( ctx context.Context, embedReq EmbedRequest, getModels func(ctx context.Context, backendTypes ...string) ([]libmodelprovider.Provider, error), resolver func(candidates []libmodelprovider.Provider) (libmodelprovider.Provider, string, error), ) (libmodelprovider.LLMEmbedClient, libmodelprovider.Provider, string, error)
Embed implements the embedding resolution workflow using the provided dependencies.
func NormalizeModelName ¶
NormalizeModelName standardizes model names for comparison
func PromptExecute ¶
func PromptExecute( ctx context.Context, req Request, getModels func(ctx context.Context, backendTypes ...string) ([]libmodelprovider.Provider, error), resolver func(candidates []libmodelprovider.Provider) (libmodelprovider.Provider, string, error), ) (libmodelprovider.LLMPromptExecClient, libmodelprovider.Provider, string, error)
PromptExecute implements the prompt execution resolution workflow using the provided dependencies.
func Randomly ¶
func Randomly(candidates []libmodelprovider.Provider) (libmodelprovider.Provider, string, error)
Randomly is a policy that selects a random provider and random backend.
This provides basic load balancing across available resources.
func Stream ¶
func Stream( ctx context.Context, req Request, getModels func(ctx context.Context, backendTypes ...string) ([]libmodelprovider.Provider, error), resolver func(candidates []libmodelprovider.Provider) (libmodelprovider.Provider, string, error), ) (libmodelprovider.LLMStreamClient, libmodelprovider.Provider, string, error)
Stream implements the streaming resolution workflow using the provided dependencies.
Types ¶
type EmbedRequest ¶
type EmbedRequest struct {
ModelName string
ProviderType string
Tracker libtracker.ActivityTracker
}
type Request ¶
type Request struct {
ProviderTypes []string
ModelNames []string
ContextLength int
Tracker libtracker.ActivityTracker
}
Source Files
Click to show internal directories.
Click to hide internal directories.