Documentation
¶
Overview ¶
Package cloudflare implements an llms.Model for Cloudflare AI Gateway and compatible APIs. Configure with an API token, server URL, and model ID.
Example
mdl, err := cloudflare.New(
cloudflare.WithToken(os.Getenv("CLOUDFLARE_API_TOKEN")),
cloudflare.WithServerURL("https://api.cloudflare.com/client/v4/accounts/<acct>/ai/run"),
cloudflare.WithModel("@cf/meta/llama-3.1-8b-instruct"),
)
_ = mdl; _ = err
Index ¶
- Variables
- type HTTPClient
- type LLM
- func (o *LLM) CreateEmbedding(ctx context.Context, inputTexts []string) ([][]float32, error)
- func (o *LLM) GenerateContent(ctx context.Context, messages []llms.Message, options ...llms.CallOption) (*llms.ContentResponse, error)
- func (o *LLM) GetName() string
- func (o *LLM) GetProviderType() llms.ProviderType
- type Option
Constants ¶
This section is empty.
Variables ¶
var ( ErrEmptyResponse = errors.New("no response") ErrIncompleteEmbedding = errors.New("not all input got embedded") )
Functions ¶
This section is empty.
Types ¶
type HTTPClient ¶ added in v0.16.114
type LLM ¶
type LLM struct {
// contains filtered or unexported fields
}
LLM is a cloudflare LLM implementation.
func (*LLM) CreateEmbedding ¶
CreateEmbedding creates embeddings for the given input texts.
func (*LLM) GenerateContent ¶
func (o *LLM) GenerateContent(ctx context.Context, messages []llms.Message, options ...llms.CallOption) (*llms.ContentResponse, error)
GenerateContent implements the Model interface.
func (*LLM) GetProviderType ¶ added in v0.10.54
func (o *LLM) GetProviderType() llms.ProviderType
GetProviderType implements the Model interface.
type Option ¶
type Option func(*options)
func WithAccountID ¶
WithAccountID Set the Account Id of the cloudflare account to use.
func WithEmbeddingModel ¶
func WithHTTPClient ¶
func WithHTTPClient(client HTTPClient) Option
WithHTTPClient Set custom http client.
func WithServerURL ¶
WithServerURL Set the URL of the cloudflare Workers AI service.
func WithSystemPrompt ¶
WithSystemPrompt Set the system prompt. This is only valid if WithCustomTemplate is not set and the cloudflare model use .System in its model template OR if WithCustomTemplate is set using {{.System}}.
Source Files
Directories