Documentation
¶
Index ¶
- Constants
- func DefaultOptions() []llm.Option
- type Provider
- func (p *Provider) CountTokens(_ context.Context, req tokencount.TokenCountRequest) (*tokencount.TokenCount, error)
- func (p *Provider) CreateStream(ctx context.Context, opts llm.Request) (llm.Stream, error)
- func (p *Provider) DefaultModel() string
- func (p *Provider) Download(ctx context.Context, models []llm.Model) error
- func (p *Provider) FetchModels(ctx context.Context) ([]llm.Model, error)
- func (p *Provider) Models() []llm.Model
- func (p *Provider) Name() string
- func (p *Provider) WithDefaultModel(modelID string) *Provider
Constants ¶
const ( ModelGLM47Flash = "glm-4.7-flash" ModelMinistral38B = "ministral-3:8b" ModelRNJ1 = "rnj-1" ModelFunctionGemma = "functiongemma" ModelDevstralSmall2 = "devstral-small-2" ModelNemotron3Nano30 = "nemotron-3-nano:30b" ModelLlama321B = "llama3.2:1b" ModelQwen317B = "qwen3:1.7b" ModelQwen306B = "qwen3:0.6b" ModelGranite31MoE1B = "granite3.1-moe:1b" ModelQwen2505B = "qwen2.5:0.5b" ModelLlama32 = "llama3.2" ModelLlama31 = "llama3.1" ModelQwen25 = "qwen2.5" ModelPhi3 = "phi3" ModelDeepSeekR1 = "deepseek-r1" ModelMistral = "mistral" ModelGemma3 = "gemma3" )
Known model IDs for Ollama. These are tested and known to work well with the chat API.
const (
ModelDefault = ModelGLM47Flash
)
Variables ¶
This section is empty.
Functions ¶
func DefaultOptions ¶ added in v0.12.0
DefaultOptions returns the default options for Ollama. Base URL defaults to http://localhost:11434. No API key is required for Ollama.
Types ¶
type Provider ¶
type Provider struct {
// contains filtered or unexported fields
}
Provider implements the Ollama (local) LLM backend.
func New ¶
New creates a new Ollama provider. Options are applied on top of DefaultOptions().
Example usage:
// Use defaults (localhost:11434)
p := ollama.New()
// Custom base URL
p := ollama.New(llm.WithBaseURL("http://remote-host:11434"))
func (*Provider) CountTokens ¶ added in v0.24.0
func (p *Provider) CountTokens(_ context.Context, req tokencount.TokenCountRequest) (*tokencount.TokenCount, error)
CountTokens estimates the number of input tokens for the given request.
This is a local approximation using the cl100k_base BPE encoding — no network call is made. Ollama does not expose a public tokenize API endpoint; the internal Tokenize function is not accessible over HTTP. Models hosted by Ollama use a variety of tokenizers (llama, qwen, phi, gemma); cl100k_base gives a rough ±10% estimate for English text.
func (*Provider) CreateStream ¶
func (*Provider) DefaultModel ¶
DefaultModel returns the configured default model ID.
func (*Provider) Download ¶
Download pulls the specified models from the Ollama registry. This method blocks until all models are downloaded. Models that are already installed will be skipped.
func (*Provider) FetchModels ¶
FetchModels retrieves the list of currently installed models from Ollama. This enumerates ALL models, including ones that may not support chat.
func (*Provider) Models ¶
Models returns a curated list of tested models that are known to work with streaming, tool calling, and conversations. These models are verified to be compatible with all features.
func (*Provider) WithDefaultModel ¶
WithDefaultModel sets the default model to use.
Source Files