Documentation
¶
Index ¶
- type LocalAIClient
- func (llm *LocalAIClient) Ask(ctx context.Context, f cogito.Fragment) (cogito.Fragment, error)
- func (llm *LocalAIClient) CreateChatCompletion(ctx context.Context, request openai.ChatCompletionRequest) (cogito.LLMReply, cogito.LLMUsage, error)
- func (llm *LocalAIClient) CreateChatCompletionStream(ctx context.Context, request openai.ChatCompletionRequest) (<-chan cogito.StreamEvent, error)
- func (llm *LocalAIClient) SetGrammar(grammar string)
- func (llm *LocalAIClient) SetMetadata(metadata map[string]string)
- func (llm *LocalAIClient) SetReasoningEffort(effort string)
- func (llm *LocalAIClient) SetTemperature(temperature float32)
- type OpenAIClient
- func (llm *OpenAIClient) Ask(ctx context.Context, f cogito.Fragment) (cogito.Fragment, error)
- func (llm *OpenAIClient) CreateChatCompletion(ctx context.Context, request openai.ChatCompletionRequest) (cogito.LLMReply, cogito.LLMUsage, error)
- func (llm *OpenAIClient) CreateChatCompletionStream(ctx context.Context, request openai.ChatCompletionRequest) (<-chan cogito.StreamEvent, error)
- type OpenAIOptions
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type LocalAIClient ¶
type LocalAIClient struct {
// contains filtered or unexported fields
}
LocalAIClient is an LLM client for LocalAI-compatible APIs. It uses the same request format as OpenAI but parses an additional "reasoning" field in the response JSON (in choices[].message) and maps it to LLMReply.ReasoningContent.
func NewLocalAILLM ¶
func NewLocalAILLM(model, apiKey, baseURL string) *LocalAIClient
NewLocalAILLM creates a new LocalAI client with the same constructor signature as NewOpenAILLM. baseURL is the API base (e.g. "http://localhost:8080/v1").
func (*LocalAIClient) Ask ¶
Ask prompts the LLM with the provided messages and returns a Fragment containing the response. Uses CreateChatCompletion so reasoning is preserved. The Fragment's Status.LastUsage is updated with the token usage.
func (*LocalAIClient) CreateChatCompletion ¶
func (llm *LocalAIClient) CreateChatCompletion(ctx context.Context, request openai.ChatCompletionRequest) (cogito.LLMReply, cogito.LLMUsage, error)
CreateChatCompletion sends the chat completion request and parses the response, including LocalAI's optional "reasoning" field, into LLMReply.ReasoningContent.
func (*LocalAIClient) CreateChatCompletionStream ¶ added in v0.10.0
func (llm *LocalAIClient) CreateChatCompletionStream(ctx context.Context, request openai.ChatCompletionRequest) (<-chan cogito.StreamEvent, error)
CreateChatCompletionStream streams chat completion events via a channel using SSE.
func (*LocalAIClient) SetGrammar ¶
func (llm *LocalAIClient) SetGrammar(grammar string)
SetGrammar sets a GBNF grammar string that constrains the model's output. When set, the grammar is included in the request body sent to LocalAI.
func (*LocalAIClient) SetMetadata ¶ added in v0.10.0
func (llm *LocalAIClient) SetMetadata(metadata map[string]string)
SetMetadata sets per-request metadata forwarded to LocalAI under the top-level "metadata" object (e.g. {"enable_thinking": "true"}). Pass nil or an empty map to clear. LocalAI uses these flags to override per-model defaults like extended thinking on a single call.
func (*LocalAIClient) SetReasoningEffort ¶ added in v0.11.0
func (llm *LocalAIClient) SetReasoningEffort(effort string)
SetReasoningEffort sets the OpenAI "reasoning_effort" field on every request (e.g. "none"/"low"/"medium"/"high") — parity with OpenAIClient, so callers can switch between the two client implementations without losing this lever. Empty leaves the field unset.
func (*LocalAIClient) SetTemperature ¶ added in v0.11.0
func (llm *LocalAIClient) SetTemperature(temperature float32)
SetTemperature sets the sampling temperature on every request — parity with OpenAIClient's Temperature option. Zero leaves the field unset (the backend's own default applies).
type OpenAIClient ¶
type OpenAIClient struct {
// contains filtered or unexported fields
}
func NewOpenAILLM ¶
func NewOpenAILLM(model, apiKey, baseURL string) *OpenAIClient
func NewOpenAILLMWithOptions ¶ added in v0.11.0
func NewOpenAILLMWithOptions(model, apiKey, baseURL string, opts OpenAIOptions) *OpenAIClient
func (*OpenAIClient) Ask ¶
Ask prompts to the LLM with the provided messages and returns a Fragment containing the response. The Fragment.GetMessages() method automatically handles force-text-reply when tool calls are present in the conversation history. The Fragment's Status.LastUsage is updated with the token usage.
func (*OpenAIClient) CreateChatCompletion ¶
func (llm *OpenAIClient) CreateChatCompletion(ctx context.Context, request openai.ChatCompletionRequest) (cogito.LLMReply, cogito.LLMUsage, error)
func (*OpenAIClient) CreateChatCompletionStream ¶ added in v0.10.0
func (llm *OpenAIClient) CreateChatCompletionStream(ctx context.Context, request openai.ChatCompletionRequest) (<-chan cogito.StreamEvent, error)
CreateChatCompletionStream streams chat completion events via a channel.
type OpenAIOptions ¶ added in v0.11.0
type OpenAIOptions struct {
Temperature float32
// Metadata is attached verbatim to every chat-completion request as the
// OpenAI "metadata" object. Backends such as LocalAI use it to carry
// per-request flags, e.g. {"enable_thinking": "false"} to disable reasoning.
Metadata map[string]string
// ReasoningEffort sets the OpenAI "reasoning_effort" field on every request
// (e.g. "none"/"low"/"medium"/"high"). This is the portable, OpenAI-standard
// control for reasoning models — unlike Metadata, it binds even when the
// model's chat template has no enable_thinking toggle (e.g. LFM2.5), so it's
// the reliable way to disable thinking. Empty leaves the field unset.
ReasoningEffort string
}
OpenAIOptions carries optional per-client settings.
Source Files