llmkit

type Config ¶

type Config struct {
	BaseURL      string
	Model        string
	APIKey       string
	Organization string
	Project      string
	Timeout      time.Duration
	IdleTimeout  time.Duration
	MaxRetries   int
	Headers      map[string]string
	Params       spec.LLMParams
}

Config is the RESOLVED endpoint + request config for one call. Build it with Default(), overlay authored blocks with Apply, then overlay the operator env with FromEnv — the same field-wise precedence the pipeline documents (env > step > entity > default).

func Default() Config

func (c Config) Apply(s spec.LLMSpec) Config

func (c Config) FromEnv() Config

func (c Config) Normalize() Config

type EmptyCompletionError ¶ added in v0.2026263.1023

type EmptyCompletionError struct {
	ReasoningBytes int
	FinishReason   string
	// Reasoning is the FULL reasoning text produced before the empty completion.
	// It is the diagnostic a runaway RCA needs: a byte count cannot distinguish a
	// model re-deriving one point (a loop) from one deliberating a genuinely long
	// chain. Kept on the typed error so no caller can drop it.
	Reasoning string
}

EmptyCompletionError is the DETERMINISTIC "the provider returned a completion with no content and no tool calls" failure. It is exported and typed so a caller can tell it apart from a transport/idle failure: it is NOT retryable with the same parameters (a `finish_reason: length` outcome is a budget decision, not a flake), and it carries the diagnostic fields an RCA needs — how much reasoning was produced and the provider's own stop reason.

func (e *EmptyCompletionError) Error() string

type Message ¶

type Message struct {
	Role       string
	Content    *string
	ToolCalls  []ToolCall
	ToolCallID string
	Reasoning  string
	// FinishReason is the provider's completion stop reason for the turn
	// ("stop", "length", "tool_calls", "content_filter", …). It is surfaced so a
	// caller can tell a TRUNCATED turn ("length") from a natural stop — the
	// distinction the unbounded-generation RCA turns on. Empty when the provider
	// reported none.
	FinishReason string
	// Usage is the provider's token accounting for the turn when it reports one
	// (llmkit requests it via stream_options.include_usage). It carries the
	// reasoning-token count that makes an unbounded reasoning generation
	// measurable rather than inferred. nil when the provider reported no usage.
	Usage *Usage
}

Message is the caller's transcript row. It is deliberately NOT a wire type (the SDK owns the wire): Content is a *string so "no content" (a pure tool-call turn) is distinguishable from "".

Reasoning carries ollama's NON-STANDARD `reasoning` output for the turn when the provider emitted one. It is SURFACED here rather than only used for the empty-completion diagnostic, so a caller (and a test) can observe that the non-standard field was actually read — the client captures it from the raw JSON because the OpenAI schema has no slot for it.

func Chat(ctx context.Context, cfg Config, msgs []openai.ChatCompletionMessageParamUnion, tools []openai.ChatCompletionToolUnionParam) (Message, error)

type StallError ¶ added in v0.2026263.1023

type StallError struct {
	Idle           time.Duration
	ContentBytes   int
	ReasoningBytes int
	ToolCalls      int
	// Reasoning is the FULL reasoning text produced before the stream stalled.
	Reasoning string
}

StallError is the typed "the stream stopped producing chunks for the idle bound" failure — the provider-unanswered class. It is TYPED (not a string to match) so a caller can classify it without coupling to the message wording, and it carries the diagnostic counters (how much content/reasoning/tool activity preceded the silence) that an RCA needs.

func (e *StallError) Error() string

type ToolCall ¶

type ToolCall struct {
	ID        string
	Name      string
	Arguments string
}

ToolCall is one assembled tool call.

func FromSDKTools(calls []openai.ChatCompletionMessageToolCallUnion) []ToolCall

type Usage ¶ added in v0.2026263.1023

type Usage struct {
	PromptTokens     int
	CompletionTokens int
	TotalTokens      int
	ReasoningTokens  int
}

Usage is one turn's token accounting. ReasoningTokens is the subset of CompletionTokens the model spent thinking (ollama/OpenAI-compatible providers report it under completion_tokens_details.reasoning_tokens); it is 0 when the provider does not break it out.