ai

package
v0.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 24, 2026 License: Apache-2.0 Imports: 4 Imported by: 0

Documentation

Overview

Package ai is the product-neutral AI contract shared by Sneat, DataTug and future products: chat requests, the normalised streaming event model, and the LLMProvider interface every adapter (the ai/cloud client, OpenAI-compatible, Anthropic) implements.

Products own their scopes, actions, prompts and controls. This package owns only what every product needs to talk to a model the same way.

Index

Constants

View Source
const (
	ToolChoiceAuto     = "auto"
	ToolChoiceNone     = "none"
	ToolChoiceRequired = "required"
)

ToolChoice values for ChatRequest.ToolChoice.

View Source
const (
	ReasoningLow    = "low"
	ReasoningMedium = "medium"
	ReasoningHigh   = "high"
)

Reasoning effort levels for ChatRequest.Reasoning.

View Source
const (
	StopReasonToolCalls = "tool_calls"
	StopReasonEnd       = "end"
	StopReasonLength    = "length"
	// StopReasonRefusal: the provider's own safety layer declined to
	// answer (e.g. Anthropic stop_reason "refusal"). Not an ai.Error --
	// the response completed normally, just with no usable content.
	StopReasonRefusal = "refusal"
	// StopReasonPauseTurn: the provider paused mid-turn expecting the
	// caller to continue the SAME turn with another request (e.g.
	// Anthropic stop_reason "pause_turn", used with long-running
	// server-side tools). Not a stopping point a caller should treat as
	// "done" the way StopReasonEnd is.
	StopReasonPauseTurn = "pause_turn"
	// StopReasonContentFilter: the provider stopped generation because a
	// content filter flagged the response (e.g. ai/openairesponses'
	// response.incomplete with incomplete_details.reason
	// "content_filter"). Like StopReasonRefusal, not an ai.Error -- the
	// response completed, just with content the provider declined to
	// finish delivering.
	StopReasonContentFilter = "content_filter"
)

StopReason values for Event.StopReason on EventCompleted.

View Source
const (
	ErrCodeAuth        = "auth"         // missing/invalid credentials
	ErrCodeQuota       = "quota"        // allowance exhausted
	ErrCodeRateLimited = "rate_limited" // retry later
	ErrCodeUpstream    = "upstream"     // provider failure
	ErrCodeInvalid     = "invalid"      // bad request
	ErrCodeCanceled    = "canceled"
)

Error codes shared by all adapters and the cloud protocol.

View Source
const ModelAuto = "auto"

ModelAuto asks the serving side to pick the model (ai/cloud routing). BYOK adapters treat it as "use the configured model".

Variables

This section is empty.

Functions

This section is empty.

Types

type Allowance

type Allowance struct {
	Unit     string    `json:"unit"` // e.g. "tokens", "requests"
	Used     int64     `json:"used"`
	Limit    int64     `json:"limit"`
	ResetsAt time.Time `json:"resetsAt,omitzero"`
}

Allowance is the caller's cloud quota after this response.

type ChatRequest

type ChatRequest struct {
	// Product identifies the consuming product ("sneat", "datatug", ...) for
	// cloud metering, limits and routing. BYOK adapters ignore it.
	Product string `json:"product"`
	// Model is a concrete model ID, ModelAuto, or "" (same as ModelAuto).
	Model string `json:"model,omitempty"`
	// System is the stable system prompt.
	System string `json:"system,omitempty"`
	// Context blocks are rendered after System, static blocks first, in the
	// given order, so the static prefix is cacheable.
	Context  []ContextBlock `json:"context,omitempty"`
	Messages []Message      `json:"messages"`
	// MaxTokens caps output; 0 means the adapter default.
	MaxTokens int `json:"maxTokens,omitempty"`
	// ResponseSchema, when set, asks for a JSON object matching this JSON
	// Schema. Adapters use native structured output where the provider has it
	// and otherwise instruct the model; either way the final object arrives as
	// an EventStructured event (text deltas may still stream first).
	ResponseSchema json.RawMessage `json:"responseSchema,omitempty"`
	// StrictSchema controls whether an adapter with a native "strict" JSON
	// Schema mode (e.g. OpenAI's response_format.json_schema.strict) turns
	// it on for ResponseSchema. Strict mode requires the schema to follow
	// stricter authoring rules (every property required, no bare optional
	// fields, additionalProperties:false throughout); a caller whose schema
	// doesn't meet them sets StrictSchema to a false pointer to opt out. Nil
	// (the default) means strict when the adapter supports it.
	StrictSchema *bool `json:"strictSchema,omitempty"`
	// Metadata is opaque key/value data forwarded to the cloud for diagnostics
	// (e.g. "path": "llm-fallback"). Never put secrets or user content here.
	Metadata map[string]string `json:"metadata,omitempty"`
	// Tools the model may call this turn.
	Tools []Tool `json:"tools,omitempty"`
	// ToolChoice: "" (adapter default) | "auto" | "none" | "required" | a
	// specific tool name.
	ToolChoice string `json:"toolChoice,omitempty"`
	// Reasoning requests extended/deliberate reasoning where the provider
	// supports it: "" | "low" | "medium" | "high". Adapters map it to their
	// own knob (OpenAI-compatible reasoning_effort, Anthropic extended
	// thinking budget) and ignore it where unsupported.
	Reasoning string `json:"reasoning,omitempty"`
}

ChatRequest is a single streamed inference request.

type ContextBlock

type ContextBlock struct {
	Scope string      `json:"scope"`
	Kind  ContextKind `json:"kind"`
	Name  string      `json:"name"`
	Text  string      `json:"text"`
}

ContextBlock is one named piece of context for a scope, e.g. the calendar skill (static) or the user's happenings this week (dynamic). Scope names are product-defined strings; this package attaches no meaning to them.

type ContextKind

type ContextKind string

ContextKind separates stable context (instructions, schemas, skills, capabilities — good prompt-cache candidates) from per-turn data (current time, entities) that changes often.

const (
	ContextStatic  ContextKind = "static"
	ContextDynamic ContextKind = "dynamic"
)

type Error

type Error struct {
	Code      string `json:"code"`
	Message   string `json:"message"`
	Retryable bool   `json:"retryable,omitempty"`
}

Error is a provider error normalised for display and fallback decisions.

func (*Error) Error

func (e *Error) Error() string

func (*Error) IsRetryable

func (e *Error) IsRetryable() bool

IsRetryable reports whether the caller may retry the request that produced this error. It lets *Error satisfy ai/internal/retry.Retryable.

type Event

type Event struct {
	Type EventType `json:"type"`
	// Started: which provider/model is actually answering.
	Provider string `json:"provider,omitempty"`
	Model    string `json:"model,omitempty"`
	// TextDelta: the next chunk of text.
	Text string `json:"text,omitempty"`
	// Structured: the final JSON object for ChatRequest.ResponseSchema.
	Structured json.RawMessage `json:"structured,omitempty"`
	// Usage / Completed: token and allowance accounting when known.
	Usage *Usage `json:"usage,omitempty"`
	// Error: a terminal or non-terminal provider error.
	Error *Error `json:"error,omitempty"`
	// ToolCall: set on EventToolCall, one fully-assembled call.
	ToolCall *ToolCall `json:"toolCall,omitempty"`
	// ToolResult: set on EventToolResult (ai/agent only).
	ToolResult *ToolResult `json:"toolResult,omitempty"`
	// StopReason: set on EventCompleted; "tool_calls" | "end" | "length".
	StopReason string `json:"stopReason,omitempty"`
	// ProviderState: set on EventCompleted when the adapter captured
	// provider-specific state (e.g. ai/anthropic's thinking/
	// redacted_thinking blocks with signatures) that MUST be attached to the
	// assistant message this turn produces — see Message.ProviderState.
	ProviderState json.RawMessage `json:"providerState,omitempty"`
}

Event is one normalised stream event. Exactly the fields relevant to Type are set. Future tool/action events add new EventType values and fields; consumers must ignore event types they do not know.

type EventType

type EventType string

EventType names a normalised stream event. The string values are also the SSE event names of the cloud protocol (see package cloudproto).

const (
	EventStarted    EventType = "response.started"
	EventTextDelta  EventType = "text.delta"
	EventStructured EventType = "output.structured"
	EventUsage      EventType = "usage"
	EventError      EventType = "error"
	EventCompleted  EventType = "response.completed"
	// EventToolCall is emitted once per call, fully assembled (adapters buffer
	// streamed argument deltas), before the terminal EventCompleted of that
	// response.
	EventToolCall EventType = "tool.call"
	// EventToolResult is emitted only by ai/agent as it feeds tool results
	// back into the loop; adapters never emit it.
	EventToolResult EventType = "tool.result"
)

type LLMProvider

type LLMProvider interface {
	// Name identifies the provider in diagnostics ("cloud",
	// "openai-compatible", "openai-responses", "anthropic").
	Name() string
	Stream(ctx context.Context, req ChatRequest) iter.Seq2[Event, error]
}

LLMProvider streams one chat response.

Stream yields events in order: EventStarted (only once the HTTP request has actually succeeded; a request that fails before any response is received produces no EventStarted at all), then any number of EventTextDelta / EventStructured / EventUsage, then exactly one of:

  • EventCompleted, ending the sequence successfully, or
  • a FATAL error: exactly one final yield of (Event{Type: EventError, Error: e}, e), where e is a non-nil *Error (wrapped in the returned error). Nothing is yielded after it, and the implementation must return immediately afterward.

An EventError yielded with a nil Go error (second return value) is NOT fatal -- it reports a problem the stream is continuing past (e.g. a dropped mid-stream diagnostic) and consumers must keep ranging. Implementations must not buffer the full response and must stop promptly when ctx is cancelled or the consumer stops iterating. A stream ended by context cancellation must yield the fatal pair with Code ErrCodeCanceled (never Retryable).

type Message

type Message struct {
	Role Role   `json:"role"`
	Text string `json:"text"`
	// ToolCalls is set on an assistant message that invoked tools.
	ToolCalls []ToolCall `json:"toolCalls,omitempty"`
	// ToolResults is set on a RoleTool message answering prior ToolCalls.
	ToolResults []ToolResult `json:"toolResults,omitempty"`
	// ProviderState is opaque, provider-specific extra content an adapter
	// attached to an assistant message it produced (e.g. ai/anthropic's
	// extended-thinking/redacted-thinking blocks, signature included) and
	// that same adapter MUST replay unmodified on a later request that
	// includes this message — some providers 400 a tool-use continuation
	// that drops or edits the thinking blocks from the turn that requested
	// the tool call. Populated from Event.ProviderState (see EventCompleted)
	// by whoever appends the assistant message (e.g. ai/agent.Loop). An
	// adapter that doesn't understand another adapter's ProviderState MUST
	// ignore it rather than error.
	ProviderState json.RawMessage `json:"providerState,omitempty"`
}

Message is one conversation turn. The system prompt and context travel separately (ChatRequest.System, ChatRequest.Context) so adapters can place them where each provider caches best.

type Role

type Role string

Role of a conversation message.

const (
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	// RoleTool carries ToolResults answering a prior assistant ToolCalls
	// message. Adapters translate it to whatever the provider needs (e.g.
	// Anthropic user messages with tool_result blocks, OpenAI-compatible
	// role:"tool" messages).
	RoleTool Role = "tool"
)

type Tool added in v0.0.3

type Tool struct {
	Name        string          `json:"name"`
	Description string          `json:"description"`
	Schema      json.RawMessage `json:"schema"`
}

Tool is a function the model may call. Schema is the JSON Schema of the arguments object (not the whole tool envelope).

type ToolCall added in v0.0.3

type ToolCall struct {
	ID        string          `json:"id"`
	Name      string          `json:"name"`
	Arguments json.RawMessage `json:"arguments"`
}

ToolCall is one invocation the model asked for, with its arguments already assembled from any streamed deltas.

type ToolResult added in v0.0.3

type ToolResult struct {
	CallID  string `json:"callId"`
	Content string `json:"content"`
	IsError bool   `json:"isError,omitempty"`
}

ToolResult answers a ToolCall by CallID. Content is provider-facing text (JSON-encode structured results yourself); IsError marks a tool-level failure (as opposed to an infrastructure failure, which aborts the run).

type Usage

type Usage struct {
	InputTokens  int64 `json:"inputTokens,omitempty"`
	OutputTokens int64 `json:"outputTokens,omitempty"`
	// CacheReadTokens counts tokens served from a prompt cache.
	// ai/openaicompat populates it from
	// usage.prompt_tokens_details.cached_tokens: this is an INFORMATIONAL
	// SUBSET already counted inside InputTokens (OpenAI's prompt_tokens
	// includes cached tokens; the details object only breaks out how many
	// of them were cache hits) — do NOT add it to InputTokens. ai/anthropic
	// populates it from usage.cache_read_input_tokens: on the Messages API
	// this is the OPPOSITE relationship — Anthropic's input_tokens counts
	// ONLY the tokens actually processed fresh, and cache_read_input_tokens
	// (billed at its own, cheaper per-token rate) is NOT included in it —
	// so for ai/anthropic, CacheReadTokens IS additive to InputTokens.
	CacheReadTokens int64 `json:"cacheReadTokens,omitempty"`
	// CacheWriteTokens counts tokens written to a prompt cache. Only
	// ai/anthropic populates it, from usage.cache_creation_input_tokens —
	// same additive relationship to InputTokens as CacheReadTokens above
	// (billed separately, at its own higher per-token rate, and not
	// included in input_tokens). ai/openaicompat never populates this
	// field: OpenAI's API has no separate cache-write concept to report.
	CacheWriteTokens int64 `json:"cacheWriteTokens,omitempty"`
	// ReasoningTokens counts provider-side reasoning/thinking tokens, ONLY
	// on adapters that report them SEPARATELY from OutputTokens.
	// ai/openaicompat populates it from
	// usage.completion_tokens_details.reasoning_tokens when the API returns
	// that field: this is an INFORMATIONAL SUBSET already counted inside
	// OutputTokens (OpenAI's completion_tokens includes reasoning tokens;
	// the details object only breaks out how many of them were spent on
	// reasoning) — do NOT add it to OutputTokens. ai/anthropic leaves it
	// zero: the Messages API's usage object has no separate thinking-token
	// count — thinking tokens are already included in OutputTokens
	// (usage.output_tokens), not broken out on top of it, so there is
	// nothing distinct to report here without double-counting.
	ReasoningTokens int64 `json:"reasoningTokens,omitempty"`
	// Allowance is set by the cloud provider (ai/cloud); nil for BYOK.
	Allowance *Allowance `json:"allowance,omitempty"`
}

Usage is token and allowance accounting for one response.

The fields below are populated from each adapter's native usage object, and their SUBSET-VS-ADDITIVE relationship to InputTokens/OutputTokens is NOT the same across adapters — summing them naively double-counts on one adapter and undercounts on the other. See CacheReadTokens/ CacheWriteTokens/ReasoningTokens doc below for the per-adapter semantics, and BillableTokens for a helper that sums correctly for a named adapter.

func Collect

func Collect(stream iter.Seq2[Event, error]) (text string, structured json.RawMessage, usage *Usage, err error)

Collect drains a stream into its concatenated text, the last structured output and the last usage seen. It is a convenience for non-interactive callers and tests; interactive UIs should consume Stream directly.

Per the LLMProvider contract, a non-nil error (the iterator's second value) is always fatal and always terminates the stream, so Collect returns as soon as it sees one. A non-fatal EventError (nil Go error) is NOT terminal: Collect keeps draining past it, since the stream itself says it is continuing.

func (Usage) BillableTokens added in v0.0.3

func (u Usage) BillableTokens(provider string) int64

BillableTokens sums u into the total tokens the named provider actually bills for this response, without double-counting a subset field (see the per-field doc on Usage) against the total it is already included in. provider should be the ai.LLMProvider.Name() that produced this Usage:

  • "openai-compatible" and "openai-responses": CacheReadTokens/ ReasoningTokens are informational subsets already counted inside InputTokens/OutputTokens — Total = InputTokens + OutputTokens. ai/openairesponses populates them from the Responses API's usage.input_tokens_details.cached_tokens and usage.output_tokens_details.reasoning_tokens, the same subset relationship as ai/openaicompat's Chat Completions prompt_tokens_details/completion_tokens_details fields.
  • "anthropic" (and any other/unrecognised provider name — see below): CacheReadTokens/CacheWriteTokens are billed separately from InputTokens/OutputTokens — Total = InputTokens + OutputTokens + CacheReadTokens + CacheWriteTokens. ReasoningTokens is not added: no current adapter populates it additively.

An unrecognised provider name falls back to the additive (Anthropic- style) formula: silently ignoring a populated Cache*Tokens field would undercount real spend, which is the worse failure mode for a billing total than adding a field a future adapter turns out to already include (there is currently no adapter where that would happen). Prefer passing the adapter's own Name() over relying on this default for a provider this function doesn't know the convention of.

Directories

Path Synopsis
Package agent implements a tool-calling agent loop over ai.LLMProvider: it streams a model's response, executes any tool calls the model asked for via caller-supplied Handlers, feeds the results back, and repeats until the model stops calling tools or a configured limit is hit.
Package agent implements a tool-calling agent loop over ai.LLMProvider: it streams a model's response, executes any tool calls the model asked for via caller-supplied Handlers, feeds the results back, and repeats until the model stops calling tools or a configured limit is hit.
Package aiconfig loads and applies the product-level configuration for which LLM and decision providers a product wires up: cloud or BYOK (bring your own key, connecting directly to the provider, never via the cloud), and which deciders run before/instead of the cloud's decision service.
Package aiconfig loads and applies the product-level configuration for which LLM and decision providers a product wires up: cloud or BYOK (bring your own key, connecting directly to the provider, never via the cloud), and which deciders run before/instead of the cloud's decision service.
Package anthropic is an ai.LLMProvider for the Anthropic Messages API, implemented directly over net/http -- no vendor SDK.
Package anthropic is an ai.LLMProvider for the Anthropic Messages API, implemented directly over net/http -- no vendor SDK.
Package cloud is the client for the cloudproto protocol (see ai/cloudproto): it implements ai.LLMProvider (chat) and exposes a separate decision.Provider via Decider(), plus a Usage lookup.
Package cloud is the client for the cloudproto protocol (see ai/cloudproto): it implements ai.LLMProvider (chat) and exposes a separate decision.Provider via Decider(), plus a Usage lookup.
Package cloudproto is the small, product-neutral wire protocol between clients and an AI cloud boundary.
Package cloudproto is the small, product-neutral wire protocol between clients and an AI cloud boundary.
Package ctxmgr selects which ai.ContextBlock values go into a ChatRequest, balancing two goals: keep the LLM within a token budget, and keep the provider-cached prefix (the leading static blocks a provider such as Anthropic prompt-caches) as stable as possible across turns so caching actually pays off.
Package ctxmgr selects which ai.ContextBlock values go into a ChatRequest, balancing two goals: keep the LLM within a token budget, and keep the provider-cached prefix (the leading static blocks a provider such as Anthropic prompt-caches) as stable as possible across turns so caching actually pays off.
Package decision defines the DecisionProvider contract and the Chain that runs providers in order until one decides.
Package decision defines the DecisionProvider contract and the Chain that runs providers in order until one decides.
llmdecider
Package llmdecider is a decision.Provider that makes ONE structured inference against an ai.LLMProvider to produce a decision.Decision.
Package llmdecider is a decision.Provider that makes ONE structured inference against an ai.LLMProvider to produce a decision.Decision.
rules
Package rules is a deterministic, table-driven decision.Provider.
Package rules is a deterministic, table-driven decision.Provider.
Package diag is the shared diagnostics record for one conversational turn: which path handled it, what the decision chain did, which LLM answered, timing and usage.
Package diag is the shared diagnostics record for one conversational turn: which path handled it, what the decision chain did, which LLM answered, timing and usage.
internal
retry
Package retry is a tiny internal helper shared by the LLMProvider HTTP adapters (ai/openaicompat, ai/anthropic, ai/cloud).
Package retry is a tiny internal helper shared by the LLMProvider HTTP adapters (ai/openaicompat, ai/anthropic, ai/cloud).
sse
Package sse is a tiny internal helper shared by every SSE reader in this module (ai/openaicompat, ai/anthropic, ai/cloudproto): a bufio.SplitFunc that behaves like bufio.ScanLines but also accepts a bare '\r' (old Mac-style line endings), which some SSE producers/proxies still emit.
Package sse is a tiny internal helper shared by every SSE reader in this module (ai/openaicompat, ai/anthropic, ai/cloudproto): a bufio.SplitFunc that behaves like bufio.ScanLines but also accepts a bare '\r' (old Mac-style line endings), which some SSE producers/proxies still emit.
Package openaicompat is an ai.LLMProvider for the OpenAI Chat Completions streaming API (and any API-compatible endpoint), implemented directly over net/http -- no vendor SDK.
Package openaicompat is an ai.LLMProvider for the OpenAI Chat Completions streaming API (and any API-compatible endpoint), implemented directly over net/http -- no vendor SDK.
Package openairesponses is an ai.LLMProvider over OpenAI's Responses API (POST {base}/responses, stream:true), implemented directly over net/http -- no vendor SDK.
Package openairesponses is an ai.LLMProvider over OpenAI's Responses API (POST {base}/responses, stream:true), implemented directly over net/http -- no vendor SDK.
Package session holds the product-neutral conversational working state that helps resolve requests such as "move it to Friday": the focused entity, the current selection, sidebar pins, and the pending and previous actions.
Package session holds the product-neutral conversational working state that helps resolve requests such as "move it to Friday": the focused entity, the current selection, sidebar pins, and the pending and previous actions.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL