Help us shape the future of Go! Take the 2026 Go Developer Survey

chat

package module
v0.38.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 27, 2026 License: MIT Imports: 27 Imported by: 0

README

chat

Multi-provider AI chat client for Go: light, framework-free, opt-in per-SDK

Go Reference Pipeline Coverage phpboyscout Go toolkit

Part of the phpboyscout Go toolkit — small, framework-free Go modules extracted from go-tool-base. Docs: chat.go.phpboyscout.uk


gitlab.com/phpboyscout/go/chat is a unified multi-provider AI chat client (Anthropic Claude, OpenAI and OpenAI-compatible servers, Google Gemini and Vertex AI, Amazon Bedrock, Azure OpenAI, vLLM, and the claude, codex and agy CLIs) with a ReAct tool-calling loop, streaming, cross-provider fallback, usage accounting, history bounding and compaction, slash commands, and encrypted conversation persistence, as a light, dependency-inverted Go library.

It is the same client behind gtb ai, extracted so any project can embed a production chat client without pulling in the go-tool-base framework, and without linking every vendor SDK. You opt into only the provider(s) you use.

Design

  • SDK-free core. The core depends only on gitlab.com/phpboyscout/go/errors, invopop/jsonschema, google/uuid, and spf13/afero. No vendor AI SDK, no web framework, no go-tool-base. A depfootprint_test.go guard enforces it.
  • Opt-in providers. Each vendor SDK lives in its own module, registered via a blank import, so a Claude-only tool never compiles the OpenAI or Gemini SDKs. The core registers no provider of its own: with no Config.Provider set the client asks for claude, and construction fails unless you have imported chat-anthropic.
  • Stdlib seams. *slog.Logger (logging), *http.Client (transport, where you inject a hardened one or take the plain bounded default), a CredentialSource func (where a provider's API key comes from), and afero.Fs (persistence). Inject your own; everything has a sane default.
  • Config-system-agnostic. The core owns provider mechanics, not config schema. A host maps its own config into the typed chat.Config; go-tool-base does this in its adapter.
  • Conversational or one-shot. By default a client is a conversation and each call re-sends the history. Set Config.Stateless and every call is independent (nothing sent, nothing retained) so one client can serve a batch of documents across goroutines instead of billing quadratically for the accumulated prefix. See Process a batch of documents.

Providers

Providers register via a blank import, so you link only the SDK(s) you use:

Provider Module SDK
claude, claude-local gitlab.com/phpboyscout/go/chat-anthropic anthropics/anthropic-sdk-go; claude-local drives the claude CLI
openai, openai-compatible, codex-local gitlab.com/phpboyscout/go/chat-openai openai/openai-go/v3; codex-local drives the codex CLI
gemini, gemini-vertex, agy-local gitlab.com/phpboyscout/go/chat-gemini google.golang.org/genai; agy-local drives the agy CLI
bedrock gitlab.com/phpboyscout/go/chat-bedrock aws/aws-sdk-go-v2 (Bedrock Runtime)
azure-openai gitlab.com/phpboyscout/go/chat-openai-azure chat-openai, plus Azure/azure-sdk-for-go/sdk/azcore
vllm gitlab.com/phpboyscout/go/chat-vllm none (its own net/http client)

claude-local and codex-local call tools only when a program also links the MCP tool bridge, gitlab.com/phpboyscout/go/chat-mcptools, with a blank import. agy-local calls no tools, because agy takes no MCP configuration per call.

The per-provider modules are thin adapters, too tightly coupled to the core to warrant their own docs sites. Each ships a detailed README, and all provider documentation lives on the core docs site.

Quick start

import (
    "gitlab.com/phpboyscout/go/chat"
    _ "gitlab.com/phpboyscout/go/chat-anthropic" // activate the Claude provider
)

client, err := chat.New(ctx, chat.Settings{
    Config: chat.Config{Provider: chat.ProviderClaude, Token: apiKey},
})
if err != nil {
    return err
}
answer, err := client.Chat(ctx, "Summarise this changelog in one line.")

See the runnable Example tests on pkg.go.dev.

Documentation

Licence

MIT. See LICENSE.

Documentation

Overview

Package chat provides a unified multi-provider AI chat client supporting Claude, OpenAI, Gemini, Claude Local (via CLI binary), and OpenAI-compatible endpoints.

The ChatClient interface exposes five methods: Add (append a message), Chat (multi-turn conversation with tool use via a ReAct loop), Ask (structured output with JSON schema validation), SetTools (register callable tools), and Usage (token accounting). Streaming-capable providers also implement StreamingChatClient (StreamChat).

Tool calling follows a JSON Schema parameter definition, and the ReAct loop automatically dispatches tool calls and feeds results back until the model produces a final text response. Per-provider token limits and maximum agent steps are configurable via Config.

Every provider lives in a separate module and is activated by a blank import, so a consumer links only the SDK(s) it uses:

import _ "gitlab.com/phpboyscout/go/chat-anthropic" // "claude", "claude-local"
import _ "gitlab.com/phpboyscout/go/chat-openai"    // "openai", "openai-compatible"
import _ "gitlab.com/phpboyscout/go/chat-gemini"    // "gemini", "gemini-vertex"

Each provider module self-registers via RegisterProvider (and a failover HTTPStatusExtractor via RegisterStatusExtractor) in its init(). Custom providers register the same way. Structured-output helpers such as GenerateSchema simplify schema generation for Ask calls.

New constructs clients from package-owned Settings; the package is config-system-agnostic and owns typed config shapes such as RuntimeConfig and FallbackConfig. A host application maps its own configuration into Settings/Config (go-tool-base ships such a Props adapter, which lives in go-tool-base, not here). A provider's key comes from a CredentialSource the host composes, through ProviderCredential; the HTTP transport and the credential source are injected via Config so the core depends on no specific HTTP or secret-store stack. Existing chat clients do not mutate provider settings live on config reload; construct a new client to pick up changed settings.

Conversation state

A client has two modes, chosen at construction. By default it is a conversation: every Chat, Ask and StreamChat call appends the exchange to the client's history and the next call re-sends all of it. Set Config.Stateless and each call becomes a one-shot instead — no prior turns sent, none retained, and one client can serve many independent calls across goroutines.

The default suits a conversation and is expensive for batch work: a caller holding one client across N independent calls pays for the accumulated prefix every time, so cost grows with the square of N, and call N is answered in the light of calls 1..N-1 without the caller asking. Stateless mode removes both. Tool calling is unaffected — the ReAct loop keeps the turns it needs within a call and discards them on return.

Providers that honour the flag implement StatelessCapable; New refuses to build a stateless client from one that does not, rather than silently ignoring the request. Under stateless mode PersistentChatClient.Save returns a snapshot with no conversation and Restore returns ErrStatelessRestore.

Generation controls

Config.Temperature and Config.TopP influence sampling; Config.Effort selects how much reasoning the model spends before answering. All are optional and default to the provider's own behaviour.

Support for these is per *model*, not per provider — two models from one vendor can accept opposite controls — so it cannot be settled at construction. A provider with no such concept at all (claude-local has no sampling flag) fails in New; a model refusing one it structurally supports yields ErrModelRejectedParameter at request time. Neither path silently drops the parameter, because a control that is quietly ignored cannot be told apart from one that does nothing.

Lower temperatures reduce variance between runs; they do not make output deterministic. Effort is the most direct cost dial here — observe it through Usage.ReasoningTokens.

Prompt caching

Providers that can cache a large stable block implement CachingChatClient; discover it by type assertion. AddCached marks content so repeated calls are billed at the provider's cached rate, and Config.CacheTTL chooses how long it is retained.

It is a hint. Every provider declines to cache content below a per-model minimum, and most do so without an error, so confirm the effect through Usage.CachedTokens rather than assuming it. Unlike Config.Stateless or Config.Temperature, being ignored here costs money rather than corrupting the result, which is why an unsupported provider simply does not implement the interface instead of failing construction.

Multimodal input

Add, Ask, Chat and StreamChat accept a trailing variadic of Media — images (and, on Gemini, PDF and A/V) sent alongside the text prompt. A text-only call passes no media and is unchanged. Each attachment's type is sniffed from its bytes (never a caller-supplied filename), cross-checked against any declared MIMEType, allowlisted, and checked against the selected provider's support before any network call; disguised or unsupported content is rejected with ErrMediaRejected or ErrMediaUnsupported. Media support is per provider: Gemini (images, PDF, A/V), Claude and OpenAI (images, PDF); ProviderClaudeLocal accepts no media.

Index

Examples

Constants

View Source
const (
	// DefaultModelGemini is the default model for the Gemini provider.
	//
	// Flash rather than Pro because Google ships no generally-available
	// Pro-tier model: gemini-3.1-pro-preview is preview-only, and both
	// gemini-3-pro-preview and gemini-2.5-pro have been withdrawn. This is the
	// most capable Gemini available at GA, and it is a genuine step below
	// [DefaultModelClaude] and [DefaultModelOpenAI] — a known gap, accepted in
	// preference to defaulting onto a preview model that may disappear.
	DefaultModelGemini = "gemini-3.8-flash"
	// DefaultModelBedrock is the Bedrock default, chosen by the formula above and
	// measured rather than assumed: on 2026-08-30 in eu-west-2 it returned
	// schema-conforming JSON on every attempt, drove a tool call, and accepts
	// image input, which are the three baseline capabilities.
	//
	// Bedrock makes the formula harder than the other providers. Model access is
	// granted per account and per region, no Amazon-owned or Anthropic model
	// there supports structured output, and a model can quietly stop being
	// available on demand. The weekly detector in chat-bedrock watches this
	// constant for all three. See spec 0018 D11.
	DefaultModelBedrock = "qwen.qwen3-vl-235b-a22b"

	// DefaultModelClaude is the default model for the Claude provider.
	//
	// Fable models are excluded although they rank above it: they double the
	// cost for a caller who never chose a model, and require 30-day data
	// retention, so a zero-data-retention organisation would get a 400. See
	// spec 0025 D4.
	DefaultModelClaude = "claude-opus-5-5"

	// DefaultModelOpenAI is the default model for the OpenAI provider.
	//
	// gpt-6.1 ships as sol alone, so the newest generation picks itself. sol
	// also continues the line chosen by judgement for gpt-5.6, whose three
	// variants nothing in the models API told apart.
	//
	// Mirrors openai.ChatModelGPT6_1Sol from openai-go/v3; inlined as a string
	// literal so the SDK-free chat core carries no OpenAI dependency. The
	// chat-openai provider module owns the mapping back onto the SDK constant.
	DefaultModelOpenAI = "gpt-6.1-sol"

	// DefaultMaxSteps is the default maximum number of ReAct loop iterations.
	DefaultMaxSteps = 20

	// DefaultMaxTokensOpenAI is the default maximum tokens per response for OpenAI.
	DefaultMaxTokensOpenAI = 4096

	// DefaultMaxTokensClaude is the default maximum tokens per response for Claude.
	//
	// Thinking shares this budget on every Opus 5.5 request. It stays under the
	// Anthropic SDK's non-streaming ceiling of about 21,333, above which the SDK
	// refuses a request that sets no timeout. See spec 0025 D8.
	DefaultMaxTokensClaude = 16000

	// DefaultMaxTokensGemini is the default maximum tokens per response for Gemini.
	DefaultMaxTokensGemini = 8192
)

Default models are chosen by one rule, applied identically to every provider:

the most capable model a provider makes generally available
that supports the module's baseline capabilities.

"Most capable" rather than a named tier, because tier names are vendor nomenclature that does not generalise — OpenAI's current generation ships as sol/terra/luna, which is not a tier ordering. "Generally available" excludes preview and experimental models: a default must not point at something a vendor may withdraw. Baseline capabilities are structured outputs, tool use and multimodal input, all of which this module exposes.

A default is what a caller gets when they have expressed no preference, so it favours capability over cost. Callers with an opinion about cost set Config.Model.

See https://gitlab.com/phpboyscout/go/chat/-/wikis/specs/0005-default-model-formula.

View Source
const DefaultChatRequestTimeout = 5 * time.Minute

DefaultChatRequestTimeout bounds a single AI request. LLM generations — especially a large single-shot code conversion on a slower flagship model like Opus — run well past the shared 30s HTTP default, so chat clients use a more generous cap. It is deliberately bounded (not unlimited): a model stuck in a loop or never returning must still fail rather than hang forever. Override per-environment with the ai.request_timeout config key (e.g. "8m") or programmatically via Config.RequestTimeout.

View Source
const DefaultCompactionTurns = 200

DefaultCompactionTurns is the retained-turn count above which a compacting policy summarises, unless WithTurnBudget says otherwise.

It is far larger than a truncation budget would be, deliberately. Dropping turns is free and summarising them is not, so a conversation should be allowed to get genuinely long before it starts paying for a round-trip it did not ask for. See spec 0014 D14.

View Source
const EnvAIProvider = "AI_PROVIDER"

EnvAIProvider is the environment variable for overriding the AI provider. The chat core honours it as an ecosystem default in applyDefaultProvider; the GTB adapter owns the ai.* / <provider>.api.* config-key schema.

View Source
const MaxBaseURLLength = 2048

MaxBaseURLLength caps the length, in bytes, of a provider BaseURL. Normal BaseURLs are well under 200 bytes; 2 KiB is generous for legitimate proxy configurations and far short of any pathological input.

View Source
const MaxTopLogprobs = 20

MaxTopLogprobs is the most alternatives LogprobsCapable.ChatWithLogprobs accepts per token. It is OpenAI's limit, kept on every provider so a caller's request means the same everywhere (spec 0032 D2).

View Source
const SnapshotVersion = 1

SnapshotVersion is the current version of the snapshot format. Increment when the format changes in a way that requires migration.

Variables

View Source
var ErrCommandUnavailable = errors.New("command not available on this client")

ErrCommandUnavailable is returned by a command the current client cannot serve — one needing a contract this provider does not implement.

It is a distinct error rather than prose so a consumer can tell "this provider cannot do that" from "that went wrong", and say so differently.

View Source
var ErrConflictingCredentials = errors.NewSentinel("chat.conflicting_credentials",
	"more than one credential supplied")

ErrConflictingCredentials reports two credentials supplied where one is read, such as Config.Token alongside Config.Credential.

View Source
var ErrInvalidBaseURL = errors.NewSentinel("chat.invalid_base_url", "invalid chat provider base URL")

ErrInvalidBaseURL is returned when Config.BaseURL fails validation. Callers can distinguish validation failures from other errors via errors.Is.

View Source
var ErrInvalidSnapshotID = errors.NewSentinel("chat.invalid_snapshot_id", "invalid snapshot identifier")

ErrInvalidSnapshotID is returned when a snapshot identifier fails validation — not a canonical UUID, contains path separators, or produces a filesystem path outside the store directory.

Callers can distinguish validation failures from I/O failures via errors.Is(err, ErrInvalidSnapshotID).

View Source
var ErrLogprobsUnavailable = errors.NewSentinel("chat.logprobs_unavailable",
	"the server accepted the request but returned no log-probabilities")

ErrLogprobsUnavailable is returned when the server accepted a log-probability request and sent none back. An empty Scored.Tokens would be indistinguishable from a reply with no tokens.

View Source
var ErrMediaRejected = errors.NewSentinel("chat.media_rejected", "media rejected")

ErrMediaRejected is returned when an attachment fails the safety filter — empty, too large, an unidentifiable/disallowed content type, or a declared type that contradicts the sniffed bytes. Wrapped with detail; test with errors.Is.

View Source
var ErrMediaUnsupported = errors.NewSentinel("chat.media_unsupported", "media not supported by provider")

ErrMediaUnsupported is returned when the selected provider (or model) cannot accept media, or cannot accept a particular attachment's type.

View Source
var ErrModelRejectedParameter = errors.NewSentinel("chat.model_rejected_parameter", "model rejected a generation parameter")

ErrModelRejectedParameter wraps a provider's refusal of a generation parameter for the selected model, as distinct from the provider not supporting that parameter at all — which is a construction error.

The two demand different fixes, which is why they are different errors: a construction error means change provider, this means change model or drop the parameter. Support is per model rather than per provider and moves between model generations, so it cannot be known before the request is made.

if errors.Is(err, chat.ErrModelRejectedParameter) {
    // this model will not take the temperature (or effort) that was set
}

The underlying provider error remains reachable by unwrapping.

View Source
var ErrProviderNotRegistered = errors.NewSentinel("chat.provider_not_registered", "unsupported provider")

ErrProviderNotRegistered is returned by New when no factory is registered under Config.Provider, which usually means the provider module's blank import is missing. The message keeps the historical "unsupported provider" wording; the name says what is actually wrong.

View Source
var ErrRefused = errors.NewSentinel("chat.refused", "model refused to answer")

ErrRefused marks a reply the provider's safety system declined or withheld: the model's own refusal, output a content filter blocked, or a prompt rejected before generation. RefusedError carries the provider's reason.

The default failover policy does not advance on it. A caller who wants a refusal to move to the next provider writes a policy that matches it:

if errors.Is(err, chat.ErrRefused) {
    return chat.FailoverNext
}
View Source
var ErrStatelessRestore = errors.NewSentinel("chat.stateless_restore", "cannot restore a snapshot into a stateless chat client")

ErrStatelessRestore is returned by PersistentChatClient.Restore on a client built with Config.Stateless. A stateless client retains no conversation, so there is nothing for a snapshot to be restored into — and silently accepting the snapshot and discarding it would hide the caller's mistake. Save still works, and returns a snapshot with no messages.

View Source
var ErrThrottled = errors.NewSentinel("chat.throttled", "provider throttled the request")

ErrThrottled wraps a provider throttling failure that survived retry.

It exists so a caller can tell a rate limit from an auth, schema or model error without matching on message text — which is fragile by construction, and worse here than usual because the vendors' own wording misleads. Gemini reports a per-minute input-token burst limit as "You exceeded your current quota, please check your plan and billing details", which sends a reader to billing for a condition that clears in two seconds.

if errors.Is(err, chat.ErrThrottled) {
    // the provider is rate-limiting; retries were already exhausted
}

The underlying provider error remains reachable by unwrapping.

View Source
var ErrTranscriptMoved = errors.New("the conversation changed while the policy was being applied")

ErrTranscriptMoved reports that the conversation changed while a policy was being applied, so the edit was computed against turns that no longer exist.

A policy may call the model — compaction does — and a client's lock must not be held across that round-trip. So an implementation copies the transcript out, runs the policy unlocked, and checks the conversation has not moved before applying the result. If another goroutine appended in the meantime, applying the edit anyway would silently drop their turns.

It is a retry, not a failure: ask again and the policy runs against the conversation as it now stands.

View Source
var ErrTruncated = errors.NewSentinel("chat.truncated", "reply truncated at the output-token limit")

ErrTruncated marks a reply cut off at the output-token limit. The partial text is on the TruncatedError that carries it.

View Source
var ErrUnableToConstruct = errors.NewSentinel("chat.unable_to_construct", "unable to construct client")

ErrUnableToConstruct marks the fatal class of construction failure: no client was returned and none could be.

It is the one question every caller must be able to ask without knowing anything else about the error:

client, err := chat.New(ctx, settings)
if errors.Is(err, chat.ErrUnableToConstruct) {
	return err // nothing was built
}
// client is usable; err, if non-nil, lists settings that were not applied

Functions

func ApplyHistoryEdit added in v0.10.0

func ApplyHistoryEdit(turns []TurnInfo, edit HistoryEdit) ([]int, error)

ApplyHistoryEdit validates an edit against the invariants every policy owes, and returns the indices to keep in order.

A provider calls it between asking its policy and rewriting its transcript. The checks live here rather than in each policy because there is one right answer and several policies: a compacting policy written later inherits them without knowing they exist.

It refuses an edit that drops a pinned turn, splits a tool group, names an index that does not exist, or replaces a turn it did not keep.

Example
package main

import (
	"fmt"

	"gitlab.com/phpboyscout/go/chat"
)

func main() {
	turns := []chat.TurnInfo{
		{Index: 0, Role: chat.RoleUser, Text: "cached reference material", Pinned: true},
		{Index: 1, Role: chat.RoleUser, Text: "an old question"},
		{Index: 2, Role: chat.RoleAssistant, Text: "an old answer"},
	}

	// Dropping the pinned turn is refused: it is billed as a cached prefix, so
	// losing it destroys every later cache hit.
	_, err := chat.ApplyHistoryEdit(turns, chat.HistoryEdit{Keep: []int{1, 2}})
	fmt.Println(err != nil)
}
Output:
true

func AssignText added in v0.10.0

func AssignText(text string, target any) error

AssignText writes a provider's raw text answer into an Ask target, per the no-schema half of the ChatClient.Ask contract: with no Config.ResponseSchema set, the model's text is what the caller asked for, and target must be a *string or a json.Unmarshaler.

Any other pointer gets a best-effort JSON unmarshal, so a caller who set no schema but whose model returned JSON anyway still works.

It lives in the core because every provider owes the same behaviour and the contract is stated on the core's own interface. Before spec 0011 only the API-based Claude provider implemented it, so schema-less Ask returned prose on `claude` and failed with "invalid character" on `claude-local` — the same code against two providers of the same model. See https://gitlab.com/phpboyscout/go/chat/-/wikis/specs/0011-chat-provider-conformance §3.2.2.

func BoundConversation added in v0.11.0

func BoundConversation(
	ctx context.Context,
	policy HistoryPolicy,
	turns []TurnInfo,
) ([]int, map[int]string, error)

BoundConversation asks a policy for an edit, validates it, and returns what the provider should send: the indices to keep, oldest first, and the replacement text for any turn the policy superseded.

It is the middle of the three-step sequence every provider runs — describe, bound, apply — and exists so that only the ends differ per provider. Three providers writing these six lines each is how they drift, which is the failure the conformance suite exists to prevent; and a provider that skips ApplyHistoryEdit gets no validation at all while appearing to work.

A nil policy returns every index unchanged and a nil map, so a provider can call it unconditionally rather than guarding at each site.

Callers pass the turns they have *retained*: the turn being sent is not among them, so a budget of forty means forty turns of history and a request carrying forty-one. That is what ChatClient.History reports, and the agreement is deliberate. Pinned turns are included and marked, never omitted — a policy budgeting against a conversation smaller than the one being sent is a policy that does not do what the caller asked.

An error from the policy, or an edit that breaks an invariant, fails the call. Sending unbounded at the moment the guard broke is the failure the caller set a policy to prevent. A caller who prefers degradation writes it into a policy that catches its own errors, where it is visible.

See https://gitlab.com/phpboyscout/go/chat/-/wikis/specs/0013-history-policy-application D2, D3, D5 and D8.

func ChatHTTPClient

func ChatHTTPClient(cfg Config) *http.Client

ChatHTTPClient returns the HTTP client a provider should use: the host-injected Config.HTTPClient verbatim when set (go-tool-base injects its hardened transport), otherwise the module's own plain bounded default. This is the seam that keeps the chat core free of any specific HTTP stack.

func Compact added in v0.12.0

func Compact(
	ctx context.Context,
	policy HistoryPolicy,
	turns []TurnInfo,
) (keep []int, replace map[int]string, changed bool, err error)

Compact bounds the conversation now, regardless of whether either budget has been reached.

A policy runs before a client is about to send something. An interactive request — a person typing "/compact" — has no request to attach to, so it needs a way in that does not wait for one. See spec 0014 D8, and https://gitlab.com/phpboyscout/go/chat/-/issues/15 for the command surface that calls this.

The caller supplies the turns because only a provider can enumerate its own transcript; the returned indices and replacements are applied exactly as a policy's would be. A policy that is not a compacting one is asked to bound normally, so a caller need not check which policy is configured.

It reports whether anything changed, so a command can say "nothing to compact" rather than claiming work it did not do.

func ExecuteTool

func ExecuteTool(ctx context.Context, l *slog.Logger, tools map[string]Tool, name string, input json.RawMessage) string

ExecuteTool looks up a tool by name from the provided registry, executes it, and returns the result as a string. If the result is not a string, it is JSON marshalled. Errors at any stage are returned as formatted error strings suitable for feeding back into the AI conversation (matching existing provider behaviour where tool errors become conversation content rather than aborting the ReAct loop).

A tool handler that panics — handlers run model-generated, adversarial input — is recovered and converted to a tool-error string rather than crashing the process. This holds for both the serial and the parallel dispatch paths, since both route through ExecuteTool.

func GenerateEncryptionKey

func GenerateEncryptionKey() ([]byte, error)

GenerateEncryptionKey returns a fresh 32-byte AES-256 key from crypto/rand, suitable for use with WithEncryption. Each snapshot store should use a distinct key obtained either from this helper or from an operator-controlled source such as a KMS or secret manager.

Closes L-2 from docs/development/reports/security-audit-2026-04-17.md — using this helper avoids the footgun of deriving keys from human-readable passphrases, which have insufficient entropy for the AES-GCM threat model.

func GenerateSchema

func GenerateSchema[T any]() *jsonschema.Schema

GenerateSchema creates a JSON schema for a given type T.

The result is assignable straight to Config.ResponseSchema and Tool.Parameters. It returned any until v0.8.0, which meant callers had to assert it back to use it — two codebases independently wrote that assertion with a dead error branch, which is what a bad default looks like from the outside. OpenAI's structured outputs feature uses a subset of JSON schema. The reflector is configured with flags to ensure the generated schema complies with this specific subset.

func MarkAs added in v0.3.2

func MarkAs(err, sentinel error) error

MarkAs tags err with a package sentinel so a caller can identify the class of failure without matching on message text, while keeping the provider's own error reachable by unwrapping.

Provider modules should use this rather than wrapping sentinels themselves. Annotation libraries have historically offered a Mark for this, and it is a trap: those markers are recognised only by the library's own Is, so a consumer writing the idiomatic standard-library check gets false. That is how this module shipped an undetectable sentinel once. Both of these hold for the result here, under the standard library:

errors.Is(chat.MarkAs(providerErr, chat.ErrThrottled), chat.ErrThrottled) // true
errors.Is(chat.MarkAs(providerErr, chat.ErrThrottled), providerErr)       // true

A nil err returns nil, so a call site can wrap unconditionally.

func ProviderHTTPStatus added in v0.10.0

func ProviderHTTPStatus(err error) (int, bool)

ProviderHTTPStatus reads the HTTP status a provider SDK error carries, using the extractors registered by whichever provider modules are linked in. The second result is false when no registered extractor recognises the error, and when none is registered at all.

It is the exported view of what FailoverPolicy classifies on, so a caller can ask why a call failed over — and so the provider-conformance suite can assert that an error chain survives a provider's own wrapping rather than being flattened to a string. See https://gitlab.com/phpboyscout/go/chat/-/wikis/specs/0011-chat-provider-conformance.

func ProviderRegistered added in v0.24.0

func ProviderRegistered(name Provider) bool

ProviderRegistered reports whether a factory is registered under name. It answers without a Settings or a construction attempt, so a tool can check that it links the provider module its configuration names.

func RegisterCapabilities added in v0.6.0

func RegisterCapabilities(name Provider, resolver CapabilityResolver)

RegisterCapabilities registers a capability resolver for a provider. Call it from an init() function alongside RegisterProvider, so an adapter cannot be linked without its capabilities coming with it.

Registering twice for the same provider replaces the previous resolver, which keeps test doubles simple.

func RegisterProvider

func RegisterProvider(name Provider, factory ProviderFactory)

RegisterProvider registers a factory function for a provider name. Call this from an init() function in your provider file or external package.

func RegisterStatusExtractor

func RegisterStatusExtractor(extractor HTTPStatusExtractor)

RegisterStatusExtractor adds a provider HTTP-status extractor to the registry. Call it from a provider module's init(); a nil extractor is ignored.

func RegisterToolServer added in v0.30.0

func RegisterToolServer(server ToolServer)

RegisterToolServer makes server the program's tool server. Call it from the init() of the module that provides it; a nil server is ignored.

func ResolveEditPolicy added in v0.27.0

func ResolveEditPolicy(policy, configured HistoryPolicy) (HistoryPolicy, BusyRule)

ResolveEditPolicy is how a provider reads the policy given to an explicit edit: the policy to run, falling back to configured when none was given, and the rule it runs under.

func ValidateBaseURL

func ValidateBaseURL(baseURL string, allowInsecure bool) error

ValidateBaseURL returns nil if baseURL is acceptable for use as a chat provider endpoint, or an error wrapping ErrInvalidBaseURL otherwise.

An empty baseURL is always accepted — callers that require a value (e.g. ProviderOpenAICompatible, ProviderVLLM) must enforce non-emptiness separately. Every non-empty URL is checked against the seven rejection rules documented at the top of this file.

Pass allowInsecure=true only for an net/http/httptest.Server or a private endpoint, such as an internal vLLM server, on a network the caller controls. The Config.AllowInsecureBaseURL field that drives this is excluded from every decoder, so config files cannot set it.

Downstream tool authors should call this at the boundary where they accept BaseURL input (their own setup wizard, CLI flag, env var) so misconfiguration surfaces early rather than at New time.

func ValidateSnapshotID

func ValidateSnapshotID(id string) error

ValidateSnapshotID returns nil if id is a canonical UUID that will be accepted by the [FileStore] methods, or an error wrapping ErrInvalidSnapshotID otherwise.

Use this at the boundary of your own system — e.g. in a CLI flag or HTTP handler that accepts a snapshot identifier from an external source — so validation happens before the value reaches Save, Load, or Delete.

Types

type BatchFunc added in v0.27.0

type BatchFunc func(ctx context.Context, turns []Turn, emit StreamCallback) (string, error)

BatchFunc sends one or more callers' turns upstream as a single request and returns the reply text. emit delivers an event to every streaming caller still listening, and fails once none is, at which point the request should stop.

type BusyRule added in v0.27.0

type BusyRule int

BusyRule says what a transcript edit does when the client is busy with another operation. See WhenBusy and spec 0026 D3.

const (
	// EditWaits queues the edit behind the operations ahead of it, and applies
	// it when it reaches the front. It is the default.
	EditWaits BusyRule = iota
	// EditRefuses returns [ErrTranscriptMoved] at once if the client is busy.
	EditRefuses
	// EditMerges applies the edit at once to the committed history, even while
	// a call runs; the call's exchange lands on top when it commits.
	EditMerges
)

func BusyRuleOf added in v0.27.0

func BusyRuleOf(policy HistoryPolicy) BusyRule

BusyRuleOf reports the rule attached by WhenBusy; any other policy, nil included, waits.

type CachingChatClient added in v0.5.0

type CachingChatClient interface {
	ChatClient

	// AddCached appends a user turn and asks the provider to cache it, so
	// later calls carrying the same prefix are billed at the cached rate.
	//
	// A hint, not a guarantee. Every provider declines to cache content below
	// a minimum size — 1024 tokens on Gemini and OpenAI, and between 512 and
	// 4096 on Claude depending on the model — and most do so silently. The
	// call still succeeds and the content is still sent; it is simply not
	// cached.
	//
	// Confirm the effect through Usage.CachedTokens rather than assuming it.
	// That is what the providers' own documentation recommends, and a number
	// that moves is worth more than a promise in this interface.
	AddCached(ctx context.Context, prompt string, media ...Media) error
}

CachingChatClient extends ChatClient with explicit prompt caching.

Providers that can carry a cache annotation implement it; discover support the same way as StreamingChatClient and PersistentChatClient:

if cc, ok := client.(chat.CachingChatClient); ok {
    err = cc.AddCached(ctx, referenceCorpus)
}

Caching changes what a call costs, never what the model is asked, so a provider that cannot do it simply does not implement this — unlike Config.Stateless or Config.Temperature, where being ignored would corrupt the result and is therefore a construction error.

type CachingSupport added in v0.6.0

type CachingSupport struct {
	// Explicit reports whether content can be marked for caching deliberately.
	Explicit Support
	// Implicit reports whether the provider caches eligible prefixes with no
	// action from the caller.
	Implicit Support
	// ImplicitMinTokens is the prefix size at which implicit caching begins to
	// engage. Zero means unknown.
	ImplicitMinTokens int
	// MinCacheableTokens is the smallest block a provider will cache at all.
	// Zero means unknown.
	MinCacheableTokens int
	// TTLs lists the retention periods the provider accepts. Empty means
	// unknown; providers offer discrete choices rather than arbitrary durations.
	TTLs []time.Duration
}

CachingSupport carries the specifics of CapCaching.

This is the capability that shows why a bare bool is not enough: "caching is supported" is not actionable, while "implicit caching engages above N tokens" tells a caller whether restructuring a prompt is worth it.

type Capabilities added in v0.6.0

type Capabilities struct {
	// Supported maps each capability to its support level. A capability absent
	// from the map reports SupportUnknown, which is the zero value, so a
	// partially-populated map is always safe to read.
	Supported map[Capability]Support

	// Sampling, Effort and Caching carry specifics where a provider supplied
	// them. Nil means the provider said nothing.
	Sampling *SamplingSupport
	Effort   *EffortSupport
	Caching  *CachingSupport
}

Capabilities reports what a provider and model support.

Read three-valued support with Capabilities.Support. The pointer fields carry per-capability specifics and are nil when the provider said nothing — which is distinct from a provider that answered with zeroes.

func (Capabilities) Support added in v0.6.0

func (c Capabilities) Support(cap Capability) Support

Support reports whether cap is supported. An unpopulated map, an absent key and an explicitly-unknown entry all report SupportUnknown.

There is deliberately no Supports() bool. Any boolean has to choose a reading of SupportUnknown, and either choice is wrong about half the time — reading it as unsupported makes claude-local look featureless, and reading it as supported promises things that may not work. A switch that ignores a case is visible in review; a misread bool is not.

type Capability added in v0.6.0

type Capability string

Capability names one thing a caller can ask a provider to do.

The set is closed and hand-chosen: the module defines the vocabulary and providers answer in it. It is deliberately not derived from whatever a vendor happens to report, because mirroring one vendor's product surface would make the vocabulary move whenever that vendor ships a feature.

A name earns its place by being something the module exposes, being meaningful for every provider (including those that lack it), and being something a caller would change behaviour over.

See https://gitlab.com/phpboyscout/go/chat/-/wikis/specs/0006-capability-reporting.

const (
	// CapSampling covers Config.Temperature and Config.TopP.
	CapSampling Capability = "sampling"
	// CapEffort covers Config.Effort.
	CapEffort Capability = "effort"
	// CapCaching covers CachingChatClient.AddCached and Config.CacheTTL.
	CapCaching Capability = "caching"
	// CapStateless covers Config.Stateless.
	CapStateless Capability = "stateless"
	// CapStreaming covers StreamingChatClient.StreamChat.
	CapStreaming Capability = "streaming"
	// CapPersistence covers PersistentChatClient save and restore.
	CapPersistence Capability = "persistence"
	// CapStructuredOutput covers Config.ResponseSchema.
	CapStructuredOutput Capability = "structured_output"
	// CapTools covers tool calling.
	CapTools Capability = "tools"
	// CapMultimodal covers image and PDF input parts.
	CapMultimodal Capability = "multimodal"
	// CapLogprobs covers LogprobsCapable.ChatWithLogprobs.
	CapLogprobs Capability = "logprobs"
)

type CapabilityResolver added in v0.6.0

type CapabilityResolver func(model string) ModelInfo

CapabilityResolver reports what a provider supports for a given model.

Provider modules register one in init() alongside their factory. The model string is whatever the caller configured, which may be empty (meaning the provider default) or a model the resolver has never heard of — in both cases the resolver should report SupportUnknown rather than guessing.

type ChatClient

type ChatClient interface {
	// Add appends a user message to the conversation history without
	// triggering a completion. The message persists for subsequent
	// Chat() or Ask() calls.
	//
	// Under Config.Stateless the message is buffered rather than retained: it
	// is sent with the next Chat/Ask/StreamChat call and then cleared.
	Add(ctx context.Context, prompt string, media ...Media) error
	// Ask sends a question and unmarshals the structured response into
	// target. If Config.ResponseSchema was set during construction, the
	// provider enforces that schema. If no schema is set, the provider
	// returns the raw text content unmarshalled into target (which must
	// be a *string or implement json.Unmarshaler).
	//
	// The question and the answer are appended to the conversation, so a
	// subsequent call re-sends them. Callers making independent calls — batch
	// classification, extraction, judging — want Config.Stateless; without it,
	// each call carries every call before it.
	//
	// Ask is tool-free: tools registered with SetTools are not offered to the
	// model. Ask promises a structured answer, and a model that replies with a
	// tool call has not delivered one. Use Chat for tool use, then Ask to shape
	// the result — Chat commits its tool calls and their results to the
	// conversation, so a following Ask can read them:
	//
	//	client.SetTools(tools)
	//	_, _ = client.Chat(ctx, "research X with the tools")
	//	var out Report
	//	_ = client.Ask(ctx, "now give me that as JSON", &out)
	//
	// That idiom costs an extra round-trip, and it does not work under
	// Config.Stateless, where the session is dropped rather than committed.
	//
	// The contract is enforced by the shared conformance suite as each provider
	// module adopts a core release carrying it, so a provider pinned to an
	// earlier core may still offer tools to Ask.
	Ask(ctx context.Context, question string, target any, media ...Media) error
	// SetTools configures the tools available to the AI. This replaces
	// (not appends to) any previously set tools.
	SetTools(tools []Tool) error
	// Chat sends a message and returns the response content. If tools
	// are configured, the provider handles tool calls internally via a
	// ReAct loop bounded by Config.MaxSteps (default 20).
	//
	// Like Ask, the exchange is appended to the conversation and re-sent by the
	// next call unless Config.Stateless is set.
	Chat(ctx context.Context, prompt string, media ...Media) (string, error)
	// Usage returns the cumulative token usage across every provider
	// round-trip made by this client instance since construction. A single
	// Chat/Ask/StreamChat call may make multiple round-trips (one per ReAct
	// step) and all are summed. Providers that do not report token counts
	// (e.g. ProviderClaudeLocal) contribute a zero, Known == false Usage.
	// Wire Config.UsageObserver to observe usage per round-trip instead.
	Usage() Usage
	// History reports the conversation this client is carrying — how many turns
	// it will re-send on the next call, and the provider's own token count for
	// the last call it made.
	//
	// Every provider appends and re-sends, so a long-lived conversation grows
	// until it overflows the model's context window. This is what makes that
	// growth visible in time to do something about it: reset the client, or set
	// Config.Stateless if the calls were independent all along.
	//
	// Check History.Known before trusting Turns.
	History() History
}

ChatClient defines the interface for interacting with a chat service.

Conversation state

By default a client is a conversation: every Chat, Ask and StreamChat call appends the exchange to the client's history, and the next call re-sends all of it. Message history from Add() calls persists the same way. To start a fresh conversation, create a new client via chat.New().

That default is expensive for batch work. A caller holding one client across N independent calls pays for the accumulated prefix on every call — cost grows with the square of N — and each call is answered in the light of the ones before it. Set Config.Stateless for one-shot calls that neither read nor retain history.

Concurrency

A conversational (default) client from a provider that runs its operations through a Lane is safe for concurrent use (spec 0026). It serves one operation at a time, in arrival order: Chat and StreamChat calls queued behind a running call go upstream as one combined request whose reply they all share, while Ask, Add, Restore and transcript edits run alone. An edit that meets a running call waits by default; see WhenBusy. A provider that has not adopted the lane makes no such guarantee, and each goroutine should use its own instance.

Stateless provider clients ARE safe for concurrent use — one client can serve a pool of workers, which is the shape batch work wants. A stateless call never waits for another call on the same client, and is never combined with one. A provider that does not implement StatelessCapable makes no such guarantee, and New refuses to build one with Config.Stateless set.

The fallback composite from NewFallback is safe for concurrent use when every member is (spec 0027).

func New

func New(ctx context.Context, settings Settings) (ChatClient, error)

New creates a ChatClient for the configured provider.

func NewFallback

func NewFallback(clients []ChatClient, opts ...FallbackOption) (ChatClient, error)

NewFallback builds a composite ChatClient that tries clients in order, advancing to the next on a retryable failure. The first client is the primary; the rest are fallbacks. At least one client is required.

Retry happens before failover

A provider's own retry runs inside the call, so the composite only sees a throttling error once those retries are exhausted. That ordering is deliberate: a per-minute burst limit clears in seconds, and waiting is cheaper than moving to a provider with a different model, different pricing and a lossy transcript replay. Failing over on the first 429 would make a burst limit look like an outage and strand a long batch on a fallback for a condition that resolves itself.

Set Config.MaxRetries to 0 for the opposite behaviour — advance immediately rather than waiting out the primary.

The returned client also satisfies StreamingChatClient, LogprobsCapable and ToolCallProposer, each only when every supplied client does. The two one-shot calls fail over like Chat but are never replayed as turns.

func NewFallbackFromConfigs

func NewFallbackFromConfigs(ctx context.Context, cfgs []Config, opts ...FallbackOption) (ChatClient, error)

NewFallbackFromConfigs constructs each provider from Config values and wraps the result in a composite. Use NewFallbackFromSettings when each provider needs distinct construction dependencies.

func NewFallbackFromSettings

func NewFallbackFromSettings(ctx context.Context, settings []Settings, opts ...FallbackOption) (ChatClient, error)

NewFallbackFromSettings constructs each provider via New and wraps the result in a composite. The first Settings value is the primary. A construction failure for a non-primary provider (e.g. a missing credential) is downgraded to a logged WARN and that provider is dropped, so one missing fallback credential does not break the whole client; if the primary fails to construct, the error is returned.

func NewWithFactory added in v0.13.0

func NewWithFactory(ctx context.Context, settings Settings, factory ProviderFactory) (ChatClient, error)

NewWithFactory creates a ChatClient from a provider's own factory, running every guard New runs around the one the registry supplies.

It exists for a provider module's second door: a constructor taking options that cannot travel through Config — a seed, an injected SDK constructor — which still has to validate the endpoint before credentials reach it, resolve the logger its factory will log through, apply provider defaults, drop what cannot be applied, and assert the built client can carry what survived.

func New(ctx context.Context, settings chat.Settings, opts ...Option) (chat.ChatClient, error) {
	return chat.NewWithFactory(ctx, settings, func(ctx context.Context, s chat.Settings) (chat.ChatClient, error) {
		return build(ctx, s, opts...)
	})
}

Handing over the factory rather than calling a validation helper first is deliberate. A helper is something a module can forget to call, and two of them did: an http:// endpoint was accepted with a nil error and the API key went over the wire in cleartext, while a call with no logger panicked. See https://gitlab.com/phpboyscout/go/chat/-/wikis/specs/0015-one-construction-path.

func NewWithFallbackSettings

func NewWithFallbackSettings(ctx context.Context, settings Settings, fallback FallbackConfig, opts ...FallbackOption) (ChatClient, error)

NewWithFallbackSettings builds a single provider client or fallback composite from package-owned settings and fallback config. Per the resolved OQ-3, fallback.Providers[0] is the primary and overrides Settings.Config.Provider.

type Command added in v0.14.0

type Command struct {
	// Name is the verb, without its leading slash.
	Name string
	// Summary is one line, shown by [HelpCommand].
	Summary string
	// Run performs it. A nil Run makes the command listable but inert, which is
	// useful for describing a command the consumer handles itself.
	Run CommandFunc
}

Command is one verb a user can invoke against a conversation.

func ClearCommand added in v0.14.0

func ClearCommand() Command

ClearCommand empties the conversation, keeping the configuration.

The system prompt and model are carried forward deliberately. Restoring a bare empty snapshot takes them with it, which would let whoever is typing delete the consumer's system instruction with a keystroke — a command changes what is in a conversation, never how the client is configured. See spec 0016 D3 and D10.

Turns added with AddCached go, because AddCached appends a user turn and a clear that leaves user content behind is not a clear. It does not waste the cache: clearing stops referencing one rather than destroying it, so re-adding the same content hits it again inside its TTL.

Requires PersistentChatClient; a client without it returns ErrCommandUnavailable.

func CompactCommand added in v0.14.0

func CompactCommand() Command

CompactCommand applies the configured history policy on demand.

It is the interactive trigger for what CompactOldest does automatically: a policy runs when the client is about to send something, and a person typing "/compact" has no request to attach to.

It applies whatever Config.HistoryPolicy holds. A caller who configured truncation gets truncation — the command asks for the conversation to be bounded now, not for a particular way of bounding it.

Requires TranscriptEditor; a provider without it returns ErrCommandUnavailable.

func HelpCommand added in v0.14.0

func HelpCommand() Command

HelpCommand lists the registered vocabulary, including whatever the consumer added.

It is the one built-in that gets more useful as a consumer registers their own, and the reason CommandRequest carries the set.

func ToolsCommand added in v0.14.0

func ToolsCommand(tools []Tool) Command

ToolsCommand lists the tools a user can expect the model to reach for.

The consumer supplies them because ChatClient.SetTools is write-only: there is no way to read a registered set back off a client, and inventing one to serve a help command would be the wrong reason to widen the interface. Pass the same slice given to SetTools.

func UndoCommand added in v0.14.0

func UndoCommand() Command

UndoCommand removes the most recent exchange.

The operation is a HistoryPolicy the core supplies, so this needs nothing from a provider beyond TranscriptEditor — the same seam `/compact` uses.

Requires TranscriptEditor; a provider without it returns ErrCommandUnavailable.

func UsageCommand added in v0.14.0

func UsageCommand() Command

UsageCommand reports what the conversation has cost so far.

type CommandFunc added in v0.14.0

type CommandFunc func(ctx context.Context, req CommandRequest) (CommandResult, error)

CommandFunc runs one command.

type CommandRequest added in v0.14.0

type CommandRequest struct {
	// Client is the client the command acts on.
	Client ChatClient
	// Args is everything after the verb, trimmed. Empty when none was given.
	Args string
	// Set is the whole registered set, so a command can describe its siblings.
	// [HelpCommand] is the reason this is here.
	Set CommandSet
}

CommandRequest is what a handler is given.

type CommandResult added in v0.14.0

type CommandResult struct {
	// Name is the command that ran, without its leading slash.
	Name string
	// Text is human-readable output. It may be empty for a command that only
	// acts.
	Text string
	// Changed reports whether the conversation was altered.
	Changed bool
}

CommandResult is what a command produced.

Data rather than a bare string: a consumer rendering /usage wants more than a sentence, and Changed is what lets a UI refresh a turn counter without guessing whether the transcript moved.

func RunCommand added in v0.14.0

func RunCommand(
	ctx context.Context,
	client ChatClient,
	set CommandSet,
	input string,
) (CommandResult, bool, error)

RunCommand dispatches input against set, and reports whether it did.

It is the consumer's own call, on input the consumer chose to treat as a candidate — the library never inspects a prompt on its way to a model. That is what makes escaping a non-problem and prompt injection impossible through this path: text the consumer did not pass here is never parsed as a command. See https://gitlab.com/phpboyscout/go/chat/-/wikis/specs/0016-slash-commands D1 and D2.

if res, handled, err := chat.RunCommand(ctx, client, commands, input); handled {
	render(res)
	continue
}

reply, err := client.Chat(ctx, input)

Input that is not a registered command returns handled false and is otherwise untouched, so it can go straight to the model. A registered command that fails is still handled: the error is the command's, not a signal to send the text on.

type CommandSet added in v0.14.0

type CommandSet []Command

CommandSet is the vocabulary a consumer offers. Order is preserved for display; lookup is by name.

type CompactOption added in v0.12.0

type CompactOption func(*compactOldest)

CompactOption configures a compacting policy. Every option has a default that works, so CompactOldest takes none in the simple case.

func WithCompactionFallback added in v0.12.0

func WithCompactionFallback(fallback HistoryPolicy) CompactOption

WithCompactionFallback bounds the conversation with fallback when summarisation fails, instead of failing the call.

Off by default. A compaction failure fails the call, per spec 0013 D3: a caller who asked for summarisation and silently got truncation learns it from the model's behaviour rather than from an error. Opting in makes the degradation visible where it was chosen.

func WithSummarizer added in v0.12.0

func WithSummarizer(cfg Config) CompactOption

WithSummarizer summarises through a client built from cfg rather than from the live client's configuration.

Use it to summarise with a cheaper model than the conversation runs on. Tools, response schema and history policy are cleared from the copy regardless: a summarisation call must not carry the caller's tools, must not be forced into their response schema, and must not be able to trigger a compaction of its own.

An empty Provider means the live client's, which keeps the summariser on the conversation's credentials and guardrail. Naming another provider builds it through the registry, so options passed to the live client's constructor do not reach it; use WithSummarizerClient when they must.

func WithSummarizerClient added in v0.12.0

func WithSummarizerClient(client ChatClient) CompactOption

WithSummarizerClient summarises through a client the caller built and owns.

The caller takes on what the derived client would otherwise guarantee: it must not carry a Config.HistoryPolicy that could re-enter compaction, and its tools and response schema are its own business. Config.Stateless is the straightforward way to be sure of the first.

func WithSummaryPrompt added in v0.12.0

func WithSummaryPrompt(prompt string) CompactOption

WithSummaryPrompt replaces the house summarisation prompt.

Set it where a domain has terms worth preserving that a general prompt would discard — order numbers, case references, the units a measurement is in. The transcript is still appended as delimited data, so the framing that keeps it from reading as instructions survives an override.

func WithTokenBudget added in v0.12.0

func WithTokenBudget(tokens int) CompactOption

WithTokenBudget sets an absolute input-token count above which compaction runs, replacing the default share of the model's reported limit.

The count compared against it is the provider's own figure for the previous call, so it is one call out of date and absent before the first call. The turn budget is the backstop for both cases.

func WithTurnBudget added in v0.12.0

func WithTurnBudget(turns int) CompactOption

WithTurnBudget sets the retained-turn count above which compaction runs.

type Config

type Config struct {
	// Provider is the AI service provider to use.
	Provider Provider
	// Model is the specific model to use (e.g. "claude-opus-5", "gpt-6.1-sol").
	//
	// Empty selects the provider's default — see [DefaultModelClaude],
	// [DefaultModelOpenAI] and [DefaultModelGemini]. Those are chosen by one
	// rule applied to every provider: the most capable model the provider makes
	// generally available that supports the module's baseline capabilities.
	//
	// The defaults therefore favour capability over cost, because a default is
	// what a caller gets having expressed no preference. Set this field if cost
	// matters, if you need a particular model's behaviour, or if you want a
	// model pinned against vendor churn.
	Model string
	// Token is the API key or token for the service.
	Token string
	// Credential yields this provider's API key. Nil leaves the provider's
	// default in place (see [ProviderCredential]), and setting it alongside
	// Token is refused.
	Credential CredentialSource `json:"-" mapstructure:"-" yaml:"-"`
	// Project is the cloud project a provider addresses. Required by
	// ProviderGeminiVertex, which falls back to GOOGLE_CLOUD_PROJECT when it is
	// empty and refuses to construct when both are. Ignored by every other
	// provider.
	//
	// It lives here rather than on a provider's constructor because this is the
	// only surface a config file decodes into, and an option is a function call.
	// See spec 0017 D7 and D10.
	Project string
	// Location is the region a provider addresses. Required by
	// ProviderGeminiVertex, which falls back to GOOGLE_CLOUD_LOCATION when it is
	// empty and refuses to construct when both are.
	//
	// Named for the concept rather than for one vendor: AWS Bedrock addresses by
	// region and means the same thing by it. See spec 0017 D7.
	Location string
	// APIVersion is the dated API version a provider requires on every request.
	// Required by ProviderAzureOpenAI, which refuses to construct without one.
	//
	// There is deliberately no default. Azure's versions are dated and models
	// appear and disappear between them, so a default shipped here would be
	// correct today and silently wrong later, surfacing as a missing model
	// rather than as a version problem. See spec 0019 D6.
	//
	// It decodes from a config file, unlike a credential: it is a version
	// string, and a config file is its natural home.
	APIVersion string
	// Capabilities is what the caller declares their endpoint supports,
	// overriding every other source.
	//
	// It exists because nobody else can know. models.dev cannot describe a
	// private deployment, a self-hosted server, or a local runner whose
	// capabilities are whatever model you pulled, and an OpenAI-compatible
	// endpoint carries no way to be asked. A caller who has pointed at their own
	// infrastructure knows more about it than any catalogue does.
	//
	// It is trusted rather than validated. A wrong declaration fails at request
	// time and is yours to correct, which is the same bargain
	// AllowInsecureBaseURL makes. Refusing to believe a caller about their own
	// endpoint is what produced the media refusal it exists to lift.
	//
	// It does not disable the media safety checks: the sniff, the allowlist and
	// the size caps still apply. You are trusted about your endpoint, not about
	// the bytes. See spec 0020 D8.
	Capabilities map[Capability]Support
	// BaseURL overrides the API endpoint. Required when using ProviderOpenAICompatible
	// or ProviderVLLM.
	// Example: "http://localhost:11434/v1" for Ollama, "https://api.x.ai/v1" for xAI.
	BaseURL string
	// SystemPrompt is the initial system prompt to set the context for the AI.
	SystemPrompt string
	// ResponseSchema is the JSON schema used to force a structured output from
	// the AI. Build one with [GenerateSchema].
	//
	// Typed since v0.8.0; it was any beforehand, so the only statement of its
	// contract was a check in New that the compiler could not read.
	ResponseSchema *jsonschema.Schema
	// SchemaName names the response schema (e.g., "error_analysis") for
	// providers whose structured-output field takes a name. Claude and Gemini
	// ignore it.
	SchemaName string
	// SchemaDescription describes the response schema for providers whose
	// structured-output field takes one. Claude and Gemini ignore it.
	SchemaDescription string
	// MaxSteps limits the number of ReAct loop iterations in Chat().
	// Zero means use the default (DefaultMaxSteps = 20).
	MaxSteps int
	// MaxTokens sets the maximum tokens per response.
	// Zero means use the provider default (OpenAI: 4096, Claude: 16000, Gemini: 8192).
	MaxTokens int
	// RequestTimeout bounds a single HTTP request to the provider. Zero falls
	// back to the ai.request_timeout config key, then DefaultChatRequestTimeout.
	// Bounded on purpose so a stuck model fails rather than hanging forever.
	RequestTimeout time.Duration
	// HTTPClient, when non-nil, is used verbatim for provider API calls. The
	// host injects a hardened transport here (go-tool-base supplies one built on
	// the go/httpclient module);
	// nil ⇒ the module builds a plain bounded client from RequestTimeout.
	HTTPClient *http.Client `json:"-" mapstructure:"-" yaml:"-"`
	// HistoryPolicy bounds the conversation before each call. Nil leaves it
	// unbounded, which is the default and what every provider did before.
	//
	// Use it for a conversation that legitimately accumulates. For independent
	// calls that never needed history at all, Stateless is the cheaper answer.
	//
	// [TruncateOldest] is the policy the module ships. A policy never drops a
	// turn added with AddCached and never splits a tool request from its
	// results; [ApplyHistoryEdit] enforces both, so a policy that gets them
	// wrong fails loudly rather than corrupting a transcript.
	HistoryPolicy HistoryPolicy `json:"-" mapstructure:"-" yaml:"-"`

	// ParallelTools enables concurrent execution of multiple tool calls
	// within a single ReAct step. Disabled by default.
	ParallelTools bool
	// MaxParallelTools limits the number of tools executing concurrently.
	// Zero means use the default (5). Only effective when ParallelTools is true.
	MaxParallelTools int

	// Temperature, when non-nil, overrides the provider's default sampling
	// temperature. Nil samples at the provider's default.
	//
	// Passed through in the provider's own units and never rescaled: ranges
	// differ (Claude 0-1, OpenAI 0-2, Gemini 0-2) and quietly rescaling what
	// the caller asked for would be worse than making them read the range. The
	// value is validated against the selected provider's range before any
	// request is made.
	//
	// Support is narrowing sharply as vendors move to reasoning models, and
	// **none of the three default models accepts it**: it is deprecated on
	// claude-opus-5, rejected by gpt-5.6-sol (which permits only its default of
	// 1), and absent from claude-local entirely. Gemini still accepts it, as do
	// older models on every provider — but reaching them means setting [Model]
	// explicitly.
	//
	// Setting it against a provider that has no sampling concept is a
	// construction error; a model that refuses it yields
	// [ErrModelRejectedParameter] at request time. See [Effort], which is the
	// forward-looking control, is supported everywhere, and is what these
	// models expect to be steered with instead.
	//
	// Lower values reduce variance between runs. They do not make output
	// deterministic — see the sampling how-to.
	Temperature *float64

	// TopP, when non-nil, overrides the provider's default nucleus-sampling
	// threshold, on a 0-1 scale. Nil samples at the provider's default.
	//
	// Providers recommend adjusting temperature or top-p, not both.
	//
	// Its support is narrowing alongside [Temperature]: gpt-5.6-sol rejects it
	// outright, where gpt-5.4 accepted it. The same guidance applies — prefer
	// [Effort] on reasoning models, or set [Model] to one that still takes it.
	TopP *float64

	// Effort selects how much reasoning the model spends before answering.
	// Empty uses the provider's default.
	//
	// This is the forward-looking sibling of Temperature: every provider
	// supports it, including claude-local, and on reasoning models it is the
	// control that replaced temperature. It is also the most direct cost dial
	// in Config — observe its effect through Usage.ReasoningTokens.
	Effort Effort

	// CacheTTL requests how long a provider retains content marked with
	// CachingChatClient.AddCached. Zero uses the provider default.
	//
	// Providers offer discrete choices rather than arbitrary durations and
	// select the nearest they support: Claude offers 5 minutes or 1 hour,
	// Gemini takes a duration. The longer option costs more to write and is
	// what makes caching pay at moderate call rates, so it is a decision for
	// the caller rather than a default.
	CacheTTL time.Duration

	// MaxRetries bounds automatic retries of transient provider failures —
	// throttling, request timeouts, 5xx. Nil uses the provider's own default,
	// which is 2 on Claude and OpenAI and is matched by Gemini.
	//
	// Zero disables retry, for a caller who would rather fail fast than have a
	// call silently take tens of seconds. That is a coherent thing to want:
	// with a fallback composite configured, MaxRetries: 0 moves to the next
	// provider on the first throttle instead of waiting out the primary.
	//
	// Retries honour the provider's own Retry-After hint where one is given —
	// a delay guessed from outside is worse than the answer already supplied.
	//
	// Note that RequestTimeout bounds a single request and nothing bounds the
	// retry sequence as a whole, so a high MaxRetries against a provider
	// supplying long hints can take a while. Cancel the context to bound it.
	MaxRetries *int

	// Stateless makes every Chat, Ask and StreamChat call a one-shot: the
	// client neither seeds the request with prior turns nor retains the turns
	// from this call. One client then serves many independent calls with no
	// per-call construction cost and no cross-call contamination.
	//
	// The zero value keeps the conversational behaviour: history accumulates
	// across calls on a single client instance. That is right for a
	// conversation and wrong for batch work, where holding one client across N
	// independent calls re-sends the whole accumulated prefix every time —
	// quadratic input cost, and call N answered in the light of calls 1..N-1
	// without the caller ever asking for it.
	//
	// Tool calling is unaffected. Stateless means "no history across calls",
	// not "no messages within one": the ReAct loop keeps the assistant and tool
	// turns it needs to make progress, and discards them when the call returns.
	//
	// Add still works, and buffers a one-shot preamble — the buffered turns go
	// out with the next call and are then cleared rather than retained. History
	// counts them until that call takes them.
	//
	// Stateless provider clients are safe for concurrent use by multiple
	// goroutines; see [ChatClient] for the exceptions. Setting this against a
	// provider module that predates the flag is a construction error rather
	// than a silent miss — see [StatelessCapable].
	Stateless bool

	// UsageObserver, when non-nil, is invoked once per provider round-trip
	// with that round-trip's token usage. A ReAct tool-calling loop fires it
	// once per step. This is the opt-in hook for emitting a telemetry event,
	// metric, or log line; the chat client never depends on a telemetry
	// collector. Providers that do not report usage still fire the observer
	// with a Known == false Usage. The callback runs synchronously on the
	// calling goroutine, so keep it fast and non-blocking.
	UsageObserver func(Usage) `json:"-" mapstructure:"-" yaml:"-"`

	// AllowInsecureBaseURL permits HTTP (non-HTTPS) BaseURLs, for an
	// httptest.Server or a private endpoint such as an internal vLLM server.
	// Anything reached over a network the caller does not control must leave
	// it false. It is settable in code only, so config files cannot enable it.
	AllowInsecureBaseURL bool `json:"-" mapstructure:"-" yaml:"-"`
}

Config holds configuration for a chat client.

type ConfigErrors added in v0.6.0

type ConfigErrors struct {
	// contains filtered or unexported fields
}

ConfigErrors aggregates everything wrong with one construction.

Construction reports every problem rather than the first, so a caller fixing three mistakes learns all three from one call instead of one round-trip at a time.

It implements multi-error unwrapping, so errors.Is finds any member sentinel whether it was the only problem or one of four.

func (*ConfigErrors) Dropped added in v0.6.0

func (e *ConfigErrors) Dropped() []DroppedSetting

Dropped reports the settings that were not applied.

It is structured on purpose: a caller deciding whether the shortfall matters needs to branch on which settings were lost, and parsing that back out of a message string is not a contract anyone should depend on.

func (*ConfigErrors) Error added in v0.6.0

func (e *ConfigErrors) Error() string

Error implements error.

func (*ConfigErrors) Fatal added in v0.6.0

func (e *ConfigErrors) Fatal() bool

Fatal reports whether construction failed outright. Equivalent to errors.Is(err, ErrUnableToConstruct), and cheaper when the type is in hand.

func (*ConfigErrors) Unwrap added in v0.6.0

func (e *ConfigErrors) Unwrap() []error

Unwrap returns the individual errors, so stdlib errors.Is and errors.As reach every member.

type ConversationStore

type ConversationStore interface {
	// Save writes a snapshot to the store.
	Save(ctx context.Context, snapshot *Snapshot) error
	// Load retrieves a snapshot by ID.
	Load(ctx context.Context, id string) (*Snapshot, error)
	// List returns summaries of all stored snapshots.
	List(ctx context.Context) ([]SnapshotSummary, error)
	// Delete removes a snapshot by ID.
	Delete(ctx context.Context, id string) error
}

ConversationStore persists and retrieves conversation snapshots.

func NewFileStore

func NewFileStore(fs afero.Fs, dir string, opts ...FileStoreOption) (ConversationStore, error)

NewFileStore creates a ConversationStore that persists snapshots as JSON files. Files are stored in dir with 0600 permissions. The directory is created with 0700 permissions if it doesn't exist.

Example
package main

import (
	"github.com/spf13/afero"

	"gitlab.com/phpboyscout/go/chat"
)

func main() {
	// Create a FileStore for persisting chat conversation snapshots.
	store, err := chat.NewFileStore(afero.NewMemMapFs(), "/conversations")
	if err != nil {
		return
	}

	// Save, Load, List, Delete snapshots
	_ = store
}
Example (WithEncryption)
package main

import (
	"github.com/spf13/afero"

	"gitlab.com/phpboyscout/go/chat"
)

func main() {
	// Encrypt stored snapshots with AES-256-GCM (key must be 32 bytes).
	key := make([]byte, 32) // In real usage, use a secure key source

	store, err := chat.NewFileStore(afero.NewMemMapFs(), "/conversations",
		chat.WithEncryption(key),
	)
	if err != nil {
		return
	}

	_ = store
}

type CredentialSource added in v0.36.0

type CredentialSource func(ctx context.Context) (string, error)

CredentialSource yields a provider's API key when a client is built.

("", nil) means not configured, which is legitimate for an endpoint that needs no key and lets FirstCredential try the next source. An error means broken, and a value returned alongside an error is discarded. The semantics match go/forge's CredentialSource, so a forge source converts with chat.CredentialSource(src). See spec 0036.

func EnvCredential added in v0.36.0

func EnvCredential(name string) CredentialSource

EnvCredential yields the value of a named environment variable. An empty name yields ("", nil) without reading anything.

func FirstCredential added in v0.36.0

func FirstCredential(sources ...CredentialSource) CredentialSource

FirstCredential yields the first non-empty key its sources produce.

A broken source is skipped, and its error is returned, joined with the others, only when nothing resolved: a good key in a later source should not be blocked by an outage in an earlier one, and a broken source should not read as an absent one. A cancelled context stops the walk.

func ProviderCredential added in v0.36.0

func ProviderCredential(cfg Config, envFallback string) CredentialSource

ProviderCredential returns the source a provider resolves its key from: cfg.Credential when set, otherwise cfg.Token, then the environment variable envFallback names. Every provider module resolves its key through it, so the order lives in one place. Pass "" for envFallback when the provider reads no well-known variable.

The returned source refuses cfg.Token set alongside cfg.Credential with ErrConflictingCredentials, and wraps a broken source's error with the provider's name. It never puts the key into an error.

func StaticCredential added in v0.36.0

func StaticCredential(key string) CredentialSource

StaticCredential yields a key the caller already holds.

type DroppedSetting added in v0.6.0

type DroppedSetting struct {
	// Fields names the Config fields that were not applied, e.g. "Temperature".
	Fields []string
	// Capability is the capability the fields required.
	Capability Capability
	// Reason is a short human-readable explanation.
	Reason string
}

DroppedSetting names a Config field that could not be applied, and why.

type Effort added in v0.3.0

type Effort string

Effort is a provider-neutral reasoning-effort level.

An ordinal ladder is neutral in a way a numeric range is not: "low" means the same thing on every provider, whereas 0.5 means different things on a 0-1 scale and a 0-2 one.

The five rungs are what Anthropic and the claude CLI accept natively, and a strict subset of what OpenAI accepts. Gemini has four levels, so EffortXHigh and EffortMax clamp to its highest — documented rather than silent.

const (
	// EffortLow spends the least reasoning the provider will accept.
	EffortLow Effort = "low"
	// EffortMedium is a middling amount of reasoning.
	EffortMedium Effort = "medium"
	// EffortHigh reasons substantially before answering.
	EffortHigh Effort = "high"
	// EffortXHigh reasons more than High. Clamps to High on Gemini.
	EffortXHigh Effort = "xhigh"
	// EffortMax spends the most reasoning the provider will accept. Clamps to
	// High on Gemini. This is the most direct cost dial in [Config] — observe
	// its effect through Usage.ReasoningTokens.
	EffortMax Effort = "max"
)

func (Effort) Rank added in v0.3.0

func (e Effort) Rank() (int, bool)

Rank returns the effort's position on the ladder, ascending from zero, and whether it is a recognised value. Provider adapters use it to clamp onto a shorter ladder without hard-coding each mapping.

func (Effort) Valid added in v0.3.0

func (e Effort) Valid() bool

Valid reports whether e is a recognised rung of the ladder.

type EffortCapable deprecated added in v0.3.0

type EffortCapable interface {
	ChatClient

	// SupportsEffort is a compile-time marker. It is never called.
	SupportsEffort()
}

EffortCapable is implemented by provider clients that can carry Config.Effort.

Deprecated: prefer CapabilitiesFor with CapEffort. This marker still gates construction and is not going away in this release.

type EffortSupport added in v0.6.0

type EffortSupport struct {
	// Levels lists the rungs the model actually implements. Empty means the
	// ladder is not known, not that no levels exist.
	Levels []Effort
}

EffortSupport carries the specifics of CapEffort.

type FailoverDecision

type FailoverDecision int

FailoverDecision is the outcome of classifying a provider error.

const (
	// FailoverFatal — do not advance; return the error to the caller.
	FailoverFatal FailoverDecision = iota
	// FailoverNext — the active provider failed transiently or is
	// unavailable; advance to the next provider.
	FailoverNext
)

type FailoverPolicy

type FailoverPolicy interface {
	Classify(err error) FailoverDecision
}

FailoverPolicy classifies a provider error into a FailoverDecision.

Implementations MUST NOT log the error's message directly. A policy only decides whether to advance; the composite logs a single coarse WARN per transition (provider names + a "status"/"network" reason, never the raw error), and reduces any endpoint detail to the host only.

var DefaultFailoverPolicy FailoverPolicy = defaultFailoverPolicy{}

DefaultFailoverPolicy advances on transient/unavailable conditions (HTTP 408, 429, 5xx, and network errors) and treats everything else — auth, bad request, unknown model, caller cancellation, and local-CLI (claude-local) failures — as fatal so an operator-fixable problem surfaces instead of being masked.

See the resolved open questions in https://gitlab.com/phpboyscout/go-tool-base/-/wikis/specs/0093-chat-provider-fallback (OQ-1: a claude-local non-zero exit is fatal).

type FallbackConfig

type FallbackConfig struct {
	Enabled   bool       `mapstructure:"enabled"`
	Providers []Provider `mapstructure:"providers"`
}

FallbackConfig is the package-owned shape for GTB's ai.fallback.* config.

type FallbackOption

type FallbackOption func(*fallbackConfig)

FallbackOption configures a composite fallback client.

func WithFailoverPolicy

func WithFailoverPolicy(policy FailoverPolicy) FallbackOption

WithFailoverPolicy overrides the error-classification policy (default DefaultFailoverPolicy).

func WithFallbackLogger

func WithFallbackLogger(log *slog.Logger) FallbackOption

WithFallbackLogger sets the logger used for the single WARN line per failover transition. Defaults to a no-op logger; NewFallbackFromConfigs wires the tool's logger automatically.

func WithOnFailover

func WithOnFailover(fn func(from, to Provider)) FallbackOption

WithOnFailover registers an observability hook invoked on each provider transition. It is called once per transition, after the WARN log and before the new provider is used, from whichever goroutine's call failed over.

func WithProviderCredentialSources added in v0.36.0

func WithProviderCredentialSources(resolve ProviderCredentialSources) FallbackOption

WithProviderCredentialSources supplies a resolver called once per provider while a chain is built, so each member gets its own CredentialSource. A resolved source replaces the token or source a member would otherwise inherit.

client, err := chat.NewWithFallbackSettings(ctx, settings, fallbackCfg,
    chat.WithProviderCredentialSources(func(p chat.Provider) (chat.CredentialSource, bool) {
        src, ok := sources[p]
        return src, ok
    }),
)

func WithStateless added in v0.2.0

func WithStateless() FallbackOption

WithStateless declares that the supplied clients are stateless, so the composite keeps no replay transcript of its own.

NewFallbackFromSettings and NewFallbackFromConfigs infer this from Config.Stateless and you do not need to pass it. It exists for NewFallback, which is handed already-built clients and cannot see their config. Passing it for clients that are *not* stateless loses cross-provider context on failover; omitting it for clients that are means the composite accumulates a transcript no one will use, and prepends it to the next one-shot call on failover.

func WithStrictToolContext

func WithStrictToolContext() FallbackOption

WithStrictToolContext makes failover fail fast once a tool call has executed in the current conversation, instead of advancing with a lossy text-only transcript replay (resolved OQ-2).

type FileStoreOption

type FileStoreOption func(*fileStoreConfig)

FileStoreOption configures a FileStore.

func WithEncryption

func WithEncryption(key []byte) FileStoreOption

WithEncryption enables AES-256-GCM encryption for stored snapshots. The key must be exactly 32 bytes and must come from a cryptographically secure source. Use GenerateEncryptionKey to generate one.

func WithLogger

func WithLogger(log *slog.Logger) FileStoreOption

WithLogger attaches a logger used for diagnostic DEBUG-level events (e.g. when [FileStore.List] skips a file whose name is not a canonical snapshot identifier). Defaults to a noop logger.

type HTTPStatusExtractor

type HTTPStatusExtractor func(err error) (status int, ok bool)

HTTPStatusExtractor pulls an HTTP status code out of a provider SDK error, unwrapping the wrapper layers the providers add. The second return is false when the error is not this provider's status-bearing type.

Each provider module registers one via RegisterStatusExtractor in its init() so the chat core can classify failover decisions without importing any vendor SDK. This mirrors the RegisterProvider registry.

type History added in v0.10.0

type History struct {
	// Turns is the number of conversation turns the client will re-send on its
	// next call, including any buffered by Add and not yet sent.
	Turns int

	// LastInputTokens is the provider-reported input-token count of the most
	// recent conversation call — the measured size of everything that call
	// sent. A one-request call outside the conversation does not move it. Zero
	// before the first call, and from providers that report no usage at all.
	//
	// It is a measurement rather than an estimate, and one call stale. See
	// [UsageTracker.LastUsage] for why the core does not compute it directly.
	LastInputTokens int

	// Known reports whether Turns is authoritative. False from a provider that
	// cannot count its own history — ProviderClaudeLocal delegates the
	// transcript to the claude CLI's own session, so it can only see what it has
	// buffered locally, not what the far side is carrying.
	Known bool
}

History reports what a client is currently carrying in its conversation, so a caller can see the growth that would otherwise only surface as a context-overflow 400 — which the failover policy correctly, and unhelpfully, classifies as fatal.

Every provider appends to its conversation and re-sends the whole of it on the next call, so a long-lived agent loop degrades monotonically. Config.Stateless answers the independent-calls case outright; this is for the conversation that legitimately accumulates and needs watching.

Check Known before treating Turns as authoritative, exactly as with Usage: a provider that delegates its transcript elsewhere cannot count it.

func (History) IsZero added in v0.10.0

func (h History) IsZero() bool

IsZero reports whether no conversation has accumulated and nothing has been sent.

type HistoryEdit added in v0.10.0

type HistoryEdit struct {
	// Keep lists the indices to retain, oldest first. An index absent from Keep
	// is dropped.
	Keep []int

	// Replace maps an index in Keep to text that supersedes that turn. It is
	// how a compacting policy substitutes a summary for the turns it folded up;
	// a truncating policy leaves it nil.
	//
	// A replaced turn becomes plain text of the same role, losing any structure
	// it had. That is why a tool group is replaced as a whole or not at all.
	Replace map[int]string
}

HistoryEdit is a policy's decision about a conversation: what to keep, and what to rewrite.

type HistoryPolicy added in v0.10.0

type HistoryPolicy interface {
	Bound(ctx context.Context, turns []TurnInfo) (HistoryEdit, error)
}

HistoryPolicy bounds a conversation before it is sent.

Bound takes a context and returns an error because a policy may call the model: compaction summarises rather than drops, which is a round-trip that can fail. It returns replacement text rather than only a subset for the same reason. Truncation uses neither, and that asymmetry is deliberate — the interface is shaped so compaction arrives as an implementation rather than as a breaking change. See https://gitlab.com/phpboyscout/go/chat/-/wikis/specs/0011-chat-provider-conformance D7.

A policy need not enforce the pinning and tool-group invariants itself: ApplyHistoryEdit checks them, so a policy that gets them wrong fails loudly instead of corrupting a transcript.

Example
package main

import (
	"context"

	"gitlab.com/phpboyscout/go/chat"
)

// keepEverySecondTurn is a deliberately silly policy, written to show the shape
// rather than to be useful: a policy reads a description of the conversation and
// returns the indices to keep.
type keepEverySecondTurn struct{}

func (keepEverySecondTurn) Bound(
	_ context.Context,
	turns []chat.TurnInfo,
) (chat.HistoryEdit, error) {
	keep := make([]int, 0, len(turns))

	for i := range turns {

		if i%2 == 0 || turns[i].Pinned || turns[i].ToolGroup != 0 {
			keep = append(keep, i)
		}
	}

	return chat.HistoryEdit{Keep: keep}, nil
}

func main() {
	cfg := chat.Config{
		Provider:      chat.ProviderClaudeLocal,
		HistoryPolicy: keepEverySecondTurn{},
	}

	_ = cfg
}

func CompactOldest added in v0.12.0

func CompactOldest(opts ...CompactOption) HistoryPolicy

CompactOldest returns a policy that summarises the older region of a conversation and substitutes the summary, rather than dropping turns as TruncateOldest does.

It calls the model, so it costs tokens and can fail. It runs only when a budget is exceeded, and deciding it has nothing to do is free.

// every default: the live model, 200 turns, 30% of the model's input limit
cfg.HistoryPolicy = chat.CompactOldest()

// a cheaper summariser, a tighter budget, and degradation rather than failure
cfg.HistoryPolicy = chat.CompactOldest(
	chat.WithSummarizer(chat.Config{Provider: chat.ProviderClaude, Model: "claude-haiku-4-5-20251001"}),
	chat.WithTokenBudget(250_000),
	chat.WithCompactionFallback(chat.TruncateOldest(200)),
)

Cached turns are never dropped or rewritten, and a tool request travels with its results — both enforced by ApplyHistoryEdit rather than by this policy.

See https://gitlab.com/phpboyscout/go/chat/-/wikis/specs/0014-history-compaction.

func DropLastExchange added in v0.14.0

func DropLastExchange() HistoryPolicy

DropLastExchange returns a policy that removes the most recent exchange: the caller's last prompt, the reply to it, and any tool traffic between them.

It is a HistoryPolicy rather than a provider method so that `/undo` needs no capability `/compact` does not already need. A pinned turn is never dropped — ApplyHistoryEdit refuses that — so undoing back into a cached prefix fails rather than quietly invalidating it.

func TruncateOldest added in v0.10.0

func TruncateOldest(maxTurns int) HistoryPolicy

TruncateOldest returns a policy that keeps the most recent turns within maxTurns, dropping oldest first.

Pinned turns are always kept and do not count against the budget in the sense that they are never candidates for dropping — a conversation whose cached prefix alone exceeds maxTurns keeps the prefix and nothing else, because dropping a cached turn is the one thing a policy may not do.

Turns are kept as a contiguous recent run rather than cherry-picked to fill the budget exactly: skipping a large tool group to fit two smaller older turns would leave the model reading a conversation that never happened.

A maxTurns of zero or less returns a policy that changes nothing.

Example
package main

import (
	"gitlab.com/phpboyscout/go/chat"
)

func main() {
	// Nil is the default: the conversation grows without bound. Set a policy for
	// a long-running loop that would otherwise fill the context window.
	cfg := chat.Config{
		Provider:      chat.ProviderClaudeLocal,
		HistoryPolicy: chat.TruncateOldest(40),
	}

	_ = cfg
}

func WhenBusy added in v0.27.0

func WhenBusy(policy HistoryPolicy, rule BusyRule) HistoryPolicy

WhenBusy attaches a BusyRule to a policy passed to an explicit edit, such as TranscriptEditor.ApplyPolicyNow. A nil policy means the configured one, under this rule.

editor.ApplyPolicyNow(ctx, chat.WhenBusy(chat.CompactOldest(), chat.EditMerges))

type Lane added in v0.27.0

type Lane struct {
	// contains filtered or unexported fields
}

Lane serialises the operations of one conversational client, so that any number of goroutines can use it (spec 0026). Operations run one at a time in arrival order; consecutive queued Lane.Converse calls run as one batch whose reply they share. The zero value is ready to use.

An operation must not call back into the client that owns the lane: it would queue behind itself.

func (*Lane) Converse added in v0.27.0

func (l *Lane) Converse(ctx context.Context, turn Turn, run BatchFunc) (string, error)

Converse runs a Chat or StreamChat turn through the lane. Run alone at an idle lane, it behaves exactly as an unlaned call; queued, it joins the batch of calls queued beside it, and returns that batch's reply.

A caller whose context ends is withdrawn: before its batch starts its prompt is never sent, and after, it stops receiving events while the others carry on.

func (*Lane) Edit added in v0.27.0

func (l *Lane) Edit(ctx context.Context, policy HistoryPolicy, apply func(context.Context) (int, error)) (int, error)

Edit runs a transcript edit under the rule WhenBusy attached to policy: queued when it waits, refused when busy if it refuses, and at once if it merges. apply reports how many turns the edit dropped.

func (*Lane) Exclusive added in v0.27.0

func (l *Lane) Exclusive(ctx context.Context, fn func(context.Context) error) error

Exclusive runs fn alone, in its turn. A caller whose context ends before its turn is withdrawn without running; once fn has started, it is given that context and Exclusive waits for it to return.

func (*Lane) TryExclusive added in v0.27.0

func (l *Lane) TryExclusive(ctx context.Context, fn func(context.Context) error) error

TryExclusive runs fn at once if the lane is idle, and otherwise returns ErrTranscriptMoved without running it.

type Limits added in v0.6.0

type Limits struct {
	MaxInputTokens  int
	MaxOutputTokens int
}

Limits carries a model's token ceilings.

Limits are deliberately not a Capability: every model has them and no provider lacks the concept, so a Support value could only ever say yes or unknown — and unknown there says nothing a nil does not. Keeping the vocabulary to things that can genuinely be absent is what makes Support mean something.

type LogprobsCapable added in v0.35.0

type LogprobsCapable interface {
	ChatClient

	// ChatWithLogprobs sends prompt after the system prompt alone and returns
	// the reply with each token's log-probability and up to top alternatives.
	// top is 0 to [MaxTopLogprobs], checked before the request. It offers no
	// tools, neither reads nor records history, runs outside the conversational
	// lane, and counts in Usage but not in History().LastInputTokens.
	//
	// A model that refuses log-probabilities is [ErrModelRejectedParameter]; a
	// reply without them is [ErrLogprobsUnavailable].
	ChatWithLogprobs(ctx context.Context, prompt string, top int, media ...Media) (Scored, error)
}

LogprobsCapable is implemented by provider clients that can return the log-probability of each generated token.

Discover it the same way as StreamingChatClient. A fallback composite offers it only when every member does.

See https://gitlab.com/phpboyscout/go/chat/-/wikis/specs/0032-token-log-probabilities.

type Media

type Media struct {
	MIMEType string
	Data     []byte
}

Media is one input attachment (an image, PDF, or supported A/V clip) sent alongside a text prompt to a multimodal model (spec 2026-07-05-chat-multimodal). Data holds the raw bytes. MIMEType is an OPTIONAL cross-check: the type is always sniffed from Data and the sniffed type is authoritative (and what is sent); when MIMEType is set it must match the sniffed family or the attachment is rejected, so a declared type can never smuggle disguised content past the safety filter.

type ModelIdentifier added in v0.6.0

type ModelIdentifier interface {
	Provider() Provider
	Model() string
}

ModelIdentifier is implemented by clients that can report the provider and model they were built with. Discover it by type assertion, as with the other optional interfaces.

CapabilitiesOf needs it because ChatClient itself carries no identity, and the core deliberately does not wrap provider clients to add one — a wrapper would have to re-advertise every optional interface or silently break capability discovery for the ones it forgot.

type ModelInfo added in v0.6.0

type ModelInfo struct {
	Capabilities Capabilities
	// Limits is nil when the ceilings are not known.
	Limits *Limits
}

ModelInfo is everything the module can say about a provider and model.

func CapabilitiesFor added in v0.6.0

func CapabilitiesFor(name Provider, model string) ModelInfo

CapabilitiesFor reports what provider supports for model, without constructing anything. Use it to choose between models before building a client.

A provider whose module is not linked reports every capability as SupportUnknown and no limits, rather than an error: not importing an adapter is a legitimate state, and Unknown is already the right answer for it.

func CapabilitiesForConfig added in v0.20.0

func CapabilitiesForConfig(cfg Config, model string) ModelInfo

CapabilitiesForConfig reports what a client built from cfg supports, with the caller's declaration applied over whatever the provider resolved.

Use it in preference to CapabilitiesFor wherever a Config is to hand. CapabilitiesFor takes only a provider and a model, so it cannot see a declaration and answers as though none was made.

func CapabilitiesOf added in v0.6.0

func CapabilitiesOf(client ChatClient) ModelInfo

CapabilitiesOf reports what an existing client supports.

Clients that do not implement ModelIdentifier report every capability as SupportUnknown, for the same reason CapabilitiesFor does on an unregistered provider.

type PersistentChatClient

type PersistentChatClient interface {
	ChatClient
	// Save captures the current conversation state as an immutable snapshot.
	// The snapshot includes provider-specific messages as opaque JSON, tool
	// metadata (without handlers), and configuration. Tokens are never saved.
	Save() (*Snapshot, error)
	// Restore replaces the current conversation state with a previously saved
	// snapshot. The snapshot's Provider must match the client's provider.
	// After restore, tools must be re-registered via SetTools with live handlers.
	Restore(snapshot *Snapshot) error
}

PersistentChatClient extends ChatClient with the ability to save and restore conversation state. Discover via type assertion (same pattern as StreamingChatClient):

if pc, ok := client.(chat.PersistentChatClient); ok {
    snapshot, err := pc.Save()
}

ClaudeLocal does not implement this interface — it delegates to an external subprocess and has no internal message state to persist.

Example
package main

import (
	"context"
)

func main() {
	// Discover persistence support via type assertion:
	//
	//   client, _ := chat.New(ctx, chat.Settings{Config: cfg, Logger: log})
	//   if pc, ok := client.(chat.PersistentChatClient); ok {
	//       snapshot, _ := pc.Save()
	//       // ... store snapshot ...
	//       pc.Restore(snapshot)
	//   }
	//
	// ClaudeLocal does not implement PersistentChatClient.

	_ = context.Background()
}

type Proposal added in v0.35.0

type Proposal struct {
	Text  string
	Calls []ToolCall
}

Proposal is a reply's text, possibly empty, and the tool calls the model made, unexecuted. Calls is empty when the model chose none.

type Provider

type Provider string

Provider defines the AI service provider.

const (
	// ProviderOpenAI uses OpenAI's API.
	ProviderOpenAI Provider = "openai"
	// ProviderOpenAICompatible uses any OpenAI-compatible API endpoint (e.g. Ollama, xAI).
	ProviderOpenAICompatible Provider = "openai-compatible"
	// ProviderClaude uses Anthropic's Claude API.
	ProviderClaude Provider = "claude"
	// ProviderClaudeLocal uses a locally installed claude CLI binary.
	ProviderClaudeLocal Provider = "claude-local"
	// ProviderCodexLocal uses a locally installed codex CLI binary, the way
	// ProviderClaudeLocal uses claude. It is registered by chat-openai.
	ProviderCodexLocal Provider = "codex-local"
	// ProviderGemini uses Google's Gemini API.
	ProviderGemini Provider = "gemini"
	// ProviderGeminiVertex uses Google's Gemini models through the Vertex AI
	// backend, addressed by Config.Project and Config.Location and authenticated
	// by Google application default credentials rather than an API key.
	//
	// It is a separate name rather than a flag on ProviderGemini because the two
	// differ in how they are addressed and how they authenticate, and because a
	// name is reachable from the registry and from a config file. See
	// https://gitlab.com/phpboyscout/go/chat/-/wikis/specs/0017-gemini-vertex-backend D5.
	ProviderGeminiVertex Provider = "gemini-vertex"
	// ProviderAgyLocal uses a locally installed agy CLI binary, which serves
	// Gemini models. It is registered by chat-gemini.
	//
	// It is the one provider in the family that cannot carry tools: agy has no
	// per-invocation MCP route, only persistent configuration, so offering them
	// would mean rewriting the user's own config around every call. It reports
	// CapTools: SupportNo rather than leaving a caller to find out. See
	// https://gitlab.com/phpboyscout/go/chat/-/wikis/specs/0022-codex-and-agy-local D4.
	ProviderAgyLocal Provider = "agy-local"
	// ProviderBedrock uses AWS Bedrock through the Converse API, addressed by
	// Config.Location and authenticated by the AWS credential chain.
	//
	// Bedrock serves models from many vendors behind one API, so Config.Model
	// carries whatever Bedrock accepts: a model ID, an inference profile ID, or
	// an ARN. This package does not inspect or rewrite it. See
	// https://gitlab.com/phpboyscout/go/chat/-/wikis/specs/0018-aws-bedrock-provider D5.
	ProviderBedrock Provider = "bedrock"

	// ProviderAzureOpenAI reaches Azure OpenAI, which serves OpenAI's models at
	// a caller-owned endpoint. It is not ProviderOpenAICompatible: the model is
	// addressed as a deployment name in the request path, an api-version query
	// parameter is required on every call, and authentication is an api-key
	// header or an Entra ID token rather than a bearer token. Pointing BaseURL
	// at an Azure endpoint under another provider name returns 404s that do not
	// explain themselves. See spec 0019 D5.
	ProviderAzureOpenAI Provider = "azure-openai"

	// ProviderVLLM reaches a vLLM server, which the caller always runs, so
	// Config.BaseURL is required. It is not ProviderOpenAICompatible: it is
	// registered by chat-vllm, which adds log-probabilities, answer scoring and
	// LoRA adapters that only vLLM serves. See
	// https://gitlab.com/phpboyscout/go/chat/-/wikis/specs/0035-vllm-adapter.
	ProviderVLLM Provider = "vllm"
)

func RegisteredProviders added in v0.24.0

func RegisteredProviders() []Provider

RegisteredProviders returns every name with a registered factory, sorted.

type ProviderCredentialSources added in v0.36.0

type ProviderCredentialSources func(provider Provider) (CredentialSource, bool)

ProviderCredentialSources resolves the credential source for one provider in a fallback chain. Returning false leaves that provider to its own default.

type ProviderFactory

type ProviderFactory func(ctx context.Context, settings Settings) (ChatClient, error)

ProviderFactory creates a ChatClient for a named provider. Register implementations via RegisterProvider in an init() function to allow external packages to add providers without modifying this file.

type RefusedError added in v0.31.0

type RefusedError struct {
	// Reason is the token the API sent, such as refusal, content_filter, SAFETY
	// or cyber_policy, or prompt_blocked where the API sends only a block
	// reason, which is then in Category.
	Reason string
	// Category is the provider's finer classification, when it gives one.
	Category string
	// Prompt reports that the prompt was rejected before anything was generated.
	Prompt bool
	// Message is the provider's own text about the refusal, if any.
	Message string
	// Err is the provider's error, when the refusal arrived as one.
	Err error
}

RefusedError is a refusal, carrying the provider's own account of it. It matches ErrRefused and, when the refusal arrived as an API error, that error too, so its HTTP status still answers ProviderHTTPStatus (spec 0029).

func (*RefusedError) Error added in v0.31.0

func (e *RefusedError) Error() string

func (*RefusedError) Unwrap added in v0.31.0

func (e *RefusedError) Unwrap() []error

type ResolvedMedia

type ResolvedMedia struct {
	MIMEType string
	Data     []byte
}

ResolvedMedia is an attachment that has passed the safety filter: its type is known, allowlisted, and provider-accepted, and it is ready to map to a provider request. Providers consume []ResolvedMedia, never raw Media.

func ValidateMediaSet

func ValidateMediaSet(provider Provider, media []Media) ([]ResolvedMedia, error)

ValidateMediaSet runs the safety choke point over a request's attachments before any network call (spec §5): it enforces the provider's media capability, the per-attachment sniff + declared cross-check + allowlist, the provider's type support, and the size/count caps. It returns the resolved, ready-to-send attachments or a typed error.

func ValidateMediaSetFor added in v0.20.0

func ValidateMediaSetFor(cfg Config, media []Media) ([]ResolvedMedia, error)

ValidateMediaSetFor is ValidateMediaSet with the caller's declaration honoured, per spec 0020 D8.

A caller who has declared CapMultimodal for their endpoint is believed, and the provider's own media table is not consulted: the table describes providers this module knows, and a declaration exists precisely for one it does not.

Everything else still applies. The sniff, the allowlist, the declared-type cross-check and the size and count caps are unchanged, because the caller is trusted about their endpoint rather than about the bytes.

type Role added in v0.10.0

type Role string

Role identifies who produced a conversation turn.

const (
	// RoleUser is a turn from the caller, including tool results, which every
	// provider sends back as user-side content.
	RoleUser Role = "user"
	// RoleAssistant is a turn from the model, including tool requests.
	RoleAssistant Role = "assistant"
	// RoleSystem is the system prompt.
	RoleSystem Role = "system"
)

type RuntimeConfig

type RuntimeConfig struct {
	Provider       Provider       `mapstructure:"provider"`
	RequestTimeout time.Duration  `mapstructure:"request_timeout"`
	Fallback       FallbackConfig `mapstructure:"fallback"`
}

RuntimeConfig is the package-owned shape for GTB's ai.* config section.

type SamplingCapable deprecated added in v0.3.0

type SamplingCapable interface {
	ChatClient

	// SupportsSampling is a compile-time marker. It is never called.
	SupportsSampling()
}

SamplingCapable is implemented by provider clients that can carry Config.Temperature and Config.TopP.

This marks a *structural* capability — whether the provider has the concept at all — not whether the selected model will accept the value. ProviderClaudeLocal does not implement it: the claude CLI exposes no sampling flag at any version, so there is nothing to plumb. A model refusing a value it structurally supports is a different failure; see ErrModelRejectedParameter.

Deprecated: prefer CapabilitiesFor with CapSampling, which reports the same structural fact three-valued and carries the model specifics with it. This marker still gates construction and is not going away in this release.

type SamplingSupport added in v0.6.0

type SamplingSupport struct {
	// MinTemperature and MaxTemperature bound Config.Temperature in the
	// provider's own units. Both zero means the range is not known.
	MinTemperature float64
	MaxTemperature float64
	// TopP reports whether nucleus sampling is separately settable.
	TopP Support
}

SamplingSupport carries the specifics of CapSampling. A zero field means "not known", never "zero".

type Scored added in v0.35.0

type Scored struct {
	Text   string
	Tokens []TokenLogprob
}

Scored is a reply with the log-probability of each generated token.

type Settings

type Settings struct {
	Config Config
	Logger *slog.Logger
}

Settings contains the package-owned construction dependencies for a chat client. Config carries provider behaviour; Logger is optional and defaults to a no-op logger.

type Snapshot

type Snapshot struct {
	// ID uniquely identifies this snapshot.
	ID string `json:"id"`
	// Provider identifies which chat provider created this snapshot.
	Provider Provider `json:"provider"`
	// Model is the AI model used in the conversation.
	Model string `json:"model"`
	// SystemPrompt is the system instruction active at snapshot time.
	SystemPrompt string `json:"system_prompt,omitempty"`
	// Messages contains provider-specific message history as opaque JSON.
	// The format varies by provider — do not parse or modify directly.
	Messages json.RawMessage `json:"messages"`
	// Tools captures tool metadata (name, description, parameters) without
	// handlers. After restoring, call SetTools to re-register live handlers.
	Tools []ToolSnapshot `json:"tools,omitempty"`
	// Metadata holds arbitrary key-value pairs for consumer use.
	Metadata map[string]string `json:"metadata,omitempty"`
	// CreatedAt is when this snapshot was taken.
	CreatedAt time.Time `json:"created_at"`
	// Version is the snapshot format version for forward compatibility.
	Version int `json:"version"`
}

Snapshot is an immutable point-in-time capture of a conversation.

func NewSnapshot

func NewSnapshot(provider Provider, model, systemPrompt string, messages json.RawMessage, tools map[string]Tool, metadata map[string]string) *Snapshot

NewSnapshot creates a Snapshot with a new UUID and the current timestamp.

The ID is generated via uuid.New so that it always satisfies the canonical-UUID contract enforced by [FileStore.Save], [FileStore.Load], and [FileStore.Delete]. Constructing a Snapshot struct directly and populating ID by hand is supported but discouraged — any value that fails ValidateSnapshotID will be rejected by the store. If you do need to accept a caller-supplied ID (for example, reconstructing a snapshot parsed from an external payload), call ValidateSnapshotID at the boundary rather than relying on Save/Load/Delete to reject it later.

Example
package main

import (
	"encoding/json"

	"gitlab.com/phpboyscout/go/chat"
)

func main() {
	snap := chat.NewSnapshot(
		chat.ProviderClaude,
		"claude-3-5-sonnet",
		"You are a helpful assistant.",
		json.RawMessage(`[{"role":"user","content":"hello"}]`),
		nil,
		map[string]string{"session": "demo"},
	)

	_ = snap.ID        // UUID
	_ = snap.CreatedAt // timestamp
}

type SnapshotSummary

type SnapshotSummary struct {
	ID           string    `json:"id"`
	Provider     Provider  `json:"provider"`
	Model        string    `json:"model"`
	CreatedAt    time.Time `json:"created_at"`
	MessageCount int       `json:"message_count"`
}

SnapshotSummary is a lightweight view of a snapshot for listing without loading the full message history.

type StatelessCapable added in v0.2.0

type StatelessCapable interface {
	ChatClient

	// SupportsStateless is a compile-time marker. It is never called.
	SupportsStateless()
}

StatelessCapable is implemented by provider clients that honour Config.Stateless.

The chat core and each provider module version independently, so a caller can pin a provider built before stateless mode existed. Without this marker, Config.Stateless on such a provider would compile, be ignored, and bill the caller for the accumulated history they asked not to send — the exact silent failure the flag exists to remove. New therefore type-asserts the constructed client and refuses to hand back one that cannot honour the request.

Discover it the same way as StreamingChatClient and PersistentChatClient:

if _, ok := client.(chat.StatelessCapable); ok {
    // one-shot calls are honoured
}

type StreamCallback

type StreamCallback func(event StreamEvent) error

StreamCallback receives streaming events. Return a non-nil error to cancel the stream.

type StreamEvent

type StreamEvent struct {
	// Type indicates the kind of event.
	Type StreamEventType

	// Delta contains the text fragment for EventTextDelta events.
	Delta string

	// ToolCall contains tool call information for EventToolCallStart/EventToolCallEnd events.
	ToolCall *StreamToolCall

	// Error contains error information for EventError events.
	Error error
}

StreamEvent represents a single event in a streaming response.

type StreamEventType

type StreamEventType int

StreamEventType identifies the kind of stream event.

const (
	// EventTextDelta is a partial text response fragment.
	EventTextDelta StreamEventType = iota
	// EventToolCallStart indicates a tool call has begun execution.
	EventToolCallStart
	// EventToolCallEnd indicates a tool call has completed execution.
	EventToolCallEnd
	// EventComplete indicates the stream has finished successfully.
	EventComplete
	// EventError indicates an error occurred during streaming.
	EventError
	// EventProgress carries a note the model wrote between tool calls, in
	// Delta. Only providers whose notes arrive outside the answer emit it.
	EventProgress
)

type StreamToolCall

type StreamToolCall struct {
	// ID is the provider-assigned identifier for the tool call.
	ID string
	// Name is the tool name.
	Name string
	// Arguments is the complete JSON argument payload (only populated on EventToolCallEnd).
	Arguments string
	// Result is the tool execution result (only populated on EventToolCallEnd).
	Result string
}

StreamToolCall contains information about a tool call within a stream.

type StreamingChatClient

type StreamingChatClient interface {
	ChatClient

	// StreamChat sends a message and streams the response via callback.
	// The callback is invoked for each event in the stream. If the callback
	// returns a non-nil error, the stream is cancelled and that error is returned.
	// The return value is the complete assembled response text (concatenation of
	// all EventTextDelta fragments) or an error if streaming failed.
	// Tool calls are handled internally via the same ReAct loop as Chat(). If
	// Config.ParallelTools is enabled, multiple tool calls are executed concurrently.
	StreamChat(ctx context.Context, prompt string, callback StreamCallback, media ...Media) (string, error)
}

StreamingChatClient extends ChatClient with streaming support. Implementations that support streaming implement this interface in addition to ChatClient. Discover support via type assertion:

if streamer, ok := client.(chat.StreamingChatClient); ok {
    result, err := streamer.StreamChat(ctx, "prompt", callback)
}

type SummariserFactory added in v0.37.0

type SummariserFactory interface {
	SummariserFactory() ProviderFactory
}

SummariserFactory is implemented by a client a provider constructor built, so compaction can summarise on the options passed there rather than refuse.

The factory carries only the options that decide where a request goes, as whom, and under what guardrail: never tool servers, CLI access, or anything shaping output. It is called again for every client a policy is wired to, so whatever it closes over must be safe to share with a second client. See https://gitlab.com/phpboyscout/go/chat/-/wikis/specs/0040-summariser-on-the-callers-credentials.

type Support added in v0.6.0

type Support uint8

Support is a three-valued answer to "does this support that".

Unknown is a first-class answer rather than a failure, and it is the reason this is not a bool. Two of the five providers cannot be interrogated at all — claude-local is a CLI with no models endpoint, and openai-compatible points at an arbitrary server — so a two-valued design would have to report those as unsupported. That is a lie which suppresses working features, and it is the worse of the two available errors: Unknown merely means "find out by trying", which is what callers do today.

Unknown is also what makes a new capability safe to add: a provider that has not been taught about one reports Unknown rather than becoming wrong.

const (
	// SupportUnknown means nobody could say — not declared, or not discoverable.
	SupportUnknown Support = iota
	// SupportNo means the provider or model does not support it.
	SupportNo
	// SupportYes means it is supported.
	SupportYes
)

func (Support) String added in v0.6.0

func (s Support) String() string

String implements fmt.Stringer.

type TokenLogprob added in v0.35.0

type TokenLogprob struct {
	Token   string
	Bytes   []byte
	Logprob float64
	Top     []TokenLogprob
}

TokenLogprob is one generated token, its log-probability and its most likely alternatives, most likely first. Top may omit the chosen token. Bytes is nil where the provider sends none.

type Tool

type Tool struct {
	Name        string                                                       `json:"name"`
	Description string                                                       `json:"description"`
	Parameters  *jsonschema.Schema                                           `json:"parameters"`
	Handler     func(ctx context.Context, args json.RawMessage) (any, error) `json:"-" mapstructure:"-" yaml:"-"`
}

Tool represents a function that the AI can call.

type ToolCall

type ToolCall struct {
	Name  string
	Input json.RawMessage
	ID    string
}

ToolCall represents a single tool invocation request. ID is the provider's call ID, empty where it sends none; execution correlates calls by position.

type ToolCallProposer added in v0.35.0

type ToolCallProposer interface {
	ChatClient

	// ProposeToolCalls offers tools to the model and returns the calls it
	// makes, in order, without running them. It refuses an empty tool list and
	// any tool carrying a Handler, before the request. It sends prompt after the
	// system prompt alone with tool choice left to the model, neither reads nor
	// records history, runs outside the conversational lane, and counts in Usage
	// but not in History().LastInputTokens.
	ProposeToolCalls(ctx context.Context, prompt string, tools []Tool, media ...Media) (Proposal, error)
}

ToolCallProposer is implemented by provider clients that can return the tool calls a model makes without running them.

Discover it the same way as StreamingChatClient. A fallback composite offers it only when every member does.

See https://gitlab.com/phpboyscout/go/chat/-/wikis/specs/0034-tool-calls-without-execution.

type ToolFailure added in v0.38.0

type ToolFailure struct {
	Content string
	Cause   error
}

ToolFailure is a tool error whose Content is exactly what the model reads, in place of the "Error: ..." text any other error becomes. Cause is for host diagnostics and never reaches the model. See https://gitlab.com/phpboyscout/go/chat/-/wikis/specs/0041-go-mcp-registry-as-chat-tools

func (*ToolFailure) Error added in v0.38.0

func (f *ToolFailure) Error() string

Error returns Content.

func (*ToolFailure) Unwrap added in v0.38.0

func (f *ToolFailure) Unwrap() error

Unwrap returns Cause.

type ToolResult

type ToolResult struct {
	Name    string
	Result  string
	IsError bool
}

ToolResult holds the result of a single tool execution. When IsError is set, Result is what the model reads about the failure; a provider carries the flag in its API's own error marker where it has one.

func DispatchToolExecution

func DispatchToolExecution(ctx context.Context, l *slog.Logger, tools map[string]Tool, calls []ToolCall, parallelTools bool, maxParallelTools int) []ToolResult

DispatchToolExecution runs tool calls sequentially or in parallel depending on parallelTools and the number of calls. It is the shared dispatch entry point for all provider ReAct loops.

func ExecuteToolResult added in v0.38.0

func ExecuteToolResult(ctx context.Context, l *slog.Logger, tools map[string]Tool, name string, input json.RawMessage) ToolResult

ExecuteToolResult is ExecuteTool with the outcome kept: IsError is set for every failure, including an unknown tool and a recovered panic. A handler error that is a ToolFailure contributes its Content as the result; any other keeps the "Error: ..." text ExecuteTool has always returned.

func ExecuteToolsParallel

func ExecuteToolsParallel(ctx context.Context, l *slog.Logger, tools map[string]Tool, calls []ToolCall, maxConcurrency int) []ToolResult

ExecuteToolsParallel executes multiple tool calls concurrently, bounded by maxConcurrency. Results are returned in the same order as the input calls. If maxConcurrency is zero or negative, it defaults to 5.

type ToolServer added in v0.30.0

type ToolServer func(ctx context.Context, log *slog.Logger, tools map[string]Tool) (*ToolSession, error)

ToolServer publishes tools for a subprocess to call, and returns where to reach them. Local-CLI providers use it to offer a caller's tools to their CLI.

The core ships none, so it links no transport for one. A module registers its server from init() with RegisterToolServer, and a program opts in with a blank import of that module (spec 0028).

func RegisteredToolServer added in v0.30.0

func RegisteredToolServer() (ToolServer, bool)

RegisteredToolServer returns the program's tool server, and false when no linked module has registered one.

type ToolSession added in v0.30.0

type ToolSession struct {
	// Endpoint is the loopback URL the CLI connects to.
	Endpoint string
	// Token is the bearer token the CLI must present.
	Token string
	// Stop takes the published tools down.
	Stop func()
}

ToolSession is one published tool set. Endpoint is empty, and Stop does nothing, when there were no tools to publish.

type ToolSnapshot

type ToolSnapshot struct {
	Name        string             `json:"name"`
	Description string             `json:"description"`
	Parameters  *jsonschema.Schema `json:"parameters,omitempty"`
}

ToolSnapshot captures tool metadata without the handler function.

type TranscriptEditor added in v0.14.0

type TranscriptEditor interface {
	// ApplyPolicyNow bounds the retained conversation immediately, rather than
	// waiting for the next request to do it, and reports how many turns went.
	//
	// A nil policy means the one on [Config.HistoryPolicy]; with none configured
	// there is nothing to apply and it returns zero. Zero with a nil error means
	// the policy kept everything, which is a legitimate outcome rather than a
	// failure — a conversation inside its budget has nothing to do.
	//
	// Implementations run the policy through [BoundConversation], so an edit
	// that would drop a cached turn or split a tool group is refused rather than
	// applied.
	//
	// It returns [ErrTranscriptMoved] when another goroutine changed the
	// conversation while the policy was running, which a caller may retry.
	ApplyPolicyNow(ctx context.Context, policy HistoryPolicy) (removed int, err error)
}

TranscriptEditor is the optional contract for a provider that can apply a HistoryPolicy to its retained conversation on demand.

It exists because the core cannot reach a transcript, and that is deliberate: Snapshot.Messages is provider JSON the core must not parse, and BoundConversation takes turns because only a provider can enumerate its own. So a consumer has no way to say "shorten this now" unless the provider offers one.

One seam, not one method per operation

The single method takes a policy rather than naming an operation, so the provider surface does not grow every time a command needs to change a transcript. `/compact` and `/undo` are both this method with a different policy, and a future operation is a policy the core writes — no adapter change, no release train.

It is also the shape a provider already runs. Before every request it describes its retained turns, calls BoundConversation, and rewrites itself from the result. This exposes that sequence rather than adding a second way to do it, so the pinning and tool-group invariants come along unchanged.

ProviderClaudeLocal will not implement it: its conversation lives in the claude CLI's own session, so it cannot enumerate turns at all. See https://gitlab.com/phpboyscout/go/chat/-/wikis/specs/0013-history-policy-application D6, and spec 0016 D5.

type TruncatedError added in v0.26.0

type TruncatedError struct {
	// Partial is the text the model produced before the limit.
	Partial string
	// MaxTokens is the limit that was reached.
	MaxTokens int
}

TruncatedError is a reply cut off at Config.MaxTokens, carrying what arrived before the cut. It matches ErrTruncated.

func (*TruncatedError) Error added in v0.26.0

func (e *TruncatedError) Error() string

func (*TruncatedError) Unwrap added in v0.26.0

func (e *TruncatedError) Unwrap() error

type Turn added in v0.27.0

type Turn struct {
	// Payload is the provider's own form of the prompt, validated before the
	// call joins the lane.
	Payload any
	// Stream receives the reply's events. Nil for a Chat call.
	Stream StreamCallback
}

Turn is one caller's prompt waiting to go upstream through a Lane.

type TurnInfo added in v0.10.0

type TurnInfo struct {
	// Index is the turn's position in the conversation, oldest first.
	Index int

	// Role is who produced the turn.
	Role Role

	// Text is the provider's best-effort rendering of the turn as plain text,
	// supplied so a policy can reason about content — a compacting policy
	// summarises from it. It is lossy by construction: a tool call rendered as
	// text is not the tool call. Nothing reconstructs a turn from it; it exists
	// to be read, and to be replaced wholesale via [HistoryEdit.Replace].
	Text string

	// Pinned marks a turn added via [CachingChatClient.AddCached]. A policy
	// must never drop one, and [ApplyHistoryEdit] refuses an edit that does.
	//
	// Caching bills a matching prefix at the cached rate, so dropping a cached
	// turn destroys every subsequent cache hit — a failure that shows up only
	// as a larger invoice.
	Pinned bool

	// ToolGroup ties a tool request to its results. Turns sharing a non-zero
	// ToolGroup are one indivisible unit; zero means the turn stands alone.
	//
	// Providers require a tool request and its result to travel together, so an
	// edit that keeps one without the other yields an API error rather than a
	// shorter conversation. [ApplyHistoryEdit] refuses to split a group.
	ToolGroup int
}

TurnInfo describes one conversation turn to a HistoryPolicy.

It is a description, not the turn itself. Provider message formats do not share a shape — an Anthropic tool_use block, a Gemini session and an OpenAI params struct have nothing useful in common — and the core deliberately treats a provider's transcript as opaque (see Snapshot.Messages). So a provider describes its turns for a policy to reason about, and applies the resulting HistoryEdit itself, in its own format.

type Usage

type Usage struct {
	// InputTokens is the number of prompt/input tokens consumed.
	InputTokens int
	// OutputTokens is the number of completion/output tokens produced.
	OutputTokens int
	// TotalTokens is the sum of input and output tokens. When a provider
	// supplies its own total it is preserved; otherwise it is computed as
	// InputTokens + OutputTokens.
	TotalTokens int
	// CachedTokens is the number of input tokens served from a provider-side
	// prompt cache, when reported. Zero when the provider does not expose it.
	CachedTokens int
	// ReasoningTokens is the number of tokens spent on internal reasoning
	// ("thinking") output, when reported. Zero when not exposed.
	ReasoningTokens int
	// Known reports whether the provider supplied token counts for this call.
	// False indicates the counts are not authoritative (e.g. ProviderClaudeLocal).
	Known bool
}

Usage reports the token consumption of one or more provider round-trips in a provider-neutral shape. Tools built on GTB use it to observe and cost their LLM calls without coupling to any one provider's SDK.

Token counts are summed across every provider round-trip made within a single Chat, Ask, or StreamChat call — a ReAct tool-calling loop makes one round-trip per step, and the usage of every step is accumulated. The Usage() accessor on a client returns the cumulative total across the lifetime of that client instance; see ChatClient.Usage for details.

Known reports whether the provider supplied token counts. Providers that do not expose usage (notably ProviderClaudeLocal, which wraps the claude CLI and returns no token data) report a zero-valued Usage with Known == false. Always check Known before treating the counts as authoritative.

func NewUsage

func NewUsage(input, output, total, cached, reasoning int) Usage

NewUsage builds a Usage from raw provider counts, computing TotalTokens when the provider did not supply one (total <= 0) and marking the result Known.

func (Usage) Add

func (u Usage) Add(other Usage) Usage

Add returns the element-wise sum of two Usage values. A result is Known if either operand is Known. TotalTokens is summed directly so a provider-supplied total (which may differ from input+output) is preserved across accumulation.

type UsageTracker

type UsageTracker struct {
	// contains filtered or unexported fields
}

UsageTracker accumulates per-round-trip usage for a single client instance and fans each round-trip out to an optional observer. It is embedded by every provider implementation. The zero value is ready to use.

It is safe for concurrent use, but note that ChatClient implementations are not themselves safe for concurrent use; the mutex guards only against an observer reading Usage() from another goroutine while a call is in flight.

func (*UsageTracker) LastUsage added in v0.10.0

func (t *UsageTracker) LastUsage() Usage

LastUsage returns the most recent conversation round-trip's usage, or a zero Usage with Known == false when no such call has been made. A round-trip recorded with RecordUsageOutsideConversation never becomes it.

This is what makes a conversation's growth observable without the core owning a tokenizer: InputTokens here is the provider's own count of everything the last call sent — system prompt, retained history and the new turn together — so it is measured rather than estimated, and stale by exactly one call.

The core cannot do better. Counting locally would need a tokenizer per model family, and there is no universal one: Anthropic publishes none (its count is an API round-trip), Gemini counts server-side, and only OpenAI can be tokenised offline. depfootprint_test.go forbids that dependency in the core for exactly this reason. See https://gitlab.com/phpboyscout/go/chat/-/wikis/specs/0011-chat-provider-conformance D7.

func (*UsageTracker) RecordUsage

func (t *UsageTracker) RecordUsage(u Usage)

RecordUsage accumulates one round-trip's usage into the lifetime total and, if an observer is configured, invokes it with that round-trip's usage. Passing a usage with Known == false (e.g. a provider that reports nothing) still fires the observer so callers can distinguish "no usage" from "no call".

func (*UsageTracker) RecordUsageOutsideConversation added in v0.34.0

func (t *UsageTracker) RecordUsageOutsideConversation(u Usage)

RecordUsageOutsideConversation records a round-trip that carried something other than the conversation, such as a one-request log-probability call, as RecordUsage does but without making it LastUsage. LastUsage feeds History.LastInputTokens, which a token-budget HistoryPolicy reads as the conversation's size.

func (*UsageTracker) SetObserver

func (t *UsageTracker) SetObserver(observer func(Usage))

SetObserver wires a per-round-trip usage observer onto the tracker. Provider modules call it during construction to forward Config.UsageObserver; the observer field stays unexported so it is only settable through this method.

func (*UsageTracker) Usage

func (t *UsageTracker) Usage() Usage

Usage returns the cumulative token usage across every provider round-trip made by this client instance since construction. It is promoted onto each provider type via embedding to satisfy ChatClient.Usage.

Directories

Path Synopsis
Package conformance is a reusable test suite that a chat provider module runs against its own client to prove it behaves like every other provider.
Package conformance is a reusable test suite that a chat provider module runs against its own client to prove it behaves like every other provider.
tools
capabilitygen module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL