llm

package
v0.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 6, 2026 License: Apache-2.0 Imports: 6 Imported by: 0

Documentation

Overview

Package llm is the single seam through which every model call passes: interview, generation, suite drafting, judging, and repair. Agent CLIs do not expose temperature, so the determinism mitigations — schema-constrained prompts, tolerant extraction with one retry, and content-hash caching — live here rather than in each caller.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CacheKey

func CacheKey(r Req) string

CacheKey derives a content-addressed key. Every field that changes the model's answer must contribute, or a cache hit serves a response to the wrong question — which would be invisible and would corrupt scores.

func CompleteJSON

func CompleteJSON[T any](ctx context.Context, g Gateway, r Req) (T, error)

CompleteJSON calls the gateway and decodes a JSON response into T, retrying once with the parse error quoted. Two attempts, not more: a model that fails twice on a schema is not going to succeed on the third try, and each attempt costs a session from the budget.

func ExtractJSON

func ExtractJSON(raw string) (json.RawMessage, error)

ExtractJSON pulls a JSON value out of a model response. Models wrap JSON in code fences and prose regardless of instructions, so extraction is tolerant by design — but it fails loudly with the raw text quoted, because a refusal, a rate-limit notice, and a truncated response are three different problems that a bare "no JSON found" would make indistinguishable.

Types

type Cache

type Cache interface {
	Get(key string) (string, bool, error)
	Put(key, value string) error
}

Cache stores completions by content hash so a given request is never asked twice within or across runs. Implemented by the store package.

type Gateway

type Gateway interface {
	Complete(ctx context.Context, r Req) (Res, error)
	// ModelFor names the concrete model that would serve a request of the
	// given class, without making a call. This exists so a caller that only
	// knows the class it asked for (the gateway resolves ModelClass to a
	// concrete model internally) can still record which model actually did
	// the work — e.g. Report.JudgeModel, which gates whether two scores are
	// comparable. A gateway that genuinely cannot know its own model (a
	// subscription CLI left to pick its own default) must return "" rather
	// than guess: an empty value means "unknown", and a guess recorded as
	// provenance would be worse than no provenance at all.
	ModelFor(class ModelClass) string
}

Gateway is a model backend.

type ModelClass

type ModelClass string

ModelClass routes a call to a capability tier rather than a model name, so gateways can map tiers onto whatever the local subscription offers.

const (
	// ClassStrong is for generation and judging.
	ClassStrong ModelClass = "strong"
	// ClassFast is for cheap classification, e.g. "did the skill trigger?".
	ClassFast ModelClass = "fast"
)

type Req

type Req struct {
	// Role tags the call for logging and cache partitioning, e.g. "interview",
	// "gen.body", "suite.draft".
	Role       string
	Prompt     string
	Schema     json.RawMessage
	ModelClass ModelClass
	// CacheKey overrides the derived content hash. Leave empty to use CacheKey.
	CacheKey string
}

Req is one model call.

type Res

type Res struct {
	Text   string
	Model  string
	Cached bool
}

Res is one model response.

Directories

Path Synopsis
Package agentcli implements the LLM gateway by shelling out to a coding-agent CLI in print mode, so every model call is billed to the operator's existing subscription rather than requiring an API key.
Package agentcli implements the LLM gateway by shelling out to a coding-agent CLI in print mode, so every model call is billed to the operator's existing subscription rather than requiring an API key.
Package api implements the LLM gateway against a direct Anthropic-compatible HTTP endpoint.
Package api implements the LLM gateway against a direct Anthropic-compatible HTTP endpoint.
Package fake provides an in-memory Gateway for tests: these tests run with no LLM subscription, no API key, and no network, so the fake is how generation and suite behaviour get exercised.
Package fake provides an in-memory Gateway for tests: these tests run with no LLM subscription, no API key, and no network, so the fake is how generation and suite behaviour get exercised.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL