Documentation
¶
Overview ¶
Package llm is the single seam through which every model call passes: interview, generation, suite drafting, judging, and repair. Agent CLIs do not expose temperature, so the determinism mitigations — schema-constrained prompts, tolerant extraction with one retry, and content-hash caching — live here rather than in each caller.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CacheKey ¶
CacheKey derives a content-addressed key. Every field that changes the model's answer must contribute, or a cache hit serves a response to the wrong question — which would be invisible and would corrupt scores.
func CompleteJSON ¶
CompleteJSON calls the gateway and decodes a JSON response into T, retrying once with the parse error quoted. Two attempts, not more: a model that fails twice on a schema is not going to succeed on the third try, and each attempt costs a session from the budget.
func ExtractJSON ¶
func ExtractJSON(raw string) (json.RawMessage, error)
ExtractJSON pulls a JSON value out of a model response. Models wrap JSON in code fences and prose regardless of instructions, so extraction is tolerant by design — but it fails loudly with the raw text quoted, because a refusal, a rate-limit notice, and a truncated response are three different problems that a bare "no JSON found" would make indistinguishable.
Types ¶
type Cache ¶
Cache stores completions by content hash so a given request is never asked twice within or across runs. Implemented by the store package.
type Gateway ¶
type Gateway interface {
Complete(ctx context.Context, r Req) (Res, error)
// ModelFor names the concrete model that would serve a request of the
// given class, without making a call. This exists so a caller that only
// knows the class it asked for (the gateway resolves ModelClass to a
// concrete model internally) can still record which model actually did
// the work — e.g. Report.JudgeModel, which gates whether two scores are
// comparable. A gateway that genuinely cannot know its own model (a
// subscription CLI left to pick its own default) must return "" rather
// than guess: an empty value means "unknown", and a guess recorded as
// provenance would be worse than no provenance at all.
ModelFor(class ModelClass) string
}
Gateway is a model backend.
type ModelClass ¶
type ModelClass string
ModelClass routes a call to a capability tier rather than a model name, so gateways can map tiers onto whatever the local subscription offers.
const ( // ClassStrong is for generation and judging. ClassStrong ModelClass = "strong" // ClassFast is for cheap classification, e.g. "did the skill trigger?". ClassFast ModelClass = "fast" )
type Req ¶
type Req struct {
// Role tags the call for logging and cache partitioning, e.g. "interview",
// "gen.body", "suite.draft".
Role string
Prompt string
Schema json.RawMessage
ModelClass ModelClass
// CacheKey overrides the derived content hash. Leave empty to use CacheKey.
CacheKey string
}
Req is one model call.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package agentcli implements the LLM gateway by shelling out to a coding-agent CLI in print mode, so every model call is billed to the operator's existing subscription rather than requiring an API key.
|
Package agentcli implements the LLM gateway by shelling out to a coding-agent CLI in print mode, so every model call is billed to the operator's existing subscription rather than requiring an API key. |
|
Package api implements the LLM gateway against a direct Anthropic-compatible HTTP endpoint.
|
Package api implements the LLM gateway against a direct Anthropic-compatible HTTP endpoint. |
|
Package fake provides an in-memory Gateway for tests: these tests run with no LLM subscription, no API key, and no network, so the fake is how generation and suite behaviour get exercised.
|
Package fake provides an in-memory Gateway for tests: these tests run with no LLM subscription, no API key, and no network, so the fake is how generation and suite behaviour get exercised. |