Documentation
¶
Overview ¶
Package llm is the single seam through which every model call passes. Determinism mitigations (schema-constrained prompts, tolerant extraction with one retry, content-hash caching) live here rather than in each caller.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrTimeout = errors.New("llm: gateway timed out")
ErrTimeout reports that a gateway call exceeded its configured timeout, distinct from context cancellation.
Functions ¶
func CacheKey ¶
CacheKey derives a content-addressed key. Every field that changes the model's answer must contribute, or a cache hit serves a wrong response.
func ExtractJSON ¶
func ExtractJSON(raw string) (json.RawMessage, error)
ExtractJSON pulls a JSON value out of a model response, tolerating code fences and surrounding prose. Fails loudly with the raw text quoted so a refusal, rate-limit notice, and truncated response stay distinguishable.
Types ¶
type Gateway ¶
type Gateway interface {
Complete(ctx context.Context, r Req) (Res, error)
// ModelFor names the concrete model for a class without making a call.
// Returns "" when unknown.
ModelFor(class ModelClass) string
}
Gateway is a model backend.
type ModelClass ¶
type ModelClass string
ModelClass routes a call to a capability tier rather than a model name.
const ( ClassStrong ModelClass = "strong" ClassFast ModelClass = "fast" )
type Req ¶
type Req struct {
Role string
Prompt string
Schema json.RawMessage
ModelClass ModelClass
CacheKey string
}
Req is one model call.
type Res ¶
Res is one model response.
func CompleteJSON ¶
CompleteJSON calls the gateway and decodes a JSON response into T, retrying once on a parse failure. Returns the Res alongside the decoded value so a caller can read the model that answered (for provenance).
Directories
¶
| Path | Synopsis |
|---|---|
|
Package api implements the LLM gateway against an HTTP endpoint that speaks either Anthropic's Messages API or OpenAI's Chat Completions API.
|
Package api implements the LLM gateway against an HTTP endpoint that speaks either Anthropic's Messages API or OpenAI's Chat Completions API. |
|
Package fake provides an in-memory Gateway for tests.
|
Package fake provides an in-memory Gateway for tests. |