Documentation
¶
Overview ¶
Package llm is the public LLM boundary of the SDD framework (20260830-234501-d-cpt-q6n): pure vocabulary plus one one-method interface, and the composition pieces every host needs around it — Bounded, RateLimited, Observed with its CallStat and StatsSink, ByPurpose (20260902-114838-d-tac-cov). No provider adapters, no configuration, no I/O: those stay at each host's composition site.
The guiding analogy is net/http: Runner is our RoundTripper. Attribution is response-carried (Result.Identity) rather than a method on the interface, because implementations may route.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type CallStat ¶
type CallStat struct {
// Purpose names what the call was for: a chat Purpose or an embed.Purpose,
// which is why the field is a plain string.
Purpose string
Identity Identity
Usage Usage
// Items is the number of inputs in the call. Embedding batches set it so
// throughput per item is derivable from Duration; chat calls leave it 0.
Items int
Duration time.Duration
// Error is the failure text when the call returned no result, empty on
// success. Failures are recorded because a call that times out or errors
// is exactly what a sink exists to make countable. Such a row carries no
// tokens, and an identity only when the failure was attributed.
Error string
}
CallStat is one call's record, handed to a StatsSink by the observing decorators (Observed here, and its embedding twin in pkg/llm/embed). The sink owns the durable shape and adds the timestamp; this is the in-process form.
type Error ¶
Error optionally attributes a failed call. An implementation that knows what it routed to wraps its error so failures measure like successes.
type Identity ¶
type Identity struct {
Provider string
Model string
// Variant is the behaviour-affecting configuration the model ran under —
// a reasoning effort, a thinking budget — carried so calls at different
// settings measure apart. Canonical form is comma-separated key=value in
// sorted key order ("reasoning_effort=high"); empty at model defaults.
//
// The boundary is what the request carries, not what the setting means:
// a value sent as its own field is a variant, a value inside the model
// identifier is the model.
Variant string
}
Identity names what served a call, reported per call on Result. It is never a static property of an implementation, because implementations may route.
type Purpose ¶
type Purpose string
Purpose names what a call is for: a routing key for implementations and an observability dimension, the same value for both so they cannot drift. The set is closed: purposes are minted only by application operations. Hosts route on these constants and never invent values.
type Request ¶
type Request struct {
Purpose Purpose
// SystemPrompt is the stable prefix, cacheable by providers that can.
SystemPrompt string
// UserPrompt is the per-call variable part.
UserPrompt string
}
Request carries everything an implementation may route on: ctx (who) and Purpose (what for), plus the two-part prompt.
type Result ¶
type Result struct {
Text string
// Identity is required on success: what actually served this call.
Identity Identity
Usage Usage
}
Result reports what a call produced and what served it.
type Runner ¶
Runner is the single port. Contract, stated RoundTripper-style: fill Result.Identity on success; report Usage, never invent it; no internal retries (retry is caller policy); bound your own calls, because deadlines are configuration and configuration is host-private; ctx carries request-scoped facts a routing implementation may use (tenant, logger).
func Bounded ¶
Bounded decorates a Runner with a per-call deadline. Bounding a call is the instance's duty under the Runner contract, so a host composes this at its site with its own configured timeout.
func ByPurpose ¶
ByPurpose dispatches each call to the runner registered for its Purpose, and to fallback for purposes not in routes. A call whose purpose has no route and no fallback fails; the framework mints purposes, so an unrouted one is a composition gap the host should see.
func Observed ¶
Observed decorates a Runner so every call, success or failure, hands one CallStat to sink. Everything recorded travels in the port data: Purpose in the Request, Identity and Usage in the Result, attribution of a failure in the typed Error. A runner that does not attribute its failure leaves the identity blank, visibly absent rather than invented. A nil sink returns r unwrapped: recording is a composition convention, and skipping it shows as an empty record, never as a broken call.
func RateLimited ¶
RateLimited decorates a Runner with a token-bucket limiter of rps requests per second, so parallel batch operations stay under provider limits. The burst is one second's worth of requests, at least one.
type RunnerFunc ¶
RunnerFunc adapts a function to Runner (test doubles, error stubs).
Directories
¶
| Path | Synopsis |
|---|---|
|
Package embed is the public embedding boundary, the twin of pkg/llm for vectors (20260902-114838-d-tac-cov): one two-method interface over pure request and result types, sharing Identity, Usage, and the call record with the chat side, plus the same three decorators.
|
Package embed is the public embedding boundary, the twin of pkg/llm for vectors (20260902-114838-d-tac-cov): one two-method interface over pure request and result types, sharing Identity, Usage, and the call record with the chat side, plus the same three decorators. |