llm

package
v0.18.0-alpha.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 12, 2026 License: MIT Imports: 5 Imported by: 0

Documentation

Overview

Package llm is the public LLM boundary of the SDD framework (20260830-234501-d-cpt-q6n): pure vocabulary plus one one-method interface, and the composition pieces every host needs around it — Bounded, RateLimited, Observed with its CallStat and StatsSink, ByPurpose (20260902-114838-d-tac-cov). No provider adapters, no configuration, no I/O: those stay at each host's composition site.

The guiding analogy is net/http: Runner is our RoundTripper. Attribution is response-carried (Result.Identity) rather than a method on the interface, because implementations may route.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type CallStat

type CallStat struct {
	// Purpose names what the call was for: a chat Purpose or an embed.Purpose,
	// which is why the field is a plain string.
	Purpose  string
	Identity Identity
	Usage    Usage
	// Items is the number of inputs in the call. Embedding batches set it so
	// throughput per item is derivable from Duration; chat calls leave it 0.
	Items    int
	Duration time.Duration
	// Error is the failure text when the call returned no result, empty on
	// success. Failures are recorded because a call that times out or errors
	// is exactly what a sink exists to make countable. Such a row carries no
	// tokens, and an identity only when the failure was attributed.
	Error string
}

CallStat is one call's record, handed to a StatsSink by the observing decorators (Observed here, and its embedding twin in pkg/llm/embed). The sink owns the durable shape and adds the timestamp; this is the in-process form.

type Error

type Error struct {
	Identity Identity
	Err      error
}

Error optionally attributes a failed call. An implementation that knows what it routed to wraps its error so failures measure like successes.

func (*Error) Error

func (e *Error) Error() string

func (*Error) Unwrap

func (e *Error) Unwrap() error

type Identity

type Identity struct {
	Provider string
	Model    string
	// Variant is the behaviour-affecting configuration the model ran under —
	// a reasoning effort, a thinking budget — carried so calls at different
	// settings measure apart. Canonical form is comma-separated key=value in
	// sorted key order ("reasoning_effort=high"); empty at model defaults.
	//
	// The boundary is what the request carries, not what the setting means:
	// a value sent as its own field is a variant, a value inside the model
	// identifier is the model.
	Variant string
}

Identity names what served a call, reported per call on Result. It is never a static property of an implementation, because implementations may route.

func (Identity) String

func (i Identity) String() string

String renders the identity for display: model, then variant in parentheses.

type Purpose

type Purpose string

Purpose names what a call is for: a routing key for implementations and an observability dimension, the same value for both so they cannot drift. The set is closed: purposes are minted only by application operations. Hosts route on these constants and never invent values.

const (
	PurposePreflight    Purpose = "preflight"
	PurposeSummarize    Purpose = "summarize"
	PurposeWritingGuide Purpose = "writing-guide"
)

type Request

type Request struct {
	Purpose Purpose
	// SystemPrompt is the stable prefix, cacheable by providers that can.
	SystemPrompt string
	// UserPrompt is the per-call variable part.
	UserPrompt string
}

Request carries everything an implementation may route on: ctx (who) and Purpose (what for), plus the two-part prompt.

func (Request) Combined

func (r Request) Combined() string

Combined returns SystemPrompt followed by UserPrompt separated by a blank line when both are non-empty. Runners without native system-prompt support use this to flatten the Request into a single payload.

type Result

type Result struct {
	Text string
	// Identity is required on success: what actually served this call.
	Identity Identity
	Usage    Usage
}

Result reports what a call produced and what served it.

type Runner

type Runner interface {
	Run(ctx context.Context, req Request) (Result, error)
}

Runner is the single port. Contract, stated RoundTripper-style: fill Result.Identity on success; report Usage, never invent it; no internal retries (retry is caller policy); bound your own calls, because deadlines are configuration and configuration is host-private; ctx carries request-scoped facts a routing implementation may use (tenant, logger).

func Bounded

func Bounded(r Runner, timeout time.Duration) Runner

Bounded decorates a Runner with a per-call deadline. Bounding a call is the instance's duty under the Runner contract, so a host composes this at its site with its own configured timeout.

func ByPurpose

func ByPurpose(routes map[Purpose]Runner, fallback Runner) Runner

ByPurpose dispatches each call to the runner registered for its Purpose, and to fallback for purposes not in routes. A call whose purpose has no route and no fallback fails; the framework mints purposes, so an unrouted one is a composition gap the host should see.

func Observed

func Observed(r Runner, sink StatsSink) Runner

Observed decorates a Runner so every call, success or failure, hands one CallStat to sink. Everything recorded travels in the port data: Purpose in the Request, Identity and Usage in the Result, attribution of a failure in the typed Error. A runner that does not attribute its failure leaves the identity blank, visibly absent rather than invented. A nil sink returns r unwrapped: recording is a composition convention, and skipping it shows as an empty record, never as a broken call.

func RateLimited

func RateLimited(r Runner, rps float64) Runner

RateLimited decorates a Runner with a token-bucket limiter of rps requests per second, so parallel batch operations stay under provider limits. The burst is one second's worth of requests, at least one.

type RunnerFunc

type RunnerFunc func(context.Context, Request) (Result, error)

RunnerFunc adapts a function to Runner (test doubles, error stubs).

func (RunnerFunc) Run

func (f RunnerFunc) Run(ctx context.Context, req Request) (Result, error)

type StatsSink

type StatsSink interface {
	RecordCall(ctx context.Context, stat CallStat)
}

StatsSink durably records per-call metrics. ctx is the call's context, so a sink can read request-scoped facts (a tenant, a logger) the way a routing Runner does. Implementations must be safe for concurrent use.

type Usage

type Usage struct {
	InputTokens       int
	OutputTokens      int
	CacheReadTokens   int
	CacheCreateTokens int
	CostUSD           float64
}

Usage is the common format for provider-reported consumption. A field the provider does not report stays zero: reported, never reconstructed.

Directories

Path Synopsis
Package embed is the public embedding boundary, the twin of pkg/llm for vectors (20260902-114838-d-tac-cov): one two-method interface over pure request and result types, sharing Identity, Usage, and the call record with the chat side, plus the same three decorators.
Package embed is the public embedding boundary, the twin of pkg/llm for vectors (20260902-114838-d-tac-cov): one two-method interface over pure request and result types, sharing Identity, Usage, and the call record with the chat side, plus the same three decorators.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL