Documentation
¶
Overview ¶
Package llm wraps a chat model for the two places sieve needs one: captioning canvas regions that yielded nothing to the scene-graph parse, and running the benchmark's question sets.
It is deliberately thin. The value it adds over calling a provider's SDK directly is in the parts that are easy to get wrong and that both callers need: refusal handling, a per-job spend ceiling, and usage accounting exact enough to put in a benchmark table.
Two providers, one shape ¶
Anthropic is the default. Setting a base URL switches to the OpenAI chat completions shape, which is what almost every other provider speaks -- Groq, OpenRouter, Together, vLLM, Ollama and the rest -- so a user can point sieve at whichever model they already pay for, or at a local one.
That matters beyond convenience. The benchmark sends a page's full HTML with every question, which on the default model costs real money per site; being able to run it against a free or self-hosted model is the difference between a benchmark anyone can reproduce and one only its author ever runs.
Index ¶
Constants ¶
const DefaultModel = "claude-opus-5"
DefaultModel is the model used unless the caller names another.
Variables ¶
var ErrBudgetExhausted = errors.New("model budget for this job is exhausted")
ErrBudgetExhausted is returned when a job's token ceiling is reached. Vision is the most expensive step in the pipeline by an order of magnitude, so the ceiling is a hard stop rather than a warning.
var ErrNoCredentials = errors.New(
"no model credentials found\n" +
" Anthropic: set ANTHROPIC_API_KEY, or run `ant auth login`.\n" +
" Any other provider: set LLM_BASE_URL, LLM_API_KEY and LLM_MODEL.\n" +
" Most providers expose an OpenAI-compatible endpoint, e.g.\n" +
" LLM_BASE_URL=https://api.groq.com/openai/v1\n" +
" Only the vision and benchmark paths need this; distillation does not.")
ErrNoCredentials is returned when no API key is configured. It explains what to do rather than surfacing a 401 from three layers down.
Functions ¶
func HasCredentials ¶
HasCredentials reports whether anything is configured to authenticate with. It is a pre-flight check so a long distillation does not run to completion and then fail on its last step.
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client is a budgeted wrapper around one provider's chat endpoint.
func New ¶
New builds a client. The API key is resolved from the options, then from ANTHROPIC_API_KEY; when neither is set the SDK's own credential chain is left to find a stored profile, and only a call will reveal whether it succeeded.
type ContentBlock ¶
type ContentBlock struct {
Text string
// MediaType and B64 are set instead of Text for an image.
MediaType string
B64 string
}
ContentBlock is a piece of a user turn: either text or an image.
It is deliberately this package's own type rather than a provider's. Callers build a request once and it goes to whichever provider is configured; making it an alias for one SDK's union would have put that SDK in the signature of every caller and made a second provider impossible to add without changing all of them.
func ImageBlock ¶
func ImageBlock(mediaType string, b64 string) ContentBlock
ImageBlock builds an image content block from base64-encoded bytes.
func TextBlock ¶
func TextBlock(s string) ContentBlock
TextBlock is a convenience for building a user turn.
func (ContentBlock) IsImage ¶
func (b ContentBlock) IsImage() bool
IsImage reports whether this block carries an image.
type Options ¶
type Options struct {
APIKey string
Model string
// BaseURL selects an OpenAI-compatible provider. Empty means Anthropic.
//
// Resolved from LLM_BASE_URL when unset, so a user can switch provider
// without every caller growing a flag.
BaseURL string
// MaxTokens bounds a single response.
MaxTokens int64
// Effort trades thoroughness against cost: low, medium, high, xhigh, max.
// Empty leaves the API default.
Effort string
// Budget caps total tokens across the client's lifetime. Zero means no cap.
Budget int64
// Timeout bounds one request.
Timeout time.Duration
}
Options configures a client.
type Result ¶
type Result struct {
Text string
Usage Usage
// Refused is true when the model's safety classifiers declined the request.
// This is a successful HTTP response, not an error, and it has to be
// checked before reading Text.
Refused bool
// RefusalCategory is the policy category, when the API supplies one.
RefusalCategory string
// Latency is wall clock for the call.
Latency time.Duration
// StopReason is the raw stop reason, for the benchmark report.
StopReason string
}
Result is one completed call.
type Usage ¶
type Usage struct {
InputTokens int64 `json:"input_tokens"`
OutputTokens int64 `json:"output_tokens"`
CacheReadTokens int64 `json:"cache_read_input_tokens,omitempty"`
CacheWriteTokens int64 `json:"cache_creation_input_tokens,omitempty"`
Calls int `json:"calls"`
}
Usage is what a call actually cost. These numbers come from the API's own accounting, not from an estimate, which is why the benchmark can report them as measurements.
Source Files