Documentation
¶
Overview ¶
Package llmclient builds the model.LLM an agent step talks to and the HTTP transports under it: retries, tool-call argument repair, OpenRouter's caching levers, and the choice of which fallback source to try next.
Index ¶
- Constants
- func CaptureContent() bool
- func HostMatches(baseURL, want string) bool
- func LookupAPIKey(envVar string, required bool) (string, error)
- func New(ri config.ResolvedInvocation, apiKey string) model.LLM
- func ProviderName(baseURL string) string
- func WithInvocation(ctx context.Context, inv *Invocation) context.Context
- func WithNewRun(ctx context.Context, jobName string) context.Context
- func WithRequestCounter(ctx context.Context, counter *RequestCounter) context.Context
- type APIError
- type CascadeAction
- type GenParams
- type Invocation
- type PinAction
- type ProbeFact
- type RequestCounter
- type SourceOutcome
- type Verdict
Constants ¶
const TelemetryScope = "github.com/jtarchie/steps/internal/agent"
TelemetryScope names the tracer and meter. It is still the agent package's path, as it was before this layer moved out of it, so a dashboard keyed on the scope keeps matching.
Variables ¶
This section is empty.
Functions ¶
func CaptureContent ¶ added in v0.2.0
func CaptureContent() bool
CaptureContent reports whether prompts, responses and tool arguments and results go on spans. Off unless asked for, as the conventions require: they can carry secrets, and nothing redacts them.
func HostMatches ¶
HostMatches reports whether baseURL's host is want or a subdomain of it.
func LookupAPIKey ¶
LookupAPIKey reads the API key from the OS environment variable named by envVar. When required, a missing/empty variable (or a missing envVar name) is a hard error — sending a blank Authorization header would just produce a confusing 401 from the endpoint. When not required (local providers with no default key), a missing key resolves to "" and no Authorization header is sent.
func New ¶
func New(ri config.ResolvedInvocation, apiKey string) model.LLM
New constructs the model.LLM used for real runs, given the already-resolved base URL/model/key (see config.ResolveAgentInvocation). Returning the model.LLM interface (not the concrete *chatModel) keeps runAgentConversation testable against an in-process fake.
Every invocation gets the package's shared HTTP client (see agentHTTPClient): the tool-call argument repair transport applies to all providers, and an OpenRouter or other caching-gateway base URL additionally gets the session/cache transport (see openrouter.go, gateway.go).
It takes the whole ResolvedInvocation rather than loose strings so the mapping from invocation fields to client settings lives in exactly one place — the three call sites (PreparedStep.Run, RunFix, buildSubAgentTool) can't individually mix up which field feeds which parameter.
func ProviderName ¶ added in v0.2.0
ProviderName is gen_ai.provider.name for an endpoint, "" when the host is not one steps recognizes.
func WithInvocation ¶ added in v0.2.0
func WithInvocation(ctx context.Context, inv *Invocation) context.Context
WithInvocation scopes ctx to inv, so every chat and execute_tool span under it says which agent emitted it.
func WithNewRun ¶
WithNewRun returns a derived context identifying a fresh run of jobName, which openRouterTransport combines with the calling agent's name to form the x-session-id header on OpenRouter chat completions (see composeSessionID).
internal/pipeline calls this once per job run. The run token is random, so two runs of the same job — including two concurrent ones under `steps web --max-concurrent` — never share a session and so never share a provider pin. It rides on the context rather than the client precisely so those concurrent runs stay separated while sharing everything else.
The session is transport-level only: it never enters a step's merkle content (see internal/merkle), so it cannot invalidate a cached step.
func WithRequestCounter ¶
func WithRequestCounter(ctx context.Context, counter *RequestCounter) context.Context
WithRequestCounter scopes a counter to one agent conversation, so the count a failure reports describes that conversation alone.
Types ¶
type APIError ¶ added in v0.2.0
APIError is a non-2xx answer from the provider: its status, and the fields of the body's "error" object when it had one. Error() is the status alone, as openai-go's was: a provider's message can echo request content.
type CascadeAction ¶
type CascadeAction int
CascadeAction is what the loop does with the attempt it just finished.
const ( // ReturnResult means hand this attempt's result back as the step's outcome. ReturnResult CascadeAction = iota )
type GenParams ¶
type GenParams struct {
Temperature *float64
TopP *float64
MaxTokens int
Reasoning string // "", "low", "medium", or "high"
}
GenParams holds the generation dials an agent configures. Unset fields (nil pointers, zero maxTokens, empty reasoning) are left off the request so the model's own defaults apply.
func (GenParams) ApplyTo ¶
func (p GenParams) ApplyTo(cfg *genai.GenerateContentConfig)
ApplyTo sets the configured dials on a genai generation config.
type Invocation ¶ added in v0.2.0
type Invocation struct {
Agent string
Conversation string
// contains filtered or unexported fields
}
Invocation is one agent invocation as the OpenTelemetry GenAI conventions see it: the agent every span under it belongs to, the conversation those spans group by, and the model and tool calls it made itself — a sub-agent is an Invocation of its own, so each call is counted once across the tree.
func InvocationFrom ¶ added in v0.2.0
func InvocationFrom(ctx context.Context) *Invocation
InvocationFrom is the invocation ctx runs under, nil outside one (preflight probes, tests).
func (*Invocation) Attributes ¶ added in v0.2.0
func (inv *Invocation) Attributes() []attribute.KeyValue
Attributes is the identity every GenAI span under inv carries: Honeycomb's agent timeline shows an "Unknown" agent for a span without gen_ai.agent.name.
func (*Invocation) CountTool ¶ added in v0.2.0
func (inv *Invocation) CountTool()
CountTool records one tool call the invocation made, rejected ones included, as gen_ai.invoke_agent.tool_calls counts them.
func (*Invocation) Counts ¶ added in v0.2.0
func (inv *Invocation) Counts() (inference, tools int64)
Counts is how many model and tool calls the invocation made itself.
type PinAction ¶
type PinAction int
PinAction is what the attempt does to the process-wide record of which source this agent runs on (see selectedSources). It is separate from CascadeAction because the two are genuinely independent: a step can end on a source worth preferring or one worth forgetting, and the same action can accompany either.
const ( // LeavePin means no evidence worth acting on. The default, and what every // ambiguous case resolves to — a pin is a durable, process-lifetime // preference, and changing it on a guess is how a run strands itself on // a source the next probe may not get to for a whole poll interval. LeavePin PinAction = iota // PinThisSource means this source served, so prefer it from here on. PinThisSource // DropPin means stop preferring the pinned source. Two decisions produce it, // for unrelated reasons: DecideCascade, when the cascade tried the // alternatives and none served, so the next step should start over from // the primary rather than keep preferring whichever was tried last; and // DecidePreflightPin, when the pre-run probe says the reason for the pin // is gone or the pinned source is. DropPin )
func DecidePreflightPin ¶
DecidePreflightPin answers the question the mid-run cascade deliberately does not: is the reason for this pin still true? Until this existed there was no answer at all — an outage lasting ninety seconds could move a `steps web` agent onto a possibly-worse model for days, and silently, since agent.failover fires once at the swap and nothing afterwards says the agent is still there.
Either fact is enough on its own to end a pin: the reason for it is gone, or the thing it points at is. Both, because releasing on a recovered primary alone would trade one blind spot for a worse one — see reconsiderPin, which gathers them.
Each fact must be FRESH to release, and this is the asymmetry the function exists to hold: keeping a pin on a stale reading changes nothing, while releasing on one destroys evidence a real conversation paid for. A cached positive from before an outage would otherwise drop a pin the cascade earned during it, and a cached negative from before a recovery would drop one a source has since served a whole conversation on — the second reachable whenever another job cascades while this probe is in flight, which is what clearSourceIf's compare-and-delete already braces for.
Pure and total for the same reason DecideCascade is: every defect this pair of functions has closed was a wrong predicate mutating durable state.
type ProbeFact ¶
ProbeFact is what one pre-run probe established: how the source answered, and whether that answer postdates the pin it is being weighed against.
Fresh is a property of the ANSWER measured against the PIN, and neither half of that is incidental. Not of the caller: several agents may share one endpoint and model, the probe cache collapses them to a single request, and the ones that read it back hold the same fact as the one that asked. And not against a clock of its own: what makes an answer able to end a pin is that it was given after the pin existed, since an older one was given before the reason for the pin did. See reconsiderPin.
The two travel together because a decision that destroys state may only rest on a fresh fact.
type RequestCounter ¶
type RequestCounter struct {
// contains filtered or unexported fields
}
RequestCounter tallies the provider requests one agent conversation makes. It is what lets a failed step report the requests it really spent rather than the turns it took.
func (*RequestCounter) Total ¶
func (c *RequestCounter) Total() int
Total is the number of provider requests counted so far.
type SourceOutcome ¶
type SourceOutcome int
SourceOutcome is what one attempt says about the SOURCE that ran it, which is a different question from what it says about the step. A step can fail on a perfectly healthy source, and a source can be unreachable without the step being wrong about anything.
Keeping the two apart is the distinction the cascade previously lacked: it pinned on "the step reached a conclusion", which is true of a provider answering 400 Bad Request.
func ClassifySource ¶
func ClassifySource(jobCtx context.Context, runErr error) SourceOutcome
ClassifySource decides what runErr says about the source that produced it.
jobCtx is the JOB's context, never the conversation's own: the two produce the identical error for opposite reasons, and only the job's says whether the run itself is still healthy.
type Verdict ¶
type Verdict struct {
Action CascadeAction
Pin PinAction
}
Verdict is one attempt's decision, whole.
func DecideCascade ¶
func DecideCascade(result SourceOutcome, hasNext, deadlineSpent, swapped bool) Verdict
DecideCascade is the whole decision, pure and total.
- hasNext: another fallback entry resolved and its credential is present.
- deadlineSpent: the step's shared deadline has already passed, so no later source could finish anything.
- swapped: the cascade has already moved off the source it started on.
swapped is what gates DropPin, and it is the correction for the case that used to drop a healthy pin: releasing means "I tried the alternatives and none served". A step that never swapped has tried no alternatives, so it has learned nothing that justifies sending the next step back to a primary preflight may already have found dead.
Source Files