Documentation
¶
Index ¶
- Variables
- func WithAttribution(ctx context.Context, a Attribution) context.Context
- type Attribution
- type ChatRequest
- type ChatResponse
- type DeltaFunc
- type DeltaKind
- type FailureKind
- type Guard
- type OpenAICompatClient
- type OpenAICompatConfig
- type PromptCachePlan
- type ProviderError
- type StreamOptions
- type ToolSpec
- type Usage
- type WireFormat
Constants ¶
This section is empty.
Variables ¶
var ( ErrInvalidProviderConfig = errors.New("provider: invalid config") ErrDirectProviderURL = errors.New("provider: direct provider url is not allowed in sandbox") ErrProviderSecret = errors.New("provider: provider api key is not allowed in sandbox") ErrProviderHTTP = errors.New("provider: sidecar http request failed") )
Functions ¶
func WithAttribution ¶
func WithAttribution(ctx context.Context, a Attribution) context.Context
WithAttribution returns a context carrying a for stamping onto the next LLM request built by a OpenAICompatClient. A zero Attribution is a no-op passthrough (keeps callers from having to guard the empty case).
Types ¶
type Attribution ¶
Attribution carries the per-turn identifiers stamped as X-Scitrera-* headers on outbound LLM requests. The platform LLM gateway reads them for tracing/attribution:
Workspace → X-Scitrera-Workspace → scitrera.workspace ThreadID → X-Scitrera-Thread-Id → mlflow.trace.session TaskID → X-Scitrera-Task-Id → scitrera.task_id
Only the values that change per turn belong here. Sandbox-static attribution (tenant/source/user) is injected by the sidecar from its proxy_config projection, not by the harness — so it stays authoritative and unspoofable. Empty fields are omitted, so a non-sandbox / non-gateway run stamps nothing.
type ChatRequest ¶
type ChatRequest struct {
Model string `json:"model,omitempty"`
Messages []protocol.ChatMessage `json:"messages"`
Tools []ToolSpec `json:"tools,omitempty"`
Stream bool `json:"stream,omitempty"`
Temperature float64 `json:"temperature,omitempty"`
MaxTokens int `json:"max_tokens,omitempty"`
ReasoningEffort string `json:"reasoning_effort,omitempty"`
// StreamOptions carries OpenAI streaming options; ChatStream sets
// include_usage so the final chunk reports token usage. Omitted on non-stream
// requests (the usage block is always present in a non-stream response).
StreamOptions *StreamOptions `json:"stream_options,omitempty"`
}
type ChatResponse ¶
type ChatResponse struct {
Message protocol.ChatMessage `json:"message"`
// Model is the provider-echoed served model ("" if the provider omits it; the
// caller falls back to the requested model).
Model string `json:"model,omitempty"`
// Usage is the provider-reported token accounting (zero when unavailable).
Usage Usage `json:"usage,omitempty"`
}
type DeltaKind ¶ added in v0.0.2
type DeltaKind int
DeltaKind identifies which streamed channel a token belongs to. Both channels share ONE ordered callback so the consumer can place them in true content order (reasoning normally precedes the answer text of the same call).
type FailureKind ¶
type FailureKind string
FailureKind classifies a provider failure so the turn loop can decide whether to retry, compact-and-retry, or surface the error immediately.
const ( FailureUnknown FailureKind = "unknown" FailureAuth FailureKind = "auth" // 401/403 — surface, do not retry FailureRateLimit FailureKind = "rate_limit" // 429 — retry with backoff FailureOverloaded FailureKind = "overloaded" // 503/529 — retry with backoff FailureServer FailureKind = "server" // 5xx — retry with backoff FailureTimeout FailureKind = "timeout" // deadline/408 — retry with backoff FailureNetwork FailureKind = "network" // dial/reset — retry with backoff FailureContextOverflow FailureKind = "context_overflow" // prompt too long — compact then retry FailureBadRequest FailureKind = "bad_request" // malformed request — surface, do not retry )
type Guard ¶
Guard validates provider configuration before a client is built; a non-nil error rejects construction. Core leaves it nil (permissive). The Scitrera distribution wires SandboxGuard to enforce the no-real-secret / no-direct-host sandbox policy.
func SandboxGuard ¶
SandboxGuard is the Scitrera-sandbox provider policy (opt-in via OpenAICompatConfig.Guard): it rejects a real provider key or a direct provider host unless allowDirect is set (sidecar-less testing). The core does not apply it by default — the sandbox distribution wires it.
type OpenAICompatClient ¶
type OpenAICompatClient struct {
// contains filtered or unexported fields
}
OpenAICompatClient talks to an OpenAI-compatible chat endpoint over HTTP. In the Scitrera distribution that endpoint is the in-sandbox sidecar (which the comments below call "the sidecar"); FormatOpenAI also lets it target any OpenAI-compatible provider directly for sidecar-less testing.
func NewOpenAICompatClient ¶
func NewOpenAICompatClient(cfg OpenAICompatConfig) (*OpenAICompatClient, error)
func (*OpenAICompatClient) BaseURL ¶
func (c *OpenAICompatClient) BaseURL() string
func (*OpenAICompatClient) Chat ¶
func (c *OpenAICompatClient) Chat(ctx context.Context, chat ChatRequest) (ChatResponse, error)
func (*OpenAICompatClient) ChatStream ¶
func (c *OpenAICompatClient) ChatStream(ctx context.Context, chat ChatRequest, onDelta DeltaFunc) (ChatResponse, error)
ChatStream issues a streaming chat request. Text and reasoning tokens are delivered to onDelta as they arrive; the accumulated final message (reasoning + text + tool calls) is returned. If the endpoint does not respond with SSE, it falls back to decoding a normal JSON response (emitting each text / reasoning part as one delta).
type OpenAICompatConfig ¶
type OpenAICompatConfig struct {
BaseURL string
AuthHeader string
// Auth supplies dynamic credentials (for example a renewable OAuth profile).
// It is mutually exclusive with AuthHeader. The host owns credential storage.
Auth llmclient.Authenticator
ChatPath string
Format WireFormat
HTTPClient *http.Client
// PrepareRequest applies an optional provider-specific request profile after
// neutral protocol conversion and before encoding.
PrepareRequest func(*llmprotocol.Request)
// StreamFirstChunk bounds the wait for the FIRST streamed chunk (time-to-first-
// token — long for big-context reasoning). StreamIdle bounds the gap between
// SUBSEQUENT chunks (once tokens flow, gaps are short). Neither is a total-duration
// cap. 0 → defaults (see NewOpenAICompatClient).
StreamFirstChunk time.Duration
StreamIdle time.Duration
// Guard, when set, validates BaseURL + AuthHeader (e.g. SandboxGuard).
Guard Guard
// PromptCaching, when true, marks the end of the stable system-prompt prefix
// with a provider prompt-cache breakpoint on the native wire path (the sidecar
// lowers it to an Anthropic cache_control:{type:"ephemeral"} block). Opt-in and
// default false so it cannot regress existing behavior; a no-op on the openai
// wire format (which drops the hint).
PromptCaching bool
// StreamUsage, when true, requests a trailing usage chunk on streamed calls
// (stream_options.include_usage) and reads it instead of short-circuiting on
// finish_reason. Opt-in and default false: it must only be enabled against
// providers that send a terminal `[DONE]`/usage chunk, else the stream waits on
// the idle-liveness timer. Non-stream calls always report usage regardless.
StreamUsage bool
}
type PromptCachePlan ¶ added in v0.0.2
type PromptCachePlan struct {
StablePrefixBytes int
StablePromptDigest string
DynamicSuffixDigest string
ToolSchemaDigest string
}
PromptCachePlan describes the request-local portions of a provider request that are expected to remain stable. It contains digests only; prompt content and tool schemas are never copied into telemetry.
func BuildPromptCachePlan ¶ added in v0.0.2
func BuildPromptCachePlan(req ChatRequest) PromptCachePlan
BuildPromptCachePlan derives stable prompt and ordered tool-schema digests from the exact request that will be sent to the provider. The plan is request-local and never mutates transcript messages.
type ProviderError ¶
type ProviderError struct {
Kind FailureKind
Status int // HTTP status, 0 for transport-level failures
Body string
// contains filtered or unexported fields
}
ProviderError wraps a classified provider failure. It preserves the ErrProviderHTTP sentinel via Unwrap (so existing errors.Is checks keep working) while exposing Kind/Status for recovery decisions.
func (*ProviderError) Error ¶
func (e *ProviderError) Error() string
func (*ProviderError) Retryable ¶
func (e *ProviderError) Retryable() bool
Retryable reports whether a fresh, identical request might succeed (transient failures). Context overflow is NOT retryable as-is — the caller must shrink the request first — so it is handled separately.
func (*ProviderError) Unwrap ¶
func (e *ProviderError) Unwrap() error
type StreamOptions ¶
type StreamOptions struct {
IncludeUsage bool `json:"include_usage,omitempty"`
}
StreamOptions mirrors OpenAI's stream_options. include_usage asks the provider to emit a final usage-only chunk on the streamed response.
type ToolSpec ¶
type ToolSpec struct {
Name string
Description string
Parameters json.RawMessage // JSON-schema object; empty => {"type":"object"}
}
ToolSpec is a tool exposed to the model. It marshals to the OpenAI function-tool shape, which both the sidecar and OpenAI-compatible providers understand.
func (ToolSpec) MarshalJSON ¶
type Usage ¶
type Usage struct {
PromptTokens int `json:"prompt_tokens,omitempty"`
CompletionTokens int `json:"completion_tokens,omitempty"`
TotalTokens int `json:"total_tokens,omitempty"`
CachedInputTokens int `json:"cached_input_tokens,omitempty"`
CacheCreationInputTokens int `json:"cache_creation_input_tokens,omitempty"`
}
Usage is the provider-reported token accounting for one chat completion. Fields are best-effort: a provider that omits usage yields a zero Usage.
func (*Usage) UnmarshalJSON ¶ added in v0.0.2
UnmarshalJSON accepts both normalized usage fields and the nested OpenAI prompt/input-token detail shapes. Providers differ on whether cache creation is called creation or write; normalize both names.
type WireFormat ¶
type WireFormat string
WireFormat selects how chat requests are encoded.
const ( // FormatNative sends native Scitrera ChatMessages (the sidecar contract). FormatNative WireFormat = "native" // FormatOpenAI sends OpenAI-style {role, content} messages, for talking to // an OpenAI-compatible provider directly (sidecar-less testing). FormatOpenAI WireFormat = "openai" // FormatResponses uses the OpenAI Responses API through the shared Scitrera // protocol/client modules. FormatResponses WireFormat = "responses" )