provider

package
v0.0.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 9, 2026 License: Apache-2.0 Imports: 23 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	ErrInvalidProviderConfig = errors.New("provider: invalid config")
	ErrDirectProviderURL     = errors.New("provider: direct provider url is not allowed in sandbox")
	ErrProviderSecret        = errors.New("provider: provider api key is not allowed in sandbox")
	ErrProviderHTTP          = errors.New("provider: sidecar http request failed")
)

Functions

func WithAttribution

func WithAttribution(ctx context.Context, a Attribution) context.Context

WithAttribution returns a context carrying a for stamping onto the next LLM request built by a OpenAICompatClient. A zero Attribution is a no-op passthrough (keeps callers from having to guard the empty case).

Types

type Attribution

type Attribution struct {
	Workspace string
	ThreadID  string
	TaskID    string
}

Attribution carries the per-turn identifiers stamped as X-Scitrera-* headers on outbound LLM requests. The platform LLM gateway reads them for tracing/attribution:

Workspace → X-Scitrera-Workspace → scitrera.workspace
ThreadID  → X-Scitrera-Thread-Id → mlflow.trace.session
TaskID    → X-Scitrera-Task-Id   → scitrera.task_id

Only the values that change per turn belong here. Sandbox-static attribution (tenant/source/user) is injected by the sidecar from its proxy_config projection, not by the harness — so it stays authoritative and unspoofable. Empty fields are omitted, so a non-sandbox / non-gateway run stamps nothing.

type ChatRequest

type ChatRequest struct {
	Model           string                 `json:"model,omitempty"`
	Messages        []protocol.ChatMessage `json:"messages"`
	Tools           []ToolSpec             `json:"tools,omitempty"`
	Stream          bool                   `json:"stream,omitempty"`
	Temperature     float64                `json:"temperature,omitempty"`
	MaxTokens       int                    `json:"max_tokens,omitempty"`
	ReasoningEffort string                 `json:"reasoning_effort,omitempty"`
	// StreamOptions carries OpenAI streaming options; ChatStream sets
	// include_usage so the final chunk reports token usage. Omitted on non-stream
	// requests (the usage block is always present in a non-stream response).
	StreamOptions *StreamOptions `json:"stream_options,omitempty"`
}

type ChatResponse

type ChatResponse struct {
	Message protocol.ChatMessage `json:"message"`
	// Model is the provider-echoed served model ("" if the provider omits it; the
	// caller falls back to the requested model).
	Model string `json:"model,omitempty"`
	// Usage is the provider-reported token accounting (zero when unavailable).
	Usage Usage `json:"usage,omitempty"`
}

type DeltaFunc

type DeltaFunc func(kind DeltaKind, text string) error

DeltaFunc receives streamed tokens as they arrive, tagged by channel.

type DeltaKind added in v0.0.2

type DeltaKind int

DeltaKind identifies which streamed channel a token belongs to. Both channels share ONE ordered callback so the consumer can place them in true content order (reasoning normally precedes the answer text of the same call).

const (
	// DeltaText is the model's answer text (OpenAI `delta.content`).
	DeltaText DeltaKind = iota
	// DeltaReasoning is the model's thinking trace (`delta.reasoning_content` /
	// `delta.reasoning`), emitted by reasoning models ahead of the answer.
	DeltaReasoning
)

type FailureKind

type FailureKind string

FailureKind classifies a provider failure so the turn loop can decide whether to retry, compact-and-retry, or surface the error immediately.

const (
	FailureUnknown         FailureKind = "unknown"
	FailureAuth            FailureKind = "auth"             // 401/403 — surface, do not retry
	FailureRateLimit       FailureKind = "rate_limit"       // 429 — retry with backoff
	FailureOverloaded      FailureKind = "overloaded"       // 503/529 — retry with backoff
	FailureServer          FailureKind = "server"           // 5xx — retry with backoff
	FailureTimeout         FailureKind = "timeout"          // deadline/408 — retry with backoff
	FailureNetwork         FailureKind = "network"          // dial/reset — retry with backoff
	FailureContextOverflow FailureKind = "context_overflow" // prompt too long — compact then retry
	FailureBadRequest      FailureKind = "bad_request"      // malformed request — surface, do not retry
)

type Guard

type Guard func(base *url.URL, authHeader string) error

Guard validates provider configuration before a client is built; a non-nil error rejects construction. Core leaves it nil (permissive). The Scitrera distribution wires SandboxGuard to enforce the no-real-secret / no-direct-host sandbox policy.

func SandboxGuard

func SandboxGuard(allowDirect bool) Guard

SandboxGuard is the Scitrera-sandbox provider policy (opt-in via OpenAICompatConfig.Guard): it rejects a real provider key or a direct provider host unless allowDirect is set (sidecar-less testing). The core does not apply it by default — the sandbox distribution wires it.

type OpenAICompatClient

type OpenAICompatClient struct {
	// contains filtered or unexported fields
}

OpenAICompatClient talks to an OpenAI-compatible chat endpoint over HTTP. In the Scitrera distribution that endpoint is the in-sandbox sidecar (which the comments below call "the sidecar"); FormatOpenAI also lets it target any OpenAI-compatible provider directly for sidecar-less testing.

func NewOpenAICompatClient

func NewOpenAICompatClient(cfg OpenAICompatConfig) (*OpenAICompatClient, error)

func (*OpenAICompatClient) BaseURL

func (c *OpenAICompatClient) BaseURL() string

func (*OpenAICompatClient) Chat

func (*OpenAICompatClient) ChatStream

func (c *OpenAICompatClient) ChatStream(ctx context.Context, chat ChatRequest, onDelta DeltaFunc) (ChatResponse, error)

ChatStream issues a streaming chat request. Text and reasoning tokens are delivered to onDelta as they arrive; the accumulated final message (reasoning + text + tool calls) is returned. If the endpoint does not respond with SSE, it falls back to decoding a normal JSON response (emitting each text / reasoning part as one delta).

type OpenAICompatConfig

type OpenAICompatConfig struct {
	BaseURL    string
	AuthHeader string
	// Auth supplies dynamic credentials (for example a renewable OAuth profile).
	// It is mutually exclusive with AuthHeader. The host owns credential storage.
	Auth       llmclient.Authenticator
	ChatPath   string
	Format     WireFormat
	HTTPClient *http.Client
	// PrepareRequest applies an optional provider-specific request profile after
	// neutral protocol conversion and before encoding.
	PrepareRequest func(*llmprotocol.Request)
	// StreamFirstChunk bounds the wait for the FIRST streamed chunk (time-to-first-
	// token — long for big-context reasoning). StreamIdle bounds the gap between
	// SUBSEQUENT chunks (once tokens flow, gaps are short). Neither is a total-duration
	// cap. 0 → defaults (see NewOpenAICompatClient).
	StreamFirstChunk time.Duration
	StreamIdle       time.Duration
	// Guard, when set, validates BaseURL + AuthHeader (e.g. SandboxGuard).
	Guard Guard
	// PromptCaching, when true, marks the end of the stable system-prompt prefix
	// with a provider prompt-cache breakpoint on the native wire path (the sidecar
	// lowers it to an Anthropic cache_control:{type:"ephemeral"} block). Opt-in and
	// default false so it cannot regress existing behavior; a no-op on the openai
	// wire format (which drops the hint).
	PromptCaching bool
	// StreamUsage, when true, requests a trailing usage chunk on streamed calls
	// (stream_options.include_usage) and reads it instead of short-circuiting on
	// finish_reason. Opt-in and default false: it must only be enabled against
	// providers that send a terminal `[DONE]`/usage chunk, else the stream waits on
	// the idle-liveness timer. Non-stream calls always report usage regardless.
	StreamUsage bool
}

type PromptCachePlan added in v0.0.2

type PromptCachePlan struct {
	StablePrefixBytes   int
	StablePromptDigest  string
	DynamicSuffixDigest string
	ToolSchemaDigest    string
}

PromptCachePlan describes the request-local portions of a provider request that are expected to remain stable. It contains digests only; prompt content and tool schemas are never copied into telemetry.

func BuildPromptCachePlan added in v0.0.2

func BuildPromptCachePlan(req ChatRequest) PromptCachePlan

BuildPromptCachePlan derives stable prompt and ordered tool-schema digests from the exact request that will be sent to the provider. The plan is request-local and never mutates transcript messages.

type ProviderError

type ProviderError struct {
	Kind   FailureKind
	Status int // HTTP status, 0 for transport-level failures
	Body   string
	// contains filtered or unexported fields
}

ProviderError wraps a classified provider failure. It preserves the ErrProviderHTTP sentinel via Unwrap (so existing errors.Is checks keep working) while exposing Kind/Status for recovery decisions.

func (*ProviderError) Error

func (e *ProviderError) Error() string

func (*ProviderError) Retryable

func (e *ProviderError) Retryable() bool

Retryable reports whether a fresh, identical request might succeed (transient failures). Context overflow is NOT retryable as-is — the caller must shrink the request first — so it is handled separately.

func (*ProviderError) Unwrap

func (e *ProviderError) Unwrap() error

type StreamOptions

type StreamOptions struct {
	IncludeUsage bool `json:"include_usage,omitempty"`
}

StreamOptions mirrors OpenAI's stream_options. include_usage asks the provider to emit a final usage-only chunk on the streamed response.

type ToolSpec

type ToolSpec struct {
	Name        string
	Description string
	Parameters  json.RawMessage // JSON-schema object; empty => {"type":"object"}
}

ToolSpec is a tool exposed to the model. It marshals to the OpenAI function-tool shape, which both the sidecar and OpenAI-compatible providers understand.

func (ToolSpec) MarshalJSON

func (t ToolSpec) MarshalJSON() ([]byte, error)

type Usage

type Usage struct {
	PromptTokens             int `json:"prompt_tokens,omitempty"`
	CompletionTokens         int `json:"completion_tokens,omitempty"`
	TotalTokens              int `json:"total_tokens,omitempty"`
	CachedInputTokens        int `json:"cached_input_tokens,omitempty"`
	CacheCreationInputTokens int `json:"cache_creation_input_tokens,omitempty"`
}

Usage is the provider-reported token accounting for one chat completion. Fields are best-effort: a provider that omits usage yields a zero Usage.

func (*Usage) UnmarshalJSON added in v0.0.2

func (u *Usage) UnmarshalJSON(data []byte) error

UnmarshalJSON accepts both normalized usage fields and the nested OpenAI prompt/input-token detail shapes. Providers differ on whether cache creation is called creation or write; normalize both names.

type WireFormat

type WireFormat string

WireFormat selects how chat requests are encoded.

const (
	// FormatNative sends native Scitrera ChatMessages (the sidecar contract).
	FormatNative WireFormat = "native"
	// FormatOpenAI sends OpenAI-style {role, content} messages, for talking to
	// an OpenAI-compatible provider directly (sidecar-less testing).
	FormatOpenAI WireFormat = "openai"
	// FormatResponses uses the OpenAI Responses API through the shared Scitrera
	// protocol/client modules.
	FormatResponses WireFormat = "responses"
)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL