Documentation
¶
Overview ¶
Package reasoninge2e provides a deterministic client-transcript model and backend-request oracle for reasoning-preservation full HTTP E2E phases.
Plans are precomputed from an explicit seed and retention policy. The package never uses package-global RNG or time.Now, returns defensive copies, and keeps oracle errors content-safe (no reasoning text, signatures, or opaque payloads).
GenerateTranscriptPlan builds immutable matrix TranscriptPlans (seed+mode+turns) with independent backend/client RNG streams and forced coverage categories.
ClientEmulator records actual proxy ChatResponse observations against Plan.Observed, then materializes the next Chat request from recorded visible/tool structure plus policy-specific submitted reasoning. Observed and submitted histories stay independent.
Responses stateful exact-reasoning harness helpers live alongside Chat (see responses.go); wire scripts/oracles remain in refbackend/refclient openairesponses packages.
AssistantTurn.Streaming is plan/client metadata; Check / BackendTurnObservation do not validate streaming wire shape (HTTP drivers assert Content-Type / SSE framing).
CheckPrefixRetention models FIFO store bounds for ModeDropped restoration; Check and CheckPrefix remain eviction-blind. CheckPrefixRetention requires maxArtifactTurns > 0.
Index ¶
- Constants
- Variables
- func AssistantTurnToChatMessage(turn AssistantTurn) map[string]any
- func Check(plan Plan, obs BackendRequestObservation) error
- func CheckPrefix(plan Plan, obs BackendRequestObservation) error
- func CheckPrefixRetention(plan Plan, obs BackendRequestObservation, maxArtifactTurns int) error
- func CheckResponsesHistoryIDs(got, want []string) error
- func ClientRetainSequence(seed uint64, n int) []bool
- func FormatFail(plan Plan, turnID string, mode RetentionMode, field, detail string) string
- func FormatMatrixFail(plan TranscriptPlan, turnIndex int, reasonCode string) string
- func FormatRetentionDiag(err error) string
- func FormatSoakFail(plan TranscriptPlan, turnIndex int, reasonCode string) string
- func MatrixReplayCommand(mode MatrixMode, seed uint64) string
- func RunSoakWorkerPoolWithStats(workers int, jobs []SoakPoolJob, run func(SoakPoolJob) SoakPoolResult) ([]SoakPoolResult, SoakPoolRunStats)
- func SoakReplayCommand(mode MatrixMode, seed uint64) string
- func ToolResultMessage(tool ToolExchange) map[string]any
- func UserMessage(text string) map[string]any
- type AssistantTurn
- type BackendRequestObservation
- type BackendTurnObservation
- type ChatCompletionRequest
- type ChatResponse
- type ChatSessionCarriers
- type ChatTurnResponse
- type ChatWireClient
- type ClientEmulator
- func (e *ClientEmulator) MaterializeChatRequest(nextUserPrompt string) ([]map[string]any, error)
- func (e *ClientEmulator) ObservedHistory() []AssistantTurn
- func (e *ClientEmulator) Plan() Plan
- func (e *ClientEmulator) Record(resp ChatResponse) error
- func (e *ClientEmulator) SubmittedHistory() []AssistantTurn
- type DecisionRecord
- type FailTrace
- type MatrixCase
- type MatrixMode
- type Plan
- type PlanConfig
- type PlannedTurn
- type ReasoningBlock
- type ResponsesMatrixCase
- type ResponsesPresenceVariant
- type RetentionMode
- type RetentionPolicy
- type ScriptedBackendTurn
- type SoakConfig
- type SoakPoolJob
- type SoakPoolResult
- type SoakPoolRunStats
- type SoakReplay
- type ToolExchange
- type TranscriptPlan
- func (p TranscriptPlan) Decisions() []DecisionRecord
- func (p TranscriptPlan) Mode() MatrixMode
- func (p TranscriptPlan) Plan() Plan
- func (p TranscriptPlan) ScriptedTurns() []ScriptedBackendTurn
- func (p TranscriptPlan) Seed() uint64
- func (p TranscriptPlan) StructuralTrace() string
- func (p TranscriptPlan) TurnCount() int
- type TurnSpec
Constants ¶
const ( DialectOpenAIChatTextV1 = "openai.chat.reasoning_text.v1" DialectAnthropicThinkingV1 = "anthropic.thinking.v1" DialectAnthropicRedactedThinkingV1 = "anthropic.redacted_thinking.v1" )
const ( EnvSoakGate = "LIP_REASONING_E2E_SOAK" EnvSoakSeeds = "LIP_REASONING_E2E_SEEDS" EnvSoakTurns = "LIP_REASONING_E2E_TURNS" EnvSoakWorkers = "LIP_REASONING_E2E_WORKERS" EnvSoakMode = "LIP_REASONING_E2E_MODE" EnvSoakSeed = "LIP_REASONING_E2E_SEED" )
Soak environment contract (opt-in; never a default/PR gate).
const ( // DefaultSoakSeeds is the default seed count (1000) for a full soak. DefaultSoakSeeds = 1000 // DefaultSoakTurns is the default HTTP turns per seed (100). DefaultSoakTurns = 100 // DefaultSoakWorkers is a conservative fixed-pool default. DefaultSoakWorkers = 4 )
Variables ¶
var ErrSoakDisabled = errors.New("reasoninge2e soak: disabled (set LIP_REASONING_E2E_SOAK=1)")
ErrSoakDisabled is returned by helpers that require an enabled soak gate.
Functions ¶
func AssistantTurnToChatMessage ¶
func AssistantTurnToChatMessage(turn AssistantTurn) map[string]any
AssistantTurnToChatMessage converts a planned/submitted assistant turn to wire JSON.
func Check ¶
func Check(plan Plan, obs BackendRequestObservation) error
Check compares a backend-bound request observation against the precomputed plan. Errors include seed, turn id, and mode, and describe structural mismatches only. Streaming on PlannedTurn / AssistantTurn is plan metadata only; this oracle does not validate stream vs non-stream wire shape on BackendTurnObservation.
func CheckPrefix ¶
func CheckPrefix(plan Plan, obs BackendRequestObservation) error
CheckPrefix validates a prefix of the plan against a backend observation. Used by multi-turn HTTP drivers where each request carries only history to date.
func CheckPrefixRetention ¶
func CheckPrefixRetention(plan Plan, obs BackendRequestObservation, maxArtifactTurns int) error
CheckPrefixRetention validates a plan prefix against a backend observation while modeling FIFO retention of artifact-producing turns (non-empty observed reasoning).
maxArtifactTurns is the store bound (newest N artifact turns retained) and must be > 0. Use CheckPrefix for unbounded/eviction-blind validation.
ModeDropped expects restoration only while retained; after eviction it expects absence. ModePreserved / ModeConflict / ModeNone keep CheckPrefix semantics. The immutable plan ExpectedBackend fields are never mutated.
func ClientRetainSequence ¶
ClientRetainSequence returns the independent client preserve bits for seed (true=preserve). Exported for independence proofs; callers must not treat this as payload-bearing.
func FormatFail ¶
func FormatFail(plan Plan, turnID string, mode RetentionMode, field, detail string) string
FormatFail wraps an oracle/driver error with content-safe seed/mode/turn context.
func FormatMatrixFail ¶
func FormatMatrixFail(plan TranscriptPlan, turnIndex int, reasonCode string) string
FormatMatrixFail builds a content-safe matrix failure line with replay command.
func FormatRetentionDiag ¶
FormatRetentionDiag extracts content-safe retention fields from an oracle error for matrix/soak fail lines. Returns "" when err is nil or carries no retention diagnostics.
func FormatSoakFail ¶
func FormatSoakFail(plan TranscriptPlan, turnIndex int, reasonCode string) string
FormatSoakFail builds a content-safe soak failure line with a single replay command.
func MatrixReplayCommand ¶
func MatrixReplayCommand(mode MatrixMode, seed uint64) string
MatrixReplayCommand returns a single content-safe seed-replay command.
func RunSoakWorkerPoolWithStats ¶
func RunSoakWorkerPoolWithStats(workers int, jobs []SoakPoolJob, run func(SoakPoolJob) SoakPoolResult) ([]SoakPoolResult, SoakPoolRunStats)
RunSoakWorkerPoolWithStats is RunSoakWorkerPool plus worker/job concurrency stats.
func SoakReplayCommand ¶
func SoakReplayCommand(mode MatrixMode, seed uint64) string
SoakReplayCommand returns one content-safe command to reproduce a soak seed.
func ToolResultMessage ¶
func ToolResultMessage(tool ToolExchange) map[string]any
ToolResultMessage builds a role=tool wire message.
func UserMessage ¶
UserMessage builds a role=user wire message.
Types ¶
type AssistantTurn ¶
type AssistantTurn struct {
ID string
VisibleText string
Reasoning []ReasoningBlock
Tool *ToolExchange
Streaming bool
}
AssistantTurn is one assistant turn in an observed or submitted transcript. Streaming is plan/client metadata (how the turn was produced); Check does not read it from BackendTurnObservation.
func ParseChatJSONAssistant ¶
func ParseChatJSONAssistant(body []byte) (AssistantTurn, error)
ParseChatJSONAssistant extracts assistant fields from a non-stream chat.completion body.
func ParseChatSSEAssistant ¶
func ParseChatSSEAssistant(body []byte) (AssistantTurn, error)
ParseChatSSEAssistant aggregates assistant fields from a chat.completion.chunk SSE body.
type BackendRequestObservation ¶
type BackendRequestObservation struct {
AssistantTurns []BackendTurnObservation
}
BackendRequestObservation is the oracle input for one backend-bound request.
func ObserveChatBackendRequest ¶
func ObserveChatBackendRequest(body []byte, turnIDs []string) (BackendRequestObservation, error)
ObserveChatBackendRequest parses a backend-bound chat/completions body into oracle input. turnIDs must align with assistant history order (may be shorter than plan).
type BackendTurnObservation ¶
type BackendTurnObservation struct {
TurnID string
VisibleText string
Reasoning []ReasoningBlock
Tool *ToolExchange
}
BackendTurnObservation is one historical assistant turn as seen on a backend-bound request. It has no Streaming field: stream vs non-stream wire shape is out of oracle scope.
type ChatCompletionRequest ¶
type ChatCompletionRequest struct {
Model string `json:"model"`
Stream bool `json:"stream,omitempty"`
Messages []map[string]any `json:"messages"`
Tools []map[string]any `json:"tools,omitempty"`
}
ChatCompletionRequest is a minimal wire request body.
type ChatResponse ¶
type ChatResponse struct {
VisibleText string
Reasoning []ReasoningBlock
Tool *ToolExchange
Streaming bool
}
ChatResponse is one proxy assistant observation recorded by the client emulator. Tool.Result is optional on the wire response; materialization uses the plan tool result.
func ChatResponseFromTurn ¶
func ChatResponseFromTurn(resp ChatTurnResponse) ChatResponse
ChatResponseFromTurn maps a wire client response into a ClientEmulator ChatResponse.
type ChatSessionCarriers ¶
ChatSessionCarriers holds authoritative session resume material from proxy responses.
type ChatTurnResponse ¶
type ChatTurnResponse struct {
Status int
RawBody []byte
ContentType string
Stream bool
VisibleText string
Reasoning []ReasoningBlock
Tool *ToolExchange
Carriers ChatSessionCarriers
}
ChatTurnResponse is the parsed proxy response for one chat turn.
type ChatWireClient ¶
type ChatWireClient struct {
BaseURL string
APIKey string
HTTPClient *http.Client
Model string
// Route, when non-empty, sets X-LIP-Route on each request (overrides proxy default_route).
Route string
Carriers ChatSessionCarriers
}
ChatWireClient is a stateful raw-wire OpenAI Chat Completions client for E2E drivers.
func (*ChatWireClient) PostChatCompletion ¶
func (c *ChatWireClient) PostChatCompletion(ctx context.Context, stream bool, messages []map[string]any, tools []map[string]any) (ChatTurnResponse, error)
PostChatCompletion sends one raw chat/completions request and updates carriers.
type ClientEmulator ¶
type ClientEmulator struct {
// contains filtered or unexported fields
}
ClientEmulator (Conversation) owns an immutable Plan plus recorded actual proxy outputs. It keeps observed and submitted histories independently, returns defensive copies, and never calls testing.T or logs payloads.
func NewClientEmulator ¶
func NewClientEmulator(plan Plan) *ClientEmulator
NewClientEmulator returns a conversation emulator bound to plan.
func (*ClientEmulator) MaterializeChatRequest ¶
func (e *ClientEmulator) MaterializeChatRequest(nextUserPrompt string) ([]map[string]any, error)
MaterializeChatRequest builds the next Chat Completions messages list. History assistant visible/tool structure comes from recorded actual responses; reasoning follows the plan retention policy (submitted). Tool results and prior user prompts are included. It is an error to materialize again while a prior materialize is still awaiting Record (unrecorded turn).
func (*ClientEmulator) ObservedHistory ¶
func (e *ClientEmulator) ObservedHistory() []AssistantTurn
ObservedHistory returns defensive copies of recorded actual assistant observations.
func (*ClientEmulator) Plan ¶
func (e *ClientEmulator) Plan() Plan
Plan returns the immutable bound plan (defensive turn copies via Plan methods).
func (*ClientEmulator) Record ¶
func (e *ClientEmulator) Record(resp ChatResponse) error
Record validates resp against the next Plan.Observed turn (structural, content-safe), then appends independent observed and submitted history entries. Visible/tool on submitted come from the actual recorded response; reasoning follows plan policy.
func (*ClientEmulator) SubmittedHistory ¶
func (e *ClientEmulator) SubmittedHistory() []AssistantTurn
SubmittedHistory returns defensive copies of policy-materialized submitted turns.
type DecisionRecord ¶
type DecisionRecord struct {
Index int
TurnID string
BackendKind string // reason | no_reason | tool
ClientDecision string // preserve | drop | none
Streaming bool
HasReasoning bool
HasTool bool
ReasonCode string // structural label only; empty unless set by callers for failures
}
DecisionRecord is one compact structural decision for a generated turn.
type FailTrace ¶
type FailTrace struct {
Seed uint64
Policy string
Mode string
TurnID string
Field string
Detail string
}
FailTrace is a content-safe structural failure description for E2E drivers. It must never carry reasoning text, signatures, opaque payloads, or anchors.
type MatrixCase ¶
type MatrixCase struct {
Mode MatrixMode
Seed uint64
}
MatrixCase is one default-matrix (mode, seed) pair.
func DefaultMatrixCases ¶
func DefaultMatrixCases() []MatrixCase
DefaultMatrixCases returns the exact 64-seed split: 16 drop-all, 16 always-reason, 32 combined. Always-reason and combined seeds are chosen so the independent client RNG stream naturally includes both preserve and drop (client bits are never mutated).
func SoakCases ¶
func SoakCases(cfg SoakConfig) []MatrixCase
SoakCases returns the mode/seed allocation for cfg. Defaults preserve 25%/25%/50% (250/250/500 of 1000). Replay returns one case.
type MatrixMode ¶
type MatrixMode string
MatrixMode selects a precomputed random-matrix scenario family.
const ( // MatrixModeRandomBackendDropAll randomizes backend reason/no-reason/tool; client drops all. MatrixModeRandomBackendDropAll MatrixMode = "random_backend_drop_all" // MatrixModeAlwaysReasonRandomClient always emits reasoning; client preserve/drop is randomized. MatrixModeAlwaysReasonRandomClient MatrixMode = "always_reason_random_client" // MatrixModeCombined randomizes backend and client retention independently. MatrixModeCombined MatrixMode = "combined" )
type Plan ¶
type Plan struct {
Seed uint64
Policy RetentionPolicy
// contains filtered or unexported fields
}
Plan is a precomputed deterministic transcript plan.
func BuildPlan ¶
func BuildPlan(cfg PlanConfig) (Plan, error)
BuildPlan precomputes a deterministic transcript plan from an explicit seed and policy.
func (Plan) ObservedTranscript ¶
func (p Plan) ObservedTranscript() []AssistantTurn
ObservedTranscript returns a defensive copy of the immutable observed assistant turns.
func (Plan) SubmittedTranscript ¶
func (p Plan) SubmittedTranscript() []AssistantTurn
SubmittedTranscript returns a defensive copy of the materialized client-submitted turns.
func (Plan) Turns ¶
func (p Plan) Turns() []PlannedTurn
Turns returns defensive copies of planned turns.
type PlanConfig ¶
type PlanConfig struct {
Seed uint64
Policy RetentionPolicy
Turns []TurnSpec
}
PlanConfig is the explicit deterministic plan input.
type PlannedTurn ¶
type PlannedTurn struct {
ID string
Mode RetentionMode
Observed AssistantTurn
Submitted AssistantTurn
ExpectedBackend AssistantTurn
}
PlannedTurn is the precomputed expectation for one assistant turn.
type ReasoningBlock ¶
ReasoningBlock is one ordered reasoning payload on an assistant turn.
type ResponsesMatrixCase ¶
type ResponsesMatrixCase struct {
Seed uint64
Variant ResponsesPresenceVariant
StreamFE bool
Trace string
}
ResponsesMatrixCase is one reproducible Responses-inclusive smoke cell.
func DefaultResponsesSmokeCases ¶
func DefaultResponsesSmokeCases() []ResponsesMatrixCase
DefaultResponsesSmokeCases returns a moderate fixed-seed set (8 cells) for local smoke.
func ResponsesSmokeCases ¶
func ResponsesSmokeCases(seedBase uint64, n int) []ResponsesMatrixCase
ResponsesSmokeCases builds n deterministic Responses presence/stream cells from seedBase.
type ResponsesPresenceVariant ¶
type ResponsesPresenceVariant string
ResponsesPresenceVariant is a deterministic exact-item presence case for seeded smokes.
const ( ResponsesPresenceEncryptedAbsent ResponsesPresenceVariant = "enc_absent" ResponsesPresenceEncryptedNull ResponsesPresenceVariant = "enc_null" ResponsesPresenceEncryptedValue ResponsesPresenceVariant = "enc_value" ResponsesPresenceWithContent ResponsesPresenceVariant = "with_content" )
type RetentionMode ¶
type RetentionMode string
RetentionMode is the planned fate of one assistant turn's reasoning.
const ( // ModeNone means the observed turn had no reasoning; nothing may be inserted later. ModeNone RetentionMode = "none" // ModePreserved means the client still carries the observed reasoning. ModePreserved RetentionMode = "preserved" // ModeDropped means the client omitted observed reasoning (restore candidate). ModeDropped RetentionMode = "dropped" // ModeConflict means the client submitted conflicting reasoning that must stay untouched. ModeConflict RetentionMode = "conflict" )
type RetentionPolicy ¶
type RetentionPolicy int
RetentionPolicy controls how the client materializes previously observed assistant reasoning.
const ( // PreserveAllReasoning keeps every observed reasoning block in the submitted transcript. PreserveAllReasoning RetentionPolicy = iota // DropAllReasoning strips all observed reasoning from the submitted transcript. DropAllReasoning // SeededPerTurnRetention keeps or drops each turn's reasoning using a seeded PRNG. SeededPerTurnRetention // ConflictReasoning submits alternate reasoning that differs from the observed blocks. ConflictReasoning )
func (RetentionPolicy) String ¶
func (p RetentionPolicy) String() string
type ScriptedBackendTurn ¶
type ScriptedBackendTurn struct {
VisibleText string
ReasoningText string
ToolID string
ToolName string
ToolArgs string
}
ScriptedBackendTurn is a content-bearing scripted assistant response for refbackends. HTTP drivers map this to emulator ScriptedTurn values; traces must not print these fields.
type SoakConfig ¶
type SoakConfig struct {
Enabled bool
Seeds int
Turns int
Workers int
Replay *SoakReplay
}
SoakConfig is the validated soak driver configuration.
func LoadSoakConfigFromEnv ¶
func LoadSoakConfigFromEnv() (SoakConfig, error)
LoadSoakConfigFromEnv parses soak settings from the process environment.
func ParseSoakConfig ¶
func ParseSoakConfig(getenv func(string) string) (SoakConfig, error)
ParseSoakConfig parses soak settings from getenv. When the soak gate is unset, overrides are ignored and Enabled is false (no error).
type SoakPoolJob ¶
type SoakPoolJob struct {
Index int
Case MatrixCase
}
SoakPoolJob is one unit of work for the fixed soak worker pool.
type SoakPoolResult ¶
type SoakPoolResult struct {
Index int
Case MatrixCase
HTTPTurns int
Err error
}
SoakPoolResult is the outcome of one soak job (content-safe Err only).
func RunSoakWorkerPool ¶
func RunSoakWorkerPool(workers int, jobs []SoakPoolJob, run func(SoakPoolJob) SoakPoolResult) []SoakPoolResult
RunSoakWorkerPool runs jobs with exactly workers long-lived goroutines. It does not spawn one goroutine per job. run must not call testing.T methods.
type SoakPoolRunStats ¶
SoakPoolRunStats captures fixed-pool execution metrics (worker spawn vs job count).
type SoakReplay ¶
type SoakReplay struct {
Mode MatrixMode
Seed uint64
}
SoakReplay identifies a single mode/seed pair for reproduction.
type ToolExchange ¶
ToolExchange is an optional tool call plus result attached to an assistant turn.
type TranscriptPlan ¶
type TranscriptPlan struct {
// contains filtered or unexported fields
}
TranscriptPlan is an immutable precomputed matrix scenario.
func GenerateTranscriptPlan ¶
func GenerateTranscriptPlan(mode MatrixMode, seed uint64, turnCount int) (TranscriptPlan, error)
GenerateTranscriptPlan precomputes a deterministic scenario from seed+mode+turnCount. It uses independent RNG streams for backend and client choices (no package-global RNG).
func (TranscriptPlan) Decisions ¶
func (p TranscriptPlan) Decisions() []DecisionRecord
Decisions returns defensive copies of structural decision records.
func (TranscriptPlan) Mode ¶
func (p TranscriptPlan) Mode() MatrixMode
Mode returns the matrix mode.
func (TranscriptPlan) Plan ¶
func (p TranscriptPlan) Plan() Plan
Plan returns the immutable reasoninge2e Plan (defensive turn copies via Plan methods).
func (TranscriptPlan) ScriptedTurns ¶
func (p TranscriptPlan) ScriptedTurns() []ScriptedBackendTurn
ScriptedTurns returns defensive copies of scripted backend turns.
func (TranscriptPlan) StructuralTrace ¶
func (p TranscriptPlan) StructuralTrace() string
StructuralTrace returns a compact content-safe decision trace.
func (TranscriptPlan) TurnCount ¶
func (p TranscriptPlan) TurnCount() int
TurnCount returns the configured turn count.
type TurnSpec ¶
type TurnSpec struct {
VisibleText string
Reasoning []ReasoningBlock
ConflictReasoning []ReasoningBlock
Tool *ToolExchange
Streaming bool
// ClientMode, when non-empty, overrides policy-derived retention for this turn.
ClientMode RetentionMode
}
TurnSpec is the explicit seed input for one observed assistant turn.