Documentation
¶
Overview ¶
Package parser is the finalize-time body parser described in ARCHITECTURE § 7.1 step 7.
Inputs: a captured-body io.Reader (typically a tmp file the drainer wrote) and the corresponding http.Header. Output: a trace.Body in one of three structural forms (body / events / body_b64) plus a parse_error when degraded to body_b64.
Project-context extraction. Pulls the project name out of an L2 system prompt by mirroring the viewer's promptSource.extractProjectContext (api-log-viewer/src/lib/promptSource.ts) so the backend-derived column stays in lockstep with what the UI computes at render time.
PHILOSOPHY § 1 carve-out 1: deterministic copy of NAMED protocol body fields (Chat messages[role=system], Messages system, Responses instructions / input items[role=system]) followed by regex over the resulting text. No header sniffing, no synthesized identifiers — the regexes match against the operator-authored injection format Claude Code / codex actually emit.
PHILOSOPHY § 2: ExtractProjectContext + ExtractSystemPrompt MUST NOT panic and MUST NOT block. Empty / unknown inputs return the zero ProjectContext (Name == "") — the caller treats that as "no project."
PHILOSOPHY § 6: SQLite columns derived from this output are rebuildable from JSONL by replaying these extractors against the recorded request body.
PHILOSOPHY § 7: small, slow surface — only the three protocols listed below (Chat / Messages / Responses). Gemini and unknown paths return "" because the contract names no project-injection path for them.
Usage extraction (T3). Pulls token / model / finish-reason fields out of a finalized trace.Trace and returns them as UsageInfo.
This is PHILOSOPHY §1 carve-out 1: a deterministic copy of NAMED protocol fields, no synthesis, no heuristic sniffing. The only "computed" field is TotalTokens, which falls back to PromptTokens+CompletionTokens when (and only when) the protocol itself did not provide a total — the comment on the struct says "provided or computed."
PHILOSOPHY §2: this function MUST NOT panic and MUST NOT block. All missing-field branches return nil pointers; all unmarshal errors are swallowed; the writer logs WARN at the call site.
PHILOSOPHY §6: SQLite columns derived from this output are rebuildable from JSONL by replaying ExtractUsage. That means the implementation must read whichever named shape the on-disk trace actually carries — for the OpenAI Responses protocol that is BOTH streaming (resp.events[-1]) and non-streaming (resp.body) shapes; otherwise a replay of a non-streaming /v1/responses trace would silently drop real named usage fields.
PHILOSOPHY §7: protocol surface is small and slow — only the four protocols listed below. Unknown paths return zero UsageInfo. Adding a new protocol is a separate piece of work.
Scope note (streaming chat / messages): the investigation contract names BODY field paths for chat / messages / gemini. The Anthropic streaming protocol spreads usage across two SSE frames (message_start carries input + cache fields; message_delta carries output_tokens), and streaming OpenAI Chat usage lives on a final chunk; neither has a single named container the contract pins down. Per §1 we do not invent paths the contract omits, so streaming chat/messages traces return nil token fields. /v1/responses is the lone exception because its terminal `response.completed` event carries a complete single- location usage block — see extractResponses below.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ExtractSystemPrompt ¶
ExtractSystemPrompt pulls the system / instructions text out of a finalized Trace's REQUEST body, dispatched by protocol. Empty when no system text is present or the body did not parse.
Per PHILOSOPHY § 1 these are named protocol fields; per § 7 the surface is small — only the three protocols whose contracts name a dedicated system-prompt path:
Chat /v1/chat/completions : req.body.messages[role=system].content
Messages /v1/messages : req.body.system (string OR array of {type,text})
Responses /v1/responses : req.body.instructions + input items[role=system]
Anthropic Messages and OpenAI Responses both ship the "system" / "instructions" / "input" fields in TWO shapes — a plain string or an array of content blocks. The dominant Claude Code → /v1/messages fixture is array-shaped (each block carries cache_control), so a string-only handler would silently drop the most important case. Both shapes are accepted; text from multiple system blocks is joined with "\n\n".
Gemini and unknown paths return "" — there is no operator-injected project file path for those protocols on this surface.
func ParseRequest ¶
ParseRequest parses a captured request body into a trace.Body.
The request side is always non-streaming (clients don't stream POST bodies to LLM gateways in practice), so this is just JSON-or-binary.
func ParseResponse ¶
ParseResponse parses a captured response body into a trace.Body.
Determines streaming vs non-streaming via the response Content-Type (text/event-stream → stream). The header check is the only place shape detection happens at finalize; the sub-shape (data-only vs event-named) is inferred inside internal/sse.
opts.ChunkTimings + opts.TsStart are used to attach per-event t_delta_ms when the body parses as SSE AND we trust the timings (i.e. response was identity-encoded so the drainer's chunk offsets correspond 1:1 to the bytes we're parsing). If the response was content-encoded, the caller passes opts.ChunkTimings = nil and every event's t_delta_ms stays nil (the sentinel per ARCHITECTURE § 3 / § 10.6).
Types ¶
type ClientInfo ¶
ClientInfo is the result of ExtractClient. Both fields nil when no rule matches; Version nil when the matched rule's discriminator doesn't expose a version.
func ExtractClient ¶
func ExtractClient(h trace.Headers) ClientInfo
ExtractClient runs the taxonomy classifier against h. It never panics — a nil or empty header map yields the zero ClientInfo.
type ParseOpts ¶
type ParseOpts struct {
ChunkTimings []capture.ChunkTiming
TsStart time.Time
}
ParseOpts carries optional timing data so finalize can attach per-event t_delta_ms (ARCHITECTURE § 3.3) to SSE events.
ChunkTimings + TsStart are typically populated only for ParseResponse on identity-encoded SSE; encoded responses leave them nil so events' TDeltaMs stays nil (the sentinel per ARCHITECTURE § 3).
type ProjectContext ¶
ProjectContext is the result of ExtractProjectContext. Name == "" means "no project signal found"; the Source sentinel distinguishes where the name came from when Name is non-empty.
Source values:
- "agents-md" — AGENTS.md injection reference adjacent to a heading
- "claude-md" — CLAUDE.md injection reference adjacent to a heading
- "first-heading" — text simply starts with `# …` (no file ref)
- "" — no match (paired with empty Name)
func ExtractProjectContext ¶
func ExtractProjectContext(systemPrompt string) ProjectContext
ExtractProjectContext returns the project name + source for the L2 portion of a system prompt. Strategy mirrors the viewer:
- AGENTS.md / CLAUDE.md injection-style file reference (path-prefixed OR "Contents of …" preamble — NOT a bare prose mention) followed by a markdown heading within a tight window → "agents-md" or "claude-md".
- Text starts with a `# …` heading (after skipping an optional leading XML close tag) → "first-heading".
- Otherwise zero value (Name == "").
Edge cases:
- Inline backticks around the name are stripped
- Empty / whitespace-only text → zero value
- HTML entities (lt / gt / amp) in the name are decoded
The function is deliberately tolerant of vendor-harness "# System" style headings on the first-heading path — that's the documented best-effort behavior the viewer ships and the source sentinel makes the weakness visible to consumers.
type UsageInfo ¶
type UsageInfo struct {
Model *string // protocol-extracted model name
FinishReason *string // protocol finish_reason / stop_reason / status
PromptTokens *int64 // input_tokens / prompt_tokens
CompletionTokens *int64 // output_tokens / completion_tokens
TotalTokens *int64 // provided or computed as prompt + completion
CachedTokens *int64 // sum of cache hits (cache_read for Anthropic, cached_tokens for OpenAI)
CacheCreationTokens *int64 // Anthropic-only: cache_creation_input_tokens (input cache misses for storage)
ReasoningTokens *int64 // OpenAI Responses only: output_tokens_details.reasoning_tokens
}
UsageInfo is the contract for usage extraction. Each field is a pointer because absence (nil) is distinct from zero. Consumers treat nil as "the protocol does not define this field" or "the field was absent in the actual response."
func ExtractUsage ¶
ExtractUsage parses usage fields from a Trace. Protocol is detected from t.Path (mirrors the viewer's detectProtocol logic in api-log-viewer/src/lib/adapters/index.ts — substring match on /v1/chat/completions, /v1/messages, /v1/responses, /v1beta/ or :generateContent).
Returns a UsageInfo with all fields populated where the protocol includes them. Nil pointers indicate either "this protocol does not define this field" or "the field was absent in this trace."
Body_b64 fallback: when JSON parsing failed at finalize, t.Resp.Body is empty (the bytes live in t.Resp.BodyB64). This function ignores body_b64 entirely — the parsed body is the source of truth, and if parsing failed there is no usage to extract.
Source Files