Documentation
¶
Overview ¶
Package parser is the finalize-time body parser described in ARCHITECTURE § 7.1 step 7.
Inputs: a captured-body io.Reader (typically a tmp file the drainer wrote) and the corresponding http.Header. Output: a trace.Body in one of three structural forms (body / events / body_b64) plus a parse_error when degraded to body_b64.
Project-context extraction. Pulls the project name out of an L2 system prompt by mirroring the viewer's promptSource.extractProjectContext (api-log-viewer/src/lib/promptSource.ts) so the backend-derived column stays in lockstep with what the UI computes at render time.
Deterministically copies named protocol body fields, then applies regexes to the resulting system text. No header sniffing, no synthesized identifiers.
ExtractProjectContext and ExtractSystemPrompt must not panic or block. Empty / unknown inputs return the zero ProjectContext (Name == "") — the caller treats that as "no project."
SQLite columns derived from this output are rebuildable from JSONL by replaying these extractors against the recorded request body.
The protocol surface is intentionally small: only the three protocols listed below (Chat / Messages / Responses). Gemini and unknown paths return "" because the contract names no project-injection path for them.
Usage extraction pulls token, model, and finish-reason fields out of a finalized trace.Trace. It deterministically copies named protocol fields; no heuristic sniffing is used. The only "computed" field is TotalTokens, which falls back to PromptTokens+CompletionTokens when (and only when) the protocol itself did not provide a total — the comment on the struct says "provided or computed."
This function must not panic or block. All missing-field branches return nil pointers; all unmarshal errors are swallowed; the writer logs WARN at the call site.
SQLite columns derived from this output are rebuildable from JSONL by replaying ExtractUsage. That means the implementation must read whichever named shape the on-disk trace actually carries — for the OpenAI Responses protocol that is BOTH streaming (resp.events[-1]) and non-streaming (resp.body) shapes; otherwise a replay of a non-streaming /v1/responses trace would silently drop real named usage fields.
The protocol surface is intentionally small: only the four protocols listed below. Unknown paths return zero UsageInfo. Adding a new protocol is a separate piece of work.
Scope note (streaming chat / messages): the investigation contract names BODY field paths for chat / messages / gemini. The Anthropic streaming protocol spreads usage across two SSE frames (message_start carries input + cache fields; message_delta carries output_tokens), and streaming OpenAI Chat usage lives on a final chunk; neither has a single named container the contract pins down. Per §1 we do not invent paths the contract omits, so streaming chat/messages traces return nil token fields. /v1/responses is the lone exception because its terminal `response.completed` event carries a complete single- location usage block — see extractResponses below.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ExtractSystemPrompt ¶
ExtractSystemPrompt pulls the system / instructions text out of a finalized Trace's REQUEST body, dispatched by protocol. Empty when no system text is present or the body did not parse.
These are named protocol fields; the supported surface is intentionally small: only protocols with a dedicated system-prompt path.
Chat /v1/chat/completions : req.body.messages[role=system].content
Messages /v1/messages : req.body.system (string OR array of {type,text})
Responses /v1/responses : req.body.instructions + input items[role=system]
Anthropic Messages and OpenAI Responses both ship the "system" / "instructions" / "input" fields in TWO shapes — a plain string or an array of content blocks. The dominant Claude Code → /v1/messages fixture is array-shaped (each block carries cache_control), so a string-only handler would silently drop the most important case. Both shapes are accepted; text from multiple system blocks is joined with "\n\n".
Gemini and unknown paths return "" — there is no operator-injected project file path for those protocols on this surface.
func ParseRequest ¶
ParseRequest parses a captured request body into a trace.Body.
The request side is always non-streaming (clients don't stream POST bodies to LLM gateways in practice), so this is just JSON-or-binary.
func ParseResponse ¶
ParseResponse parses a captured response body into a trace.Body.
Determines streaming vs non-streaming via the response Content-Type (text/event-stream → stream). The header check is the only place shape detection happens at finalize; the sub-shape (data-only vs event-named) is inferred inside internal/sse.
opts.ChunkTimings + opts.TsStart are used to attach per-event t_delta_ms when the body parses as SSE AND we trust the timings (i.e. response was identity-encoded so the drainer's chunk offsets correspond 1:1 to the bytes we're parsing). If the response was content-encoded, the caller passes opts.ChunkTimings = nil and every event's t_delta_ms stays nil (the sentinel per ARCHITECTURE § 3 / § 10.6).
Types ¶
type ClientInfo ¶
ClientInfo is the result of ExtractClient. Both fields nil when no rule matches; Version nil when the matched rule's discriminator doesn't expose a version.
func ExtractClient ¶
func ExtractClient(h trace.Headers) ClientInfo
ExtractClient runs the taxonomy classifier against h. It never panics — a nil or empty header map yields the zero ClientInfo.
type ParseOpts ¶
type ParseOpts struct {
ChunkTimings []capture.ChunkTiming
TsStart time.Time
}
ParseOpts carries optional timing data so finalize can attach per-event t_delta_ms (ARCHITECTURE § 3.3) to SSE events.
ChunkTimings + TsStart are typically populated only for ParseResponse on identity-encoded SSE; encoded responses leave them nil so events' TDeltaMs stays nil (the sentinel per ARCHITECTURE § 3).
type ProjectContext ¶
ProjectContext is the result of ExtractProjectContext. Name == "" means "no project signal found"; the Source sentinel distinguishes where the name came from when Name is non-empty.
Source values:
- "agents-md" — AGENTS.md injection reference adjacent to a heading
- "claude-md" — CLAUDE.md injection reference adjacent to a heading
- "first-heading" — text simply starts with `# …` (no file ref)
- "" — no match (paired with empty Name)
func ExtractProjectContext ¶
func ExtractProjectContext(systemPrompt string) ProjectContext
ExtractProjectContext returns the project name + source for the L2 portion of a system prompt. Strategy mirrors the viewer:
- AGENTS.md / CLAUDE.md injection-style file reference (path-prefixed OR "Contents of …" preamble — NOT a bare prose mention) followed by a markdown heading within a tight window → "agents-md" or "claude-md".
- Text starts with a `# …` heading (after skipping an optional leading XML close tag) → "first-heading".
- Otherwise zero value (Name == "").
Edge cases:
- Inline backticks around the name are stripped
- Empty / whitespace-only text → zero value
- HTML entities (lt / gt / amp) in the name are decoded
The function is deliberately tolerant of vendor-harness "# System" style headings on the first-heading path — that's the documented best-effort behavior the viewer ships and the source sentinel makes the weakness visible to consumers.
type UsageInfo ¶
type UsageInfo struct {
Model *string // protocol-extracted model name
FinishReason *string // protocol finish_reason / stop_reason / status
PromptTokens *int64 // input_tokens / prompt_tokens
CompletionTokens *int64 // output_tokens / completion_tokens
TotalTokens *int64 // provided or computed as prompt + completion
CachedTokens *int64 // sum of cache hits (cache_read for Anthropic, cached_tokens for OpenAI)
CacheCreationTokens *int64 // Anthropic-only: cache_creation_input_tokens (input cache misses for storage)
ReasoningTokens *int64 // OpenAI Responses only: output_tokens_details.reasoning_tokens
}
UsageInfo is the contract for usage extraction. Each field is a pointer because absence (nil) is distinct from zero. Consumers treat nil as "the protocol does not define this field" or "the field was absent in the actual response."
func ExtractUsage ¶
ExtractUsage parses usage fields from a Trace. Protocol is detected from t.Path (mirrors the viewer's detectProtocol logic in api-log-viewer/src/lib/adapters/index.ts — substring match on /v1/chat/completions, /v1/messages, /v1/responses, /v1beta/ or :generateContent).
Returns a UsageInfo with all fields populated where the protocol includes them. Nil pointers indicate either "this protocol does not define this field" or "the field was absent in this trace."
Body_b64 fallback: when JSON parsing failed at finalize, t.Resp.Body is empty (the bytes live in t.Resp.BodyB64). This function ignores body_b64 entirely — the parsed body is the source of truth, and if parsing failed there is no usage to extract.
Source Files