parser

type Agent ¶

type Agent string

Agent identifies which on-disk format a session uses.

const (
	AgentClaude Agent = "claude"
	AgentCodex  Agent = "codex"
	AgentPi     Agent = "pi"
)

type Attachment ¶

type Attachment struct {
	MessageOrdinal int
	SHA256         string
	Bytes          int
	MediaType      string
	Filename       string
	Content        string
}

Attachment is one binary blob attached to a message: today a lifted image (a Codex image-generation result or a pasted image). The bytes live in the CAS keyed by SHA256; Bytes is the raw (decoded) size and MediaType its semantic type, so the UI can render it without fetching. Content carries the decoded bytes only on the inline path (a small image with no sentinel, used by the batch/test parser); when the client lifted the image it is empty and SHA256 names the already-uploaded blob.

type AttachmentOp ¶

type AttachmentOp struct {
	MessageOrdinal int
	SHA256         string
	Content        string
	Bytes          int
	MediaType      string
	Filename       string
}

AttachmentOp records one binary attachment (today a lifted image) against a message. Like a tool body it reaches the CAS by one of two paths: when the client lifted the image and left a sentinel, SHA256 is set and the server records the reference with no blob write; otherwise Content holds the decoded bytes inline for the server to store. Bytes and MediaType describe the decoded image either way, so the row's metadata is the same whichever path delivered the bytes.

type Body ¶

type Body struct {
	SHA256      string
	Bytes       int
	MediaType   string
	Stored      []byte
	ContentType string
	Kind        string
}

Body is one tool body the client lifts out of the transcript. Stored holds the exact bytes the CAS keeps, which are the raw canonical bytes for a small body and the zstd-compressed form for a large one; ContentType says which. SHA256 is the key, the sha256 of Stored. Bytes is the RAW (uncompressed) canonical length, the size the transcript sentinel and the tool_calls row record (what a reader thinks of as the body's size), independent of how the bytes are stored. Kind is "input" or "result", for diagnostics and tests.

func ExtractBodies(agent Agent, region []byte, enc BodyEncoder) ([]byte, []Body, error)

func RewriteLine(agent Agent, line []byte, enc BodyEncoder) ([]byte, []Body)

type BodyEncoder ¶

type BodyEncoder interface {
	// EncodeBody returns the CAS key (sha256 of the stored bytes), the stored bytes,
	// and their storage content type (ContentRaw or ContentZstd) for a body's raw
	// canonical bytes.
	EncodeBody(raw []byte) (sha string, stored []byte, contentType string)
}

BodyEncoder turns a tool body's canonical raw bytes into the bytes the CAS actually stores, deciding per body whether compression is worth it. The CAS key is the sha256 of the STORED bytes, so the same raw body must always encode to the same stored bytes and key; the client supplies a deterministic implementation. The parser depends only on this interface, never on a compression library, which is what keeps the server (which links the parser but never compresses) free of one. The streaming big-line path lives in the client and drives the same encoder directly, so a body encodes identically whether buffered or streamed.

type BodyKind ¶

type BodyKind int

BodyKind selects how the raw located value is canonicalized into the bytes the CAS stores. The three kinds mirror the branches of bodyContent: a value copied verbatim, a JSON string whose decoded contents are the body, and an array of blocks flattened to its text.

const (
	// BodyRaw copies the value's raw bytes verbatim. This is the canonical form
	// for genuine objects (a result whose body is a JSON object) and for the
	// claude/pi tool input, where the stored body is exactly input.Raw.
	BodyRaw BodyKind = iota
	// BodyJSONString treats the value as a JSON string and emits its decoded
	// contents, matching gjson .String(). This is the canonical form for a string
	// result body and for codex arguments (a JSON-encoded string).
	BodyJSONString
	// BodyArrayText treats the value as an array of typed blocks and emits the
	// blockText flattening: the decoded text of each contributing element joined
	// by a single newline.
	BodyArrayText
	// BodyBase64 treats the value as a JSON string carrying a base64-encoded binary
	// blob (optionally wrapped in a `data:<media>;base64,` URI) and emits the decoded
	// binary bytes. This is the canonical form for the image payloads Codex inlines
	// (image_generation results, data-URI image_url blocks, pasted images): the CAS
	// stores the real PNG/JPEG bytes, not the base64 text, so a reader serves the blob
	// directly under its image media type. The decode ignores \r and \n exactly as
	// encoding/base64 does, so the streamed result is byte-identical to the buffered
	// base64.StdEncoding.DecodeString the extractor uses.
	BodyBase64
)

func ClassifyResultBody(firstByte byte) (BodyKind, string)

type BodyLocation ¶

type BodyLocation struct {
	Span     ValueSpan
	Kind     BodyKind
	Media    string
	FilePath string
	Detail   string
}

BodyLocation is one tool body found in a transcript line by streaming, ready to be lifted to the CAS without ever buffering the body. Span is the raw byte range of the value within the line (relative to the line's first byte), the bytes the sentinel replaces. Kind and Media say how to canonicalize the raw value into the bytes the CAS stores (CanonicalBodyReader), so the streamed body is byte identical to what the server records inline today. FilePath is the top-level file_path string of a JSON tool input (empty otherwise), and Detail is the input's short human-scannable summary (a command, pattern, URL, or description; empty otherwise), both extracted here so the sentinel the caller builds carries them, exactly as the buffered RewriteLine path does via sentinelFilePath and sentinelDetail.

type Delta ¶

type Delta struct {
	Messages    []MessageOp
	ToolCalls   []ToolCall
	ToolResults []ToolResultOp
	Usage       []Usage
	Attachments []AttachmentOp

	Started time.Time
	Ended   time.Time

	// Cwd and GitBranch are the last values seen in the region. The store ignores
	// them (announce owns those columns); the test-facing Parse wrapper uses them.
	Cwd       string
	GitBranch string
}

Delta is everything one Reduce call produces for one raw region: the rows to write and the region's timestamp span. It carries operations, not a whole session, so applying it is append-only work proportional to the region, never to the session. It deliberately carries no message/token counters: the session rollups are derived downstream from the rows that actually persist (the store dedups messages and usage on insert), so a counter here would only risk drifting from that surviving set.

type Idx ¶

type Idx int

Idx selects an array element by its 0-based position.

type Key ¶

type Key string

Key selects an object member by name.

type LocatedSpan ¶

type LocatedSpan struct {
	PathIndex int
	Span      ValueSpan
}

LocatedSpan pairs a located value's span with the index of the path that produced it, so callers can correlate results back to their request even when some paths are absent and skipped.

func LocateValues(ctx context.Context, paths [][]Step, next func() ([]byte, error)) ([]LocatedSpan, error)

type Message ¶

type Message struct {
	Ordinal      int
	Role         Role
	Content      string
	ThinkingText string
	Model        string
	Timestamp    time.Time
	HasThinking  bool
	HasToolUse   bool
}

Message is one turn. Content holds the conversational text (stored inline and searchable); ThinkingText holds concatenated reasoning blocks.

type MessageOp ¶

type MessageOp struct {
	Ordinal      int
	Role         Role
	Content      string
	ThinkingText string
	Model        string
	HasThinking  bool
	HasToolUse   bool
	Timestamp    time.Time
}

MessageOp is one message write in a Delta. Each ordinal is written exactly once: a turn is folded whole within the region that contains it (the ingest protocol keeps a turn inside one chunk), so Content and ThinkingText are the complete text, not a fragment to append.

type Role ¶

type Role string

Role is a normalized message role.

const (
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	RoleSystem    Role = "system"
	RoleTool      Role = "tool"
	// RoleContext marks an injected-context turn: agent framing (project
	// instructions, the environment block) that an agent prepends to a session
	// rather than a human prompt. It is a distinct role so every role='user'
	// reader (the session title, user_message_count, the prompt-hygiene
	// aggregate) excludes it, and the transcript renders it in its own Context
	// section instead of as the opening turn. Only Codex records such framing as a
	// transcript message today (Claude and pi keep their framing in the system
	// prompt, which akari never ingests); the role is agent-agnostic so any
	// reducer can classify into it.
	RoleContext Role = "context"
)

type Session ¶

type Session struct {
	Cwd         string
	GitBranch   string
	StartedAt   time.Time
	EndedAt     time.Time
	Messages    []Message
	ToolCalls   []ToolCall
	UsageEvent  []Usage
	Attachments []Attachment
}

Session is the parsed projection of one session file.

func Parse(agent Agent, raw []byte) (Session, error)

type State ¶

type State struct {
	// NextOrdinal is the ordinal the next message will take.
	NextOrdinal int `json:"next_ordinal"`
	// Model is the sticky current model (Codex carries it across lines, and a
	// later region's usage line may need the model named in an earlier region).
	Model string `json:"model"`
}

State is the carry-over a parser needs to resume from a byte cursor. It is serialized to the session_raw row between chunks and is bounded in size: it holds only counters, never any per-message accumulation or an open turn. A Codex turn folds a run of reasoning/function_call items and the final text into one assistant message, but that fold never crosses a region: the ingest protocol cuts chunks on turn boundaries, so a whole turn always lands in one region. That is what lets the open turn live in the reducer for the span of a single Reduce call and never be serialized here.

func DecodeState(b []byte) (State, error)

func (s State) Encode() ([]byte, error)

type Step ¶

type Step interface {
	// contains filtered or unexported methods
}

Step is one segment of a path into a JSON document: either an object Key or an array Idx. The marker method keeps the two step kinds in a single closed set so callers cannot smuggle in an arbitrary type.

type ToolCall ¶

type ToolCall struct {
	MessageOrdinal  int
	CallIndex       int
	ToolName        string
	Category        string
	FilePath        string
	Detail          string
	InputJSON       string
	InputSHA256     string
	InputBytes      int
	InputMediaType  string
	ResultBody      string
	ResultSHA256    string
	ResultBytes     int
	ResultMediaType string
	ResultStatus    string // "ok" | "error" | "" (pending)
	CallUID         string
}

ToolCall is one tool invocation attached to a message. InputJSON and ResultBody carry the bulky bodies the CAS stores; ResultBytes and ResultMediaType describe ResultBody exactly, so the recorded size and media type always match the stored content. InputJSON is the raw tool-input JSON; ResultBody is the result body (a tool result that is an array of text blocks is flattened to its text).

CallUID is the agent's own call id; the incremental pipeline records it on the row so a tool result arriving in a later line (Claude delivers tool_result in the following user entry, which may land in a later region) is back-patched by an UPDATE keyed on it rather than by a parser-held id->row map. It is carried on inserts only; a Session assembled for tests ignores it. InputSHA256, InputBytes, and InputMediaType carry a CAS reference when the client lifted the input body to the CAS and left a sentinel in the transcript: the server records the reference instead of re-storing the body. InputJSON is empty in that case. When the body travels inline, InputJSON holds it and the SHA/bytes/media fields are unset (the server hashes and sizes the inline body).

Detail is a bounded, human-scannable summary of the input the UI shows when a call has no file_path: a shell command, a search pattern, a fetched URL, or an agent's description, derived from the input's top-level JSON keys. On the inline path it is derived here from the raw input; on the sentinel path the body is no longer readable, so it rides the sentinel and comes back through the casRef, exactly the way FilePath does.

type ToolResultOp ¶

type ToolResultOp struct {
	CallUID    string
	Body       string
	BodySHA256 string
	Bytes      int
	MediaType  string
	Status     string
}

ToolResultOp back-patches a tool call's result, matched by the agent's call id.

A result body normally travels inline (Body holds it) and the server writes it to the CAS. When the client has already lifted the body to the CAS and left a sentinel in the transcript, BodySHA256 is set instead: the server records the reference and writes no blob, but Bytes and MediaType still describe the body exactly so the row's metadata is unchanged.

type Usage ¶

type Usage struct {
	MessageOrdinal *int
	Model          string
	Input          int
	Output         int
	CacheWrite     int
	CacheRead      int
	Reasoning      int
	OccurredAt     time.Time
	DedupKey       string
	SourceOffset   int64
	SourceIndex    int
}

Usage is one token-accounting record. MessageOrdinal is nil for session-level totals not tied to a single message. SourceOffset and SourceIndex identify the originating line (and its position within it) so incremental inserts are idempotent even for agents whose usage carries no native dedup key.

type ValueSpan ¶

type ValueSpan struct {
	Start int64
	End   int64
}

ValueSpan is the byte range [Start,End) of one located JSON value within a single JSONL line, relative to the line's first byte (offset 0 = first byte of the line). It matches gjson's value Index and Index+len(Raw) exactly.

func LocateValue(ctx context.Context, path []Step, next func() ([]byte, error)) (ValueSpan, bool, error)