sqlite

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 31, 2026 License: MIT Imports: 9 Imported by: 0

Documentation

Overview

Package sqlite is the derived-cache store described in ARCHITECTURE § 4.

The SQLite database mirrors JSONL columns and adds writer-computed columns (model, tokens, key_hash, prefix_canonical_hash, parent_id, session_root_id). Everything in here is fully rebuildable from data/**/*.jsonl{,.gz} — see PHILOSOPHY § principle 6.

v0 uses modernc.org/sqlite (pure Go, no cgo). Slightly slower than mattn/go-sqlite3 but trivial to build / cross-compile.

Index

Constants

This section is empty.

Variables

View Source
var ErrNotFound = errors.New("trace not found")

ErrNotFound is returned by GetByID when no row matches the given id.

Functions

This section is empty.

Types

type ListFilters

type ListFilters struct {
	Since         time.Time // ts_start >= Since (zero = no bound)
	Until         time.Time // ts_start <  Until (zero = no bound)
	Status        *int      // exact match if set
	StatusBucket  int       // 0 = no constraint; 2/4/5 = match 2xx/4xx/5xx range
	Model         string    // exact match if set
	Path          string    // exact match if set
	PathPrefix    string    // matched as `LIKE <prefix>%`; mutually exclusive with Path
	KeyHashPrefix string    // accepts 8- or 16-char prefix; matches by LIKE
	SessionRootID string    // exact match if set
	Project       string    // exact match on client_project if set
	Limit         int       // 1..500; <=0 means default 100

	// Cursor is opaque base64 of "<ts_start_ms>:<id>"; rows strictly older
	// than the cursor (ts_start DESC, id DESC ordering) are returned.
	// Empty = first page.
	CursorTsStart int64
	CursorID      string
}

ListFilters bounds and filters /api/traces queries per ARCHITECTURE § 6.2. All fields are optional; zero values mean "no constraint".

type ListPage

type ListPage struct {
	Rows         []Row
	NextCursorMs int64  // 0 if no next page
	NextCursorID string // empty if no next page
}

ListPage is the paginated result.

type Row

type Row struct {
	ID       string
	TsStart  time.Time
	TsEnd    time.Time
	Client   string
	Method   string
	Path     string
	Upstream string
	Status   int

	Disconnected  bool
	TruncatedReq  bool
	TruncatedResp bool

	// Protocol field copies (may be NULL if absent). Populated from
	// trace.Trace by the writer; pass nil for unknown.
	Model            *string
	Stream           *bool
	PromptTokens     *int64
	CompletionTokens *int64
	TotalTokens      *int64
	FinishReason     *string

	// Extracted usage fields (T3). Deterministic copies of named protocol
	// usage fields — PHILOSOPHY § 1 carve-out 1. Nil when the upstream
	// response didn't carry the field; nil is distinct from a real zero.
	CachedTokens        *int64 // sum of cache-hit tokens across protocols
	CacheCreationTokens *int64 // Anthropic cache_creation_input_tokens (cache miss writes)
	ReasoningTokens     *int64 // OpenAI Responses reasoning tokens from output

	// Client-identity fields (R5a). Deterministic copies of named request-
	// header fields routed through the taxonomy table in ExtractClient.
	// PHILOSOPHY § 1 + § 7: no heuristic UA parsing — nil when the header
	// doesn't match a known taxonomy row.
	ClientKind    *string // taxonomy row key (e.g. "claude-code-desktop")
	ClientVersion *string // version string from the matching header (if any)

	// Project-context field (W4.1 Phase 2). Deterministic copy of the L2
	// project name parsed out of the request body's system / instructions
	// text by parser.ExtractProjectContext. PHILOSOPHY § 1: nil when the
	// trace carried no project signal (distinct from a real empty string).
	ClientProject *string // project name (e.g. "my-repo" from `# my-repo` heading)

	// Identifiers.
	KeyHash string

	// Session fields. Populated by AppendTrace (writer side) or by
	// scanRow (read side). Callers should NOT set these on the writer
	// path — they are derived inside the transaction.
	PrefixLen           *int
	PrefixCanonicalHash *string
	ParentID            *string
	SessionRootID       string

	// JSONL location.
	JSONLPath   string
	JSONLOffset int64

	// MediaCount is the number of extracted media files for this trace
	// (Phase K). Filled by the writer after extraction runs; default 0
	// when extraction is disabled or the trace has no extractable media.
	// PHILOSOPHY § 1: deterministic count of named protocol fields only.
	MediaCount int
}

Row is the Go-side projection of one traces row. Session-inference columns (PrefixLen, PrefixCanonicalHash, ParentID, SessionRootID) are filled by AppendTrace, not by the caller — pass them empty.

func FromTrace

func FromTrace(t trace.Trace, keyHash, jsonlPath string, jsonlOffset int64) Row

FromTrace builds a Row from a trace.Trace plus the writer's known extras. Session columns are left to AppendTrace; derived columns (model, tokens, finish_reason) are left nil — caller fills them.

type SessionSummary

type SessionSummary struct {
	SessionRootID    string
	NTurns           int64
	FirstTs          time.Time
	LastTs           time.Time
	LastPath         string
	LastStatus       int
	LastModel        string
	DistinctKeyCount int64
	OkCount          int64
	ErrCount         int64
}

SessionSummary is one row of the /api/sessions response — an aggregate over a session_root_id. Useful for "what conversations are happening" without paging through every per-turn trace.

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store is the writer-side handle into index.sqlite. One Writer holds one Store. Read-API handlers will get their own Store handles in M4 (separate connections, WAL allows concurrent reads).

func Open

func Open(path string) (*Store, error)

Open opens or creates index.sqlite at path, runs migration, and applies the pragmas from ARCHITECTURE § 4.

func (*Store) AllMatching

func (s *Store) AllMatching(filters ListFilters, hardCap int) ([]Row, error)

AllMatching returns every row matching filters, ordered chronologically (ts_start ASC, id ASC for a deterministic tiebreak).

Unlike List, this method ignores filters.Limit (the per-page cap) and any cursor fields, and orders ASC so callers (the exporter) can stream results into a reproducible zip. The hardCap argument optionally bounds the total rows fetched; pass 0 (or any non-positive value) for unlimited — the export path uses this. The cap parameter is kept on the signature for future callers that want a guard (rate-limit, preview).

The WHERE-clause is built from the same fields as List (status, status bucket, model, path/prefix, key_hash prefix, session_root_id, since, until) — keep them in sync if List grows new filters.

func (*Store) AppendTrace

func (s *Store) AppendTrace(r Row, sessionPrefix []json.RawMessage) error

AppendTrace inserts a row and runs session inference atomically.

sessionPrefix is the result of session.Build(path, req.body). Pass nil if the trace has no session concept (embeddings, image gen, etc.); the row's session_root_id defaults to id (self-root) and parent_id stays NULL.

Everything happens in one transaction: this gives us one fsync per trace (instead of two for INSERT + UPDATE) and atomic session-inference reads. On commit failure the row is not inserted; the caller's JSONL line is still on disk and will be picked up by startup rebuild.

func (*Store) Close

func (s *Store) Close() error

Close closes the underlying *sql.DB.

func (*Store) CountRows

func (s *Store) CountRows() (int64, error)

CountRows is used by the startup rebuild path to detect a "fresh" index.

func (*Store) FindParent

func (s *Store) FindParent(keyHash string, strictHashes []string) (parentID, sessionRootID string, prefixLen int, ok bool, err error)

FindParent runs the session-inference IN-clause query described in ARCHITECTURE § 5.2. Given a trace's strict-prefix canonical hashes (in descending-length order), it returns the longest-matching prior trace's (id, session_root_id, prefix_len). If no prior trace matches, ok is false.

strictHashes MUST be ordered by length descending so the SQL's ORDER BY prefix_len DESC stably returns the longest match. The caller's responsibility — sqlite has no information about prefix semantics, only the hashes.

func (*Store) GetByID

func (s *Store) GetByID(id string) (Row, error)

GetByID returns the row with the given trace ID. Returns ErrNotFound when no such row exists.

func (*Store) List

func (s *Store) List(filters ListFilters) (ListPage, error)

List returns at most filters.Limit (default 100, max 500) rows ordered by (ts_start DESC, id DESC) that match the filters.

Always orders by ts_start DESC for cursor stability. To page, pass the previous page's NextCursorMs / NextCursorID back as filters.CursorTsStart / filters.CursorID.

func (*Store) ListSessions

func (s *Store) ListSessions(since time.Time, limit int) ([]SessionSummary, error)

ListSessions returns the most recent sessions, latest-activity first. limit defaults to 100, max 500. Optional since filter applies to the session's last-activity timestamp.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL