Help us shape the future of Go! Take the 2026 Go Developer Survey

obsdb

package
v0.20.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 27, 2026 License: MIT Imports: 28 Imported by: 6

Documentation

Overview

Package obsdb is the observability database: what weft/otel's local sink writes and Weft Studio reads, in one module so neither owns the schema (ADR 0024, S3).

The model is OTLP-shaped — spans and log records, resource included — so a stock OpenTelemetry pipeline's data and weft's own land in the same tables. Weft identity (the run → session → public-id chain) is derived from attributes, never parsed out of ids, and is stored as indexed columns through the Weft struct.

Three moving parts:

  • DB is the backend interface: Write (idempotent on (run, record kind, position) and (trace, span)), run/session queries, the event page with its gap detector, the transcript a replay reads, and the spans of a run or trace. Status is derived from the four-row table (DeriveStatus), never stored.
  • The hub (Hub, Frame) is the live lane: every Write publishes its frames before returning, so an in-process subscriber — Studio's SSE handler in setup A — sees a record without a network hop.
  • obsdbtest is the conformance table every backend runs (the storetest pattern); sqlite is the default backend, clickhouse the hosted one (its own module).

Two capabilities are optional interfaces a backend implements beside DB, both first-party backends included: Searcher (runs found by what they said and did) and CommandStore (the playground link's commands, run routes, breakpoints and runtimes, so Studio's runtime server survives a restart; Studio's bookkeeping, never touched by the run deletions).

Deltas travel but are not stored (their own counter means their absence never looks like a lost event); heartbeats are never stored, they only move a run's last-seen. Both rules are Write's, shared by every backend.

Index

Examples

Constants

View Source
const (
	CommandQueued   = "queued"
	CommandAccepted = "accepted"
	CommandRejected = "rejected"
	CommandFinished = "finished"
	CommandLost     = "lost"
)

Command states: the runtime server's lifecycle (WEFT-PLAYGROUND §10.5). Queued and accepted are live; the other three are terminal.

View Source
const (
	CompactionRun     = "run"
	CompactionSession = "session"
	ReasonCompacted   = "compacted"
	// RecordCompaction is the weft.record of thread's session marker.
	RecordCompaction = "compaction"
	// RecordThread is the weft.record of thread's session records (the
	// state log's family, event names weft.thread.<kind>; record
	// contract §4). Stored as plain records under the run they concern.
	RecordThread = "thread"
	// AttrChatRole is the chat span attribute naming a model call that
	// is not the run's own step (record contract §3); ChatRoleSummarizer
	// is weft/thread's summarizer call — its usage the session row's
	// SummarizerUsage, never the run's.
	AttrChatRole       = "weft.chat.role"
	ChatRoleSummarizer = "summarizer"
)

The two compaction scopes (weft.compaction.scope) and the one weft.messages.reason ADR 0028 §8 defines.

View Source
const (
	LineageChild = "child"
	LineageFork  = "fork"
)

SessionQuery.Lineage's values.

View Source
const (
	FrameRecord = "record"
	FrameRun    = "run"
	// FrameAttempt is an `attempt` span (one try inside a model call,
	// docs/record-contract.md §3) published as it lands (O17, UX-E-30):
	// the live lane's word that a step is retrying or falling back
	// before its chat span ends. Live only: the span is stored as any
	// span is, and the frame is never backfilled.
	FrameAttempt = "attempt"
)

Frame kinds.

View Source
const (
	ManifestSourceFile    = "file"    // the weft.json Studio serves
	ManifestSourceRuntime = "runtime" // a runtime registration
)

The sources Studio writes manifests from.

View Source
const (
	MaxSearchTerms     = 16
	MaxSearchTermBytes = 256
	MaxSearchOffset    = 100_000
)

Search bounds: a query is a few words, never a document.

View Source
const (
	// StatsBuckets is the series' default bucket count: 24 — an hour a
	// bucket over a day's window, seven hours over a week's.
	StatsBuckets = 24
	// MaxStatsBuckets bounds StatsQuery.Buckets.
	MaxStatsBuckets = 366
	// MaxStatsGroups bounds a StatsPage's groups: past it the groups
	// with the fewest runs (then calls) are cut, and StatsPage.GroupsCut
	// says how many.
	MaxStatsGroups = 200
)

Stats bounds.

View Source
const (
	MinStatsYear = 1678
	MaxStatsYear = 2262
)

The window's bounds: the years Unix nanoseconds hold in an int64 (2262 up to its April 11th; past that instant a time is refused too).

View Source
const BusyCancelCause = "thread.interrupted"

BusyCancelCause is weft.cancel.cause on a turn thread's Interrupt or Rollback busy policy cancelled — a newer message ended it on purpose: SessionRow.EndedTurns, never FailedTurns.

View Source
const ChildRunsCap = 10_000

ChildRunsCap bounds one TreeReader.ChildRuns call: 10,000 child rows in all, over every parent it names. A call whose parents hold more answers ErrChildRunsCapped and no map: the caller reads those parents' children as not known, never as a part.

View Source
const ErrorTypeCancelled = "cancelled"

ErrorTypeCancelled is the run row's error_type for a run whose invoke_agent span names a context error (context.Canceled, context.DeadlineExceeded): the run's context ended — a caller's cancel, a deadline, a session interrupt — which the row reads as one kind, the cancel cause (cancel_cause) naming which.

View Source
const InterruptedAfter = 30 * time.Second

InterruptedAfter is the "last seen" age that turns a non-terminal run interrupted: three missed heartbeats.

View Source
const LogSkew = time.Minute

LogSkew widens a run's time window for the app-log scan (the otel_logs read on ClickHouse, the same bound on SQLite): clock skew between the emitters of one run.

View Source
const MaxErrorClassBytes = 240

MaxErrorClassBytes caps an error class: a longer one is cut at a rune boundary and ends in "…" — which keeps every class inside the search filters' MaxSearchTermBytes, so it is usable as a filter value.

View Source
const MaxErrorTexts = 2000

MaxErrorTexts bounds the distinct error texts ErrorCounts reads for one facet or class filter: past it the rarest texts are not read, and the answer says capped.

View Source
const MaxGaps = 1000

MaxGaps bounds EventPage.Gaps: positions are the sender's numbers, and one stray high position must not turn a page read into a list of every position below it.

View Source
const MaxLogCandidates = 10_000

MaxLogCandidates caps how many log records one DB.OtherLogs read loads: the first non-weft records of the run's traces in its time window, by time — before attribution, so a subagent's or a sibling run's lines in the same trace (and duplicates) count against it. A run whose traces hold more reads LogPage.Truncated.

View Source
const MaxOutcomes = 40

MaxOutcomes bounds SessionRow.Outcomes: a strip, not the turn list.

View Source
const MaxSessionEventLimit = 1000

MaxSessionEventLimit bounds one SessionEvents page.

View Source
const MaxTies = 500

MaxTies bounds how far a page may run past its Limit to finish a group of rows sharing the cursor's time (RunPage, SessionPage): one page never grows unbounded however many rows share one timestamp.

View Source
const PromptPreviewRunes = 200

PromptPreviewRunes caps RunRow.PromptPreview.

View Source
const RecordRaw = "raw"

RecordRaw is the raw record kind (ADR 0028 §12, plan A6): one model call attempt's wire exchange — the provider request and response a first-party adapter sent and received — emitted only under otel.RawBodies(true) with WEFT_ENV=dev (or otel.DevMode()), to the Local and Studio destinations. It is positioned by weft.raw.index and stored beside the request records; DB.Requests attaches it to its attempt's row (RequestRecord.Wire).

View Source
const SessionForkedFromKey = "weft.session.forked_from"

SessionForkedFromKey is the run metadata a forked thread session's turns carry: <origin session>#<entry> (SessionRow.ForkedFrom).

View Source
const SessionTitleKey = "weft.session.title"

SessionTitleKey is the run metadata a thread session stamps with its current title (docs/record-contract.md §2): it lands in RunRow.Meta and is read back as SessionRow.Title.

View Source
const StockErrNoDescription = "the root span's status is Error (no description)"

StockErrNoDescription is a failed derived run's error when its root span's status is Error with no description (§9): a failed run never reads an empty error.

View Source
const SweepBatch = 500

SweepBatch bounds one pass of DB.Sweep: at most this many root runs (each with its subtree of not-running children) are deleted per writer transaction on SQLite, per round of DELETE mutations on ClickHouse. Between passes the writer is free: a sweep of a large backlog (the first sweep of `weft studio --retain` on an old database) interleaves with ingest instead of blocking it.

View Source
const TreeCutParent = "\x00cut"

TreeCutParent is the parent a backend maps a run to in the parents it hands TreeMembers (and TreeReadIDs, CutRunTree, CapTree) when its read found the run but may not show it — a run past the backend's retention cut. No run has it, so the run is no member and neither is a run whose stored parent it is; and the id rule stops at it, so an id-shaped descendant with no stored parent stays out with it rather than joining the nearest member above (TreeMembers). A NUL byte leads it: no run id holds one.

View Source
const TreeMissingCap = 100

TreeMissingCap bounds RunTree.Missing: the first 100 by start, the whole count in RunTree.MissingCount.

View Source
const TreeRowCap = 10_000

TreeRowCap bounds the run rows one RunTreeSpans read takes of a weft run's tree: 10,000, in RunRow order. A tree of more runs is read that far and says so (RunTree.Truncated, as past TreeSpanCap): the runs past it, and their spans, are not in the answer.

View Source
const TreeSpanCap = 10_000

TreeSpanCap bounds the spans one RunTreeSpans answer holds: 10,000, the first in start order. A tree with more is cut there and says so (RunTree.Truncated; Studio's truncated hole, cause response_cap): each child run's own tree is a read away.

View Source
const TreeTraceCap = 50_000

TreeTraceCap bounds the spans a derived run's tree (§9 of docs/record-contract.md: its spans carry no run id) reads of its trace before it cuts the run's subtree out: 50,000, the first in start order. A trace with more is read that far and says so (RunTree.TraceTruncated). The subtree is cut from what was read, then capped at TreeSpanCap, so a run late in a large trace keeps its own spans while they fall inside the read.

Variables

View Source
var ErrChildRunsCapped = errors.New("obsdb: more child runs than ChildRunsCap")

ErrChildRunsCapped is ChildRuns' answer past ChildRunsCap.

View Source
var ErrClosed = errors.New("obsdb: closed")

ErrClosed is returned by a backend used after Close.

View Source
var ErrInvalidCommand = errors.New("obsdb: invalid command store row")

ErrInvalidCommand is returned by a CommandStore write whose row lacks its key, or whose command State is not one of the five.

View Source
var ErrInvalidManifest = errors.New("obsdb: invalid manifest")

ErrInvalidManifest is PutManifest's refusal of a row CheckStoredManifest rejects.

View Source
var ErrInvalidSearch = errors.New("obsdb: invalid search")

ErrInvalidSearch is returned by ParseSearch and Search for a query outside the grammar or its bounds; Studio answers it 400.

View Source
var ErrInvalidStats = errors.New("obsdb: invalid stats query")

ErrInvalidStats is returned by CheckStats and Stats for a query outside its bounds; Studio answers it 400.

View Source
var ErrInvalidStatus = errors.New("status must be running, succeeded, failed, interrupted, parked or cancelled")

ErrInvalidStatus is ParseStatus's error: the word is not one of StatusWords. Its message names them.

View Source
var ErrNotFound = errors.New("obsdb: not found")

ErrNotFound is returned by Run, Session, Events, Transcript, RunSpans, OtherLogs, Requests, Prompt, Tools, Catalogs, Experiment, DeleteRun, DeleteSession and ResolvePublicID for an id the database does not hold (Prompt and Tools also for a hash, possibly as a *HoleError). Trace answers an unknown trace with no spans instead: a trace is only ever the spans that arrived.

View Source
var ErrStepMessages = errors.New("obsdb: the records do not rebuild the step's messages")

ErrStepMessages is matched (errors.Is) by every *StepMessagesError: the stored records do not rebuild the messages a request names.

StatusWords is every word RunQuery.Status takes, in the order an error names them: the four derived statuses, then the read-time words.

Functions

func ApplyView added in v0.13.0

func ApplyView(transcript []core.Message, v Compaction) ([]core.Message, error)

ApplyView rebuilds the messages a compacted request carried (ADR 0028 §8): transcript — the growth records below the view's index, concatenated — with the range [FromSeq, ToSeq) replaced by the view's messages. A range that does not fit the transcript, or a body that is not messages, is an error.

func AttachRaw added in v0.14.0

func AttachRaw(reqs []RequestRecord, raws []StoredRecord)

AttachRaw sets each request record's Wire to the raw record of its attempt — the same step and attempt — and leaves it nil where the run holds none (RawBodies off, a third-party adapter that does not report, a run written before the raw record). Where an SDK's own transport retries put several exchanges on one attempt, the last (the highest weft.raw.index) is the attempt's. raws are stored raw records in any order; every backend answers through it.

func AttemptIndex added in v0.19.0

func AttemptIndex(attrs map[string]any) (int64, bool)

AttemptIndex is an attempt span's weft.attempt.index as every aggregate reads it: an integer (a whole float64 is one — JSON decodes numbers so), or a string of decimal digits, the spelling a stock collector's stringified attributes carry. Anything else is no index (ok false). SQLite's span_stats derivation and ClickHouse's statsSpans restate the rule; the conformance suite's Stats case pins all three.

func AttrTrue added in v0.18.0

func AttrTrue(m map[string]any, k string) bool

AttrTrue reports a boolean span or record attribute set true: a bool true, or the string "true" a store that stringifies attributes reads back (ClickHouse's Map(String, String) behind a stock collector). A reader of a weft flag — weft.tool.denied, weft.tool.approved — takes it through here, so both spellings read the same.

Example

A status filter as the read API takes it (I3): the four derived statuses and the read-time parked and cancelled are filters; any other word is an error naming them, never an empty list. A read-time word narrows its derived status (Stored) with a condition of its own. A weft flag read the same in both spellings: the bool a fixture or the OTLP wire carries, and the string "true" a store that stringifies attributes (ClickHouse behind a stock collector) reads back.

package main

import (
	"fmt"

	"github.com/weftgo/weft/obsdb"
)

func main() {
	fmt.Println(
		obsdb.AttrTrue(map[string]any{"weft.tool.denied": true}, "weft.tool.denied"),
		obsdb.AttrTrue(map[string]any{"weft.tool.denied": "true"}, "weft.tool.denied"),
		obsdb.AttrTrue(map[string]any{}, "weft.tool.denied"),
	)
}
Output:
true true false

func CheckSearch added in v0.14.0

func CheckSearch(q SearchQuery) ([]string, error)

CheckSearch validates the parts of a SearchQuery a backend cannot read safely: the sort, the offset and the text's terms (returned).

func CheckStats added in v0.19.0

func CheckStats(q StatsQuery) (buckets int, width time.Duration, err error)

CheckStats validates a StatsQuery: a known By, a window, a bucket count in bounds. It returns the bucket count and width — the window cut into Buckets equal parts, the width rounded up to a whole nanosecond so the last bucket ends at or after Until.

func CheckStoredCommand added in v0.15.0

func CheckStoredCommand(c StoredCommand) error

CheckStoredCommand is the validation every backend's PutCommand runs: an ID and one of the five states.

func CheckStoredManifest added in v0.16.0

func CheckStoredManifest(m StoredManifest) error

CheckStoredManifest is the rule both backends apply before a write: the body is a JSON object and the hash is its sha256 (ManifestHash).

func CheckToolArgs added in v0.13.0

func CheckToolArgs(tool string, schema, args json.RawMessage) error

CheckToolArgs checks a tool call's arguments against the tool's input schema as a tools record stores it (ToolEntry.Schema): the check a playground `tool_args` transcript edit passes before it is accepted (ADR 0029 §8), on both of its surfaces — weft/runtime's authoritative copy and Studio's mirror — so the two refuse in one wording. A nil error means the arguments fit.

The failure is a *core.ToolError coded INVALID_INPUT whose cause is core.ErrInvalidToolInput, worded like the loop where the loop has a sentence for it — `INVALID_INPUT: tool "lookup_order": field "days": expected integer, got string` — naming the field by its JSON path (dotted; an array element by its index). The loop's decode does not check required fields or enums; their sentences ("missing required field …", "… is not one of the schema's values") are this check's own. A null field is a no-op, as encoding/json decodes it; an integer must be written as one (3.0 is refused, as the loop refuses it).

The arguments must be one JSON object (empty or null reads as {}, as the loop decodes them). The schema walk covers the vocabulary weft's reflector and a hand-written schema share: type (one or a list), properties, required, additionalProperties (false refuses an undeclared field; a schema checks each extra one), items and enum. Any other keyword is not checked. An empty or unreadable schema checks the object shape alone.

Example

A playground tool_args edit is checked against the schema the run's tools record stored before the replay is accepted: a wrong type is refused in the loop's own words, naming the field.

package main

import (
	"encoding/json"
	"fmt"

	"github.com/weftgo/weft/obsdb"
)

func main() {
	schema := json.RawMessage(`{"type":"object","properties":{"order_id":{"type":"string"}},"required":["order_id"]}`)
	fmt.Println(obsdb.CheckToolArgs("lookup_order", schema, json.RawMessage(`{"order_id":"7"}`)))
	fmt.Println(obsdb.CheckToolArgs("lookup_order", schema, json.RawMessage(`{"order_id":7}`)))
}
Output:
<nil>
INVALID_INPUT: tool "lookup_order": field "order_id": expected string, got number

func CommandTerminal added in v0.15.0

func CommandTerminal(state string) bool

CommandTerminal reports whether state is terminal — rejected, finished or lost: nothing acts on the command any more, and PruneCommands may delete it.

func DedupTranscript

func DedupTranscript(bodies []json.RawMessage) []json.RawMessage

DedupTranscript makes a run's messages bodies concatenate to the transcript the run actually held. bodies are the messages records' bodies in index order, as a backend's Transcript reads them; every backend's Transcript returns its result through this.

One stored shape needs it: a run recorded before ADR 0028 §8's partial-resume shape. A resume whose input already holds a partial tool message (some calls of the step answered, the rest parked) was recorded with that input as fed — index 0 — and then the tool message it rebuilt over the partial one, at index 1. Concatenated naively the two versions of the one message sit side by side; the later record is the authoritative one. The current core never writes that shape (its input record stops at the last assistant message with tool calls and the rebuilt message is the next growth record, step 0); this stays for runs stored in the old one. So when the second body is a lone tool message and the first holds, directly after its last assistant message with tool calls, a tool message whose results the lone one repeats, the partial message is dropped from the first body. Every other message stays byte-for-byte, the number of bodies never changes (a body's place is still its record's index), and any other shape is returned untouched.

func DeriveSettling added in v0.16.0

func DeriveSettling(rootEnded, finishRecorded bool, lastSeen, now time.Time) bool

DeriveSettling is RunRow.Settling's rule: the invoke_agent span is stored (rootEnded), the run_finish record is not (finishRecorded false), and now - lastSeen ≤ InterruptedAfter — the window in which a destination that stored the root before the records behind it (a log exporter's retry or backlog; Studio's destination batches logs faster than spans, but a user's Exporters pair need not) may still deliver them. Pass rootEnded false for a derived run — stock spans have no records to wait for — and for a failed one: a failed root is terminal, its event page done at once (EventPage.Done), so the row never says "settling" while the page says done. A backend derives it at read time beside DeriveStatus, with the same clock.

func ErrorClass added in v0.19.0

func ErrorClass(err string) string

ErrorClass normalises an error text into its class. The text is read as words — runs of letters, digits and _ . - — and the rest kept as it stands:

  • a word that is a number (digits with optional . or , groups and a sign) becomes N, one with a unit suffix (ms, s, KiB, %, …) N followed by the unit: "input 30/25, output 15/0" reads "input N/N, output N/N", "after 1.5s" "after Ns";
  • a word that is an id becomes <id>: a UUID, a hex string of 8 or more characters with a digit (a run id, a trace id), or a word one of whose _ / - separated parts is 8 or more letters and digits mixed (call_8f3a9b2c11, toolu_01ABCdefGH);
  • any other word stays — a model name like glm-5.3-flash keeps its digits;
  • a double-quoted or backquoted span is kept verbatim, quotes included (a tool's name in %q), an unclosed quote to the end.

The class is trimmed and cut at MaxErrorClassBytes. "" for "".

Example

ErrorClass reads an error text as its class: numbers N, ids <id>, quoted strings and names kept.

package main

import (
	"fmt"

	"github.com/weftgo/weft/obsdb"
)

func main() {
	fmt.Println(obsdb.ErrorClass("weft: run failed at step 2: weft: run exceeded its usage limit (input 30/25, output 15/0)"))
	fmt.Println(obsdb.ErrorClass(`tool "lookup_42" retried 3 times, past MaxModelRetries(2)`))
}
Output:
weft: run failed at step N: weft: run exceeded its usage limit (input N/N, output N/N)
tool "lookup_42" retried N times, past MaxModelRetries(N)

func ErrorClassTexts added in v0.19.0

func ErrorClassTexts(counts []ErrorCount, class string) []string

ErrorClassTexts is the texts among counts whose class is class, in counts' order: RunQuery.Errs for an error_class filter. Never nil — no text of the class is an empty set, which matches no run.

func EventsLostHole added in v0.16.0

func EventsLostHole(r RunRow) (Hole, HoleCause)

EventsLostHole is the counted gap (U5): a weft run whose root span is stored and says it emitted EventsEmitted event records, of which the store holds fewer — HoleGap with CauseEventsLost, worded by EventsLostReason. "" while the run settles (its records may still land), for a derived run, a run whose root is not stored or carries no count, and a run whose events all arrived.

func EventsLostReason added in v0.16.0

func EventsLostReason(r RunRow, below int) string

EventsLostReason words a counted gap (EventsLostHole) with its numbers: "N of M events were lost", and how many of them are the run's tail — positions above the highest one stored, which no counter-hole reader can see. below is the number of positions missing below the highest stored one (EventPage.Gaps' length; a reader that did not walk them passes -1 and the split is omitted). "" when nothing is lost.

func ExplainMissing added in v0.10.0

func ExplainMissing(ctx context.Context, db DB, runID, kind, hash string) error

ExplainMissing is the error DB.Prompt and DB.Tools return for a hash the run holds no record of (kind "prompt" or "tools"): a *HoleError when the reason is known (HoleError's table), a plain ErrNotFound otherwise, and the read's own error when the run or its requests cannot be read. Every backend answers through it, so both say the same thing for the same records.

func HoleNote added in v0.10.0

func HoleNote(h Hole) (reason, fix string)

HoleNote is a badge's one-line reason and, where one exists, its one-line fix ("" when the hole has none: a redaction is by design, a derived value has nothing to repair) — ADR 0028 §11's table in words, the one source both Studio surfaces render (its golden, studio/testdata/holes.golden.json, is what Studio's TypeScript table is checked against). A reader that knows more (a step's own gap, a run built from spans) says so in its own reason and keeps this fix. An unknown hole has neither.

func HoleNoteFor added in v0.10.0

func HoleNoteFor(h Hole, cause HoleCause) (reason, fix string)

HoleNoteFor is HoleNote for a hole whose cause the reader knows: the cause's own reason and fix where the table words it, HoleNote(h) otherwise (CauseDefault included). The one table for every surface (run page, panel, export).

func IsSessionEventKind added in v0.18.0

func IsSessionEventKind(kind string) bool

IsSessionEventKind reports whether a record of weft.record kind feeds the session_events index: thread's state log and its compaction marker. A backend indexes such a record when it carries a session id.

func IsStockRoot added in v0.15.0

func IsStockRoot(s Span) bool

IsStockRoot reports whether a span opens a derived run on its own (§9): an invoke_agent span (by gen_ai.operation.name, or without it by a name starting with invoke_agent) with no weft.run.id and a gen_ai.conversation.id — the run's session. An invoke_agent span without a conversation id opens a run (with no session) only when no other invoke_agent span is stored above it in its trace — a top-level agent — which DeriveStockRuns decides from the trace; a nested one folds into the enclosing run.

func LastAssistantText added in v0.14.0

func LastAssistantText(body []byte) (string, bool)

LastAssistantText is the text of a body's last assistant message with non-empty text: the run's answer when the body is its latest messages record holding one.

func LimitOf

func LimitOf(n int) int

LimitOf normalizes a query limit: 0 means the default 50, values above 500 clamp to 500, negatives read as the default.

func LostRecordsHole added in v0.16.0

func LostRecordsHole(r RunRow, hasSpans bool) (Hole, HoleCause)

LostRecordsHole is the hole of a weft run whose spans were stored and none of whose records was (no event, messages or request record). When the root says the core emitted records (EventsEmitted or MessagesEmitted > 0, U5) every one of them was lost on the way: HoleGap with CauseEventsLost. Otherwise — the root says it emitted none, or predates the count — the app exported spans only: not_recorded with CauseNoLogger. "" for any other run: one with a record, one with no span (hasSpans false: the reader's, from DB.RunSpans), a running one (its records may be in flight), a settling one (RunRow.Settling: its root span was stored moments ago and its records may still be on the way — decided once InterruptedAfter has passed) and a derived run (RunRow.Derived: stock spans are no weft app, its badge is derived's own, docs/record-contract.md §9).

func ManifestHash added in v0.16.0

func ManifestHash(body []byte) string

ManifestHash is the hash a manifest body is stored under: sha256 of its bytes, lowercase hex — the core's weft.manifest.hash of the same bytes.

func MessagesLostHole added in v0.16.0

func MessagesLostHole(r RunRow) (Hole, HoleCause)

MessagesLostHole is the counted transcript gap: a weft run whose root span is stored and says it emitted MessagesEmitted growth messages records (weft.run.messages), of which the store holds some but fewer — HoleGap with CauseMessagesLost, worded by MessagesLostReason. Some, not none: a content-off destination stores no messages record by design (docs/record-contract.md §5) and nothing on the row says a destination was content-off, so a store holding none is never called lost — a content-on destination that lost every messages record reads like a content-off one (§7 says so). "" while the run settles, for a derived run, a run whose root is not stored or carries no count, and a run whose transcript batches all arrived.

func MessagesLostReason added in v0.16.0

func MessagesLostReason(r RunRow) string

MessagesLostReason words a counted transcript gap (MessagesLostHole) with its numbers: "K of N transcript batches were lost: …". "" when MessagesLostHole raises none.

func MetaOf

func MetaOf(attrs map[string]any) map[string]string

MetaOf returns the caller metadata a set of attributes carries: every string value under a key outside the record/span contract (what core.Metadata put there, enduser.id included). Keys under "weft." that are not part of the contract — thread's weft.session.parent, a future module's own — are metadata and are kept. An empty string is "none" (core.Metadata's rule: a session stamps its own identity keys, "" when it has none) and is never kept: a weft.session.parent = "" names no parent.

func MetadataCapReason added in v0.16.0

func MetadataCapReason(n int64) string

MetadataCapReason is the metadata_cap hole's reason with the count: "N metadata pairs dropped (limit 64 keys, 128-byte keys, 1 KiB values)".

func NonMetaKeys added in v0.16.0

func NonMetaKeys() []string

NonMetaKeys lists, sorted, the record/span contract keys MetaOf excludes from a run row's metadata — for a backend that must copy the set into its own query language (the ClickHouse views' filter) and test its copy whole.

func ParseSearch added in v0.14.0

func ParseSearch(text string) ([]string, error)

ParseSearch splits search text into its terms: whitespace separates terms, and a double-quoted span is one term, spaces kept (a phrase: "refund failed" matches those two words adjacent, in that order). An unclosed quote runs to the end of the text; an empty quoted span is no term. Terms are matched case-insensitively as substrings — no stemming, no operators: AND, OR and * are ordinary text. More than MaxSearchTerms terms, or a term longer than MaxSearchTermBytes, is ErrInvalidSearch.

func ParseSeverity added in v0.10.0

func ParseSeverity(s string) (n int, ok bool)

ParseSeverity reads a severity filter: a level name (trace, debug, info, warn or warning, error, fatal — any case, the range's first number), a SeverityText spelling ("WARN2"), or a number 1–24. ok is false for anything else.

func ParseStockRunID added in v0.15.0

func ParseStockRunID(id string) (traceID, spanID string, ok bool)

ParseStockRunID splits a derived run id into its trace and root span ids; ok is false for any id not of StockRunID's exact shape — every weft run id among them.

func PercentileRank added in v0.19.0

func PercentileRank(p, n int64) int64

PercentileRank is the 1-based nearest rank of percentile p over n sorted values, ⌈p·n/100⌉ in integers: the rank both backends' SQL reads ((n·p + 99) / 100).

func PromptPreview added in v0.14.0

func PromptPreview(s string) string

PromptPreview cuts a prompt to its first PromptPreviewRunes runes — the cut both backends make in SQL (substr, leftUTF8), here for a writer that holds the text in Go.

func PromptText added in v0.14.0

func PromptText(body []byte, input bool) (string, bool)

PromptText is the turn's own prompt as one messages record body holds it. The input record (weft.messages.input: what the run was fed — on a continued conversation the whole history, the turn's question last) gives its last user message with non-empty text; any other record its first. A record with no user text gives none.

A run's prompt (TurnPrompt) is its input record's when that holds one, else the lowest-index record's that does — so a message steered in at a later step is never the prompt — with one exception, the resume: an input record that ends at an assistant message carrying tool calls was cut there by the core (core/loop.go), and what the run was fed after that message — the rebuilt tool message, then a new prompt if the resume carried one — is the next growth record (index + 1). Its user text before its first assistant message is the turn's prompt when there is any; a plain resume (Approve or Deny, no new prompt) keeps the input record's.

An accepted edge of the resume rule: on a plain Approve or Deny resume, a user message steered in before the resumed batch's first assistant reply lands in the same next record, before that reply, and is indexed as the prompt. It is structurally indistinguishable from a resume carrying a new prompt — the transcript carries no steered mark — so the rule cannot tell the two apart.

func RunErrorType added in v0.16.0

func RunErrorType(errorType string) string

RunErrorType is the run row's error_type for an invoke_agent span's error.type: the span's value — max_steps, usage_limit, loop_detected, model_retries_exceeded, model_contract, model_error, … (the core's closed table, record contract §3) — with the two context errors read as ErrorTypeCancelled. "" for "".

func SessionContentOff added in v0.16.0

func SessionContentOff(first RunRow) bool

SessionContentOff is SessionRow.ContentOff's rule over the session's first turn, a run row with its Status and Settling derived (DeriveStatus, DeriveSettling): events stored, not one messages record, not Derived, and over — not running, not settling, whose messages records may still arrive. Every backend and Studio's scoped session row read it; the web client's lib/title.ts contentOff is its twin.

func SessionEventKind added in v0.18.0

func SessionEventKind(e SessionEvent) string

SessionEventKind is the short kind a session event names: the event name past weft.thread. for a thread record (approval.decision, receipt, …), "compaction" for a marker, the event name itself else.

func SessionEventName added in v0.18.0

func SessionEventName(kind, eventName string, body []byte) string

SessionEventName is a session event's event name: the record's own when it carried one, else the one its kind and body imply — weft.compaction for a marker, weft.thread.<body kind> for a thread record ("" when the body names no kind). Every backend derives it so.

func SessionTurnFacts added in v0.18.0

func SessionTurnFacts(row *SessionRow, turns []RunRow)

SessionTurnFacts sets row's turn facts (Pending, FailedTurns, EndedTurns, CancelledTurns, Resumes, Outcomes, ForkedFrom) from its turns, given in any order: they are read by turn, then start, then id, the newest last. Every backend and Studio's scoped session row read it; the SessionQuery filters state the same rule in SQL.

Example

A session's state over time (U16): a busy-policy interrupt is ended, and another cancel cancelled (U16-fix) — neither a failure; a decided park resumed; the newest turn's open park is the session's Pending — it awaits a decision.

package main

import (
	"fmt"

	"github.com/weftgo/weft/obsdb"
)

func main() {
	turns := []obsdb.RunRow{
		{ID: "s-t1", Turn: 1, Status: obsdb.StatusSucceeded, Pending: 1, ResumedBy: "s-t2"},
		{ID: "s-t2", Turn: 2, Status: obsdb.StatusSucceeded, ResumedFrom: "s-t1"},
		{ID: "s-t3", Turn: 3, Status: obsdb.StatusFailed, ErrorType: obsdb.ErrorTypeCancelled, CancelCause: obsdb.BusyCancelCause},
		{ID: "s-t4", Turn: 4, Status: obsdb.StatusFailed, ErrorType: "model_error"},
		{ID: "s-t5", Turn: 5, Status: obsdb.StatusFailed, ErrorType: obsdb.ErrorTypeCancelled, CancelCause: "thread.closed"},
		{ID: "s-t6", Turn: 6, Status: obsdb.StatusSucceeded, Pending: 2},
	}
	var row obsdb.SessionRow
	obsdb.SessionTurnFacts(&row, turns)
	fmt.Println(row.Outcomes)
	fmt.Println("pending", row.Pending, "failed", row.FailedTurns, "ended", row.EndedTurns,
		"cancelled", row.CancelledTurns, "resumes", row.Resumes)
}
Output:
[resumed succeeded ended failed cancelled parked]
pending 2 failed 1 ended 1 cancelled 1 resumes 1

func SeverityText added in v0.10.0

func SeverityText(n int) string

SeverityText is an OTLP severity number's short name: "TRACE", "DEBUG", "INFO", "WARN", "ERROR" or "FATAL" for the first number of each range, with the range's ordinal after it for the rest ("WARN2" is 14); "" for 0 (unspecified) and anything outside 1–24.

func SortCompactions added in v0.10.0

func SortCompactions(cs []Compaction)

SortCompactions orders DB.Compactions' answer: the views by index, then the session markers in the order given (a backend reads them in emission order). A marker is filed under the run that produced the compacted context, so it follows that run's own views.

func SpanProblems added in v0.16.0

func SpanProblems(s Span) []string

SpanProblems derives the problem tokens one weft span contributes to its run's Problems (record contract §6): an attempt (every one, keyed by its chat span and index, so the fold counts the failed ones a later attempt followed), a fallback layer's answer, an execute_tool span that failed (tool_failed, or denied for error.type DENIED; none for RETRY or a context error), whose result the cap cut or whose arguments a middleware repaired, and an invoke_agent span's weft.run.denied, weft.tap.panics and weft.metadata.dropped. Nil for a span that contributes none. The rules are restated in ClickHouse's traces view (migrations 0017, 0018, 0029) and SQLite's backfills (0016, 0017); the conformance suite pins all three agree.

func StepRange added in v0.14.0

func StepRange(reqs []RequestRecord) (lo, hi int, ok bool)

StepRange is the smallest and largest Step of reqs, for a backend's raw-record read; ok is false for an empty page.

func StockCandidate added in v0.15.0

func StockCandidate(s Span) bool

StockCandidate reports whether a span may take part in a derived run: it carries no weft.run.id (weft's own rows always win) and is one of the GenAI operations a derivation reads (invoke_agent, chat, execute_tool, by stockOp's rule). The writers re-derive a trace when a batch brings one: a trace of plain HTTP spans costs no read. A generic span between a root and its calls is part of the run's tree all the same; it arrives, like every child, before the root it sits under ends, and the root's batch re-derives the trace.

func StockHasMessages added in v0.20.0

func StockHasMessages(attrs map[string]any) bool

StockHasMessages reports whether a chat span carries the conventions' message content (docs/record-contract.md §9): any key of the current family or the legacy ones. A derived run's MessageCount counts its chat spans that do.

func StockNotRun added in v0.20.0

func StockNotRun(spans []Span) (reason, fix string)

StockNotRun says why a trace's spans make no run (§9): "" when any span carries weft.run.id (a weft run) or a derived run comes of them (DeriveStockRuns); else notRunParentMissing when a GenAI span's parent span is not stored (its agent span may still be on the way), notRunChatOnly when GenAI chat or execute_tool spans are there with no invoke_agent span to open a run, and notRunNoGenAI when no span is a GenAI operation. fix is the reason's.

func StockRunID added in v0.15.0

func StockRunID(traceID, spanID string) string

StockRunID is the id of the run derived from the stock root span spanID of trace traceID: otel-<trace id>-<root span id>, lowercase hex — deterministic from the spans, so a retried export lands on the same row. It never contains "/" and cannot collide with an id the core generates (32 hex digits, a thread turn's -t<n>, a Subagent child's /); a caller-chosen core.RunID must not use the prefix (the record contract's §2 says so; nothing enforces it).

func StockRunPrefix added in v0.20.0

func StockRunPrefix(traceID string) string

StockRunPrefix is the prefix every derived run id of a trace starts with (StockRunID): "otel-<trace id>-", lowercase. A store keyed by run id reads a trace's derived rows by this prefix — its primary key — instead of scanning for the trace id.

func StockRunsOf added in v0.15.0

func StockRunsOf(all []SpanLink, keys []SpanKey, deleted map[string]bool) []string

StockRunsOf is the derived run ids (StockRunID) of the deleted span keys that are non-weft spans (no run id in all) and not already in deleted: the derived runs whose root a deletion takes — a stock agent under a deleted weft run's tool, whose row is no child of it (§9: the walk stops at a weft span) yet would outlive every span it was derived from. A key that is no root names no row; deleting by its id deletes nothing.

func StockToolContent added in v0.20.0

func StockToolContent(attrs map[string]any) (args json.RawMessage, result string, hasArgs, hasResult bool)

StockToolContent reads a derived run's execute_tool span's call content (docs/record-contract.md §9): its gen_ai.tool.call.arguments as JSON — a JSON string value as the JSON it holds, any other string as a JSON string, a structured value marshalled — and its gen_ai.tool.call.result as text (a string as is, a structured value as its JSON). hasArgs and hasResult say whether the span carried the attribute: absent, the producer captured no content, and the call's arguments and result stay unknown — never an empty value.

Example

A stock agent with no gen_ai.conversation.id is a run with no session (docs/record-contract.md §9); its tool span's captured content is the call's arguments and result.

const trace = "6a000000000000000000000000000001"
spans := []obsdb.Span{
	{TraceID: trace, SpanID: "6a00000000000001", Name: "invoke_agent planner", Start: stockAt(0), End: stockAt(900),
		Attrs: map[string]any{"gen_ai.operation.name": "invoke_agent", "gen_ai.agent.name": "planner"}},
	{TraceID: trace, SpanID: "6a00000000000002", ParentSpanID: "6a00000000000001", Name: "execute_tool lookup_order",
		Start: stockAt(100), End: stockAt(200),
		Attrs: map[string]any{"gen_ai.operation.name": "execute_tool", "gen_ai.tool.name": "lookup_order",
			"gen_ai.tool.call.arguments": `{"id":"4411"}`, "gen_ai.tool.call.result": `{"status":"shipped"}`}},
}
runs := obsdb.DeriveStockRuns(spans)
fmt.Printf("%s session=%q\n", runs[0].Row.ID, runs[0].Row.SessionID)
args, result, _, _ := obsdb.StockToolContent(runs[0].Spans[1].Attrs)
fmt.Println(string(args), "=", result)
Output:
otel-6a000000000000000000000000000001-6a00000000000001 session=""
{"id":"4411"} = {"status":"shipped"}

func StockTraces added in v0.15.0

func StockTraces(ids []string) []string

StockTraces is the trace ids the derived run ids among ids name, in order, once each: a deletion reads a derived run's spans by its trace, since none of them carries a run id (DeletedSpans).

func TailLostHole added in v0.16.0

func TailLostHole(r RunRow) (Hole, HoleCause)

TailLostHole is the hole of a weft run whose event records a destination dropped, read from its row alone. Where the root span carries the count (EventsEmitted > 0, U5) the rule is the count's: fewer events stored than emitted is HoleGap with CauseEventsLost (EventsLostHole) — a lost tail, a lost middle, a failed run's lost records alike. Where it does not (a weft before U5), the old reading: its invoke_agent span is stored (RootEnded, an Ok or Unset root — the run reads succeeded), some of its events are (EventCount > 0) and its run_finish is not (FinishRecorded false) — HoleGap with CauseTailLost. "" for any other run: a derived one, a settling one (RunRow.Settling: its tail may still be on the way — the gap is raised once InterruptedAfter has passed), one whose events all arrived, and one with no record at all (LostRecordsHole's).

func TranscriptBodies added in v0.10.0

func TranscriptBodies(batches []TranscriptBatch) []json.RawMessage

TranscriptBodies returns the batches' bodies in order — Transcript's answer, for a backend that reads both through one query.

func TreeChildID added in v0.17.0

func TreeChildID(parent, id string) bool

TreeChildID reports whether id has the shape core gives a child run of parent (core's childRunID): parent + "/" + a step (one or more ASCII digits) or "resume", then "/" + a non-empty call id.

func TreeIDRule added in v0.17.0

func TreeIDRule(runID, id string, known func(string) bool) (parent string, ok bool)

TreeIDRule is the parent core's id rule gives id inside runID's subtree: a Subagent child's id is <parent>/<step>/<call id> (and <parent>/resume/<call id> under Approve), so an id of that shape under runID (TreeChildID: the step all digits or "resume", a non-empty call id) descends from runID; an app's own hierarchical id ("job/1", or "job/1/retry/2" under "job/1") is not of it. The parent is the nearest run above id that known holds and of which id has the shape — id trimmed at each "/" from the end, so a call id holding a "/" of its own still finds its run — else runID. ok is false for an id outside the subtree (runID itself included). The rule is a fallback for a run no row or span names a parent of (TreeMembers): a stored parent always wins over the id.

func TreeMemberIDs added in v0.17.0

func TreeMemberIDs(runID string, parents map[string]string) []string

TreeMemberIDs is TreeMembers' run ids, sorted: the ids a backend reads the tree's spans by.

func TreeMembers added in v0.17.0

func TreeMembers(runID string, parents map[string]string) map[string]string

TreeMembers is runID's subtree from what a backend knows before it reads a span: parents maps each run it found — the rows its parent-chain and id-rule reads returned, the trace's runs with no row (their spans' weft.parent.run.id, "" when none) — to its parent. A run is in the subtree when its parent chain reaches runID, or, when no row or span names its parent (mapped to ""), when the id rule names it a descendant (TreeIDRule; its parent then the rule's): a stored parent outside the subtree wins over the id. The rule's parent is a member, never merely a run the backend found: the nearest run above the id that is (or becomes) a member, so an app's own "j/1" (no member: not of core's shape, no stored parent) leaves "j/1/0/c" to j — whichever superset of rows a backend read. The answer maps every member, runID included, to its parent; a parent chain that loops (never written by the core) ends where it repeats. A run mapped to TreeCutParent is found but cut: never a member, and the id rule stops at it — a descendant whose nearest run is a cut one stays out, never re-attached to a member above it.

Example

A run tree's members from the rows alone: the parent chain first, then core's id rule for a run no chain places — here a grandchild whose run_start was lost (its row names no parent) and a nested turn whose parent chain leads elsewhere. A backend reads the members' spans by run id after this.

package main

import (
	"fmt"

	"github.com/weftgo/weft/obsdb"
)

func main() {
	parents := map[string]string{
		"d":             "",
		"d/0/c1":        "d",
		"d/0/c1/1/c2":   "", // a row with no parent: the id rule places it
		"nested_turn":   "d/0/c1",
		"sibling":       "",
		"elsewhere/0/x": "elsewhere",
	}
	for _, id := range obsdb.TreeMemberIDs("d", parents) {
		p, _ := obsdb.TreeIDRule("d", id, func(s string) bool { _, ok := parents[s]; return ok })
		fmt.Printf("%s (parent %q, id rule %q)\n", id, obsdb.TreeMembers("d", parents)[id], p)
	}
}
Output:
d (parent "", id rule "")
d/0/c1 (parent "d", id rule "d")
d/0/c1/1/c2 (parent "d/0/c1", id rule "d/0/c1")
nested_turn (parent "d/0/c1", id rule "")

func TreeReadIDs added in v0.17.0

func TreeReadIDs(q TreeQuery, parents map[string]string, rows []TreeRun) []string

TreeReadIDs is the run ids a backend reads a weft run tree's spans by: TreeMembers' ids, sorted, and under a scope (TreeQuery.PublicID, Meta) only the members the rows show visible — the root, and each run whose row carries the scope under a visible parent — so a run the scope leaves out is never read and never takes a visible run's place under the read's limit. A member with no row is read: its spans decide (CapTree).

func TurnPrompt added in v0.14.0

func TurnPrompt(input, next []byte) (string, bool)

TurnPrompt is a run's prompt from its input record and the record at the next index (nil when none is stored): PromptText's resume rule over the pair — the next record's first user text before its first assistant message when the input record ends at an assistant message with tool calls, else the input record's last user text. A user message steered in before the first assistant reply of a plain resume's batch reads as that resume's new prompt (PromptText names the edge: the transcript carries no steered mark).

Types

type Aggregator added in v0.19.0

type Aggregator interface {
	// Stats answers StatsQuery: one StatsGroup per key of q.By over the
	// runs started in [q.Since, q.Until) that q.RunQuery's filters keep,
	// with a per-bucket series. ErrInvalidStats for a query outside
	// CheckStats' bounds.
	Stats(ctx context.Context, q StatsQuery) (StatsPage, error)
	// ErrorCounts counts the distinct error texts (RunRow.Err, never
	// "") of the runs q matches — Searcher.Search's match exactly, its
	// sort, offset and limit not read — most frequent first (ties by
	// text), at most MaxErrorTexts of them; capped is true when the
	// match held more distinct texts than that. ErrorClasses folds the
	// answer into the error_class facet, ErrorClassTexts resolves one
	// class back to its texts (RunQuery.Errs).
	ErrorCounts(ctx context.Context, q SearchQuery) (counts []ErrorCount, capped bool, err error)
}

Aggregator is the optional aggregates capability of a DB (UX plan D7, A4, A2): numbers across runs rather than one run's records. Both first-party backends implement it — obsdb/sqlite over its span_stats rollup (a derived index written at ingest beside every stored weft span, rebuildable from the spans table: migration 0023), obsdb/clickhouse with native GROUP BY over otel_traces and weft_runs, no table of its own — and the obsdbtest conformance table holds them to the same answers. A backend that does not implement it is a database without aggregates: Studio then leaves its stats capability off and says why.

Example

An Aggregator answers numbers across runs: here, per agent over a day, from the spans the writer stored — SQLite's span_stats rollup (ClickHouse groups otel_traces natively). The error class folds the failures that differ only in their numbers.

package main

import (
	"context"
	"fmt"
	"log"
	"time"

	"github.com/weftgo/weft/obsdb"
	"github.com/weftgo/weft/obsdb/sqlite"
)

func main() {
	db, err := sqlite.Open(":memory:")
	if err != nil {
		log.Fatal(err)
	}
	defer func() { _ = db.Close() }()
	t0 := time.Date(2026, 10, 1, 9, 0, 0, 0, time.UTC)
	run := func(id string, at time.Duration, in int64, failure string) {
		root := map[string]any{"weft.run.id": id, "gen_ai.operation.name": "invoke_agent", "gen_ai.agent.name": "researcher"}
		chat := map[string]any{"weft.run.id": id, "gen_ai.operation.name": "chat", "weft.step.index": int64(0),
			"gen_ai.provider.name": "zai", "gen_ai.request.model": "glm-5.3-flash",
			"gen_ai.usage.input_tokens": in, "gen_ai.usage.output_tokens": int64(5), "weft.ttft_ms": int64(400)}
		status := 1
		if failure != "" {
			status, root["error.type"] = 2, "usage_limit"
		}
		trace := fmt.Sprintf("%032x", at)
		if err := db.Write(context.Background(), obsdb.Batch{Spans: []obsdb.Span{
			{TraceID: trace, SpanID: "0000000000000002", ParentSpanID: "0000000000000001", Name: "chat glm-5.3-flash",
				Start: t0.Add(at), End: t0.Add(at + time.Second), StatusCode: 1, Attrs: chat},
			{TraceID: trace, SpanID: "0000000000000001", Name: "invoke_agent researcher",
				Start: t0.Add(at), End: t0.Add(at + 2*time.Second), StatusCode: status, StatusMessage: failure, Attrs: root},
		}}); err != nil {
			log.Fatal(err)
		}
	}
	run("run_1", 0, 20, "")
	run("run_2", time.Hour, 30, "weft: run exceeded its usage limit (input 30/25, output 5/0)")
	run("run_3", 2*time.Hour, 41, "weft: run exceeded its usage limit (input 41/25, output 5/0)")

	agg := db.(obsdb.Aggregator)
	page, err := agg.Stats(context.Background(), obsdb.StatsQuery{RunQuery: obsdb.RunQuery{ParentRunID: "*"},
		By: obsdb.StatsByAgent, Since: t0, Until: t0.Add(24 * time.Hour)})
	if err != nil {
		log.Fatal(err)
	}
	for _, g := range page.Groups {
		fmt.Printf("%s: %d runs, %d failed, %d input tokens, p50 %gms, TTFT p50 %gms\n",
			g.Key, g.Runs, g.Failed, g.Usage.InputTokens, g.Duration.P50, g.TTFT.P50)
	}
	counts, _, err := agg.ErrorCounts(context.Background(), obsdb.SearchQuery{RunQuery: obsdb.RunQuery{ParentRunID: "*"}})
	if err != nil {
		log.Fatal(err)
	}
	for _, c := range obsdb.ErrorClasses(counts, 0) {
		fmt.Println(c.Count, c.Class)
	}
}
Output:
researcher: 3 runs, 2 failed, 91 input tokens, p50 2000ms, TTFT p50 400ms
2 weft: run exceeded its usage limit (input N/N, output N/N)

type Assembler added in v0.13.0

type Assembler struct {
	// contains filtered or unexported fields
}

Assembler is AssembleStep over one run's growth batches decoded once: each Step call answers exactly what AssembleStep answers for the same records and step, without re-reading every batch (a reader that walks every step of a run — the wefttest fixture export — would otherwise decode the transcript once per step). The messages Step returns share their parts with the Assembler's decoded batches: read them, or copy before patching. Not safe for concurrent use.

func NewAssembler added in v0.13.0

func NewAssembler(batches []TranscriptBatch) *Assembler

NewAssembler decodes batches once for Step.

func (*Assembler) Step added in v0.13.0

func (a *Assembler) Step(requests []RequestRecord, compactions []Compaction, step int) (StepMessages, error)

Step is AssembleStep for one step over the decoded batches.

type Batch

type Batch struct {
	Spans   []Span
	Records []Record
}

Batch is one write: spans and records as ingested, in arrival order. Write is idempotent on (run, record kind, pos) and (trace, span), so a retried transport's duplicate is a no-op.

type BreakpointProvenanceStore added in v0.20.0

type BreakpointProvenanceStore interface {
	// PutBreakpointSet is PutBreakpoints keeping set.SetAt and set.SetBy
	// (a clear — empty Tools — keeps neither).
	PutBreakpointSet(ctx context.Context, runtimeID string, set BreakpointSet) error
	// BreakpointSet is Breakpoints with the stored provenance (zero
	// SetAt, empty SetBy when none was kept). Tools never nil when stored.
	BreakpointSet(ctx context.Context, runtimeID string) (BreakpointSet, bool, error)
}

BreakpointProvenanceStore is the optional breakpoints-with-provenance capability of a CommandStore (U30.2, UX-A-33): the set together with when and through which surface Studio delivered it, so a restarted Studio still says "set by <surface> at <time>". Both first-party backends implement it; the runtime server type-asserts it and falls back to PutBreakpoints / Breakpoints (no provenance) without it. The rows are the same: PutBreakpoints is PutBreakpointSet with Tools alone, and Breakpoints reads a provenance row's tools.

type BreakpointSet added in v0.20.0

type BreakpointSet struct {
	Tools []string
	// SetAt is when Studio delivered the set (zero: not known).
	SetAt time.Time
	// SetBy is the surface the set came through: the Studio page that
	// sent it ("runtimes"), else "api" for a PUT that named none ("":
	// not known).
	SetBy string
}

BreakpointSet is one runtime's breakpoints as the store keeps them (BreakpointProvenanceStore): the tools, and — for a set Studio delivered — when and through which surface it was set. A set adopted from a runtime's registration carries neither.

type CommandQuery added in v0.15.0

type CommandQuery struct {
	ID        string
	RuntimeID string
	RunID     string
	States    []string // any of these states; nil and empty both mean no filter
	Limit     int
}

CommandQuery selects commands. Every set field must match; an empty query matches every row. Limit ≤ 0 answers every match.

type CommandStore added in v0.15.0

type CommandStore interface {
	// PutCommand writes c, replacing any row under c.ID whole (an
	// upsert: the caller writes the row it holds). A zero Created or
	// Updated is stamped with the current time — on every write, so
	// the caller carries the Created it read back into each upsert; a
	// zero one would move the row to the front of the newest-first
	// order. ErrInvalidCommand for an empty ID or a State outside the
	// five.
	PutCommand(ctx context.Context, c StoredCommand) error
	// Commands answers the rows q selects, newest Created first (ties
	// by ID, descending). An empty query answers every row.
	Commands(ctx context.Context, q CommandQuery) ([]StoredCommand, error)
	// PutRoute writes r, replacing any route of r.RunID. ErrInvalidCommand
	// for an empty RunID or RuntimeID. A zero Seen is stamped with the
	// current time on every write.
	PutRoute(ctx context.Context, r RunRoute) error
	// Route answers the route of runID; false when none is stored.
	Route(ctx context.Context, runID string) (RunRoute, bool, error)
	// PutBreakpoints replaces runtimeID's breakpoint set whole; an empty
	// tools stores the empty set (a clear), which Breakpoints then
	// answers as stored. ErrInvalidCommand for an empty runtimeID.
	PutBreakpoints(ctx context.Context, runtimeID string, tools []string) error
	// Breakpoints answers runtimeID's stored set (never nil when
	// stored); false when none was ever stored — distinct from a stored
	// empty set, which a registering runtime's own set must not replace.
	Breakpoints(ctx context.Context, runtimeID string) ([]string, bool, error)
	// PutRuntime writes r, replacing any row under r.ID whole. Zero
	// Registered or LastSeen is stamped with the current time on every
	// write, so the caller carries Registered into each upsert.
	// ErrInvalidCommand for an empty ID.
	PutRuntime(ctx context.Context, r StoredRuntime) error
	// Runtimes answers every stored runtime, most recently seen first
	// (ties by ID).
	Runtimes(ctx context.Context) ([]StoredRuntime, error)
	// DeleteRuntime removes runtimeID's row and its breakpoint set —
	// Studio's "forget" of a runtime that is gone (U20.1). Its commands
	// and routes stay: they are history, and PruneCommands bounds them.
	// An id with nothing stored is no error. ErrInvalidCommand for an
	// empty runtimeID.
	DeleteRuntime(ctx context.Context, runtimeID string) error
	// PruneCommands bounds the store, the in-memory runtime server's
	// rule made durable, and answers how many commands it deleted:
	//
	//   - a terminal command (CommandTerminal) whose Updated is before
	//     before is deleted;
	//   - then, while more than keep commands remain (keep ≤ 0: no
	//     count bound), the
	//     oldest terminal ones by Created go — a queued or accepted
	//     command is never deleted, whatever its age or the count;
	//   - a route goes when it names a command (CommandID) no longer
	//     stored, or when its Seen is before before and no queued or
	//     accepted command names its run;
	//   - a runtime goes when its LastSeen is before before, no queued
	//     or accepted command is addressed to it and its breakpoint set
	//     is absent or empty (that set goes with it);
	//   - an empty breakpoint set no runtime row names goes when it was
	//     last written before before; a non-empty one stays, as a
	//     runtime holding one does.
	//
	// The terminal commands go in bounded passes (SweepBatch rows on
	// SQLite, each its own transaction, ctx checked between them); on
	// an error the passes before it stay deleted and are counted. On
	// ClickHouse the prune is not atomic with concurrent Put* calls —
	// a Put racing it keeps its own version, but each rule decides
	// from what it read — so the caller serialises PruneCommands with
	// its own writes. On ClickHouse the count names the commands a
	// rule selected; a command a racing Put rewrote is counted though
	// it stays.
	PruneCommands(ctx context.Context, before time.Time, keep int) (int, error)
}

CommandStore is the optional playground-link capability of a DB (plan I3): what Studio's runtime server asked the runtimes it links to do, so a restarted Studio still knows. It holds four small sets of mutable rows beside the run tables:

  • commands: every command Studio enqueued to a runtime — its wire body, its state (CommandQueued … CommandLost) and the times a restarted Studio decides "lost" from (AcceptedAt, LastBeat), never re-issuing one;
  • routes: run id → the runtime that started it (an ack named it), what cancel, steer and the debugger's verbs are routed by;
  • breakpoints: each runtime's debugger tool set, replaced whole;
  • runtimes: each registration, so a restarted Studio can list what was registered before it.

Both first-party backends implement it — obsdb/sqlite (migration 0007) and obsdb/clickhouse (migration 0008) — and the obsdbtest conformance table holds them to the same answers. A backend that does not implement it keeps the runtime server in memory only, as before.

These rows are Studio's own bookkeeping, not run data: DB.DeleteRun, DB.DeleteSession, DB.Sweep and DB.Vacuum never touch them; PruneCommands is their retention.

Example

A CommandStore keeps what Studio asked a runtime to do across a restart: the command with its state and the times a restarted Studio decides "lost" from, the route from the run an ack named back to the runtime, and the runtime's breakpoint set.

package main

import (
	"context"
	"fmt"
	"log"
	"time"

	"github.com/weftgo/weft/obsdb"
	"github.com/weftgo/weft/obsdb/sqlite"
)

func main() {
	db, err := sqlite.Open(":memory:")
	if err != nil {
		log.Fatal(err)
	}
	defer func() { _ = db.Close() }()
	cs, ok := db.(obsdb.CommandStore)
	if !ok {
		log.Fatal("this backend keeps commands in memory only")
	}
	ctx := context.Background()
	t0 := time.Unix(0, 1790845923120000000).UTC()

	cmd := obsdb.StoredCommand{ID: "cmd_1", RuntimeID: "rt_1", State: obsdb.CommandQueued,
		Created: t0, Body: []byte(`{"command_id":"cmd_1","agent":"support"}`)}
	if err := cs.PutCommand(ctx, cmd); err != nil {
		log.Fatal(err)
	}
	// The accepted ack names the run: the row moves on, the route is learned.
	cmd.State, cmd.RunID, cmd.Updated, cmd.AcceptedAt = obsdb.CommandAccepted, "run_1", t0.Add(time.Second), t0.Add(time.Second)
	if err := cs.PutCommand(ctx, cmd); err != nil {
		log.Fatal(err)
	}
	if err := cs.PutRoute(ctx, obsdb.RunRoute{RunID: "run_1", RuntimeID: "rt_1", CommandID: "cmd_1", Seen: cmd.Updated}); err != nil {
		log.Fatal(err)
	}
	if err := cs.PutBreakpoints(ctx, "rt_1", []string{"issue_refund"}); err != nil {
		log.Fatal(err)
	}

	live, err := cs.Commands(ctx, obsdb.CommandQuery{States: []string{obsdb.CommandQueued, obsdb.CommandAccepted}})
	if err != nil {
		log.Fatal(err)
	}
	for _, c := range live {
		fmt.Println(c.ID, c.State, c.RunID, "accepted at", c.AcceptedAt.Sub(c.Created))
	}
	route, _, _ := cs.Route(ctx, "run_1")
	tools, _, _ := cs.Breakpoints(ctx, "rt_1")
	fmt.Println("run_1 →", route.RuntimeID, "parks on", tools)
}
Output:
cmd_1 accepted run_1 accepted at 1s
run_1 → rt_1 parks on [issue_refund]

type Compaction added in v0.10.0

type Compaction struct {
	Scope        string
	Hash         string // weft.compaction.hash
	Index        int64
	Step         int
	FromSeq      int64
	ToSeq        int64
	Messages     json.RawMessage
	Reason       string
	Replaced     int
	Entries      int
	TokensBefore int64
	TokensAfter  int64
}

Compaction is one compaction a run's records name (ADR 0028 §8), read by DB.Compactions. Two scopes:

  • CompactionRun: a compaction view, the messages record the core emits when a PrepareStep sent a request whose messages are not the run's transcript. The request whose messages_ref names Index saw the growth records up to Index, concatenated, with the transcript range [FromSeq, ToSeq) replaced by Messages. It is never part of the plain transcript (Transcript and TranscriptBatches hold growth records only) and never applies to another request.
  • CompactionSession: the informational marker weft/thread emits (a record of kind compaction) when the session compacts, filed under the last run that produced the compacted context (or, for a context no run produced, under the next run). The next run's input record holds the compacted context literally; the marker carries no messages and a reader never applies it. Index and Step are -1, FromSeq and ToSeq 0.

Replaced and Entries are the change in messages: for a view, ToSeq-FromSeq and the length of Messages; for a session marker, the counts thread reported over the session's context. TokensBefore and TokensAfter are thread's estimates (session scope only; 0 for a view, whose tokens the core does not estimate). Reason is thread's compaction reason (manual, threshold, overflow, from_hook, trim); "" for a view.

func CompactionOf added in v0.10.0

func CompactionOf(kind string, index int64, step int, attrs map[string]any, body []byte) (c Compaction, ok bool, err error)

CompactionOf reads one stored record as a Compaction, so every backend parses alike. kind is the record's weft.record, index its position, step its weft.step.index, attrs its attributes (numbers as numbers or numeric strings) and body its body. ok is false for a record that names no compaction — a growth messages record, any other kind. A messages record whose weft.messages.reason is not one this build knows is an error: ADR 0028 §8 has readers fail loudly rather than mistake a view they cannot apply for something else.

func ViewOf added in v0.13.0

func ViewOf(rec RequestRecord, compactions []Compaction) (Compaction, bool)

ViewOf returns the run-scope compaction view a request record names: the one whose index is the request's messages_ref.index. ok is false for a request that carried the plain transcript (its ref names a growth record) or names nothing.

type DB

type DB interface {
	// Write stores one batch, idempotent on (run, record kind, pos)
	// and (trace, span): a retried transport's duplicate is a no-op.
	// Deltas are counted, never stored; heartbeats are never stored —
	// they only move the run's last-seen (DeriveStatus's clock).
	Write(ctx context.Context, b Batch) error

	Runs(ctx context.Context, q RunQuery) (RunPage, error)
	Run(ctx context.Context, id string) (RunDetail, error) // ErrNotFound
	// Events pages a run's durable events in position order. after is
	// exclusive — pass -1 to read from the start (0 would skip position
	// 0, the run_start) and EventPage.NextAfter to continue. limit: 0 =
	// 100, max 1000.
	Events(ctx context.Context, runID string, after int64, limit int) (EventPage, error)
	// Transcript returns the messages bodies in index order, one per
	// growth messages record (a compaction view, ADR 0028 §8, is not
	// transcript: see Compactions), read through DedupTranscript — so
	// they concatenate to the transcript the run held.
	Transcript(ctx context.Context, runID string) ([]json.RawMessage, error)
	// TranscriptBatches is Transcript with what each messages record
	// stored beside its body: its index, the step it joined
	// (weft.step.index, -1 when the record carried none) and whether
	// it is the run's input (weft.messages.input). The bodies are
	// Transcript's, through DedupTranscript, one per record.
	TranscriptBatches(ctx context.Context, runID string) ([]TranscriptBatch, error)
	// Compactions returns the compactions a run's records name (ADR
	// 0028 §8): every run-scope view in index order, then the session
	// markers thread filed under the run, in emission order — what a reader needs to draw the compaction and
	// rebuild a request's messages. Transcript and TranscriptBatches
	// never include a view: the plain transcript is growth records
	// only. A messages record with an unknown weft.messages.reason is
	// an error. ErrNotFound for an unknown run; empty, not an error,
	// for a run that never compacted.
	Compactions(ctx context.Context, runID string) ([]Compaction, error)

	// The request record (ADR 0028). Requests pages a run's request
	// records in index order, one per model-call attempt
	// (RequestQuery). Prompt and Tools return the run's prompt or tools
	// record of one hash: ErrNotFound when the run holds none, as a
	// *HoleError when the reason is known (not_recorded, stripped,
	// gap — ExplainMissing). Catalogs returns every tools record of the
	// run, one per hash, in index order; a content-off run has none.
	// All four answer ErrNotFound for an unknown run.
	Requests(ctx context.Context, runID string, q RequestQuery) ([]RequestRecord, error)
	Prompt(ctx context.Context, runID, hash string) (PromptRecord, error)
	Tools(ctx context.Context, runID, hash string) (ToolsRecord, error)
	Catalogs(ctx context.Context, runID string) ([]ToolsRecord, error)
	RunSpans(ctx context.Context, runID string) ([]Span, error)
	// OtherLogs pages a run's app log records — the non-weft records
	// the writers store beside weft's (other_logs on SQLite, otel_logs
	// on ClickHouse), attributed to the run through the spans they were
	// emitted under — in time order (LogQuery, ReadOtherLogs), with
	// what the read could not show (LogPage: partial while running,
	// lines under a never-stored span, the candidate cap).
	// ErrNotFound for an unknown run; empty Logs, not nil, for a run
	// with no app logs; a *HoleError (Kind "logs", HoleNotRecorded) for
	// a finished run with no span to attribute them through.
	OtherLogs(ctx context.Context, runID string, q LogQuery) (LogPage, error)
	// CountLogs counts a run's app log records at or above minSeverity
	// (an OTLP severity number; 13 is WARN) with OtherLogs'
	// attribution, in one scan the backend narrows by severity
	// (CountOtherLogs): the run document's log_warn / log_error without
	// paging every line's body. Capped past MaxLogCandidates such lines
	// (the counts are floors). OtherLogs' errors.
	CountLogs(ctx context.Context, runID string, minSeverity int) (LogCounts, error)
	Trace(ctx context.Context, traceID string) ([]Span, error) // empty, not an error, for an unknown trace
	// Traces lists traces — weft and non-weft alike, one row per trace
	// id the spans table holds — newest first (TraceQuery, TracePage):
	// the row is computed from the trace's spans alone, so a trace
	// whose root never arrived is listed with Root nil.
	Traces(ctx context.Context, q TraceQuery) (TracePage, error)

	Sessions(ctx context.Context, q SessionQuery) (SessionPage, error)
	Session(ctx context.Context, id string) (SessionDetail, error)
	ResolvePublicID(ctx context.Context, publicID string) (sessionID string, err error)

	// The playground's saved experiments (WEFT-PLAYGROUND §10.4,
	// PQ4): the definition rows keyed by the weft.experiment.id the
	// runs carry — one small table beside runs.
	SaveExperiment(ctx context.Context, e Experiment) error // upsert by ID
	Experiments(ctx context.Context) ([]Experiment, error)  // newest update first
	Experiment(ctx context.Context, id string) (Experiment, error)

	// Deletion and retention (plan A7, §13.13). Every one touches the
	// observability tables alone — never another module's tables
	// sharing the file (thread's sessions in one SQLite file stay
	// byte-identical) and never the experiments table (its definitions
	// name runs, they do not hold them).
	//
	// DeleteRun removes a run and its subagent children — every run
	// whose ParentRunID chain reaches id — with everything stored under
	// them: records (deltas kept by KeepDeltas included), the search
	// index rows, the spans named by DeletedSpans and the app log
	// records emitted under those spans; the CommandStore rows
	// (commands, run_routes, breakpoints, runtimes) are never touched.
	// The derived run of a stock agent under the run's tool goes too
	// (docs/record-contract.md §9): it is no child of the run — the
	// derivation stops at weft spans — but its root is among the spans
	// deleted, and its row would otherwise outlive every span it was
	// derived from (StockRunsOf).
	// ErrNotFound for an unknown id.
	// A run still being written is deleted as it stands: a later write
	// for it starts a new run row from what that write carries.
	DeleteRun(ctx context.Context, id string) error
	// DeleteSession is DeleteRun for every run carrying the session id
	// (weft.session.id), children included. The app's own conversation
	// store is not obsdb's and is never touched. ErrNotFound when no
	// run carries the id.
	DeleteSession(ctx context.Context, id string) error
	// Sweep deletes, as DeleteRun does, the runs whose LastSeen is
	// before the cutoff and whose status is not running (succeeded,
	// failed or interrupted — a run silent past InterruptedAfter is
	// interrupted, so it counts as not running and is swept), with
	// their children that are not running either; a running child keeps
	// its own subtree and is swept on its own once it ends. Until then
	// that child of a swept parent is listed only under RunQuery's
	// ParentRunID "*" (the API's all=1 / parent=*), never among the
	// top-level runs. It works in passes of at most SweepBatch roots
	// (each with its subtree), so a large backlog never holds the
	// writer for the whole sweep; ctx is checked between passes. It
	// returns how many runs it deleted, children and the derived runs
	// DeleteRun takes beside them counted, over every pass — on an
	// error, the passes before it stay deleted.
	Sweep(ctx context.Context, before time.Time) (int, error)
	// Vacuum gives the space deletions freed back: SQLite runs VACUUM
	// (the whole file is rewritten, other modules' tables with their
	// contents unchanged); ClickHouse runs OPTIMIZE TABLE … FINAL on
	// the tables deletions mutate, which merges every part — an
	// operator's command, heavy on a large database.
	Vacuum(ctx context.Context) error

	Close() error
}

DB is the observability database every backend implements (sqlite by default, clickhouse hosted). Implementations must be safe for concurrent use: writers are rare (the local sink's synchronous processors), readers many (Studio).

DeleteRun, DeleteSession, Sweep and Vacuum are the run tables' alone. A CommandStore's rows — commands, run routes, breakpoints, runtimes — are Studio's record of what it asked a runtime to do, not run data: deleting a run leaves the commands and the route that name it, and CommandStore.PruneCommands is their only retention.

type ErrorClassCount added in v0.19.0

type ErrorClassCount struct {
	Class string
	Count int64
}

ErrorClassCount is one error class and how many runs' texts fold into it.

func ErrorClasses added in v0.19.0

func ErrorClasses(counts []ErrorCount, limit int) []ErrorClassCount

ErrorClasses folds error counts into classes (ErrorClass), most runs first, ties by class; at most limit of them (0 = all).

type ErrorCount added in v0.19.0

type ErrorCount struct {
	Err   string
	Count int64
}

ErrorCount is one distinct error text and how many runs carry it.

type EventPage

type EventPage struct {
	Events    []PosEvent
	NextAfter *int64
	// Done: every stored event was returned and the run is terminal —
	// its run_finish is stored, its invoke_agent span ended with Error,
	// or its invoke_agent span is stored (RunRow.RootEnded) and nothing
	// was heard from the run for over InterruptedAfter (the settling
	// window, DeriveSettling: records that trail a stored root are given
	// that long to land, so a run whose run_finish was lost still ends
	// a poll that waits for Done). An interrupted run (no root, stale)
	// is not Done here: a reader that follows it decides from the row's
	// derived Status.
	Done bool
	Gaps []int64
}

EventPage is one page of a run's durable events. Gaps are durable positions missing below the high-water mark: a lost batch, never a delta — deltas are on their own counter, so their absence cannot open a hole here (D3). At most MaxGaps are listed, lowest first.

type Experiment

type Experiment struct {
	ID, Name, Agent  string
	Created, Updated time.Time
	Variants         []ExperimentVariant
	Inputs           []ExperimentInput
	// PassPredicate is the experiment's pass predicate (UX plan D8), as
	// the playground sent it: obsdb stores and returns it verbatim and
	// reads nothing into it (Studio bounds its length). "" when none,
	// and on an experiment saved before obsdb kept it.
	PassPredicate string
	// ExperimentCommand is the rest of the §5.1 command the matrix
	// sent, so a saved experiment reopens as the form that ran it.
	// Zero on an experiment saved before obsdb kept it.
	ExperimentCommand
}

Experiment is one saved playground group: a name, the variants and inputs that define it, keyed by the id its runs carry as weft.experiment.id (§10.4). Variants and inputs are the wire shapes verbatim (overrides JSON included); the runs themselves stay in the runs table.

type ExperimentCommand added in v0.14.0

type ExperimentCommand struct {
	Engine          string          `json:"engine,omitempty"`           // live | scripted
	SideEffects     string          `json:"side_effects,omitempty"`     // substitute | park | allow
	Thread          string          `json:"thread,omitempty"`           // ephemeral | fork
	FromStep        *int            `json:"from_step,omitempty"`        // the step every cell continues from
	SourceRunID     string          `json:"source_run_id,omitempty"`    // the run the cells replay from
	TranscriptEdits json.RawMessage `json:"transcript_edits,omitempty"` // the list as sent
}

ExperimentCommand is the experiment-wide part of the §5.1 run command, kept as sent: the modes, the step the cells continue from, the source run and the transcript edits. Every field is optional — zero is "not recorded" (an older row) or the command's default, and a reader does not tell the two apart. obsdb stores it as one JSON document beside the definition and does not validate it (Studio does, on the way in).

type ExperimentInput

type ExperimentInput struct {
	Key         string `json:"key"`
	SourceRunID string `json:"source_run_id,omitempty"`
	Text        string `json:"text,omitempty"`
	FromStep    *int   `json:"from_step,omitempty"`
}

ExperimentInput is one row of the matrix: a key, a source run to take the turn from, or a literal text. FromStep is the row's own step (UX-C-37's multi-source matrix: each source run continues from its own failing step, keeping its own prefix); nil, the experiment's ExperimentCommand.FromStep.

type ExperimentVariant

type ExperimentVariant struct {
	Key         string          `json:"key"`
	Overrides   json.RawMessage `json:"overrides,omitempty"`
	Engine      string          `json:"engine,omitempty"`
	SideEffects string          `json:"side_effects,omitempty"`
	Thread      string          `json:"thread,omitempty"`
}

ExperimentVariant is one column of the matrix: a key, the overrides object it applies (verbatim — instructions, tools_enabled, model, thinking, options, params, tool_choice, park_on, only_tools, and any key a later command adds), and the modes it runs under when they differ from the experiment's (empty: the experiment's own).

type Frame

type Frame struct {
	Kind   string  // "record" | "run" | "attempt"
	Record *Record // Kind == record; Weft derived
	Weft   Weft
	Run    *RunRow // Kind == run
	Span   *Span   // Kind == attempt; Weft derived
	Seq    uint64  // hub-wide, monotonic: the SSE id and the resume cursor
}

Frame is one live-lane message: a record as ingested (deltas included), a run row that changed, or an attempt span as it landed.

func AttemptFrame added in v0.17.0

func AttemptFrame(s Span) (Frame, bool)

AttemptFrame builds the frame a write publishes for an `attempt` span (FrameAttempt): the span with its derived identity. ok is false for every other span, and for an attempt span that names no run.

func RecordFrame

func RecordFrame(r Record) Frame

RecordFrame builds the frame a Write publishes for one record: the record as ingested with its derived identity, deltas included.

func RunFrame

func RunFrame(r RunRow) Frame

RunFrame builds the frame a Write publishes for a run row that changed.

type Hole added in v0.10.0

type Hole string

Hole is one badge of ADR 0028 §11's closed table: the one word a reader shows wherever a record is missing, cut or derived. The vocabulary is closed — a new badge is an amendment to that ADR — and Holes lists it. The zero value means no hole.

const (
	HoleTruncated   Hole = "truncated"    // a destination's cap cut the content (weft.content.truncated_bytes)
	HoleStripped    Hole = "stripped"     // the destination's chain is content-off (weft.content = stripped)
	HoleRedacted    Hole = "redacted"     // the destination's Redact changed the content (reserved: weft's pipeline does not mark a redaction)
	HoleMaxTokens   Hole = "max_tokens"   // the step finished on the output token limit
	HoleInterrupted Hole = "interrupted"  // the run stopped reporting (DeriveStatus)
	HoleGap         Hole = "gap"          // a hole in a contiguous counter, or a record a destination dropped
	HoleNotRecorded Hole = "not_recorded" // the record kind did not exist in the version that wrote the run
	HoleDerived     Hole = "derived"      // the value was computed by the reader, not recorded
	HoleHidden      Hole = "hidden"       // the reader's scope may not see it
	HoleCompacted   Hole = "compacted"    // the model saw a compacted view
)

The closed table, in ADR 0028 §11's order.

func Holes added in v0.10.0

func Holes() []Hole

Holes returns the closed table in its order: the one list a badge renderer (Studio's TypeScript twin included) is checked against.

type HoleCause added in v0.10.0

type HoleCause string

HoleCause narrows a hole to the cause a reader established, when the table words that cause apart: CauseDefault is the hole's own note.

const (
	CauseDefault HoleCause = ""
	// CauseNoSpans: the run's records arrived but no span did — it ran
	// without a tracer (not_recorded: attempts, the answering model).
	CauseNoSpans HoleCause = "no_spans"
	// CauseResultCap: the core's own tool-result cap cut the content
	// before any destination saw it (truncated: a tool result the model
	// itself read as a prefix plus the marker, weft.MaxResultBytes).
	CauseResultCap HoleCause = "result_cap"
	// CauseLogCap: the app-log reader scanned its candidate cap
	// (MaxLogCandidates) before attributing lines to the run, so later
	// lines may be missing (truncated: no destination setting changes
	// it).
	CauseLogCap HoleCause = "log_cap"
	// CauseNoPublicID: the session's turns carry no weft.public_id —
	// a thread session created without thread.PublicID, or an app that
	// never stamped one (not_recorded: Studio's
	// GET /api/sessions/{id}/public_id).
	CauseNoPublicID HoleCause = "no_public_id"
	// CauseDevTokenOnly: the route answers the dev (server) token and
	// setup A's open API only; a panel token, scoped to one public id,
	// may not learn another (hidden: the session-to-public-id lookup).
	CauseDevTokenOnly HoleCause = "dev_token_only"
	// CauseNotServed: the Studio asked serves no request record (its
	// requests capability is off, or it predates the record), so the
	// run's request, prompt and tools records were not read — the run
	// may hold them (not_recorded: the panel's Request tab).
	CauseNotServed HoleCause = "not_served"
	// CauseResponseCap: the route bounds how much one response reads
	// (Studio's GET /api/diff compares a bounded number of steps), and
	// the run holds more than it read (truncated: nothing was lost; the
	// rest is a read away).
	CauseResponseCap HoleCause = "response_cap"
	// CauseRawOff: the attempt has no raw record (ADR 0028 §12) — wire
	// bodies are recorded only under otel.RawBodies(true) with
	// WEFT_ENV=dev (or otel.DevMode()), by an adapter that reports them (not_recorded: the
	// requests row's raw, the wire tab).
	CauseRawOff HoleCause = "raw_off"
	// CauseStockSpans: the run was built by the reader from stock OTel
	// GenAI spans (RunRow.Derived, docs/record-contract.md §9) — its
	// producer is not weft, so it recorded no events, transcript or
	// request records (derived: every pane those would fill).
	CauseStockSpans HoleCause = "stock_spans"
	// CauseNoLogger: a weft run's spans were stored and none of its
	// records was — the app exported spans only, a tracer wired and no
	// logger (not_recorded: events, the transcript and the request
	// records; LostRecordsHole decides it).
	CauseNoLogger HoleCause = "no_logger"
	// CauseTailLost: a weft run's invoke_agent span was stored and its
	// run_finish record was not, though earlier records of it were — a
	// destination dropped the run's record tail (gap: the run reads
	// succeeded from its root, its last steps' records missing;
	// TailLostHole decides it).
	CauseTailLost HoleCause = "tail_lost"
	// CauseEventsLost: a weft run's invoke_agent span says how many
	// event records the core emitted (weft.run.events, U5) and fewer
	// were stored — a destination dropped them (gap: EventsLostHole
	// decides it, EventsLostReason counts them, "N of M events were
	// lost"). It replaces tail_lost and no_logger's guess wherever the
	// root carries the count.
	CauseEventsLost HoleCause = "events_lost"
	// CauseMessagesLost: a weft run's invoke_agent span says how many
	// growth messages records the core emitted (weft.run.messages, U5)
	// and the store holds some, but fewer — a destination dropped
	// transcript batches (gap: MessagesLostHole decides it,
	// MessagesLostReason counts them, "K of N transcript batches were
	// lost"). A store holding none is not called lost: a content-off
	// destination stores no messages record by design and obsdb cannot
	// tell it from one that lost every one (docs/record-contract.md §7).
	CauseMessagesLost HoleCause = "messages_lost"
	// CauseMetadataCap: the core's metadata limits dropped pairs of the
	// run's Metadata (64 keys, a key over 128 bytes, a value over
	// 1 KiB — dropped whole, never cut; weft.metadata.dropped on its
	// invoke_agent span, Problems.MetadataDropped; truncated:
	// MetadataCapReason counts them).
	CauseMetadataCap HoleCause = "metadata_cap"
)

func HoleCauses added in v0.10.0

func HoleCauses() []HoleCause

HoleCauses lists the causes the table words, in order.

type HoleError added in v0.10.0

type HoleError struct {
	Kind string // "prompt", "tools" or "logs"
	Hash string
	Hole Hole
}

HoleError is the ErrNotFound DB.Prompt and DB.Tools return when the reader knows why the record is missing: errors.Is(err, ErrNotFound) holds, and errors.As yields the badge. Hole is one of:

  • HoleNotRecorded: the run was written before ADR 0028 (its InstructionsHash is ""), so no such record ever existed;
  • HoleStripped: a request naming the hash came through a content-off chain, which drops prompt and tools records;
  • HoleGap: a request naming the hash was stored as emitted, yet the record it names is absent — a destination dropped it.

A hash no request of the run names is a plain ErrNotFound.

DB.OtherLogs returns one with Kind "logs", no Hash and HoleNotRecorded for a run that has no span to attribute app logs through (ReadOtherLogs).

func (*HoleError) Error added in v0.10.0

func (e *HoleError) Error() string

func (*HoleError) Is added in v0.10.0

func (e *HoleError) Is(target error) bool

Is makes a HoleError an ErrNotFound.

type Hub

type Hub interface {
	Publish(ctx context.Context, f Frame)
	Subscribe(ctx context.Context, sel Selector, after uint64) (<-chan Frame, error)
}

Hub is the live lane. Publish never blocks the caller beyond a bounded per-subscriber queue; a slow subscriber is dropped with Overflow.

func NewHub

func NewHub(opts ...HubOption) Hub

NewHub returns the in-process hub: one Seq counter for every frame, one bounded queue per subscriber (QueueSize, default 1000). Hosted Studio swaps in a Redis/NATS implementation of the same interface; this one is what setup A's live lane runs on.

type HubOption

type HubOption interface {
	// contains filtered or unexported methods
}

HubOption configures NewHub.

func QueueSize

func QueueSize(n int) HubOption

QueueSize sets the per-subscriber queue depth. Publish to a full queue drops the subscriber (its channel closes) rather than blocking the writer: the durable lane is the database's job, and a live tail can always resume from it.

type LogCandidates added in v0.10.0

type LogCandidates func(ctx context.Context, traceIDs []string, from, to time.Time, limit int) ([]OtherLog, error)

LogCandidates is a backend's half of DB.OtherLogs: the non-weft log records (no weft.run.id) of the given traces whose time is within [from, to]: the first limit of them, returned in time order (ties in a stable order of the backend's). ReadOtherLogs does the rest.

type LogCounts added in v0.16.0

type LogCounts struct {
	Total, Warn, Error int64
	Capped             bool
}

LogCounts is one DB.CountLogs answer: the run's app log records at or above the asked severity (Total), split into WARN (13–16) and ERROR or above (17–24). Capped says the scan stopped at MaxLogCandidates lines (LogPage.Truncated's rule, over the asked severity's lines only): every count is then a floor — a reader marks it "≥".

func CountOtherLogs added in v0.16.0

func CountOtherLogs(ctx context.Context, db DB, runID string, minSeverity int, candidates LogCandidates) (LogCounts, error)

CountOtherLogs is DB.CountLogs for every backend: ReadOtherLogs' attribution and deduplication over the candidates at or above minSeverity, which the backend selects in its query (candidates is that filtered scan), counted rather than paged — one scan, no indexing, no page walk. Its errors are ReadOtherLogs': an unknown run is ErrNotFound, a run with no span a *HoleError (nothing to attribute through); a running run counts what is attributable now.

type LogPage added in v0.10.0

type LogPage struct {
	Logs      []OtherLog
	Partial   bool
	Gap       int
	Truncated bool
	NoBridge  bool
}

LogPage is one DB.OtherLogs answer: the page and what the reader knows about what it could not show.

  • Partial: the run is running. Spans are exported when they end, so a line under an in-flight span (invoke_agent, an open execute_tool) is not attributable yet: it appears once its span arrives, and it may then take an index below lines already served — indexes are stable only once the run has ended.
  • Gap: lines in the run's traces (deduplicated, in its window) that name a span no stored span has — dropped (a lost batch, a process that exited before exporting), or still open (an app span that encloses the run and has not ended). They may belong to this run or to another in the same trace, and are not in Logs. Always 0 while Partial (the span may still be on its way).
  • Truncated: the run's traces hold more than MaxLogCandidates log lines in its window — anyone's, the cap applies before attribution — so only lines among the first MaxLogCandidates by time were read: later lines of this run may be missing, and Logs may hold far fewer than MaxLogCandidates.
  • NoBridge: the run's root span (invoke_agent) is stored without weft.logs.bridged and no app log line is attributed to the run: the pipeline that recorded it had no slog bridge (otel.Slog, otel.SlogHandler) when it began, so its slog lines — weft.Logger's, mw.Log's, the app's own — never reached the store. A reader shows it as a hint beside the empty page, never as a loss. Never set while the root is not stored (a running run), for a run derived from stock spans, once any line is attributed, or beside Gap or Truncated (lines exist that may be the run's).

func ReadOtherLogs added in v0.10.0

func ReadOtherLogs(ctx context.Context, db DB, runID string, q LogQuery, candidates LogCandidates) (LogPage, error)

ReadOtherLogs is DB.OtherLogs for every backend, so both attribute, order, index and page alike. An app log names no run — it carries no weft.run.id, or the writer would have taken it for a weft record — so it is attributed through the span it was emitted under: a record belongs to the run when its (trace, span) is one of the run's spans (invoke_agent, chat, execute_tool: a tool handler's log line), or a non-weft span below one of them (the app's own span inside a tool), never one below another run's span (a subagent's lines are the child's). A record emitted with no span context cannot be attributed and is in no run's logs.

The scan reads the run's traces, bounded by its time window (Started − LogSkew to LastSeen + LogSkew). Duplicates — a retried transport's batch, which neither backend's log table collapses — are dropped (same time, trace, span, severity, event name, body and attributes). The order is time, then span id, severity and body; the filter applies after indexing (LogQuery). An unknown run is ErrNotFound; a run with spans and no app logs has an empty Logs, never nil. A run with no span at all has nothing to attribute logs through: a running one reads empty and Partial (its spans may not have arrived yet), any other is a *HoleError{Kind: "logs", Hole: HoleNotRecorded} — it was recorded without a tracer.

While the run is running the page is Partial and its indexes may shift (LogPage): a client paging with From may re-read lines and miss as many. TODO(phase 2): a stable cursor — (time, span, seq) rather than an index — so a live walk neither repeats nor skips.

At most MaxLogCandidates records are read (Truncated past it), chosen before attribution. TODO(phase 2): filter by the attributed span set in the backends' SQL, so the cap counts this run's lines only. Lines naming a span never stored are counted, not shown (Gap).

type LogQuery added in v0.10.0

type LogQuery struct {
	From        int64
	Limit       int
	MinSeverity int
}

LogQuery selects a run's app log records for DB.OtherLogs. The zero value reads every record from index 0, 100 at a time. From is the first index read (inclusive): pass the last returned Index + 1 to continue; a page shorter than PageLimit is the last. MinSeverity, when above 0, keeps the records whose OTLP severity number is at least that (ParseSeverity reads a level name); Index is the record's place among all of the run's app logs, so a filtered page's indexes skip and From continues a filtered walk alike. Limit: 0 = 100, max 1000.

func (LogQuery) PageLimit added in v0.10.0

func (q LogQuery) PageLimit() int

PageLimit is the query's normalized limit: 100 for 0 (or negative), 1000 at most.

type ManifestStore added in v0.16.0

type ManifestStore interface {
	// PutManifest stores m unless its hash is stored already (then it
	// is a no-op answering nil). A zero FirstSeen is stamped with the
	// current time. ErrInvalidManifest when CheckStoredManifest
	// refuses m.
	PutManifest(ctx context.Context, m StoredManifest) error
	// Manifest answers the manifest stored under hash; false when none
	// is.
	Manifest(ctx context.Context, hash string) (StoredManifest, bool, error)
}

ManifestStore is the optional manifest capability of a DB (U27): each agent's configuration by its hash, so a run's ManifestHash — the weft.manifest.hash its root span carried (docs/record-contract.md §2) — resolves to the configuration it ran with, after that configuration stopped being the current one.

A row is one agent's one-agent core.Manifest document, byte for byte: the bytes weft.Manifest writes for that agent alone, whose sha256 is the hash its runs carry. The store is content-addressed — the hash is the body's sha256 (CheckStoredManifest) — and immutable: a second PutManifest of a stored hash changes nothing, so FirstSeen and Source are the first write's.

Both first-party backends implement it — obsdb/sqlite (migration 0014) and obsdb/clickhouse (migration 0016) — and the obsdbtest conformance table holds them to the same answers. Studio writes it from the weft.json it serves and from runtime registrations.

Retention: manifests are kept forever. They are Studio's bookkeeping, not run data — one small row per distinct agent configuration — and DB.DeleteRun, DB.DeleteSession, DB.Sweep and DB.Vacuum never touch them, so a run that remains always resolves its hash.

type MessageText added in v0.14.0

type MessageText struct {
	Role, Text string
	ToolCall   bool
}

MessageText is one message of a messages record body reduced to what search reads: its role and the text of its text parts, joined by "\n" (every part of type "text", an empty one included, in order), and whether it carries a tool call (a part of type "tool_call").

func MessagesText added in v0.14.0

func MessagesText(body []byte) []MessageText

MessagesText reads a messages record body — a JSON array of core.Message — into its messages' text, in order. A body that does not parse reads as no messages: search indexes what it can read and never fails a write over it.

type ModelUsage added in v0.18.0

type ModelUsage struct {
	Provider, Model string
	Usage           core.Usage
}

ModelUsage is one model's part of a summed usage figure: the provider and model a span named ("" when it named none) and the usage of the spans that named them.

func SumModelUsage added in v0.18.0

func SumModelUsage(parts []ModelUsage) (core.Usage, []ModelUsage)

SumModelUsage is parts' total and the parts sorted by provider, then model (nil when there are none): the backends' one rule for a SessionRow's SummarizerUsage and SummarizerModels.

type OtherLog added in v0.10.0

type OtherLog struct {
	Index     int64
	Time      time.Time
	Severity  int
	EventName string
	Body      string
	Attrs     map[string]any
	TraceID   string
	SpanID    string
	Service   string
}

OtherLog is one app log record of a run: a non-weft record (no weft.run.id — the app's own slog or OTel Logs API lines, which the writers keep in other_logs on SQLite and in otel_logs on ClickHouse) attributed to the run through the span it was emitted under (DB.OtherLogs). Index is its place in the run's app logs, in time order; Severity the OTLP severity number (0 unspecified, 1–24; SeverityText names it). Attrs are the record's attributes: typed on SQLite, strings on ClickHouse, whose otel_logs keeps the collector's string map only.

type Percentiles added in v0.19.0

type Percentiles struct {
	P50, P95 float64
	N        int64
}

Percentiles are nearest-rank percentiles over N values (the value at rank ⌈p·N/100⌉ of the sorted list — both backends compute exactly this, so their answers are equal, not close). Zero N: no value, P50 and P95 zero.

type PosEvent

type PosEvent struct {
	Pos            int64
	Time           time.Time
	Event          json.RawMessage // the body, verbatim
	Content        string
	TruncatedBytes int64
}

PosEvent is one positioned event of a run: the body verbatim, at its durable position, with the two content attributes the destination's chain stamped on it (ADR 0028 §11): Content is weft.content ("stripped" from a content-off chain, "" as emitted) and TruncatedBytes weft.content.truncated_bytes (what a content-on chain's cap cut, 0 for none). A ClickHouse row written before migration 0004 carries neither.

type Problems added in v0.16.0

type Problems struct {
	// AttemptsRetried counts the model attempts that failed and were
	// followed by another attempt of the same model call: attempt spans
	// with status Error whose weft.attempt.index is below the highest
	// index among their chat span's attempts (mw.Retry's retries,
	// mw.Fallback's fallen-back-from tries, an adapter's reported SDK
	// retries). The last attempt of a call is never counted — failed, it
	// failed the call, and nothing retried it.
	AttemptsRetried int64
	// Fallbacks counts the model calls a fallback layer answered:
	// attempt spans that succeeded carrying weft.attempt.layer
	// "fallback" (mw.Fallback's report for a backup). Never a comparison
	// of model names.
	Fallbacks int64
	// ToolErrors counts the tool calls that ended in an error result:
	// execute_tool spans with status Error, except DENIED (Denied),
	// RETRY (core.ModelRetry: the intended repair loop) and a context
	// error (the run's cancellation reaching the call — the run's
	// error_type, not the tool's failure). NO_SUCH_TOOL counts.
	ToolErrors int64
	// Denied counts the refused tool calls: execute_tool spans whose
	// error.type is DENIED (a policy middleware, mw.Allow), plus the
	// calls a resume denied — Deny, or no decision — which have no span
	// and are counted by the invoke_agent span's weft.run.denied.
	Denied int64
	// Truncated counts the tool results the result cap cut:
	// execute_tool spans carrying weft.tool.truncated_bytes > 0.
	Truncated int64
	// Repaired counts the tool calls whose arguments a model
	// middleware repaired before the loop saw them (mw.RepairJSON):
	// execute_tool spans carrying weft.tool.args_repaired true (U23.1).
	Repaired int64
	// TapPanics counts the observer panics the run contained — taps,
	// OnMessages observers, a record the pipeline failed to build, and
	// every call into the run's TracerProvider or LoggerProvider that
	// panicked (P9-core-panic) — from its invoke_agent span's
	// weft.tap.panics (U32.1, UX-E-51). The run is unchanged; an
	// observer the app relies on saw less.
	TapPanics int64
	// MetadataDropped counts the metadata pairs the core's limits
	// dropped (64 keys, 128-byte keys, 1 KiB values), from the
	// invoke_agent span's weft.metadata.dropped (U32.1, UX-E-42): the
	// run document's truncated hole, cause metadata_cap — not a
	// problem badge.
	MetadataDropped int64
	// LogWarn and LogError count the app's own log records under the
	// run's spans at WARN (13–16) and at ERROR or above (17–24). They
	// are read time, not ingest time — an app log carries the trace and
	// span ids, not the run id, so it can only be attributed once the
	// run's spans are known (DB.CountLogs) — and filled on the run
	// document alone (Studio's run route), never on a list row.
	LogWarn, LogError int64
}

Problems are a run's problem counters (docs/record-contract.md §6; U4): what went wrong inside a run that may still have succeeded — the runs list badges them, the run page's stats strip counts them. The span-derived counters are folded at read time from a set of tokens the writers derive at ingest (SpanProblems: one token per problem span, keyed by its span id), so a retried batch that stores a span twice never counts it twice, on either backend — SQLite merges the set into runs.problems, ClickHouse into weft_runs.Problems (groupUniqArrayArray).

func CountProblems added in v0.16.0

func CountProblems(tokens []string) Problems

CountProblems folds a run's problem tokens into its counters (LogWarn and LogError stay 0: read time). Duplicate tokens count once; an unknown kind counts nowhere (a newer writer's, or one of U4's replaced kinds). A failed attempt counts when its chat span has an attempt of a higher index in the set.

func (Problems) Any added in v0.16.0

func (p Problems) Any() bool

Any reports whether any counter is non-zero.

type PromptRecord added in v0.10.0

type PromptRecord struct {
	Index          int64
	Time           time.Time
	Hash           string
	Text           string
	Segments       []PromptSegment
	Content        Hole
	TruncatedBytes int64
}

PromptRecord is one stored prompt record (ADR 0028 §4): the composed system text of one distinct system hash. Index is weft.prompt.index. Content is HoleTruncated when a destination's cap cut Text (TruncatedBytes > 0), HoleDerived when the body did not parse (a malformed producer; Hash is then the weft.system.hash attribute). Hash is over the text before the cut. Segments attribute Text to its sources — the instructions and each tool's PromptSnippet (UX-S2-11; docs/record-contract.md §4) — nil when the producer sent none (no tool contributed a snippet, a cap cut the text, or a writer before the field).

func PromptRecordOf added in v0.10.0

func PromptRecordOf(r StoredRecord) PromptRecord

PromptRecordOf builds a PromptRecord from a stored prompt record. Content is HoleTruncated when a cap cut the text; a body that does not parse (a malformed producer) reads HoleDerived, with no text and the hash of the record's weft.system.hash attribute.

type PromptSegment added in v0.16.0

type PromptSegment struct {
	Source string `json:"source"`
	Text   string `json:"text"`
}

PromptSegment is one source's share of a composed system text: Source "instructions" or "tool:<name>".

type RawRecord added in v0.14.0

type RawRecord struct {
	Index           int64     `json:"index"`
	Step            int       `json:"step"`
	Attempt         int64     `json:"attempt"`
	Time            time.Time `json:"time"`
	Request         RawSide   `json:"request"`
	Response        *RawSide  `json:"response"`
	RedactedHeaders []string  `json:"redacted_headers"`
	RedactedQuery   []string  `json:"redacted_query,omitempty"`
	Content         Hole      `json:"content"`
	TruncatedBytes  int64     `json:"truncated_bytes"`
}

RawRecord is one stored raw record: the attempt's wire exchange, as the core shaped it — credential headers stripped and named in RedactedHeaders, credential query values replaced and named in RedactedQuery, each body capped at 256 KiB with the cut counted. Index is weft.raw.index, Step the stored weft.step.index, Attempt the body's attempt (the attempt in flight when the adapter reported). TruncatedBytes is weft.content.truncated_bytes, both bodies' cut; Content is HoleTruncated when it is above 0, HoleDerived when the body did not parse (a malformed producer: the sides are zero).

func RawRecordOf added in v0.14.0

func RawRecordOf(r StoredRecord) RawRecord

RawRecordOf builds a RawRecord from a stored raw record.

type RawSide added in v0.14.0

type RawSide struct {
	Method         string              `json:"method,omitempty"`
	URL            string              `json:"url,omitempty"`
	Status         int                 `json:"status,omitempty"`
	Headers        map[string][]string `json:"headers"`
	MediaType      string              `json:"media_type,omitempty"`
	Body           string              `json:"body"`
	Encoding       string              `json:"encoding,omitempty"`
	Bytes          int64               `json:"bytes"`
	TruncatedBytes int64               `json:"truncated_bytes,omitempty"`
	// Complete is the response side's: false when the body was not read
	// to its end (closed unread by a transport retry, a timeout, a
	// reader that stopped) — the body is what was read, never the whole;
	// nil on the request side and on records that do not say.
	Complete *bool `json:"complete,omitempty"`
}

RawSide is one side of the exchange. Body is the bytes as text, or base64 when Encoding is "base64" (the bytes were not UTF-8); Bytes is the whole body's size before the cap, TruncatedBytes what the cap removed. Method and URL are the request's, Status the response's (absent on the other side). A nil Response is an exchange that got no response (a transport error).

type Record

type Record struct {
	Time, Observed  time.Time
	TraceID, SpanID string
	Severity        int
	EventName       string
	Body            string
	Service         string
	Attrs           map[string]any
	Resource        map[string]any
}

Record is one log record, OTLP-shaped.

func FromOTLPLogs

func FromOTLPLogs(req *collogspb.ExportLogsServiceRequest) []Record

FromOTLPLogs decodes an OTLP/HTTP log export request into the model. weft records (weft.run.id present) and non-weft records (slog lines, other services) both pass through; the kind they carry decides where Write puts them.

type RequestBody added in v0.10.0

type RequestBody struct {
	Step            int                `json:"step"`
	Attempt         int64              `json:"attempt"`
	SystemHash      string             `json:"system_hash"`
	MessagesRef     RequestMessagesRef `json:"messages_ref"`
	Tools           RequestTools       `json:"tools"`
	ToolChoice      *RequestToolChoice `json:"tool_choice,omitempty"`
	Thinking        *RequestThinking   `json:"thinking,omitempty"`
	SequentialTools bool               `json:"sequential_tools"`
	// Parallelism is the step's Parallelism dial (UX-S2-07, record
	// contract §4): how many of its tool calls may run at once. 0 on a
	// record written before it.
	Parallelism int           `json:"parallelism,omitempty"`
	Params      RequestParams `json:"params"`
	Model       RequestModel  `json:"model"`
	Stream      bool          `json:"stream"`
	// Prepared names the parts the run's PrepareStep chain changed
	// (O9, record contract §4): system, messages, tools, tool_choice,
	// params, thinking, sequential_tools. Absent when none did.
	Prepared []string `json:"prepared,omitempty"`
	// Wire is how the adapter's escape hatches shape the request
	// (UX-S2-09, record contract §4): names only. Nil when the model
	// declares none.
	Wire *RequestWire `json:"wire,omitempty"`
}

RequestBody is the request record's body (ADR 0028 §3), keys as the core writes them; an absent field is unset.

type RequestMessagesRef added in v0.10.0

type RequestMessagesRef struct {
	Index *int64 `json:"index,omitempty"`
	Count int    `json:"count"`
}

RequestMessagesRef says which messages the request carried: the run's view as of messages record Index, Count messages long. Index is nil where no messages record exists (capture off).

type RequestModel added in v0.10.0

type RequestModel struct {
	Provider string `json:"provider,omitempty"`
	Name     string `json:"name,omitempty"`
}

RequestModel is the model the attempt requested.

type RequestParams added in v0.10.0

type RequestParams struct {
	Temperature *float64 `json:"temperature,omitempty"`
	TopP        *float64 `json:"top_p,omitempty"`
	MaxTokens   *int     `json:"max_tokens,omitempty"`
	Stop        []string `json:"stop,omitempty"`
	Seed        *int64   `json:"seed,omitempty"`
}

RequestParams are the sampling parameters; nil is the adapter's default.

type RequestQuery added in v0.10.0

type RequestQuery struct {
	Step  *int
	From  int64
	Limit int
}

RequestQuery selects a run's request records for DB.Requests, in request-index order. The zero value reads every step from index 0, 100 at a time. Step, when set, keeps step *Step's attempts only. From is the first index read (inclusive): pass the last returned Index + 1 to continue; a page shorter than PageLimit is the last. Limit: 0 = 100, max 1000.

func (RequestQuery) PageLimit added in v0.10.0

func (q RequestQuery) PageLimit() int

PageLimit is the query's normalized limit: 100 for 0 (or negative), 1000 at most.

type RequestRecord added in v0.10.0

type RequestRecord struct {
	Index          int64
	Step           int
	Attempt        int64
	Time           time.Time
	SystemHash     string
	CatalogHash    string
	Content        Hole
	TruncatedBytes int64 // informational only; a request record is never capped by weft
	Body           RequestBody
	Raw            json.RawMessage
	Wire           *RawRecord
}

RequestRecord is one stored request record (ADR 0028 §3): one model call attempt. Index is weft.request.index, Step the stored weft.step.index (-1 when absent), Attempt the body's attempt. The record's weft.system.hash and weft.catalog.hash attributes win over the body's system_hash and tools.catalog_hash when set and they disagree; SystemHash and CatalogHash fall back to the body's where the attribute is absent. Raw is the body verbatim.

Content is the record's hole, from ADR 0028 §11's table:

  • "": stored as emitted;
  • HoleStripped: it came through a content-off chain (weft.content = stripped: its params.stop and messages_ref.index were removed, and that chain dropped the prompt and tools records it names) — whether or not its body parses;
  • HoleDerived: a malformed producer — an unstripped record whose body did not parse, so Body is zero, Attempt is 0, and the two hashes come from the record's attributes.

A request record is never capped (only prompt and tools records are), and a destination's Redact over params.stop is not marked on the record: HoleRedacted is reserved for the readers that can tell (ADR 0028 §11, A3).

Wire is the attempt's raw record (ADR 0028 §12: the provider request and response on the wire, recorded only under otel.RawBodies in development), nil when the run holds none for this attempt (AttachRaw).

func RequestRecordOf added in v0.10.0

func RequestRecordOf(r StoredRecord) RequestRecord

RequestRecordOf builds a RequestRecord from a stored request record: the hashes are the attributes' where set, the body's otherwise; a stripped mark is HoleStripped, and only an unstripped record whose body does not parse is HoleDerived (RequestRecord).

type RequestThinking added in v0.10.0

type RequestThinking struct {
	Level  string `json:"level"`
	Budget int64  `json:"budget,omitempty"`
	// Wire is how the adapter sends the level (O16): the provider field
	// it writes, or "none"; absent when the model declares none.
	Wire string `json:"wire,omitempty"`
}

RequestThinking is the thinking configuration in force.

type RequestToolChoice added in v0.10.0

type RequestToolChoice struct {
	Mode string `json:"mode"`
	Name string `json:"name,omitempty"`
}

RequestToolChoice is the tool choice in force.

type RequestTools added in v0.10.0

type RequestTools struct {
	CatalogHash string   `json:"catalog_hash"`
	Names       []string `json:"names"`
}

RequestTools is the catalog a request offered: its hash ("" for no tools) and the names in the order offered.

type RequestWire added in v0.20.0

type RequestWire struct {
	ServerAddress string   `json:"server_address,omitempty"`
	ExtraBodyKeys []string `json:"extra_body_keys,omitempty"`
	ExtraHeaders  []string `json:"extra_headers,omitempty"`
	Dropped       int      `json:"dropped,omitempty"`
}

RequestWire is a request's wire facts: the endpoint's host[:port], ExtraBody's top-level keys and ExtraHeaders' names (lowercase), each list sorted and bounded (32 names), Dropped counting the names left out.

type RunDetail

type RunDetail struct {
	RunRow
	Children []RunRow
}

RunDetail is one run with its subagent children (runs whose ParentRunID is this run, by Started).

type RunPage

type RunPage struct {
	Runs         []RunRow
	Total        int
	NextBefore   *time.Time
	NextBeforeID string
}

RunPage is one Runs result, newest Started first (ties by ID, descending). NextBefore and NextBeforeID are the next page's RunQuery.Before and BeforeID, nil and "" on the last page. A page does not end inside a group of runs sharing one Started: the rest of the group rides along, so a caller paging on Before alone skips nothing — up to MaxTies rows past Limit. A longer tie (a bulk import at one truncated timestamp) is cut there, and only the (Before, BeforeID) pair resumes inside it. Total counts the whole match, cursor aside.

type RunQuery

type RunQuery struct {
	Agent, SessionID, PublicID, ParentRunID string            // ParentRunID: "" top-level only (no parent, no caller link), "*" all
	Status                                  Status            // "" any; a StatusWords word (parked, cancelled read time)
	Playground                              *bool             // nil any
	ExperimentID                            string            // the runs of one experiment
	Meta                                    map[string]string // subset match on metadata
	Before                                  time.Time         // cursor on Started; zero = newest
	// LastSeenAfter keeps the runs last seen at or after it (zero = no
	// bound). A running run is always among them once the bound is
	// older than InterruptedAfter: running means seen that recently.
	LastSeenAfter time.Time
	// BeforeID, with Before, makes the cursor the pair (Started, ID):
	// the runs after that one in the list order — exact inside a group
	// sharing one Started. Pass RunPage.NextBeforeID.
	BeforeID string
	Limit    int // 0 = 50, max 500
	// StoppedBy keeps the runs whose RunRow.StoppedBy is exactly this
	// ("" any); a value ending in ':' is a prefix ("has_tool_call:"
	// matches every has_tool_call:<tools> value).
	StoppedBy string
	// ErrorType keeps the runs whose RunRow.ErrorType is exactly this
	// ("" any): max_steps, cancelled, model_error, …
	ErrorType string
	// Errs keeps the runs whose RunRow.Err is exactly one of these
	// texts: nil any, an empty non-nil slice none. The error_class
	// filter resolves a class to its texts (ErrorClassTexts) and reads
	// them through it.
	Errs []string
	// IDPrefix keeps the runs whose ID starts with it, byte for byte
	// ("" any; no wildcard, no case folding): a pasted short id.
	IDPrefix string
	// CallerRunID keeps the runs started from one of that run's tool
	// calls without being its Subagent children (RunRow.CallerRunID;
	// "" any): a delegating turn's thread/pool child turns, its nested
	// Generates. Pair it with ParentRunID "*" — a caller-linked run is
	// top-level by ParentRunID.
	CallerRunID string
	// SessionIDs keeps the runs of any of these sessions (nil any; with
	// SessionID both apply): one read for a set of sessions — Studio
	// prices a level of delegated sessions with one.
	SessionIDs []string
	// TurnBelow and TurnAbove keep the runs whose RunRow.Turn is below
	// / above them (0 = no bound): with SessionID, a turn's neighbours —
	// TurnBelow N, Limit 1 the newest-started turn before N, and the
	// page's Total how many there are; TurnAbove N with Oldest the
	// first after it (U15.1-fix: a turn page's stepper reads O(1) rows,
	// never the whole session).
	TurnBelow, TurnAbove int
	// Oldest turns the list around: oldest first, in start order (run
	// id breaking ties, ascending), and the (Before, BeforeID) cursor
	// then reads the runs after that one — RunPage.NextBefore pages
	// forward. A run page's caller_linked rows are its first page. Runs
	// alone: Search keeps its own sort.
	Oldest bool
}

RunQuery selects runs for Runs. The zero value lists top-level runs, newest first, 50 at a time.

type RunRoute added in v0.15.0

type RunRoute struct {
	RunID, RuntimeID, CommandID, PublicID string
	Fork                                  bool
	Seen                                  time.Time
}

RunRoute is one run → runtime route: the runtime an ack said started the run, the command whose ack named it last, the public id and fork mark the command carried, and when an ack last named it.

type RunRow

type RunRow struct {
	ID, ParentRunID, ParentCallID, TraceID                     string
	Agent, Provider, Model, ManifestHash, WeftVersion, Service string
	SessionID, PublicID                                        string
	Turn                                                       int
	Playground                                                 bool
	ExperimentID, ForkedFrom                                   string
	// ResumedFrom is the run this one resumes — its weft.resumed_from
	// (core.ResumeOf; thread and weft/runtime pass it, an app passes
	// the parked RunResult.ID) — and ResumedBy, on a parked run, the
	// run that resumed it: the latest run by start time whose
	// ResumedFrom names this run (a tie: the larger id), derived at read time on both backends, so a resumer
	// stored before or after the parked run links it alike (record
	// contract §6, UX-E-17). "" when none.
	ResumedFrom, ResumedBy string
	// CallerRunID / CallerCallID are the tool call this run was started
	// from when it is no core Subagent child (weft.caller.run.id /
	// weft.caller.call.id, O11; record contract §2): a thread/pool child
	// session's turn, another agent's Generate inside a handler. A link,
	// never parenthood: ParentRunID names a Subagent's parent alone, the
	// run whose usage includes this one's (rule 13). "" when none, and
	// on every run recorded before weft 0.18 (which named such a run's
	// caller in ParentRunID instead).
	CallerRunID, CallerCallID string
	Meta                      map[string]string
	Started                   time.Time
	Finished                  *time.Time
	LastSeen                  time.Time
	Status                    Status
	Err                       string
	Steps, Pending            int
	StopReason                string
	Usage                     core.Usage
	EventCount, MessageCount  int64
	DeltaCount                int64 // deltas are counted, never stored (Q4)
	// ADR 0028 §10: run_start's weft.instructions.hash ("" on a run
	// written before the request record — RequestsHole reads the
	// table), request index 0's weft.catalog.hash ("" when it offered
	// no tools) and the request records' high-water mark (max
	// weft.request.index + 1). The API names them instructions_hash,
	// catalog_hash and request_count.
	InstructionsHash, CatalogHash string
	RequestCount                  int64
	// PromptPreview is the run's prompt — the turn's own, PromptText's
	// rule with TurnPrompt's resume clause (Searcher's prompt field) — cut to
	// PromptPreviewRunes runes; "" when the records carried no content
	// (a content-off run stores no messages) or no user text.
	PromptPreview string
	// Derived marks a run the reader built from stock GenAI spans
	// (DeriveStockRuns; docs/record-contract.md §9): an invoke_agent
	// span with gen_ai.conversation.id and no weft.run.id, its id
	// otel-<trace id>-<root span id>. Such a run has no events, no
	// transcript and no request records (its counts and hashes are
	// zero, its status succeeded or failed only); RequestsHole reads
	// HoleDerived for it, and the API badges it "derived from spans ·
	// no events" (HoleNoteFor(HoleDerived, CauseStockSpans)).
	Derived bool
	// Removed marks a run frame's row that no longer exists: a derived
	// run the latest derivation folded into another (a top-level agent
	// without a conversation id whose enclosing agent arrived in a later
	// export, docs/record-contract.md §9). Only a live frame carries it
	// (StockFoldedRuns, the writers' dropped rows) — so an open list drops
	// the row; no store returns such a row.
	Removed bool
	// RootEnded marks a run whose invoke_agent span was stored: the span
	// is exported only once it ended, so the run is over whatever records
	// are missing (docs/record-contract.md §6). Its error status is
	// failed; any other status reads succeeded with or without a
	// run_finish (DeriveStatus's ended). LostRecordsHole reads it too.
	RootEnded bool
	// FinishRecorded marks a run whose run_finish record is stored (the
	// column both backends already keep: runs.finished_ok,
	// weft_runs.FinishedOK; a derived run's reads its succeeded status).
	// A RootEnded run without it lost its record tail on the way
	// (TailLostHole) or never had records (LostRecordsHole).
	FinishRecorded bool
	// Settling marks a RootEnded weft run (not Derived, not Failed) whose run_finish
	// is not stored and that was heard from within InterruptedAfter
	// (DeriveSettling): a destination may store the root span before the
	// records that trail it (a log exporter's retry or backlog), so for
	// that window the reader neither calls the records lost
	// (LostRecordsHole, TailLostHole) nor the event stream done
	// (EventPage.Done). Derived at read time with Status, never stored;
	// Status itself already reads succeeded or failed from the root.
	Settling bool
	// StoppedBy names what ended the run, in the manifest's stop_when
	// vocabulary (has_tool_call:<tools>, step_count_is:N,
	// output_submitted, final_answer, parked, max_steps, usage_limit,
	// loop_detected, model_retries_exceeded, canceled, a custom
	// condition's name): the invoke_agent span's weft.run.stopped_by, or
	// run_finish's stopped_by — one core value on both carriers, so a
	// failed run (no run_finish) still names it from its span. "" when
	// neither carried one: a failure no control decided (a model error),
	// a run still going, or one recorded before the field existed.
	StoppedBy string
	// CancelCause is a canceled run's weft.cancel.cause
	// (thread.interrupted, thread.closed, studio.cancel, deadline, or a
	// caller's own core.CancelCause kind); "" on a plain cancel and on
	// every other run.
	CancelCause string
	// EventsEmitted and MessagesEmitted are what the core emitted (U5):
	// the invoke_agent span's weft.run.events and weft.run.messages — the
	// durable event records and growth messages records the run handed
	// to its LoggerProvider. 0 when the root carried neither (not stored
	// yet, or a weft before them) and when the run emitted none (no
	// logger). Fewer stored than emitted is a loss (EventsLost,
	// TailLostHole, LostRecordsHole); emitted 0 with nothing stored is
	// no_logger. Fewer messages stored than emitted, some stored, is a
	// lost transcript batch (MessagesLostHole); none stored is not
	// called lost: a content-off destination legitimately stores no
	// messages record (contract §5), and the row cannot tell it from one
	// that lost them all (contract §7).
	EventsEmitted, MessagesEmitted int64
	// ErrorType is a failed run's kind (U4): the invoke_agent span's
	// error.type — max_steps, usage_limit, loop_detected,
	// model_retries_exceeded, model_contract, model_error, … (the
	// core's closed table, record contract §3) — with the context
	// errors read as "cancelled" (RunErrorType); "" for a run that did
	// not fail, a failure whose root carried none, or a run still
	// going. The larger when several roots differ (both backends).
	ErrorType string
	// Problems are the run's problem counters (U4; Problems): failed
	// attempts, fallbacks, tool errors, denials, truncated results —
	// derived at ingest from its spans; the log counters are filled on
	// the run document only.
	Problems Problems
}

RunRow is one run in a list or detail: the identity chain, timing, derived status and the denormalised usage and counts the list view reads without a second query.

func StockFoldedRuns added in v0.20.0

func StockFoldedRuns(spans []Span) []RunRow

StockFoldedRuns is the derived runs the spans no longer make that an earlier derivation of the same trace may have (§9): one row per non-weft invoke_agent span without a conversation id that is no root of the latest derivation — an agent that was top-level while its parent was not stored, folded now — each Removed, with the id, trace and agent a live frame needs (RunFrame) and Derived set. A publisher announces them so an open list drops the row; an id no list ever had is a no-op there.

func (RunRow) RequestsHole added in v0.10.0

func (r RunRow) RequestsHole() Hole

RequestsHole is ADR 0028 §10's reading table over a run row: HoleNotRecorded when InstructionsHash is "" and no request record exists (written before the request record existed: no request, prompt or tools records), "" otherwise — with RequestCount 0 the run made no model call, above 0 its requests are recorded. A row with request records but no instructions hash lost the batch that carried run_start (or has not received it yet): its requests are recorded, and a prompt or catalog they name but the store lacks is a gap (ExplainMissing), never not_recorded — "upgrade weft" would be the wrong fix for a run this weft wrote. A derived run (RunRow.Derived: built from stock spans, docs/record-contract.md §9) reads HoleDerived instead: its records are absent because its producer is not weft, so "upgrade weft" would be the wrong fix there too (the badge's words are HoleNoteFor(HoleDerived, CauseStockSpans)). A weft run whose root span is stored and none of whose records is reads HoleNotRecorded too — its InstructionsHash comes from the span — with the cause RequestsHoleCause names (CauseNoLogger).

func (RunRow) RequestsHoleCause added in v0.16.0

func (r RunRow) RequestsHoleCause() (Hole, HoleCause)

RequestsHoleCause is RequestsHole with the cause a reader words it by (HoleNoteFor): CauseStockSpans for a derived run; CauseNoLogger for a weft run whose invoke_agent span is stored (RootEnded) and no record of it is (LostRecordsHole: the app exported spans only — its requests pane would otherwise list empty with no badge, the hash the span carries reading as recorded); CauseDefault for a run written before the request record. A settling run (RunRow.Settling) has no hole yet: the records behind its root may still land.

type RunTree added in v0.17.0

type RunTree struct {
	// Spans is the tree's spans in start order, then span id: at most
	// TreeSpanCap, the first ones.
	Spans []TreeSpan
	// Truncated says the tree holds more than TreeSpanCap spans: Spans
	// is its first TreeSpanCap.
	Truncated bool
	// TraceTruncated says a derived run's trace held more than
	// TreeTraceCap spans: the tree was cut from the trace's first
	// TreeTraceCap, so a span of it after them was not read. Never set
	// for a weft run (its spans are read by run id).
	TraceTruncated bool
	// Missing is the descendant runs the tree knows from their rows
	// that have no span in Spans and are not running — a child that
	// ran without a tracer, or whose spans a destination dropped —
	// ordered by Started, then id: at most TreeMissingCap, the first
	// ones. A running child (StatusRunning: spans export when they
	// end) is not missing yet. Empty when Truncated: a cut tree cannot
	// tell a missing run from one past the cap.
	Missing []TreeRun
	// MissingCount is how many runs are missing: Missing's length
	// before TreeMissingCap.
	MissingCount int
	// Hidden says TreeQuery's scope (PublicID, Meta) left runs out.
	Hidden bool
	// HiddenCount is how many runs it left out: each descendant run the
	// tree knows (from its rows' parents or its spans) that the scope
	// cut, its subtree's runs included. Never their ids. 0 unless Hidden.
	HiddenCount int
}

RunTree is TreeReader.RunTreeSpans' answer.

func CapTree added in v0.17.0

func CapTree(read []Span, more bool, q TreeQuery, parents map[string]string, rows []TreeRun) RunTree

CapTree is a backend's last step: read is the spans it read in start order — a weft run's by member id (TreeReadIDs), at most TreeSpanCap + 1, more true when it read that many; a derived run's from its trace, more false; parents is CutRunTree's; rows is the member runs it read a row of. The answer is the read cut to q's subtree, then to q's scope (TreeQuery.PublicID, Meta), then to TreeSpanCap — so a run the scope leaves out never takes a visible one's place — and Missing: each of the visible rows but the root's that is a member (TreeMembers, its parent the members'), not running, with no span in the answer, ordered by Started, then id, the first TreeMissingCap. Missing stays empty on a cut tree.

type SearchQuery added in v0.14.0

type SearchQuery struct {
	RunQuery
	Text, Tool, Model, Error string
	Since, Until             time.Time
	MinTokens                int64
	Sort                     SearchSort
	Offset                   int
}

SearchQuery selects runs for Search. Its RunQuery carries the filters /api/runs reads — agent, status, session, public id (the panel token's scope), parent, playground, experiment, metadata — with the same meaning, so a scope applied to one applies to the other; RunQuery.Before and BeforeID are not read (a search pages by Offset) and RunQuery.Limit is the page size (0 = 50, max 500).

The search fields:

  • Text: the search terms (ParseSearch's grammar). A run matches when every term occurs, case-insensitively, as a substring of at least one of the run's search fields — its prompt (the turn's own: PromptText), its answer (the text of its last assistant message that carries text), its error, or the name of a tool it called (tool_start events) — each term free to match a different field. A content-off run has no prompt or answer: only its error and tool names can match.
  • Tool: the exact name of a tool the run called.
  • Model: the exact model name the run row carries.
  • Error: a case-insensitive substring of the run's error.
  • Since, Until: Started ≥ Since, Started < Until (zero = open).
  • MinTokens: input + output tokens ≥ MinTokens (0 = any).
  • Sort: the order (SearchSort; "" is SortStarted).
  • Offset: rows to skip in that order.

type SearchSort added in v0.14.0

type SearchSort string

SearchSort is a search's order. Every order breaks ties by Started, then run id, both descending, so a result is total and stable.

const (
	SortStarted  SearchSort = "started"  // newest first (the default)
	SortDuration SearchSort = "duration" // longest first: Finished (LastSeen while unfinished) − Started
	SortTokens   SearchSort = "tokens"   // most input + output tokens first
)

type Searcher added in v0.14.0

type Searcher interface {
	// Search answers the runs SearchQuery matches, sorted as it asks,
	// with Total counting the whole match. NextBefore and NextBeforeID
	// stay nil and "": a search pages by SearchQuery.Offset.
	Search(ctx context.Context, q SearchQuery) (RunPage, error)
}

Searcher is the optional search capability of a DB (plan A7): runs found by what they said and did, not only by who ran them. Both first-party backends implement it — obsdb/sqlite through an FTS5 trigram index maintained on write, obsdb/clickhouse through case-insensitive substring scans over the stored records — and the obsdbtest conformance table holds them to the same answers. A backend that does not implement it is a database that cannot search: Studio then leaves its search capability off and says why.

Example

A Searcher finds runs by what they said and did: every term a case-insensitive substring of the run's prompt, answer, error or a tool name, each term in any of them. The run row carries its prompt preview.

package main

import (
	"context"
	"fmt"
	"log"
	"time"

	"github.com/weftgo/weft/obsdb"
	"github.com/weftgo/weft/obsdb/sqlite"
)

func main() {
	db, err := sqlite.Open(":memory:")
	if err != nil {
		log.Fatal(err)
	}
	defer func() { _ = db.Close() }()
	t0 := time.Unix(0, 1790845923120000000).UTC()
	write := func(run, prompt, answer string, at time.Duration) {
		rec := func(idx int64, body string) obsdb.Record {
			return obsdb.Record{Time: t0.Add(at), Body: body, Attrs: map[string]any{
				"weft.record": "messages", "weft.run.id": run, "weft.messages.index": idx}}
		}
		if err := db.Write(context.Background(), obsdb.Batch{Records: []obsdb.Record{
			rec(0, `[{"role":"user","content":[{"type":"text","text":"`+prompt+`"}]}]`),
			rec(1, `[{"role":"assistant","content":[{"type":"text","text":"`+answer+`"}]}]`),
		}}); err != nil {
			log.Fatal(err)
		}
	}
	write("run_1", "Refund order 7", "The refund failed: card declined.", 0)
	write("run_2", "Refund order 9", "Refunded.", time.Second)

	page, err := db.(obsdb.Searcher).Search(context.Background(), obsdb.SearchQuery{Text: "refund failed"})
	if err != nil {
		log.Fatal(err)
	}
	for _, r := range page.Runs {
		fmt.Println(r.ID, "-", r.PromptPreview)
	}
}
Output:
run_1 - Refund order 7

type Selector

type Selector struct {
	RunID, SessionID, PublicID, Agent string
	// All matches every run's frames (subagent children included; a
	// record that names no run is no run's, and is not matched). It is
	// a scope of its own: set with any other field, the selector is
	// over-set and matches nothing.
	All bool
}

Selector scopes a subscription: exactly one field set. A run-scoped subscription follows one run; a session- or public-id-scoped one spans the many runs of one conversation, which is why the cursor is hub-wide rather than per run. All is the explicit every-run scope (Studio's Live page): it must be asked for, never implied.

type SessionDetail

type SessionDetail struct {
	SessionRow
	Runs []RunRow
}

SessionDetail is one session with its turns — all of them: top-level runs by Turn, then Started; experiments excluded (they hang off runs via ForkedFrom).

type SessionEvent added in v0.18.0

type SessionEvent struct {
	SessionID string
	RunID     string
	Kind      string
	Name      string
	PublicID  string
	Content   string
	Time      time.Time
	Body      json.RawMessage
}

SessionEvent is one stored session event: a thread record or a compaction marker as it arrived. RunID is the run it is filed under ("" for a record held at Close: a session-level fact); Kind its weft.record; Name its event name (weft.thread.<kind> or weft.compaction — derived from the body's kind when the record carried none); PublicID its weft.public_id; Content its weft.content; Body the record's body verbatim.

type SessionEventPage added in v0.18.0

type SessionEventPage struct {
	Events []SessionEvent
	Total  int
}

SessionEventPage is one SessionEvents result: Events in order, Total the session's whole count.

type SessionEventQuery added in v0.18.0

type SessionEventQuery struct {
	SessionID string
	PublicID  string
	RunIDs    []string
	Kinds     []string
	Offset    int
	Limit     int
}

SessionEventQuery selects one session's events for SessionEvents. PublicID, when set, keeps only the events carrying that weft.public_id (a panel token's scope): the page, the offsets and Total all count those alone. Offset is the number of events skipped (pass the previous offset plus the page's length to continue); Limit: 0 = 100, max MaxSessionEventLimit. RunIDs, when set, keeps only the events filed under one of those runs (a runless record is under none of them) — one turn's facts without paging the whole session; the page, the offsets and Total count those alone, as under PublicID. Kinds, when set, keeps only the events whose SessionEventKind is one of them ("compaction", "interrupt", "rewind", …) — a reader after a few kinds pages those alone (Studio's turn marks, U16-fix); the page, the offsets and Total count those alone, as under RunIDs.

func (SessionEventQuery) PageLimit added in v0.18.0

func (q SessionEventQuery) PageLimit() int

PageLimit is the query's normalized limit: 100 for 0 (or negative), MaxSessionEventLimit at most.

type SessionEventReader added in v0.18.0

type SessionEventReader interface {
	// SessionEvents pages a session's events in order (SessionEventQuery):
	// by record time, then run id, kind and body — one total order on
	// every backend, stable across reads, two records of one time, run
	// and kind included. Total counts every event the query selects (the
	// session's, or its public id's under PublicID). An unknown session
	// is an empty page, not an error: the caller decides whether the
	// session exists (DB.Session).
	SessionEvents(ctx context.Context, q SessionEventQuery) (SessionEventPage, error)
}

SessionEventReader is the session timeline's read (U12.2, D6), an optional capability of a DB: a session's state log — the thread records (RecordThread) and compaction markers (RecordCompaction) its records carry — in time order, from the session_events index (SQLite migration 0022's table, ClickHouse 0025's weft_session_events, both keyed by session id and time). Both first-party backends implement it and the obsdbtest conformance table holds them to the same answers; a backend that does not is a database without the timeline — Studio then leaves its timeline capability off and says why.

The index is fed at ingest from every record of those two kinds that carries a weft.session.id, a record filed under no run included (a thread record held at Close, weft.run.id ""): it is the session's, never a run's, and never an app log line (OtherLogs, CountLogs). DeleteRun and Sweep remove a run's events with the run; Sweep also removes, after each pass, the runless events older than its cutoff of a session no run is left of; DeleteSession removes every event of the session, the runless ones included.

Example

A SessionEventReader reads a session's state log — thread's records and its compaction markers — in record-time order from the session_events index, a record filed under no run (held at Close) among them: the session timeline's one read.

package main

import (
	"context"
	"fmt"
	"log"
	"time"

	"github.com/weftgo/weft/obsdb"
	"github.com/weftgo/weft/obsdb/sqlite"
)

func main() {
	db, err := sqlite.Open(":memory:")
	if err != nil {
		log.Fatal(err)
	}
	defer func() { _ = db.Close() }()
	t0 := time.Unix(0, 1790845923120000000).UTC()
	fact := func(run, kind, body string, at time.Duration) obsdb.Record {
		return obsdb.Record{Time: t0.Add(at), EventName: "weft.thread." + kind, Body: body, Attrs: map[string]any{
			"weft.record": obsdb.RecordThread, "weft.run.id": run, "weft.session.id": "s_1", "weft.content": "none"}}
	}
	if err := db.Write(context.Background(), obsdb.Batch{Records: []obsdb.Record{
		fact("s_1-t1", "approval.request", `{"kind":"approval.request","session":"s_1","entry":"e3","call_id":"c1","tool":"refund"}`, 0),
		fact("s_1-t1", "approval.decision", `{"kind":"approval.decision","session":"s_1","entry":"e4","call_id":"c1","outcome":"deny","via":"user"}`, time.Second),
		fact("", "grant.revoke", `{"kind":"grant.revoke","session":"s_1","entry":"e9","grant_id":"g1"}`, 2*time.Second),
	}}); err != nil {
		log.Fatal(err)
	}
	page, err := db.(obsdb.SessionEventReader).SessionEvents(context.Background(), obsdb.SessionEventQuery{SessionID: "s_1"})
	if err != nil {
		log.Fatal(err)
	}
	for _, e := range page.Events {
		fmt.Printf("%q %s\n", e.RunID, obsdb.SessionEventKind(e))
	}
	fmt.Println(page.Total)
	// RunIDs narrows the read to the named runs' events (one turn's
	// decisions); the runless record is under none of them.
	turn, err := db.(obsdb.SessionEventReader).SessionEvents(context.Background(),
		obsdb.SessionEventQuery{SessionID: "s_1", RunIDs: []string{"s_1-t1"}})
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(turn.Total)
}
Output:
"s_1-t1" approval.request
"s_1-t1" approval.decision
"" grant.revoke
3
2

type SessionPage

type SessionPage struct {
	Sessions     []SessionRow
	Total        int
	NextBefore   *time.Time
	NextBeforeID string
}

SessionPage is one Sessions result, paged as RunPage is: newest LastSeen first (ties by ID, descending), NextBefore/NextBeforeID are the next SessionQuery.Before/BeforeID, sessions sharing the last row's LastSeen ride along up to MaxTies, and Total counts the whole match.

type SessionQuery

type SessionQuery struct {
	Agent, PublicID string
	Q               string
	Parents         []string
	IDs             []string
	// IDPrefix keeps the sessions whose id starts with it, byte for
	// byte (RunQuery.IDPrefix's rule: no wildcard, no case folding) —
	// the ⌘K palette's lookup of a pasted short session id (U30.2). ""
	// any.
	IDPrefix  string
	Meta      map[string]string
	Awaiting  bool
	Failed    bool
	Compacted bool
	Status    Status    // "" any; running, succeeded, failed or interrupted
	Lineage   string    // "", LineageChild or LineageFork
	Since     time.Time // zero = any
	Before    time.Time // cursor on LastSeen; zero = newest
	BeforeID  string    // with Before: the (LastSeen, ID) pair, as RunQuery.BeforeID
	Limit     int       // 0 = 50, max 500
}

SessionQuery selects sessions. The zero value lists them newest activity first, 50 at a time. PublicID scopes the runs a session is grouped over, before the grouping (RunQuery.PublicID's rule): a session is listed when one of its turns carries the public id, and its turns, usage, first/last seen, agent, status, the page's order, Total and the cursor are those turns' alone — a run of another app sharing the session id (a colliding derived run) moves none of them.

Q finds sessions by text: every whitespace-separated term must be a case-insensitive substring of one of the session's title, its first turn's prompt (the whole prompt, not the preview's cut), its id, its public id or its agent — terms AND, fields OR. Case folding is ASCII-only on SQLite (lower()), Unicode on ClickHouse (positionCaseInsensitiveUTF8).

Parents keeps the sessions delegated by any of those sessions (SessionRow's ParentSession: thread/pool child sessions, ADR 0022), empty any: one read lists a whole level of a delegation tree.

Meta scopes the runs a session is grouped over as PublicID does (RunQuery.Meta's subset match, U12.2 — the API's tag.<k>=<v>: thread.WithRunMetadata's tags, core.Metadata's pairs): a session is listed when one of its own turns carries every pair, and its turns, usage, first/last seen, agent, status, title, first prompt, turn facts, the page's order, Total and the cursor are those turns' alone (I4: a tag-scoped panel token's session lists its tagged turns, never the rest). Compacted still reads the session's compaction markers, which carry no metadata.

IDs keeps the sessions whose id is one of them, empty any: one read gives a known set of sessions their rows (Studio's timeline reads the sessions a session forked so, U15.2-fix) — every aggregate still over all of each one's turns.

Awaiting, Failed, Compacted, Lineage and Since filter the groups (U16, UX-B-28), each SessionTurnFacts' rule stated in SQL over the scoped turns: Awaiting keeps the sessions whose newest turn is an open park (SessionRow.Pending > 0), Failed those with a failed turn (SessionRow.FailedTurns > 0: a turn thread's busy policy ended, or a cancelled one, is no failure), Compacted those thread compacted (a compaction marker in the session_events index, under PublicID that public id's), Lineage "child" the delegated sessions (ParentSession set) and "fork" the forked ones (ForkedFrom set) — any other value keeps none — and Since those last seen at or after it. Status keeps the sessions whose newest turn's outcome (OutcomeOf, the session page's word) is it: one of the four derived statuses, so failed leaves out a newest turn that was cancelled or busy-ended, and succeeded one with pending calls (parked, decided or not), though SessionRow.Status reads them failed and succeeded; a read-time word (parked, cancelled) keeps none — Awaiting is the session's parked.

Example (Meta)

SessionQuery.Meta scopes the turns a session is grouped over (I4): a session whose turns carry a tag only partly — a batch that started mid-conversation — lists its tagged turns alone, as PublicID scopes a session to one public id's turns.

package main

import (
	"context"
	"fmt"
	"log"
	"time"

	"github.com/weftgo/weft/obsdb"
	"github.com/weftgo/weft/obsdb/sqlite"
)

func main() {
	db, err := sqlite.Open(":memory:")
	if err != nil {
		log.Fatal(err)
	}
	defer func() { _ = db.Close() }()
	ctx := context.Background()
	at := time.Now().UTC().Add(-time.Minute)
	var recs []obsdb.Record
	for turn, batch := range []string{"", "b1", "b1"} { // turn 1 untagged
		id := fmt.Sprintf("s_mix-t%d", turn+1)
		attrs := map[string]any{"weft.record": "event", "weft.event.type": "run_start", "weft.event.pos": int64(0),
			"weft.run.id": id, "weft.session.id": "s_mix", "weft.turn": int64(turn + 1)}
		if batch != "" {
			attrs["arena.batch"] = batch
		}
		recs = append(recs, obsdb.Record{Time: at.Add(time.Duration(turn) * time.Second), EventName: "weft.event",
			Body: `{"type":"run_start","id":"` + id + `"}`, Attrs: attrs, Service: "ex", Severity: 9})
	}
	if err := db.Write(ctx, obsdb.Batch{Records: recs}); err != nil {
		log.Fatal(err)
	}
	for _, q := range []obsdb.SessionQuery{{}, {Meta: map[string]string{"arena.batch": "b1"}}} {
		page, err := db.Sessions(ctx, q)
		if err != nil {
			log.Fatal(err)
		}
		fmt.Println(page.Sessions[0].ID, page.Sessions[0].Turns)
	}
}
Output:
s_mix 3
s_mix 2
Example (States)

The sessions list's state filters (U16, U16-fix) and a session's events of a few kinds: a session thread compacted, its newest turn succeeded — SessionQuery's Compacted and Status keep it — and its compaction marker read alone among its events (SessionEventQuery.Kinds, what Studio's turn marks read).

package main

import (
	"context"
	"fmt"
	"log"
	"time"

	"github.com/weftgo/weft/obsdb"
	"github.com/weftgo/weft/obsdb/sqlite"
)

func main() {
	db, err := sqlite.Open(":memory:")
	if err != nil {
		log.Fatal(err)
	}
	defer func() { _ = db.Close() }()
	ctx := context.Background()
	at := time.Now().UTC().Add(-time.Minute)
	ident := map[string]any{"weft.run.id": "s_ex-t1", "weft.session.id": "s_ex", "weft.public_id": "pub_ex", "weft.turn": int64(1)}
	span := obsdb.Span{TraceID: "0102030405060708090a0b0c0d0e0f10", SpanID: "0a0b0c0d0e0f0102",
		Name: "invoke_agent ex", Kind: 1, Start: at, End: at.Add(time.Second), StatusCode: 1, Service: "ex",
		Attrs: map[string]any{"gen_ai.operation.name": "invoke_agent"}}
	label := obsdb.Record{Time: at, EventName: "weft.thread.label", Body: `{"kind":"label","entry":"e1","name":"start"}`,
		Attrs: map[string]any{"weft.record": obsdb.RecordThread}, Service: "ex", Severity: 9}
	marker := obsdb.Record{Time: at.Add(time.Second), EventName: "weft.compaction", Body: `{"scope":"session","hash":"sha256:ex","replaced":4}`,
		Attrs:   map[string]any{"weft.record": obsdb.RecordCompaction, "weft.compaction.scope": "session", "weft.compaction.hash": "sha256:ex"},
		Service: "ex", Severity: 9}
	for k, v := range ident {
		span.Attrs[k], label.Attrs[k], marker.Attrs[k] = v, v, v
	}
	if err := db.Write(ctx, obsdb.Batch{Spans: []obsdb.Span{span}, Records: []obsdb.Record{label, marker}}); err != nil {
		log.Fatal(err)
	}
	page, err := db.Sessions(ctx, obsdb.SessionQuery{Compacted: true, Status: obsdb.StatusSucceeded})
	if err != nil {
		log.Fatal(err)
	}
	for _, r := range page.Sessions {
		fmt.Println(r.ID, r.Status)
	}
	evs, err := db.(obsdb.SessionEventReader).SessionEvents(ctx, obsdb.SessionEventQuery{SessionID: "s_ex", Kinds: []string{obsdb.RecordCompaction}})
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(len(evs.Events), evs.Total, obsdb.SessionEventKind(evs.Events[0]))
}
Output:
s_ex succeeded
1 1 compaction

type SessionRow

type SessionRow struct {
	ID, PublicID, Agent string
	Turns               int
	FirstSeen, LastSeen time.Time
	Status              Status // the newest turn's
	Usage               core.Usage
	// SummarizerUsage is what weft/thread's summarizer model calls — a
	// compaction's summary, a SummarizeLeft branch summary — cost the
	// session: the summed usage of the chat spans marked weft.chat.role
	// summarizer under its runs (audit O15, record contract §6). A
	// separate, labelled figure, never part of Usage (the turns' own)
	// nor of any run's usage: a reader adds it for the session's whole
	// cost. Zero when the session summarized nothing (or before weft
	// recorded the spans).
	SummarizerUsage core.Usage
	// SummarizerModels is SummarizerUsage split by the model that
	// summarized — each summarizer span's gen_ai.provider.name and
	// gen_ai.request.model (Provider and Model "" when the span named
	// none) — sorted by provider, then model: what a reader prices each
	// part at (U15.1-fix, UX-B-13). Its parts sum to SummarizerUsage;
	// nil when the session summarized nothing.
	SummarizerModels []ModelUsage
	// Title is the newest turn's (by turn, then start) non-empty
	// SessionTitleKey — thread's Title as that turn started, a label
	// that survives content-off — "" when no turn carried one.
	Title string
	// FirstPrompt is the first turn's (lowest turn, then earliest
	// start) RunRow.PromptPreview: "" when that turn's records carried
	// no content (content-off) or no user text — never a later turn's.
	FirstPrompt string
	// ContentOff marks a session whose first turn (FirstPrompt's) was
	// recorded without content: events stored and not one messages
	// record (RunRow.EventCount > 0, MessageCount 0), and not Derived,
	// nor running or settling — its messages records may still arrive
	// (SessionContentOff). The list says "(content not captured)" for it
	// instead of the id.
	ContentOff bool
	// ParentSession, ParentCall and ParentRun are a delegated session's
	// lineage (thread/pool, ADR 0022): the session that delegated it
	// (its turns' weft.session.parent), the delegating tool call
	// (weft.session.parent_call) and the parent's run that call
	// belonged to (its first turn's CallerRunID, else — recorded before
	// weft 0.18 — its ParentRunID). "" for a session nothing delegated,
	// and ParentRun "" for a child submitted outside a run (p.Submit).
	ParentSession, ParentCall, ParentRun string
	// The turns' state over time (U16; UX-B-15, UX-B-16), read by
	// SessionTurnFacts over the same turns as every aggregate: Pending
	// is the newest turn's pending calls when it is an open park
	// (succeeded, pending calls, no resumer: the session awaits a
	// decision), else 0; FailedTurns counts the failed turns that are
	// neither ended nor cancelled; EndedTurns, apart, the turns thread's
	// busy policy ended (Interrupt or Rollback: a cancel cause of
	// BusyCancelCause, never a failure); CancelledTurns the other failed
	// turns whose error type is ErrorTypeCancelled (a caller's cancel, a
	// deadline, thread.closed, studio.cancel: no failure of the agent);
	// Resumes the turns that resume a parked one (a stored ResumedFrom),
	// so Turns - Resumes is the session's exchanges; Outcomes the newest
	// MaxOutcomes turns' outcomes, oldest first (Turns - len(Outcomes)
	// older ones are not in it); ForkedFrom the weft.session.forked_from
	// (<origin>#<entry>) its turns carry (the greatest, should they
	// differ), "" for a session no Fork made. A backend counts in SQL and
	// reads only the newest MaxOutcomes turns for the strip and Pending.
	Pending, FailedTurns, EndedTurns, CancelledTurns, Resumes int
	Outcomes                                                  []TurnOutcome
	ForkedFrom                                                string
}

SessionRow is one session: a GROUP BY over its own turns (D3; record contract §6) — the non-playground runs carrying its weft.session.id that are turns of it: a run whose id is <session>-t<turn> (thread's own id rule, a turn started inside a tool handler included), or one linked to no run at all (neither ParentRunID nor CallerRunID: a run that set weft.session.id itself). A Subagent child or a Generate inside a handler inherits the session id through ctx metadata and is no turn; a thread/pool child session is a session of its own, its lineage on ParentSession.

type SkippedTool added in v0.20.0

type SkippedTool struct {
	Name   string `json:"name"`
	Reason string `json:"reason"`
	Source string `json:"source,omitempty"`
}

SkippedTool is one tool an importer left out, and why: the tools record's skipped[] entry. Source is the source it would have come from ("mcp:billing").

type SourceDetail added in v0.20.0

type SourceDetail struct {
	Server     string `json:"server,omitempty"`
	Transport  string `json:"transport,omitempty"`
	Prefix     string `json:"prefix,omitempty"`
	RemoteName string `json:"remote_name,omitempty"`
}

SourceDetail is a tool's origin in parts (record contract §4, O8): weft/mcp's server name, transport, import prefix and the tool's name on the server.

type Span

type Span struct {
	TraceID, SpanID, ParentSpanID string // lowercase hex
	Name                          string
	Kind                          int // OTLP SpanKind
	Start, End                    time.Time
	StatusCode                    int // 0 unset, 1 ok, 2 error
	StatusMessage                 string
	Service                       string         // resource service.name
	Attrs                         map[string]any // string, bool, int64, float64, []any
	Resource                      map[string]any
	Events                        []SpanEvent // exceptions etc.
}

Span is one finished span, OTLP-shaped.

func FromOTLPTraces

func FromOTLPTraces(req *coltracepb.ExportTraceServiceRequest) []Span

FromOTLPTraces decodes an OTLP/HTTP trace export request into the model. The receiver (studio/ingest) and a stock collector's dump produce the same rows the local sink does; non-weft spans pass through unchanged — a polyglot trace is a first-class citizen.

func StockRunSpans added in v0.15.0

func StockRunSpans(spans []Span, id string) []Span

StockRunSpans is the spans of the derived run id among spans (one trace's, as DB.Trace reads them): StockRun.Spans, or nil when no root of that id is among them.

type SpanEvent

type SpanEvent struct {
	Time  time.Time
	Name  string
	Attrs map[string]any
}

SpanEvent is one event recorded on a span (an exception, a marker).

type SpanKey added in v0.14.0

type SpanKey struct {
	TraceID, SpanID string
}

SpanKey names one span.

func DeletedSpans added in v0.14.0

func DeletedSpans(all []SpanLink, deleted map[string]bool) []SpanKey

DeletedSpans is the rule both backends delete spans by (DB.DeleteRun, DeleteSession, Sweep): the deleted runs' own spans and, transitively, the non-weft spans below them — a span carrying a run id stops the walk, so a run that is kept keeps its spans. A derived run (§9 of the record contract: no run id on any span) is named by its id: its root span seeds the walk, which takes its subtree — a nested root's too, which is its child run, deleted with it (StockTraces names the traces to read). It is the rule a run's app logs are attributed by (DB.OtherLogs), so a deleted run takes exactly the log records its logs page showed: the backends delete the non-weft log records emitted under the returned spans. all holds every span of the traces the deleted runs' spans are in; deleted is the set of run ids being deleted. The keys come back sorted.

type SpanLink struct {
	TraceID, SpanID, ParentSpanID, RunID string
}

SpanLink is one stored span's place in its trace, what DeletedSpans walks: its key, its parent and the weft.run.id it carries ("" for a non-weft span).

type StatsBucket added in v0.19.0

type StatsBucket struct {
	Runs, Failed, Calls, FailedCalls int64
	InputTokens, OutputTokens        int64
}

StatsBucket is one bucket of a group's series: the sums StatsGroup names, over the runs that started in the bucket.

type StatsBy added in v0.19.0

type StatsBy string

StatsBy is what Stats groups by.

const (
	// StatsByAgent groups runs by their agent name (RunRow.Agent; ""
	// for an unnamed agent).
	StatsByAgent StatsBy = "agent"
	// StatsByErrorType groups the failed runs by RunRow.ErrorType (""
	// for a failure whose root carried none).
	StatsByErrorType StatsBy = "error_type"
	// StatsByModel groups model calls (chat spans) by the model they
	// asked for (gen_ai.request.model). The key is the model name alone:
	// two providers' models of the same name are one group (its cost is
	// still priced per provider/model part, StatsGroup.Models).
	StatsByModel StatsBy = "model"
	// StatsByTool groups tool calls (execute_tool spans) by tool name.
	StatsByTool StatsBy = "tool"
	// StatsByStep groups model calls by their step index
	// (weft.step.index, the key its decimal string). A replay's calls
	// key on the replay's own step index, counted from its from_step's
	// 0 — not the source's step it re-asks (the playground diff's
	// aligned step): a replay from step 3 adds to step 0's group. Studio
	// leaves playground runs out by default, which removes the common
	// case.
	StatsByStep StatsBy = "step"
)

func (StatsBy) RunLevel added in v0.19.0

func (by StatsBy) RunLevel() bool

RunLevel says the groups of by are runs (agent, error_type) rather than spans (model, tool, step).

type StatsCell added in v0.19.0

type StatsCell struct {
	Key                                       string
	Bucket                                    int
	Runs, Failed, Calls, FailedCalls, Retried int64
	Denials                                   int64
	InputTokens, OutputTokens                 int64
	CachedInputTokens, CacheWriteTokens       int64
	ReasoningTokens                           int64
	// DelegatedRuns is summed into the group, never into a bucket.
	DelegatedRuns int64
}

StatsCell is one group's sums in one bucket, as a backend's GROUP BY answers them; BuildStats adds the cells of one (Key, Bucket) together (a run-side cell and a span-side cell, say). Bucket is clamped into the series.

type StatsGroup added in v0.19.0

type StatsGroup struct {
	Key                         string
	Runs, Failed                int64
	Calls, FailedCalls, Retried int64
	Denials                     int64
	Usage                       core.Usage
	Models                      []ModelUsage
	Delegated                   core.Usage
	DelegatedModels             []ModelUsage
	DelegatedRuns               int64
	Duration, TTFT              Percentiles
	Series                      []StatsBucket
}

StatsGroup is one key's numbers over the window. What a field counts depends on the query's By:

  • Runs: the runs in the group (agent, error_type: the runs; model, tool, step: the distinct runs that made a call in it). Failed: those of them whose run failed (RunRow.Status failed).
  • Calls: model calls (chat spans) — the group's runs' for agent and error_type, the group's own for model and step — or tool calls (execute_tool spans) for tool; a summarizer's call (weft.chat.role) and a call a resume denied (weft.tool.denied, never run) are none. FailedCalls: those that failed — status Error; for a tool call the record contract's tool_errors rule (DENIED, RETRY and the context errors are no failure).
  • Retried: the model calls that took more than one attempt (their attempt spans' highest weft.attempt.index above 1); 0 for tool.
  • Denials: the record contract's problems.denied rule over the group's spans — execute_tool spans with error.type DENIED plus each invoke_agent span's weft.run.denied (agent, error_type, tool; 0 for model and step, whose spans carry no denial).
  • Usage and Models: the model calls' own usage (chat spans' gen_ai.usage.*), split by the provider and model each call asked for, sorted by provider then model — never a run's rolled-up total, so a subagent's tokens are its own agent's, not its parent's too. Zero for tool.
  • Duration: run durations (Finished − Started, the runs that finished) for agent and error_type; the calls' span durations for model, tool and step. TTFT: the model calls' weft.ttft_ms (none for tool). Both in milliseconds, nearest-rank percentiles.
  • Delegated, DelegatedModels and DelegatedRuns (by=agent alone): the model calls of the runs below the group's runs in their Subagent trees (core child ids: <run id>/<step>/<call>, at any depth) whose own agent is another — what the agent's subagents spent for it, apart from its own Usage — each call once however many of the group's runs it sits under, and how many such runs made a call. Under a scope (RunQuery.PublicID, Meta: a panel token's) a descendant is read when its own row is in the scope and so is every run between it and the group's run — the run tree's rule (TreeQuery's scope): a run under a hidden one is hidden with its subtree, so the figure matches what the run document and the span tree show. Zero for every other By. A call can sit in several groups' delegated figures (boss → helper → boss: the grandchild's call is helper's delegated, and the helper's call boss's), so delegated figures are never summed across groups — the groups' own Usage sums to the true total.
  • Series: the same sums per bucket of the window, Buckets of them.

type StatsModelPart added in v0.19.0

type StatsModelPart struct {
	Key       string
	Delegated bool
	ModelUsage
}

StatsModelPart is one group's usage on one provider/model; with Delegated, its subagents' (StatsGroup.DelegatedModels).

type StatsPage added in v0.19.0

type StatsPage struct {
	By           StatsBy
	Since, Until time.Time
	Buckets      int
	Bucket       time.Duration // each bucket's width
	Groups       []StatsGroup
	GroupsCut    int
}

StatsPage is one Stats answer: the window and its buckets as read, the groups — most runs first, then most calls, then by key (by=step: by step index, ascending) — and how many groups MaxStatsGroups cut.

func BuildStats added in v0.19.0

func BuildStats(q StatsQuery, cells []StatsCell, models []StatsModelPart, pcts []StatsPercentiles) StatsPage

BuildStats is a backend's answer assembled from its GROUP BY rows: the cells summed per key and bucket, the model parts and percentiles attached, the groups sorted and cut at MaxStatsGroups. A key present in models or pcts alone still gets its group. One rule for every backend, so their pages differ only where their rows do.

type StatsPercentiles added in v0.19.0

type StatsPercentiles struct {
	Key            string
	Duration, TTFT Percentiles
}

StatsPercentiles is one group's duration and TTFT percentiles.

type StatsQuery added in v0.19.0

type StatsQuery struct {
	RunQuery
	By           StatsBy
	Since, Until time.Time // both required, Since before Until
	Buckets      int       // 0 = StatsBuckets, max MaxStatsBuckets
}

StatsQuery selects what Stats aggregates. Its RunQuery carries the filters /api/runs reads — agent, public id (the panel token's scope), metadata (tag.<k>), parent ("*" every run, children included; "" the top-level runs only), playground, experiment, status — with the same meaning (Playground nil: both; Studio's /api/stats sends false unless asked, so health numbers never mix in replays); RunQuery's Before, BeforeID, Limit and Oldest are not read. A derived run (RunRow.Derived: built from stock spans) counts as a run — so a group's Runs is the runs list's count under the same filters — with no calls: it has no weft spans to aggregate.

The window is the runs' start: a run is in when Since ≤ Started < Until, and every span of a run in the window is counted with it, in its run's bucket — so a run's model calls are never split across buckets and Runs sums over the series.

type Status

type Status string

Status is a run's derived state. running, succeeded and failed are what the data says; interrupted is what a reader concludes from a stale last-seen (DeriveStatus) — never stored, so a crash leaves evidence and a live run in another process is not mistaken for a corpse (the store's rule, moved to last-seen).

const (
	StatusRunning     Status = "running"
	StatusSucceeded   Status = "succeeded"
	StatusFailed      Status = "failed"
	StatusInterrupted Status = "interrupted"
)
const (
	// StatusParked is an open park: succeeded, pending > 0, and no run
	// whose resumed_from names it (RunRow.ResumedBy empty).
	StatusParked Status = "parked"
	// StatusCancelled is failed with error_type cancelled (RunErrorType:
	// the run's context ended).
	StatusCancelled Status = "cancelled"
)

The read-time status words (I3; Studio's lib/status.ts vocabulary): never a row's Status, only a RunQuery filter, each mapped onto the stored columns by both backends (docs/record-contract.md §6).

func DeriveStatus

func DeriveStatus(spanFailed, ended bool, lastSeen, now time.Time) Status

DeriveStatus applies the four-row table every backend reads through:

the invoke_agent span ended with status error          →  failed
a run_finish record exists, or the invoke_agent span
ended with any other status (Ok, Unset)                 →  succeeded
neither, and now - LastSeen ≤ InterruptedAfter          →  running
neither, and older                                      →  interrupted

spanFailed is the invoke_agent span's error status; ended is the run_finish record's presence or the invoke_agent span's (RunRow.RootEnded) — a span is exported only once it ended, so a run whose root arrived is over even when its run_finish was lost (a tracer-only app's run, or one whose record tail a destination dropped). LastSeen is the newest record or span time for the run, heartbeats included. failed outranks succeeded: a run whose span errored failed, whatever records raced behind it. Pending > 0 on a succeeded run means parked at an approval, not stuck.

func ParseStatus added in v0.17.0

func ParseStatus(s string) (Status, error)

ParseStatus reads a status filter as the read API takes it: "" is any, a word of StatusWords is itself, anything else wraps ErrInvalidStatus — never an empty answer for a typo.

Example
package main

import (
	"errors"
	"fmt"

	"github.com/weftgo/weft/obsdb"
)

func main() {
	st, err := obsdb.ParseStatus("parked")
	fmt.Println(st, st.Stored(), err)
	_, err = obsdb.ParseStatus("completed")
	fmt.Println(errors.Is(err, obsdb.ErrInvalidStatus))
	fmt.Println(err)
}
Output:
parked succeeded <nil>
true
status must be running, succeeded, failed, interrupted, parked or cancelled ("completed" given)

func (Status) Stored added in v0.17.0

func (s Status) Stored() Status

Stored is the derived status a filter word narrows: parked reads succeeded runs, cancelled failed ones (each with its own condition beside it, the backends'); the four derived statuses are themselves.

type StepMessages added in v0.13.0

type StepMessages struct {
	Step     int
	Messages []core.Message
	View     *Compaction
	Derived  bool
}

StepMessages is the messages one step's model call carried — what the model saw at that step (ADR 0028 §8, ADR 0029): the replay prefix for from_step Step, the "compacted messages" a fixture keys on. Step is the step asked for. View is the run-scope compaction view the step's request named (its messages_ref.index), nil when the request carried the plain transcript; Messages is then the growth records up to the view's index with its range replaced, else the growth records up to the request's own index. Derived is true when no request record placed the messages — a run written before ADR 0028, or a request whose messages_ref a content-off chain stripped — and the from_step cut rule did instead: the transcript before step Step's assistant message (no view can apply there: nothing names one).

func AssembleStep added in v0.13.0

func AssembleStep(batches []TranscriptBatch, requests []RequestRecord, compactions []Compaction, step int) (StepMessages, error)

AssembleStep is MessagesAsOf over records already read: the run's growth batches (TranscriptBatches), its request records (any steps; the answering — last — attempt of step is used) and its compactions. A reader assembling several steps of one run decodes the batches once with NewAssembler instead.

func MessagesAsOf added in v0.13.0

func MessagesAsOf(ctx context.Context, db DB, runID string, step int) (StepMessages, error)

MessagesAsOf answers the messages step step of run runID's model call carried (StepMessages), read through TranscriptBatches, Requests and Compactions — one assembly (AssembleStep) for every backend and every reader: Studio's transcript route with ?step=, weft/runtime's replay prefix, the wefttest fixtures. ErrNotFound for an unknown run and for a step the run never reached (no request record and no assistant message of that step); an error wrapping ErrStepMessages when the records do not rebuild what the request names.

A session-scope compaction marker is never applied: the run's input record already holds the compacted context literally. A run-scope view applies only to the request that names it, never to a later step's.

Example

ADR 0028 §8's worked example, read back as the model saw it: growth records 0 (the input, u1), 1 (a1) and 2 (t1), then step 1's PrepareStep sent [u1, s] — the core recorded view 3 replacing seqs [1, 3) with s, and step 1's request names it. MessagesAsOf answers each step's messages: step 0 the plain transcript, step 1 the view applied (the replay prefix for from_step 1, ADR 0029).

package main

import (
	"context"
	"errors"
	"fmt"
	"log"
	"time"

	"github.com/weftgo/weft/obsdb"
	"github.com/weftgo/weft/obsdb/sqlite"
)

func main() {
	db, err := sqlite.Open(":memory:")
	if err != nil {
		log.Fatal(err)
	}
	defer func() { _ = db.Close() }()
	ctx := context.Background()
	at := time.Unix(0, 1790845923120000000).UTC()
	text := func(role, t string) string {
		return `{"role":"` + role + `","content":[{"type":"text","text":"` + t + `"}]}`
	}
	msgs := func(index, step int64, body string, extra map[string]any) obsdb.Record {
		attrs := map[string]any{"weft.record": "messages", "weft.run.id": "run_1",
			"weft.messages.index": index, "weft.step.index": step}
		for k, v := range extra {
			attrs[k] = v
		}
		return obsdb.Record{Time: at, EventName: "weft.messages", Body: body, Attrs: attrs}
	}
	request := func(index, step int64, ref int64, count int) obsdb.Record {
		return obsdb.Record{Time: at, EventName: "weft.request",
			Body: fmt.Sprintf(`{"step":%d,"attempt":1,"messages_ref":{"index":%d,"count":%d},"tools":{"catalog_hash":"","names":[]},"params":{},"model":{"name":"m"}}`, step, ref, count),
			Attrs: map[string]any{"weft.record": "request", "weft.run.id": "run_1",
				"weft.request.index": index, "weft.step.index": step, "weft.attempt.index": int64(1)}}
	}
	err = db.Write(ctx, obsdb.Batch{Records: []obsdb.Record{
		{Time: at, EventName: "weft.event", Body: `{"type":"run_start","id":"run_1"}`,
			Attrs: map[string]any{"weft.record": "event", "weft.run.id": "run_1", "weft.event.type": "run_start", "weft.event.pos": int64(0)}},
		msgs(0, 0, "["+text("user", "u1")+"]", map[string]any{"weft.messages.input": true}),
		request(0, 0, 0, 1),
		msgs(1, 0, `[{"role":"assistant","content":[{"type":"tool_call","id":"c1","name":"lookup","args":{}}]}]`, nil),
		msgs(2, 0, `[{"role":"tool","content":[{"type":"tool_result","call_id":"c1","name":"lookup","content":"t1"}]}]`, nil),
		msgs(3, 1, "["+text("user", "s")+"]", map[string]any{"weft.messages.reason": "compacted",
			"weft.messages.from_seq": int64(1), "weft.messages.to_seq": int64(3),
			"weft.compaction.scope": "run", "weft.compaction.hash": "h1"}),
		request(1, 1, 3, 2),
	}})
	if err != nil {
		log.Fatal(err)
	}
	for _, step := range []int{0, 1} {
		sm, err := obsdb.MessagesAsOf(ctx, db, "run_1", step)
		if err != nil {
			log.Fatal(err)
		}
		var roles []string
		for _, m := range sm.Messages {
			roles = append(roles, string(m.Role))
		}
		fmt.Println(step, roles, sm.View != nil)
	}
	_, err = obsdb.MessagesAsOf(ctx, db, "run_1", 5)
	fmt.Println(errors.Is(err, obsdb.ErrNotFound))
}
Output:
0 [user] false
1 [user user] true
true

type StepMessagesError added in v0.13.0

type StepMessagesError struct {
	Step int
	Hole Hole
	Msg  string
}

StepMessagesError is MessagesAsOf's and AssembleStep's refusal, with the hole that explains it (ADR 0028 §11's table): HoleGap when a record the step's request names is missing or does not fit — a growth record below its index, the growth record its plain ref names, a view that does not fit or was never written (a rewritten request with no view index), a body that is not messages — and HoleStripped when the run stored no messages at all (content capture off, or a content-off chain). errors.Is(err, ErrStepMessages) holds; a reader badges Hole and never guesses.

func (*StepMessagesError) Error added in v0.13.0

func (e *StepMessagesError) Error() string

func (*StepMessagesError) Is added in v0.13.0

func (e *StepMessagesError) Is(target error) bool

Is makes a StepMessagesError an ErrStepMessages.

type StockRun added in v0.15.0

type StockRun struct {
	Row   RunRow
	Spans []Span
}

StockRun is one run derived from stock spans: its row and its spans.

Row is the run row of §9: Derived true; ID StockRunID's; TraceID, Service, Agent and SessionID (the conversation id, "" when the root carries none) the root's; Provider and Model the root's gen_ai.provider.name and gen_ai.request.model, else the first chat span's; Started and Finished the root's start and end; LastSeen the latest time in the subtree; Status StatusFailed with the root's status description as Err (StockErrNoDescription when it has none) when the root's status is Error, else StatusSucceeded — never running or interrupted: a span is exported only once it ended, so a stored root is a finished run, and there is no heartbeat to read a liveness from; Steps the subtree's chat span count; MessageCount the subtree's chat spans that carry the conventions' message content (StockHasMessages: each a batch of the messages the model was sent and answered, as a messages record is a weft run's); Usage the root's gen_ai.usage.* when it carries any (verbatim, taken as its own figure), else the sum over the subtree's chat spans; ParentRunID and ParentCallID, for a root nested in another root's subtree, that root's derived id and the gen_ai.tool.call.id of the nearest execute_tool span above it. Turn, PublicID, StopReason, Pending, Meta, the other counts and the hashes are not derived (zero).

Spans is the run's subtree — the root and every span under it by parent span id, down to (not into) a nested root or a span carrying weft.run.id — in start order (ties by span id), copied. The chat spans carry weft.step.index (their 0-based ordinal by start) and the execute_tool spans weft.tool.seq (their 1-based ordinal) and weft.step.index (the latest chat span that started before them; absent when none did): the reader's positions under weft's own keys, so every consumer of a run's spans (the waterfall, the tools list) reads one shape — and gen_ai.operation.name on both, the operation stockOp resolved (from the span name when the producer set none or set it empty), so a reader that switches on the attribute sees what the derivation saw. The producer's attributes are otherwise as stored; the row's Derived flag (the derived badge) says who numbered them.

func DeriveStockRuns added in v0.15.0

func DeriveStockRuns(spans []Span) []StockRun

DeriveStockRuns derives every run the spans hold (§9): one per stock root (IsStockRoot, or a top-level invoke_agent span without a conversation id), its subtree the spans under it down to a nested root. A span is counted toward a root only through a complete chain of stored parents: one whose chain reaches a span not (yet) stored, a span with no parent, or a span carrying weft.run.id belongs to no derived run — so as later batches complete the tree a run's steps and usage only grow, and the whole set converges on the full tree's answer whatever order the spans arrived in. The spans may hold several traces; a duplicate (trace, span) is read once. The runs come back ordered by start, then id.

Example

A Python agent's stock GenAI spans — no weft.* attribute — are a run (docs/record-contract.md §9): the invoke_agent span with a conversation id is its root, its chat spans its steps, its execute_tool spans its tool calls.

package main

import (
	"fmt"
	"time"

	"github.com/weftgo/weft/obsdb"
)

var stockEpoch = time.Date(2026, 10, 1, 10, 0, 0, 0, time.UTC)

func stockAt(ms int) time.Time { return stockEpoch.Add(time.Duration(ms) * time.Millisecond) }

func main() {
	const trace = "4bf92f3577b34da6a3ce929d0e0e4736"
	spans := []obsdb.Span{
		{TraceID: trace, SpanID: "00f067aa0ba902b7", Name: "invoke_agent support",
			Start: stockAt(0), End: stockAt(3200), StatusCode: 1, Service: "py-support",
			Attrs: map[string]any{"gen_ai.operation.name": "invoke_agent", "gen_ai.agent.name": "support",
				"gen_ai.conversation.id": "conv_7f3a"}},
		{TraceID: trace, SpanID: "a1b2c3d4e5f60001", ParentSpanID: "00f067aa0ba902b7", Name: "chat gpt-5",
			Start: stockAt(50), End: stockAt(1250),
			Attrs: map[string]any{"gen_ai.operation.name": "chat", "gen_ai.provider.name": "openai",
				"gen_ai.request.model": "gpt-5", "gen_ai.usage.input_tokens": int64(812), "gen_ai.usage.output_tokens": int64(41)}},
		{TraceID: trace, SpanID: "a1b2c3d4e5f60002", ParentSpanID: "00f067aa0ba902b7", Name: "execute_tool lookup_order",
			Start: stockAt(1300), End: stockAt(1420),
			Attrs: map[string]any{"gen_ai.operation.name": "execute_tool", "gen_ai.tool.name": "lookup_order",
				"gen_ai.tool.call.id": "call_abc"}},
	}
	for _, r := range obsdb.DeriveStockRuns(spans) {
		fmt.Println(r.Row.ID)
		fmt.Println(r.Row.Derived, r.Row.Status, r.Row.SessionID, r.Row.Provider+"/"+r.Row.Model, r.Row.Steps, r.Row.Usage.InputTokens)
		fmt.Println(r.Row.RequestsHole())
		for _, s := range r.Spans {
			fmt.Println(s.Name, s.Attrs["weft.step.index"], s.Attrs["weft.tool.seq"])
		}
	}
}
Output:
otel-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7
true succeeded conv_7f3a openai/gpt-5 1 812
derived
invoke_agent support <nil> <nil>
chat gpt-5 0 <nil>
execute_tool lookup_order 0 1

type StoredCommand added in v0.15.0

type StoredCommand struct {
	ID        string
	RuntimeID string // the runtime the command was enqueued to
	// Kind is the frame kind the caller names — "" or run, cancel,
	// approve, breakpoints, steer; stored verbatim, never interpreted.
	Kind  string
	State string // one of the five Command* states
	// RunID is the run an ack named ("" until one did).
	RunID string
	// PublicID is the public id the command is scoped to; Fork marks a
	// thread=fork command (its runs are sessions the runtime owns).
	PublicID string
	Fork     bool
	// Agent, SourceRunID and ExperimentID are the command's own, copied
	// out of Body for the reader that lists without decoding it.
	Agent, SourceRunID, ExperimentID string
	// Status is a finished command's run outcome (succeeded | failed);
	// Error why, when there is a why (a rejection's reason, a failed
	// run's error, what made it lost).
	Status, Error string
	// Created and Updated bound the row's life; AcceptedAt is when the
	// accepted ack arrived and LastBeat the last heartbeat (a repeated
	// accepted ack) — zero when none did. A restarted Studio decides a
	// silent accepted command lost from these two, never re-issuing it.
	Created, Updated, AcceptedAt, LastBeat time.Time
	Body                                   json.RawMessage
	Ack                                    json.RawMessage
}

StoredCommand is one command row. The indexed columns are what the queries select by and what a restart decides from; Body is the whole wire command (the runtime link's frame data) and Ack the last ack the runtime posted for it, both stored verbatim (nil reads back nil).

func CommandByRun added in v0.15.0

func CommandByRun(ctx context.Context, s CommandStore, runID string) (StoredCommand, bool, error)

CommandByRun answers the newest command whose RunID is runID; false when none names it.

type StoredManifest added in v0.16.0

type StoredManifest struct {
	// Hash is sha256 of Body, lowercase hex.
	Hash string
	// Body is the one-agent core.Manifest document, verbatim.
	Body json.RawMessage
	// Source is where Studio first saw it: ManifestSourceFile or
	// ManifestSourceRuntime ("" when the writer did not say).
	Source string
	// FirstSeen is when it was first stored.
	FirstSeen time.Time
}

StoredManifest is one stored agent configuration.

type StoredRecord added in v0.10.0

type StoredRecord struct {
	Index          int64
	Step           int
	Time           time.Time
	Body           []byte
	Content        string
	TruncatedBytes int64
	SystemHash     string
	CatalogHash    string
}

StoredRecord is one request, prompt or tools record as a backend stored it: its index, stored step (-1 when absent), time and body, and the four attributes the readers need — weft.content (the mark), weft.content.truncated_bytes, weft.system.hash and weft.catalog.hash. A backend fills it from its columns (or its attribute column) and builds the read type through RequestRecordOf, PromptRecordOf or ToolsRecordOf, so every backend parses alike.

type StoredRuntime added in v0.15.0

type StoredRuntime struct {
	ID, Service, Host, Cwd string
	Pid                    int
	Agents                 []string
	Registered, LastSeen   time.Time
	Body                   json.RawMessage
}

StoredRuntime is one runtime's latest registration. Agents are its agents' names; Body is the whole registration as the runtime posted it. Cwd is the app's working directory when the registration carries one ("" otherwise).

type ToolEntry added in v0.10.0

type ToolEntry struct {
	Name           string          `json:"name"`
	Description    string          `json:"description"`
	Schema         json.RawMessage `json:"schema"`
	TimeoutMS      int64           `json:"timeout_ms"`
	Approval       bool            `json:"approval"`
	Replay         string          `json:"replay"`
	MaxResultBytes int             `json:"max_result_bytes"`
	Sequential     bool            `json:"sequential"`
	Source         string          `json:"source"`
	// Output marks the tool Output registered (submit_output): the
	// run's typed answer (record contract §4, UX-S1-06).
	Output bool `json:"output,omitempty"`
	// SourceDetail is the tool's origin in parts (O8): nil when the
	// producer set none.
	SourceDetail *SourceDetail `json:"source_detail,omitempty"`
}

ToolEntry is one tool of a catalog: the model-visible triple and the policy chips (ADR 0028 §5).

type ToolsRecord added in v0.10.0

type ToolsRecord struct {
	Index          int64
	Time           time.Time
	Hash           string
	Tools          []ToolEntry
	Content        Hole
	TruncatedBytes int64
	// Skipped are the tools an importer left out of the offered
	// sources (record contract §4, O8: weft/mcp's ImportError), at most
	// 64; SkippedMore counts the rest. Nil when none was skipped.
	Skipped     []SkippedTool
	SkippedMore int
}

ToolsRecord is one stored tools record (ADR 0028 §5): the catalog of one distinct catalog hash, tools in name order. Content is HoleTruncated when a destination's cap dropped entries from the end (TruncatedBytes > 0), HoleDerived when the body did not parse (a malformed producer; Hash is then the weft.catalog.hash attribute). Hash is over the whole catalog.

func ToolsRecordOf added in v0.10.0

func ToolsRecordOf(r StoredRecord) ToolsRecord

ToolsRecordOf builds a ToolsRecord from a stored tools record. Content is HoleTruncated when a cap dropped entries; a body that does not parse (a malformed producer) reads HoleDerived, with no tools and the hash of the record's weft.catalog.hash attribute.

type TracePage added in v0.14.0

type TracePage struct {
	Traces       []TraceRow
	Total        int
	NextBefore   *time.Time
	NextBeforeID string
}

TracePage is one Traces result, paged as RunPage is: newest Start first (ties by TraceID, descending), NextBefore/NextBeforeID are the next TraceQuery.Before/BeforeID (nil and "" on the last page), traces sharing the last row's Start ride along up to MaxTies, and Total counts the whole match, cursor aside. A trace's Start is its earliest span, so a trace whose earlier span arrives between page reads can be listed again on a later page (never skipped); clients dedupe by TraceID.

type TraceQuery added in v0.14.0

type TraceQuery struct {
	// Service keeps the traces holding at least one span whose resource
	// service name is exactly this ("" any).
	Service string
	// PublicID keeps the traces every span of which carries
	// weft.public_id = PublicID — the rule a panel token's
	// /api/traces/{id} read applies, so a scoped list offers no trace
	// its detail would refuse (a non-weft span or another public id's
	// run in the trace excludes it).
	PublicID string
	// Meta keeps the traces holding, for each pair, a span whose
	// attributes carry it — run metadata rides every weft span verbatim
	// (core.Metadata, core.WithMetadata: tag.<k>=<v> and flow= on
	// /api/traces, U22). Empty: no filter.
	Meta map[string]string
	// ScopeMeta keeps the traces every span of which carries each pair
	// — PublicID's rule for a tag-scoped panel token (I4), so a scoped
	// list offers no trace its /api/traces/{id} read would refuse (a
	// span without the pair, a non-weft span included, excludes it).
	// Empty: no filter.
	ScopeMeta map[string]string
	// NotRun keeps the GenAI traces that make no run
	// (docs/record-contract.md §9, UX-H-02): a span without
	// weft.run.id whose GenAI operation StockCandidate reads
	// (invoke_agent, chat, execute_tool, by attribute or name), no span
	// carrying weft.run.id, and no derived run row of the trace — what
	// Studio's /runs counts as "GenAI traces that are not runs".
	NotRun bool
	// IDPrefix keeps the traces whose id starts with it, byte for byte
	// (RunQuery.IDPrefix's rule: no wildcard, no case folding) — the ⌘K
	// palette's lookup of a pasted short trace id. "" any.
	IDPrefix string
	Since    time.Time // Start at or after; zero = no bound
	Until    time.Time // Start before; zero = no bound
	Before   time.Time // cursor on Start; zero = newest
	// BeforeID, with Before, makes the cursor the pair (Start, TraceID),
	// as RunQuery.BeforeID: pass TracePage.NextBeforeID. A late earlier
	// span can re-list a trace on a later page (TracePage).
	BeforeID string
	Limit    int // 0 = 50, max 500
}

TraceQuery selects traces for Traces. The zero value lists every trace, newest first, 50 at a time. A trace's time is its Start: the earliest start of the spans it holds.

type TraceRoot added in v0.14.0

type TraceRoot struct {
	SpanID, Name string
}

TraceRoot is a trace's root span: its id and name.

type TraceRow added in v0.14.0

type TraceRow struct {
	TraceID    string
	Root       *TraceRoot
	Service    string
	Start, End time.Time
	SpanCount  int
	Error      bool
	RunIDs     []string
}

TraceRow is one trace in a list, computed from its spans: the time range they cover, how many there are (each (trace, span) once, whatever a retried batch duplicated), whether any ended with status error, the distinct weft.run.id values they carry and the ids of the runs derived from its stock GenAI spans (docs/record-contract.md §9; sorted: a subagent child shares its parent's trace, so both are here) and the root — the earliest span with no parent span id, nil when none arrived (only child spans were stored: a hole the reader badges). Service is the root's, or the earliest span's when the root is missing.

type TranscriptBatch added in v0.10.0

type TranscriptBatch struct {
	Index        int64
	Step         int
	Input        bool
	InputDerived bool // Input was inferred by the backend, not read from the record
	Messages     json.RawMessage
}

TranscriptBatch is one messages record as stored (ADR 0028 §8): readers place a batch by its Step, never by its neighbours. Step is -1 when the record carried no weft.step.index — a ClickHouse row written before migration 0004, or a producer that never stamped it; a reader that places such a batch anyway infers the step and says so (the derived badge, ADR 0028 §11). Input is the record's weft.messages.input: the input record — on a partial resume a prefix of what the run was fed, ending at the last assistant message with tool calls; the tail (the rebuilt tool message and what follows it) is the next batch, at step 0, not flagged. A backend that did not store the flag for a row (ClickHouse rows written before weft_records.Input existed) infers Input on index 0 — true unless the body is a lone assistant message, which is step 0 of a run fed no messages (the core writes the input record first, and none for an empty input) — and sets InputDerived on that batch so readers badge it HoleDerived.

func DedupBatches added in v0.10.0

func DedupBatches(batches []TranscriptBatch) []TranscriptBatch

DedupBatches runs DedupTranscript over the batches' bodies; the number of batches and their stored fields never change.

type TreeQuery added in v0.17.0

type TreeQuery struct {
	// RunID is the tree's root run.
	RunID string
	// PublicID, when not "", scopes the tree to one public id (a panel
	// token's, U9.1-fix): the root run's spans stay, and a descendant
	// run stays when its row and each of its spans carry PublicID and
	// its parent run stays; any other descendant is left out with its
	// whole subtree, unnamed but counted (RunTree.Hidden, HiddenCount). The
	// backend leaves the rows' outsiders out of its span read, and the
	// cap counts the visible spans alone.
	PublicID string
	// Meta, when not empty, scopes the tree to the runs carrying every
	// pair (a tag-scoped panel token's, I4) by PublicID's rule: a
	// descendant run stays when its row's metadata and each of its spans'
	// attributes carry every pair — run metadata rides every weft span
	// verbatim — and its parent run stays. With PublicID both apply.
	Meta map[string]string
}

TreeQuery names the run tree TreeReader.RunTreeSpans reads.

func (TreeQuery) Scoped added in v0.20.0

func (q TreeQuery) Scoped() bool

Scoped reports whether q scopes the tree (PublicID or Meta set).

type TreeReader added in v0.17.0

type TreeReader interface {
	// RunTreeSpans returns the spans of the run and of every run that
	// descends from it (RunTree): the runs whose parent chain reaches
	// it — RunRow.ParentRunID, else a span's weft.parent.run.id for a
	// run with no row — and, whatever else is known of them, the runs
	// core's id rule names its descendants (TreeIDRule: a Subagent
	// child's id is <parent>/<step>/<call id>). The descendants are
	// found from the run rows first and their spans read by run id, in
	// any trace — the run's trace is consulted only for its runs that
	// have no row — so a leaf of a large app-level trace reads its own
	// subtree, never the whole trace; a derived run (§9 of
	// docs/record-contract.md), whose spans carry no run id, is cut
	// from its trace. At most TreeSpanCap spans, in start order then
	// span id; RunTree.Truncated says the tree holds more. Each span
	// names the run it belongs to and that run's parent: the key a
	// reader draws one run's spans under another's by is (RunID,
	// SpanID). ErrNotFound for an unknown run; empty, not an error, for
	// a run with no span stored. TreeQuery.PublicID and Meta scope the
	// answer (a panel token's): the cut, then the scope, then
	// the cap, so a run the scope leaves out never takes a visible
	// run's place under the cap (RunTree.Hidden).
	RunTreeSpans(ctx context.Context, q TreeQuery) (RunTree, error)
	// ChildRuns returns the direct children (RunRow.ParentRunID) of each
	// of the parent ids, in one read per call however many ids it names
	// (the backend may split a very long list into chunks): each
	// parent's children ordered by Started, then id; a parent with no
	// child (or no row) is absent from the map. RunDetail.Children's
	// rows, for a page of runs at once — the runs list prices a page's
	// subagent trees level by level instead of one read per row. At
	// most ChildRunsCap rows in all; past it ErrChildRunsCapped.
	ChildRuns(ctx context.Context, parentIDs []string) (map[string][]RunRow, error)
}

TreeReader is the run tree's reads (U9.1), an optional capability of a DB: a run's spans with every descendant run's beside them, and the children of many runs at once. Both first-party backends implement it and the obsdbtest conformance table holds them to the same answers; a backend that does not is a database without the tree — Studio then leaves its spans_tree capability off, says why, and reads each run's children one run at a time.

type TreeRun added in v0.17.0

type TreeRun struct {
	ID, ParentRunID, PublicID string
	Meta                      map[string]string
	Started                   time.Time
	Status                    Status
}

TreeRun names one run of a tree: its id, its parent, its public id (weft.public_id, "" when none — what a scoped reader filters by), its metadata (RunRow.Meta: TreeQuery.Meta's scope reads it), when it started (RunRow.Started, the order Missing lists runs in) and its derived status (DeriveStatus with the read's clock: a running run is never Missing).

type TreeSpan added in v0.17.0

type TreeSpan struct {
	Span
	RunID, ParentRunID string
}

TreeSpan is one span of a run tree (TreeReader.RunTreeSpans): the span, the run it belongs to — its weft.run.id, or the derived run (§9 of docs/record-contract.md) whose subtree holds it — and that run's parent run ("" for the tree's root run when it has none, and when no row, span or id names one).

func CutRunTree added in v0.17.0

func CutRunTree(spans []Span, runID string, parents map[string]string) []TreeSpan

CutRunTree cuts spans, in the order a backend read them, to runID's subtree: the spans of runID and of every run whose parent chain reaches it. parents maps run ids to their stored parent (RunRow.ParentRunID; a backend reads it beside the spans); a run absent from it, or mapped to "", takes its parent from its spans' weft.parent.run.id (the root span carries it), and a derived run from its derivation's ParentRunID; a run none of those names a parent of is in the subtree when the id rule names it a descendant (TreeIDRule) — a running child whose run_start was lost, a run with no row. A span belongs to the run its weft.run.id names, else to the derived run whose subtree holds it; a span of neither (an app's own span with no run) is left out. A parent chain that loops (never written by the core) ends where it repeats. The result keeps the input order; nil spans or a run with no span in them cut to an empty, non-nil slice.

Example

A run tree: the spans of a trace cut to one run's subtree, each keyed by its run and that run's parent. The run rows' parents come in beside the spans (a backend reads them in the same query); a run without a row takes its parent from its root span's weft.parent.run.id.

package main

import (
	"fmt"

	"github.com/weftgo/weft/obsdb"
)

func main() {
	spans := []obsdb.Span{
		{TraceID: "t1", SpanID: "01", Name: "invoke_agent director", Attrs: map[string]any{"weft.run.id": "d"}},
		{TraceID: "t1", SpanID: "02", ParentSpanID: "01", Name: "execute_tool ops", Attrs: map[string]any{"weft.run.id": "d"}},
		{TraceID: "t1", SpanID: "03", ParentSpanID: "02", Name: "invoke_agent ops",
			Attrs: map[string]any{"weft.run.id": "d/0/c1", "weft.parent.run.id": "d"}},
		{TraceID: "t1", SpanID: "04", ParentSpanID: "03", Name: "chat", Attrs: map[string]any{"weft.run.id": "d/0/c1/0/c2"}},
		{TraceID: "t1", SpanID: "05", Name: "invoke_agent sibling", Attrs: map[string]any{"weft.run.id": "s"}},
	}
	parents := map[string]string{"d/0/c1/0/c2": "d/0/c1"} // the worker's run row
	for _, s := range obsdb.CutRunTree(spans, "d", parents) {
		fmt.Printf("%s %s (run %s, parent %q)\n", s.SpanID, s.Name, s.RunID, s.ParentRunID)
	}
}
Output:
01 invoke_agent director (run d, parent "")
02 execute_tool ops (run d, parent "")
03 invoke_agent ops (run d/0/c1, parent "d")
04 chat (run d/0/c1/0/c2, parent "d/0/c1")

type TurnOutcome added in v0.18.0

type TurnOutcome string

TurnOutcome is one turn's mark on a session row's strip (SessionRow.Outcomes): a run's derived status, with a park told open from decided and a busy-policy end told from a failure.

const (
	OutcomeSucceeded   TurnOutcome = "succeeded"
	OutcomeFailed      TurnOutcome = "failed"
	OutcomeRunning     TurnOutcome = "running"
	OutcomeInterrupted TurnOutcome = "interrupted" // the heartbeat went stale
	OutcomeCancelled   TurnOutcome = "cancelled"   // failed, error type cancelled
	OutcomeEnded       TurnOutcome = "ended"       // thread's busy policy cancelled it
	OutcomeParked      TurnOutcome = "parked"      // pending calls, no resumer
	OutcomeResumed     TurnOutcome = "resumed"     // pending calls a resumer decided
)

The outcomes.

func OutcomeOf added in v0.18.0

func OutcomeOf(r RunRow) TurnOutcome

OutcomeOf is a turn's TurnOutcome: a failed turn with BusyCancelCause ended, else cancelled with that error type; a succeeded turn with pending calls parked, or resumed once a run resumed it (ResumedBy); else its status.

type Weft

type Weft struct {
	RunID, ParentRunID, ParentCallID string
	SessionID, PublicID              string
	Turn                             int
	Agent                            string
	Record                           string // event | delta | messages | heartbeat | "" (non-weft)
	EventType                        string
	Pos                              int64 // event pos, messages index, delta pos, or request/prompt/tools index
	Step                             int   // -1 when absent
	ToolSeq                          int64 // -1 when absent
	Playground                       bool
	ExperimentID, ForkedFrom         string
	// ResumedFrom is a resuming run's weft.resumed_from (core.ResumeOf,
	// UX-E-17): the parked run it resumes. On its invoke_agent span and
	// run_start record only.
	ResumedFrom string
	// CallerRunID / CallerCallID are a run's weft.caller.run.id /
	// weft.caller.call.id (O11, U14): the tool call another run was
	// started from when it is no core Subagent child — a thread/pool
	// child session's turn, a Generate inside a handler. A link, never
	// parenthood (ParentRunID is the Subagent child's alone). On the
	// invoke_agent span and run_start record only.
	CallerRunID, CallerCallID string
	// Reason is a messages record's weft.messages.reason: "" for
	// transcript growth, "compacted" for a compaction view (ADR 0028
	// §8), which is never counted or concatenated as transcript. A
	// non-string reason reads as its text (never "": the record is not
	// growth), as ClickHouse's stringified attribute column reads it.
	Reason string
	// Input is a messages record's weft.messages.input: the run's input
	// record (ADR 0028 §8) — what search reads the turn's prompt from.
	Input bool
}

Weft is what obsdb derives from a span or record's attributes, stored as indexed columns. Zero values mean absent.

func DeriveRecord

func DeriveRecord(r Record) Weft

DeriveRecord returns the weft identity a log record carries. Pos is the event position for event records, the messages index for transcript records, the delta position for delta records, and the request, prompt, tools or raw index for those kinds (ADR 0028) — the value the (run, kind, pos) key and the live lane's dedup need. A record with no weft.run.id is non-weft and lands in other_logs.

func DeriveSpan

func DeriveSpan(s Span) Weft

DeriveSpan returns the weft identity a span carries, from its attributes. A span with no weft.run.id is non-weft (a stock OTel application's span): every field reads absent.

Directories

Path Synopsis
Package clickhouse — see doc.go for the package documentation.
Package clickhouse — see doc.go for the package documentation.
internal
bpset
Package bpset is the breakpoints row's one text form, shared by obsdb's two backends (obsdb.BreakpointSet): the set's JSON in the tools column.
Package bpset is the breakpoints row's one text form, shared by obsdb's two backends (obsdb.BreakpointSet): the set's JSON in the tools column.
caps
Package caps holds obsdb's read and sweep bounds as obsdb and its backends apply them: the exported constants (obsdb.TreeSpanCap, TreeTraceCap, TreeMissingCap, TreeRowCap, ChildRunsCap, SweepBatch, MaxLogCandidates), the same values, as variables the conformance table (obsdbtest) lowers around a past-the-cap case, so that case writes tens of rows, not tens of thousands, and proves the same behaviour at the lowered bound.
Package caps holds obsdb's read and sweep bounds as obsdb and its backends apply them: the exported constants (obsdb.TreeSpanCap, TreeTraceCap, TreeMissingCap, TreeRowCap, ChildRunsCap, SweepBatch, MaxLogCandidates), the same values, as variables the conformance table (obsdbtest) lowers around a past-the-cap case, so that case writes tens of rows, not tens of thousands, and proves the same behaviour at the lowered bound.
reqread
Package reqread holds what obsdb's two backends share to answer DB.Prompt, DB.Tools and DB.Catalogs over the records they read: the lookups by hash and the one-per-hash catalog list.
Package reqread holds what obsdb's two backends share to answer DB.Prompt, DB.Tools and DB.Catalogs over the records they read: the lookups by hash and the one-per-hash catalog list.
Package obsdbtest is the shared conformance table for obsdb.DB backends — the executable form of S3.5's promises, the storetest pattern.
Package obsdbtest is the shared conformance table for obsdb.DB backends — the executable form of S3.5's promises, the storetest pattern.
Package sqlite is obsdb's default backend: the observability schema in a single SQLite file on the CGO-free modernc.org/sqlite driver (thread/sqlite's choice).
Package sqlite is obsdb's default backend: the observability schema in a single SQLite file on the CGO-free modernc.org/sqlite driver (thread/sqlite's choice).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL