Documentation
¶
Overview ¶
Package backfill extracts token usage from Claude Code transcripts and backfills historical nodes in the tapes database.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ScanTranscriptDir ¶
ScanTranscriptDir finds all JSONL files under the given directory.
Types ¶
type APIRunRequest ¶ added in v0.5.0
type APIRunRequest struct {
TranscriptDir string `json:"transcript_dir"`
DryRun bool `json:"dry_run,omitempty"`
Verbose bool `json:"verbose,omitempty"`
Sessions bool `json:"sessions,omitempty"`
OrgID string `json:"org_id,omitempty"`
AuthSubject string `json:"auth_subject,omitempty"`
}
APIRunRequest is the request payload for the API-backed usage sync flow.
type Backfiller ¶
type Backfiller struct {
// contains filtered or unexported fields
}
Backfiller matches Claude Code transcript usage data to tapes DB nodes.
func NewBackfillerWithDriver ¶ added in v0.5.0
func NewBackfillerWithDriver(driver storage.Driver, opts Options) *Backfiller
NewBackfillerWithDriver creates a Backfiller using an existing storage driver.
type Result ¶
type Result struct {
Matched int `json:"matched"`
Skipped int `json:"skipped"`
Unmatched int `json:"unmatched"`
TotalTokensBackfilled int `json:"total_tokens_backfilled"`
TranscriptFiles int `json:"transcript_files"`
TranscriptEntries int `json:"transcript_entries"`
SessionsBackfilled int `json:"sessions_backfilled,omitempty"`
NodesLinked int `json:"nodes_linked,omitempty"`
}
Result contains statistics from a backfill run.
type TranscriptBlock ¶
TranscriptBlock represents a content block in a transcript message.
type TranscriptEntry ¶
type TranscriptEntry struct {
Type string `json:"type"`
UUID string `json:"uuid"`
ParentUUID *string `json:"parentUuid"`
Timestamp string `json:"timestamp"`
SessionID string `json:"sessionId"`
Cwd string `json:"cwd"`
Version string `json:"version"`
Message *TranscriptMessage `json:"message"`
SourcePath string `json:"-"`
}
TranscriptEntry represents a single line in a Claude Code JSONL transcript.
func ParseTranscript ¶
func ParseTranscript(path string) ([]TranscriptEntry, error)
ParseTranscript reads a JSONL file and returns assistant entries with usage data. It deduplicates by message ID, keeping the last (most complete) entry per message.
func (*TranscriptEntry) TextContent ¶
func (e *TranscriptEntry) TextContent() string
TextContent extracts the concatenated text from all text content blocks.
type TranscriptMessage ¶
type TranscriptMessage struct {
ID string `json:"id"`
Role string `json:"role"`
Model string `json:"model"`
Content []TranscriptBlock `json:"content"`
Usage *TranscriptUsage `json:"usage"`
StopReason json.RawMessage `json:"stop_reason"`
}
TranscriptMessage represents the message field within a JSONL entry.
type TranscriptUploadOptions ¶ added in v0.16.0
type TranscriptUploadOptions struct {
// ProjectDir is one Claude Code project directory
// (~/.claude/projects/<flattened-cwd>) holding <session>.jsonl
// files and per-session subagents/ directories.
ProjectDir string
// SessionIDs filters which sessions upload; empty means every
// .jsonl in the directory.
SessionIDs []string
// IngestURL is the tapes-ingest base URL.
IngestURL string
// HarnessID tags the session envelope; Claude Code transcripts are
// "claude".
HarnessID string
Verbose bool
Logf func(format string, args ...any)
}
TranscriptUploadOptions configures a harness-transcript upload run.
type TranscriptUploadResult ¶ added in v0.16.0
type TranscriptUploadResult struct {
Sessions int `json:"sessions"`
Files int `json:"files"`
Uploaded int `json:"uploaded"`
Deduped int `json:"deduped"`
Failed int `json:"failed"`
Failures []string `json:"failures,omitempty"`
}
TranscriptUploadResult summarizes an upload run.
func UploadTranscripts ¶ added in v0.16.0
func UploadTranscripts(ctx context.Context, opts TranscriptUploadOptions) (*TranscriptUploadResult, error)
UploadTranscripts pushes harness transcripts (main + subagents) into the tapes raw layer via POST /v1/ingest/transcript. Idempotent: the server dedups by content version, so re-running uploads nothing for unchanged files and a new version for grown ones.
type TranscriptUsage ¶
type TranscriptUsage struct {
InputTokens int `json:"input_tokens"`
OutputTokens int `json:"output_tokens"`
CacheCreationInputTokens int `json:"cache_creation_input_tokens"`
CacheReadInputTokens int `json:"cache_read_input_tokens"`
}
TranscriptUsage contains token counts from a Claude Code transcript entry.
type WireTraceOptions ¶ added in v0.16.0
type WireTraceOptions struct {
// CapturesDir is the paperd wire-trace root holding turn-* bundles
// (request.json + response.sse + meta.json per turn).
CapturesDir string
// IngestURL is the base URL of a tapes-ingest server, e.g.
// "http://127.0.0.1:8090". The backfill POSTs one envelope per
// captured turn to {IngestURL}/v1/ingest.
IngestURL string
// SessionIDs filters replay to bundles whose captured
// X-Tapes-Harness-Session-Id matches; empty replays everything.
SessionIDs []string
// DryRun parses and reduces every bundle but skips the POST.
DryRun bool
// Verbose logs each turn's outcome to Logf.
Verbose bool
// Logf receives progress output; defaults to a no-op.
Logf func(format string, args ...any)
}
WireTraceOptions configures a wire-trace → ingest backfill run.
type WireTraceResult ¶ added in v0.16.0
type WireTraceResult struct {
Scanned int `json:"scanned"`
Posted int `json:"posted"`
RawOnly int `json:"raw_only"`
Skipped int `json:"skipped"`
Failed int `json:"failed"`
Failures []string `json:"failures,omitempty"`
}
WireTraceResult summarizes a wire-trace backfill run.
func WireTrace ¶ added in v0.16.0
func WireTrace(ctx context.Context, opts WireTraceOptions) (*WireTraceResult, error)
WireTrace replays paperd wire-trace capture bundles through a tapes-ingest server, reconstructing the envelope tapes-extproc would have dispatched live: verbatim request bytes, response reduced with the same shared capture reducer, meta rebuilt from the bundle, and the session block recovered from the captured X-Tapes-* headers.
The whole flow is idempotent end to end: the raw layer dedupes on (org, request_id) — the turn directory name, stable across re-runs — and node inserts are content-addressed ON CONFLICT DO NOTHING, so turns that already landed via live capture reproduce their existing nodes byte for byte and only gain the previously-missing raw row.
Only provider chat-completion calls are replayed (…/v1/messages); auxiliary traffic in the trace (count_tokens, tapes API reads) is skipped.