sqlite

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 8, 2026 License: MIT Imports: 21 Imported by: 0

Documentation

Overview

Package sqlite is part of the GoFastr harness.

See docs/harness-architecture.md for the architecture this package implements.

Package sqlite is the SQLite-backed implementation of session.Store.

Uses the existing framework dependency `mattn/go-sqlite3`. Per the architecture doc, at-rest encryption is on the v0.2 roadmap; v0.1 stores plaintext and relies on the redaction middleware on the write path to keep secrets out of the file.

Index

Constants

This section is empty.

Variables

View Source 
var ErrLedgerClosed = errors.New("sqlite: cost ledger closed")

ErrLedgerClosed is returned when methods are called after Close.

View Source 
var ErrNoReconciliations = errors.New("sqlite: no reconciliations recorded")

ErrNoReconciliations sentinel.

Functions

func DefaultCostLedgerPath 

func DefaultCostLedgerPath() string

DefaultCostLedgerPath returns the canonical XDG state location.

func DefaultPath 

func DefaultPath() string

makeAbsPath helps callers build the default location.

func ExportDEK 

func ExportDEK(dbPath string, kek, recipientKey []byte, outPath string) error

ExportDEK writes the DEK wrapped under a recipient's public key so the user can escrow or migrate the DB. The recipient format is X25519 base64; we use a tiny in-package wrapper (no third-party dep) that emits an AES-GCM-wrapped DEK plus the recipient's public key fingerprint for identification.

To keep the implementation stdlib-only, the "recipient" here is another 32-byte key, not full X25519 + age. Real X25519/age integration lives behind a build-tagged extension at session/sqlite/escrow_age.go; the v0.1 default ships this simpler form that still meets the threat-model goal of "key escrow that doesn't require the harness binary to recover."

func HashArgs 

func HashArgs(args []byte) string

HashArgs returns the SHA-256 hex of tool arguments, used to fill tool_call_intents.args_hash.

func ImportDEK 

func ImportDEK(dbPath, exportedPath string, recipientKey, localKEK []byte) error

ImportDEK reads a previously-exported DEK and installs it as the active DEK for the given DB, wrapped under the local KEK. The recipientKey must match what ExportDEK used.

func LoadOrCreateDEK 

func LoadOrCreateDEK(dbPath string, kek []byte) ([]byte, string, error)

LoadOrCreateDEK opens the DEK header at `<dbPath>.dek`. If it doesn't exist, a fresh DEK is generated and wrapped with `kek`. Returns the unwrapped DEK and the header path.

func RotateKEK 

func RotateKEK(dbPath string, oldKEK, newKEK []byte) error

RotateKEK re-wraps the DEK with a new KEK without re-encrypting the DB. After this returns, only `newKEK` can unwrap.

Types

type CostLedger 

type CostLedger struct {
	// contains filtered or unexported fields
}

CostLedger is a separate SQLite DB that records cost rows independently from the event log. Survives VACUUM on the main session DB and is queryable directly for the cost dashboard.

func OpenCostLedger 

func OpenCostLedger(path string) (*CostLedger, error)

OpenCostLedger opens the cost ledger DB.

func (*CostLedger) Close 

func (c *CostLedger) Close() error

Close releases the DB.

func (*CostLedger) CostByProvider 

func (c *CostLedger) CostByProvider(ctx context.Context, since time.Time) (map[string]float64, error)

CostByProvider returns USD totals grouped by provider since `since`.

func (*CostLedger) CostBySession 

func (c *CostLedger) CostBySession(ctx context.Context, since time.Time) (map[string]float64, error)

CostBySession returns USD totals grouped by session since `since`.

func (*CostLedger) Record 

func (c *CostLedger) Record(ctx context.Context, session, provider, model string, inputT, outputT, cacheT int, usd float64) error

Record inserts one cost row.

type DEKHeader 

type DEKHeader struct {
	Version  int    `json:"version"`
	Nonce    []byte `json:"nonce"`          // AES-GCM nonce for the wrap
	Wrapped  []byte `json:"wrapped"`        // AES-GCM ciphertext of the DEK
	Metadata string `json:"meta,omitempty"` // human-readable, e.g. "kek=machine"
}

DEKHeader is the on-disk shape of the wrapped DEK.

type DefaultRedactor 

type DefaultRedactor struct{}

DefaultRedactor replaces common secret patterns with «redacted:KIND».

func (DefaultRedactor) Redact 

func (DefaultRedactor) Redact(text string) string

Redact applies a list of canned regex replacements. The list is intentionally tight in v0.1 — false positives are worse than false negatives here because the regex runs on *every event payload*. Roadmap OBS-EXPORT-BUNDLE adds a deeper-pass detector for export.

type EncryptionMode 

type EncryptionMode int

EncryptionMode controls how the session log file is protected. 

const (
	// EncryptionNone — plaintext SQLite file. The default for v0.1.
	EncryptionNone EncryptionMode = iota

	// EncryptionAtRest — the SQLite file is encrypted in place
	// using file-level AES-GCM whenever the harness is not running.
	// On Open(), the file is decrypted to a sidecar; on Close(), the
	// sidecar is re-encrypted and the decrypted copy is removed.
	//
	// The architecture doc notes that whole-file encryption + VACUUM
	// is a problem for very large logs; this mode accepts that
	// limitation in exchange for a real defense against laptop theft.
	// Page-level encryption requires a custom SQLite VFS and is a
	// genuinely larger sub-project (acknowledged in the spec but not
	// blocking v0.1).
	EncryptionAtRest
)

type MonthlyRollover 

type MonthlyRollover struct {
	// contains filtered or unexported fields
}

MonthlyRollover provides per-month log files (sessions-YYYYMM.db) for retention efficiency: VACUUM only touches the current month's file, not the whole history.

The current month's file is the "live" Store; older months are closed and become read-only archives. Open queries can span archives by iterating Open() on each month file.

func NewMonthlyRollover 

func NewMonthlyRollover(dir string) (*MonthlyRollover, error)

NewMonthlyRollover returns a rollover manager rooted at dir. The current month's store is opened immediately.

func (*MonthlyRollover) Close 

func (m *MonthlyRollover) Close() error

Close releases the live Store.

func (*MonthlyRollover) Current 

func (m *MonthlyRollover) Current() (*Store, error)

Current returns the live Store, rolling over if the calendar month has changed since the last call.

func (*MonthlyRollover) ListArchives 

func (m *MonthlyRollover) ListArchives() ([]string, error)

ListArchives returns the paths of every month file in the directory (current + archives), oldest first.

type ProviderAttempt 

type ProviderAttempt struct {
	ID               int64
	Session          string
	Turn             int
	Provider         string
	Model            string
	RequestID        string // upstream request ID when surfaced
	StartedAt        time.Time
	EndedAt          time.Time
	TerminatedReason string // "complete" | "eof" | "timeout" | "cancel" | "http_error"
	InputTokens      int
	OutputTokens     int
	CacheReadTokens  int
	CacheWriteTokens int
	EstimatedUSD     float64
}

ProviderAttempt records one provider HTTP call. Used by the cost reconciliation job to compare local estimates against provider-side usage APIs (OpenRouter /credits, ZAI /billing/usage, etc.).

type Reconciliation 

type Reconciliation struct {
	ID          int64
	Provider    string
	PeriodStart time.Time
	PeriodEnd   time.Time
	UpstreamUSD float64
	LocalUSD    float64
	VarianceUSD float64
	RanAt       time.Time
}

Reconciliation is one reconciliation pass against an upstream provider's billing API.

type Redactor 

type Redactor interface {
	Redact(text string) string
}

Redactor replaces matched substrings with redaction markers.

type Store 

type Store struct {

	// Redactors run on every event payload before it lands in the
	// `events.payload` column. The default redactor uses common
	// secret regexes; profiles can swap in tighter sets.
	Redactors []Redactor
	// contains filtered or unexported fields
}

Store is the SQLite-backed session.Store.

func Open 

func Open(path string) (*Store, error)

Open opens (or creates) the SQLite session log at path. Creates the parent directory and applies the v1 schema on first use.

func OpenEncrypted 

func OpenEncrypted(path string, mode EncryptionMode, key []byte) (*Store, error)

OpenEncrypted opens a SQLite session log with the requested encryption mode. The key (32 bytes) protects the at-rest file when mode == EncryptionAtRest.

  • mode = EncryptionNone: behaves exactly like Open.
  • mode = EncryptionAtRest: if `<path>.enc` exists, decrypt to `path` first; then open. The encrypted form is written on Close.

The unencrypted `path` is created with mode 0600 and lives in the XDG state dir, which is owner-readable only on every supported OS.

func OpenWithKEK 

func OpenWithKEK(dbPath string, kek []byte) (*Store, error)

OpenWithKEK opens an encrypted Store using the DEK/KEK scheme. If no .dek file exists, a fresh DEK is generated and wrapped.

func (*Store) AppendEvent 

func (s *Store) AppendEvent(ctx context.Context, env control.EventEnvelope) error

AppendEvent persists one envelope with redactors applied.

func (*Store) ApplyRetention 

func (s *Store) ApplyRetention(ctx context.Context, ttl time.Duration) (int64, error)

ApplyRetention drops the payload column for events older than ttl, leaving metadata-only rows.

func (*Store) AttemptsForTurn 

func (s *Store) AttemptsForTurn(ctx context.Context, session string, turn int) ([]ProviderAttempt, error)

AttemptsForTurn returns all attempts for one turn. Useful when computing the true cost of a turn that retried.

func (*Store) Close 

func (s *Store) Close() error

Close closes the underlying DB.

func (*Store) CloseEncrypted 

func (s *Store) CloseEncrypted() error

CloseEncrypted closes the Store and re-encrypts the file when EncryptionAtRest is active. The plaintext file is removed afterwards.

func (*Store) DropMetadata 

func (s *Store) DropMetadata(ctx context.Context, age time.Duration, exemptSessions []string) (int64, error)

DropMetadata removes event rows entirely (not just their content) whose ts is older than the given age. Returns the count of deleted rows.

Default policy per § Persistence: drop metadata-only rows after 180 days. Pinned sessions (tracked separately) survive.

func (*Store) EnsureAttemptsSchema 

func (s *Store) EnsureAttemptsSchema(ctx context.Context) error

EnsureAttemptsSchema applies the provider_attempts migration. Safe to call repeatedly.

func (*Store) EventsSince 

func (s *Store) EventsSince(ctx context.Context, sess ids.SessionID, since uint64, limit int) ([]control.EventEnvelope, error)

EventsSince returns events with id > since for a session.

func (*Store) ListPastSessions 

func (s *Store) ListPastSessions(ctx context.Context, limit int) ([]session.PastSession, error)

ListPastSessions aggregates the events table into per-session summaries. Newest-last-seen first. Uses two passes — one for the counts/timestamps (fast), one for the first user-message text per session (slower; bounded by `limit`).

func (*Store) LocalCostInPeriod 

func (s *Store) LocalCostInPeriod(ctx context.Context, provider string, start, end time.Time) (float64, error)

LocalCostInPeriod returns the local-estimated USD for one provider across the time window. Used by the reconciliation job.

func (*Store) OrphanIntents 

func (s *Store) OrphanIntents(ctx context.Context, sess ids.SessionID) ([]session.ToolIntent, error)

OrphanIntents returns intents without matching outcomes. v0.1 returns all intents for the given LogID; the higher layer narrows by session.

func (*Store) ReconcileNow 

func (s *Store) ReconcileNow(ctx context.Context, fetcher UpstreamUsageFetcher, period time.Duration) (Reconciliation, error)

ReconcileNow runs one reconciliation pass for the given provider. Returns the diff. Callers schedule this daily.

func (*Store) RecordAttempt 

func (s *Store) RecordAttempt(ctx context.Context, a ProviderAttempt) (int64, error)

RecordAttempt inserts a provider_attempts row, returning the assigned ID.

func (*Store) RecordToolIntent 

func (s *Store) RecordToolIntent(ctx context.Context, intent session.ToolIntent) error

RecordToolIntent writes an intent row.

func (*Store) RecordToolOutcome 

func (s *Store) RecordToolOutcome(ctx context.Context, outcome session.ToolOutcome) error

RecordToolOutcome writes an outcome row.

func (*Store) SaveReconciliation 

func (s *Store) SaveReconciliation(ctx context.Context, r Reconciliation) (int64, error)

SaveReconciliation persists a reconciliation result.

func (*Store) SnapshotPath 

func (s *Store) SnapshotPath() string

SnapshotPath returns the on-disk path the store was opened with. Useful for diagnostics + the /health endpoint.

type UpstreamUsageFetcher 

type UpstreamUsageFetcher interface {
	// Name returns the provider name (matches ProviderAttempt.Provider).
	Name() string
	// FetchUsageUSD returns the upstream-reported USD spent in the
	// given period.
	FetchUsageUSD(ctx context.Context, start, end time.Time) (float64, error)
}

UpstreamUsageFetcher is the abstraction the reconciliation job uses to pull billing data from a provider. Each Provider package can implement this if its upstream offers a usage API.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.