Documentation
¶
Overview ¶
Package backup implements the portable logical snapshot of the Cortex memory store: observations, graph relations (edges), and sessions, packaged as JSONL parts inside a tar.gz with a sha256-verified manifest.
This is the portable logical layer, distinct from the physical SQLite snapshot (`cortex backup <path>` → VACUUM INTO). It follows the projection/obsidian package pattern: a pure library with narrow inputs, wired to the store bundle at the CLI composition root.
Scope: local SQLite mode. Server (PostgreSQL) mode full-database backup remains the domain of pg_dump; this package's schema-projection queries are SQLite-specific and restore refuses non-v2 databases via the schema identity check.
Security invariants:
- Only a fixed allow-list of columns is exported. Configuration secrets (API keys, DSNs, web keys, bearer tokens) never enter the projection.
- Defense in depth: the caller may supply known secret values; Export fail-closes if any of them appears in the serialized archive.
- Embedding vectors are NOT exported (regenerable, and potentially re-identifying); per-observation embedding state (model/dimensions) is.
Index ¶
- Constants
- Variables
- type EdgeRecord
- type EmbeddingState
- type ErrIncompatibleSchema
- type ErrSecretDetected
- type ErrTampered
- type ExportOptions
- type ExportResult
- type Exporter
- type Manifest
- type ObservationRecord
- type Part
- type RestoreOptions
- type RestoreResult
- type Restorer
- type Secret
- type SessionRecord
- type Source
Constants ¶
const ( // Format identifies the archive family in the manifest. Format = "cortex-backup" // FormatVersion is the manifest/record layout version of this package. FormatVersion = 1 // ManifestName is the manifest entry name inside the tar.gz. ManifestName = "manifest.json" // Part names (JSONL, one record per line). ObservationsPart = "observations.jsonl" EdgesPart = "edges.jsonl" SessionsPart = "sessions.jsonl" )
Variables ¶
var KnownEnvSecrets = []string{
"CORTEX_EMBEDDING_API_KEY",
"CORTEX_LLM_API_KEY",
"CORTEX_RERANK_API_KEY",
"CORTEX_HTTP_TOKEN",
"CORTEX_REMOTE_TOKEN",
"CORTEX_WEB_KEY",
"CORTEX_SYNC_TOKEN",
"CORTEX_POSTGRES_DSN",
"CORTEX_TEST_POSTGRES_DSN",
"CORTEX_TEST_POSTGRES_MIGRATION_DSN",
"CORTEX_TEST_POSTGRES_AUTHZ_ADMIN_DSN",
}
KnownEnvSecrets is the ordered list of Cortex credential environment variables whose values must never leak into a logical backup.
Functions ¶
This section is empty.
Types ¶
type EdgeRecord ¶
type EdgeRecord struct {
ID int64 `json:"id"`
FromObsID int64 `json:"from_obs_id"`
ToObsID int64 `json:"to_obs_id"`
RelationType string `json:"relation_type"`
Weight float64 `json:"weight"`
Confidence float64 `json:"confidence"`
Source sql.NullString `json:"source,omitempty"`
Reasoning sql.NullString `json:"reasoning,omitempty"`
ValidFrom sql.NullString `json:"valid_from,omitempty"`
InvalidAt sql.NullString `json:"invalid_at,omitempty"`
ValidUntil sql.NullString `json:"valid_until,omitempty"`
TenantID sql.NullString `json:"tenant_id,omitempty"`
WorkspaceID sql.NullString `json:"workspace_id,omitempty"`
EvolutionID sql.NullInt64 `json:"evolution_id,omitempty"`
EvolutionType string `json:"evolution_type"`
FactState string `json:"fact_state"`
ChangeReason sql.NullString `json:"change_reason,omitempty"`
CreatedAt sql.NullString `json:"created_at,omitempty"`
}
EdgeRecord is one portable graph relation row.
type EmbeddingState ¶
EmbeddingState records the per-observation embedding metadata. The vector itself is intentionally NOT exported: it is regenerable and potentially re-identifying.
type ErrIncompatibleSchema ¶
ErrIncompatibleSchema reports a restore refusal caused by schema-version incompatibility (the archive is newer than the target database).
func (*ErrIncompatibleSchema) Error ¶
func (e *ErrIncompatibleSchema) Error() string
type ErrSecretDetected ¶
type ErrSecretDetected struct{ Name string }
ErrSecretDetected reports that a configured secret value appeared in the exported archive; the export fail-closes rather than leak it.
func (*ErrSecretDetected) Error ¶
func (e *ErrSecretDetected) Error() string
type ErrTampered ¶
type ErrTampered struct{ Part string }
ErrTampered reports a sha256 mismatch between manifest and part payload.
func (*ErrTampered) Error ¶
func (e *ErrTampered) Error() string
type ExportOptions ¶
type ExportOptions struct {
// Out is the destination file path (a tar.gz archive).
Out string
// Tenant optionally restricts the export to one tenant id.
Tenant string
// ToolVersion is recorded in the manifest (the CLI injects its version).
ToolVersion string
// Secrets are credential values that must never appear in the archive
// (defense-in-depth; see package doc).
Secrets []Secret
// Now overrides the manifest timestamp (tests).
Now func() time.Time
}
Options control a logical export.
type ExportResult ¶
type ExportResult struct {
Path string
SizeBytes int64
Observations int
Edges int
Sessions int
SchemaVer string
}
ExportResult summarizes a completed export.
type Exporter ¶
Exporter produces logical snapshots from the SQLite memory store.
The projection deliberately reads schema-owned columns through a fixed allow-list of SELECTs rather than the domain model: tenant/workspace/owner columns and verbatim stored timestamps are part of the portable snapshot but are deliberately absent from the local domain types. All other writes in the codebase keep flowing through the store bundle; this package is a read-only dumper plus an ID-preserving restorer (see restore.go).
func (*Exporter) Export ¶
func (e *Exporter) Export(ctx context.Context, opts ExportOptions) (*ExportResult, error)
Export writes the tar.gz archive to opts.Out.
type Manifest ¶
type Manifest struct {
Format string `json:"format"`
FormatVersion int `json:"format_version"`
SchemaFamily string `json:"schema_family"`
SchemaVersion string `json:"schema_version"`
ToolVersion string `json:"tool_version"`
CreatedAt string `json:"created_at"`
Source Source `json:"source"`
Parts map[string]Part `json:"parts"`
Notes map[string]string `json:"notes,omitempty"`
}
Manifest is the self-describing index stored inside every archive.
type ObservationRecord ¶
type ObservationRecord struct {
ID int64 `json:"id"`
SessionID string `json:"session_id"`
Type string `json:"type"`
Title string `json:"title"`
Content string `json:"content"`
ToolName sql.NullString `json:"tool_name,omitempty"`
Project sql.NullString `json:"project,omitempty"`
Scope string `json:"scope"`
TopicKey sql.NullString `json:"topic_key,omitempty"`
Confidence float64 `json:"confidence"`
Source string `json:"source"`
Tags sql.NullString `json:"tags,omitempty"`
TenantID sql.NullString `json:"tenant_id,omitempty"`
WorkspaceID sql.NullString `json:"workspace_id,omitempty"`
OwnerID sql.NullString `json:"owner_id,omitempty"`
CreatedAt string `json:"created_at"`
UpdatedAt string `json:"updated_at"`
ValidUntil sql.NullString `json:"valid_until,omitempty"`
EmbeddingState *EmbeddingState `json:"embedding_state,omitempty"`
}
ObservationRecord is one portable observation row. Timestamps are carried verbatim as stored (TEXT) so a restore is byte-faithful.
type RestoreOptions ¶
type RestoreOptions struct {
// From is the source archive path (tar.gz produced by Export).
From string
// Secrets are credential values that must never have been exported; if
// any appears inside the archive the restore refuses (the archive was
// produced by a non-Cortex tool or tampered with).
Secrets []Secret
}
Options control a logical restore.
type RestoreResult ¶
type RestoreResult struct {
ObservationsRestored int
ObservationsSkipped int
EdgesRestored int
EdgesSkipped int
SessionsRestored int
SessionsSkipped int
SchemaVer string
}
RestoreResult summarizes a completed restore. Restored/Skipped counts make re-running a restore idempotent and observable.
type Restorer ¶
Restorer applies a logical snapshot to a v2 SQLite database, preserving original observation/edge IDs so graph relations and idempotent re-runs stay coherent.
Restore semantics:
- manifest validation: format, format version, per-part sha256 (tamper rejection) and schema-version compatibility (refuse newer, see checkSchemaCompatible);
- single transaction; any failure rolls the whole restore back;
- idempotent: existing IDs are skipped (ON CONFLICT DO NOTHING), never overwritten;
- embeddings are NOT restored. The target's observation_vectors are untouched; absent vectors mean the observation is pending re-embed and `cortex reindex` (or the embedding worker) self-heals.
func (*Restorer) Restore ¶
func (r *Restorer) Restore(ctx context.Context, opts RestoreOptions) (*RestoreResult, error)
type Secret ¶
Secret is a named credential value the caller asserts must never appear in an exported archive. Names are only used in error messages.
func CollectEnvSecrets ¶
CollectEnvSecrets gathers non-trivial (>= minSecretLen runes) values for the known credential environment variables plus any extra (name, value) pairs supplied by the caller (e.g. tokens read from configuration).
type SessionRecord ¶
type SessionRecord struct {
ID string `json:"id"`
Project string `json:"project"`
Directory string `json:"directory"`
StartedAt string `json:"started_at"`
EndedAt sql.NullString `json:"ended_at,omitempty"`
Summary sql.NullString `json:"summary,omitempty"`
TenantID sql.NullString `json:"tenant_id,omitempty"`
WorkspaceID sql.NullString `json:"workspace_id,omitempty"`
}
SessionRecord is one portable session row.