idmigrate

package
v0.0.21 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 18, 2026 License: MIT Imports: 22 Imported by: 0

Documentation

Overview

Package idmigrate owns pasture's one-time OFFLINE identity-format cutover: the `pasture migrate ids [--dry-run]` command's production path and the cutover write the first open of a legacy store performs. It is not a runtime reader and adds no compatibility mode. After it runs, the retired two-dash representation never resolves anywhere in pasture.

The dependency (github.com/dayvidpham/provenance) owns the format, the parser and every Provenance-owned table, canonical mutation and allocation receipt. This package owns only what pasture owns: the audit trail, the pasture-side registry and projection tables, the shared lifecycle replay that must run inside the same transaction, and the cutover receipt. It calls the dependency's published conversion APIs; it never patches or duplicates them.

The cutover is one fenced pipeline with named phases:

P0 read-only state check (EvaluateCutoverOnHandle); the dry run stops here.
P1 archive (ArchiveStore): a consistent logical copy through a read-only
   source, verified, with the archive's own format marker recorded.
P2 audit catch-up with the DEFERRED ceiling (audit.WithDeferredCeiling).
P3 conversion transaction (Convert): ONE BEGIN IMMEDIATE on a pinned
   connection, bracketed by the dependency's verified FK-off/restore.
P4 reopen proof (the command frontend only).

Inside P3 the phases are: (a) the under-lock EVIDENCE verdict; (b) the runtime-state refusal; (c) the raw blob inventory; (d) schema completion; (e) the conversions; (f) owner verification; (g) the transaction-bound non-reclaiming lifecycle replay and comparison; (h) inventory equality; (i) the marker; (j) the receipt; (k) the deferred ceiling — then COMMIT.

Any failure in (b)-(k) rolls the entire transaction back: no key, canonical byte, digest, projection row, blob row, marker, receipt or ceiling survives. No normal open or pooled read runs inside the lock. Reclaim is structurally absent from this path, so the conversion reports zero reclaimed bodies.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CutoverGuidanceRefusal

func CutoverGuidanceRefusal(storePath string) error

CutoverGuidanceRefusal is the plain-migrate refusal for a markerless store (CUTOVER). Plain `pasture migrate` keeps its schema meaning only for a NORMAL store; a markerless store must be cut over by `pasture migrate ids` or by a normal open, both of which archive and write the fence evidence.

func IsRetiredIdentity

func IsRetiredIdentity(raw string) bool

IsRetiredIdentity reports whether raw is a recognizable retired "<namespace>--<uuid>" identity: a final two-dash separator with a non-empty namespace before it and a canonical 36-character hyphenated UUID after it. Arbitrary text, Git SHAs and session names are not retired identities. It is the read-path detector: after the cutover the retired form never resolves, so a command that accepts a free string must refuse a retired identity by name rather than treat it as an unknown free context and return an empty result.

func OutcomeRefusal

func OutcomeRefusal(state CutoverState, populatedConvertible bool) error

OutcomeRefusal is the ONE Outcome->refusal mapping every frontend consumes — the normal-open guard, the plain-migrate precheck and both identity-cutover command paths — so they can never disagree about which refusal a verdict earns. populatedConvertible records whether the caller CONVERTS a populated markerless store (REFUSE-POPULATED): the conversion command does, so it passes true and the verdict falls through to the pipeline; the guard and the plain migrate precheck do not, so they pass false and refuse it. Every other refusal text is shared unconditionally, and the non-refusing verdicts (Normal, Cutover) return nil.

func RecordedSchemaVersionPath

func RecordedSchemaVersionPath(storePath string) int

RecordedSchemaVersionPath reads the audit schema version through a short-lived read-only handle, reporting 1 (legacy pre-v2) when the file or the meta table is absent. It is used to count how many catch-up steps a run applied.

func RefusePopulatedRefusal

func RefusePopulatedRefusal(storePath string) error

RefusePopulatedRefusal is the storage refusal a populated, markerless store earns on a normal open. It names the path, the converting command, the stopped operation, the unchanged state and the backup guidance.

func RetiredIdentityRefusal

func RetiredIdentityRefusal(raw string) error

RetiredIdentityRefusal is the validation refusal a read command returns when its argument is a recognizable retired two-dash identity. It names the slash form of the same identity so the caller can retry with the resolved value.

func StatStore

func StatStore(storePath string) bool

StatStore reports whether the store file exists, without creating it.

func UnbackedMarkRefusal

func UnbackedMarkRefusal(storePath string) error

UnbackedMarkRefusal is the storage refusal a marker-bearing store with an incomplete fence (missing/mismatched receipt, receipt without marker, or a recorded version below the receipt's floor) earns on a normal open or a command precheck. The state line is a DIAGNOSIS from the observed evidence, not a certainty: a genuine cutover store whose receipt row was lost or damaged presents identically.

Types

type Archive

type Archive struct {
	Path   string
	SHA256 string
	// Skipped is true when the store did not exist at P1's own check, so no
	// archive was made (archive_path is empty and no pre-cutover content is
	// recoverable through this run's artifacts).
	Skipped bool
	Marker  ArchiveMarker
}

Archive is the P1 result: where the consistent archive is, its SHA-256, and the archive's OBSERVED identity-format marker. The marker is the fact the scoped P3a refusal states; it is read at the archive's verify reopen and is never inferred from timing.

func ArchiveStore

func ArchiveStore(ctx context.Context, storePath, backupDir string, now time.Time) (Archive, error)

ArchiveStore runs P1: a consistent logical copy of the committed store, taken through a READ-ONLY source connection so no journal-mode switch or any other write precedes it. The copy is staged as `<final>.<pid>.<seq>.partial`, fsynced, verified (integrity_check + foreign_key_check + the format marker + SHA-256), and only then published to its final name with a link that refuses to overwrite another archive. An unverified file can never sit at the name a recovery instruction points at, and one archiver can never clear, overwrite, or rename away another's bytes mid-flight.

It is skipped only when the store did not exist at this function's own read-only observation of the path: the first open of a brand-new store has no pre-cutover content to archive.

type ArchiveMarker

type ArchiveMarker struct {
	Present bool
}

ArchiveMarker is the marker evidence P1 records by reading the staged archive. It is the observed fact the scoped P3a refusal states.

type CatchUpOptions

type CatchUpOptions struct {
	// StorePath is the store the pipeline converts.
	StorePath string
	// BackupDir overrides where P1 writes the consistent archive. Empty uses the
	// store's own directory.
	BackupDir string
	// Now is the timestamp source for the archive name. Nil means the wall clock.
	Now func() time.Time
	// TimeoutProfile, when non-zero, is applied to the audit catch-up trail. The
	// offline command leaves it zero (the audit subsystem's own default); the
	// first-open auto cutover passes the resolved store profile.
	TimeoutProfile timeouts.Profile
	// Logger owns warnings from the returned trail. Nil retains the process default.
	Logger *slog.Logger
}

CatchUpOptions configures the shared P1+P2 orchestration.

type CatchUpResult

type CatchUpResult struct {
	// Archive is the P1 result (its Path is empty when P1 skipped a store that
	// did not exist).
	Archive Archive
	// AppliedCatchUpSteps is how many audit steps P2 applied this run.
	AppliedCatchUpSteps int
	// Trail is the catch-up audit trail, left OPEN and current: the first-open
	// auto cutover reuses it as the tracker's trail, and the offline command
	// closes it immediately after the catch-up.
	Trail *audit.SqliteAuditTrail
}

CatchUpResult is the P1+P2 state both frontends hand to Convert.

func RunCatchUp

func RunCatchUp(ctx context.Context, opts CatchUpOptions) (CatchUpResult, error)

RunCatchUp is the single P1+P2 orchestration both frontends call, so they can never diverge on the order or on how the applied-step count is computed:

P1 archive (ArchiveStore): a consistent copy through a read-only source,
   skipped only when the store did not exist at P1's own read-only check.
P2 audit catch-up with the DEFERRED identity-format ceiling, so a populated
   legacy store stops at the cutover target and the conversion applies the
   remaining ceiling step atomically with the marker and the receipt.

Order is load-bearing: the archive precedes ANY write, and the catch-up precedes the conversion. The returned trail is left open; the caller owns closing it.

type ConversionMirrors

type ConversionMirrors struct {
	ProvenanceOwned        []IdentityColumn
	ProvenancePreserved    []IdentityColumn
	ProvenanceOwnedContext []IdentityColumn
	PastureOwned           []IdentityColumn
	PastureOwnedArray      []IdentityColumn
	PasturePreserved       []IdentityColumn
	PastureSpecial         []IdentityColumn
}

ConversionMirrors is the read surface the cross-module classification gate needs: every declared mirror that decides what the dry run and the conversion treat as identities. Production code reads the package-private slices directly; this accessor exists so a black-box gate can compare the mirrors to the dependency's schema and to pasture's own schema.

func DeclaredConversionMirrors

func DeclaredConversionMirrors() ConversionMirrors

DeclaredConversionMirrors returns the complete set of declared conversion mirrors. It is the gate's read surface, not a production entry point.

type ConvertOptions

type ConvertOptions struct {
	// StorePath is the resolved database path. It labels errors and names the
	// store in refusals.
	StorePath string
	// Archive is the P1 result this run recorded. P3's scoped refusals name it.
	Archive Archive
	// AppliedCatchUpSteps is the number of audit steps this run's P2 catch-up
	// applied (zero when the store was already current). P3's scoped refusals
	// name it.
	AppliedCatchUpSteps int
	// Now is the timestamp source for the receipt and the ceiling. Nil means
	// the wall clock.
	Now func() time.Time
	// contains filtered or unexported fields
}

ConvertOptions carries the inputs to the P3 conversion transaction.

type CutoverState

type CutoverState struct {
	StorePath string
	// FileExists is the observation that gates P1's archive skip.
	FileExists bool
	// Format is the dependency's inspected identity format.
	Format provenance.IDFormat
	// MarkerPresent is true when the store carries any identity-format marker.
	MarkerPresent bool
	// ReceiptPresent and ReceiptMatches describe the pasture receipt.
	ReceiptPresent bool
	ReceiptMatches bool
	Receipt        Receipt
	// RecordedVersion is the audit schema version (1 when no meta table exists).
	RecordedVersion int
	// Populated is the dependency's EXISTS probe over Provenance identity tables.
	// It is a P0 ROUTING input only; P3a must not re-dispose on it.
	Populated bool
	Outcome   Outcome
}

CutoverState is the full read-only evidence the state machine disposes over.

func EvaluateCutoverOnHandle

func EvaluateCutoverOnHandle(ctx context.Context, db *sql.DB, storePath string, fileExists bool) (CutoverState, error)

EvaluateCutoverOnHandle evaluates the state machine in ONE deferred read transaction, so the marker, the receipt and the recorded version come from a single snapshot and cannot be torn by a concurrent commit.

func InspectOpenState

func InspectOpenState(ctx context.Context, storePath string) (CutoverState, error)

InspectOpenState evaluates the state machine against a store path, opening its own short-lived READ-ONLY handle. An absent store is CUTOVER (the first open creates and cuts it over). A read-only open failure that is NOT a missing file fails closed as a storage error rather than being deferred.

func (CutoverState) EvidenceVerdict

func (s CutoverState) EvidenceVerdict() EvidenceVerdict

EvidenceVerdict classifies the state for the under-lock P3a decision.

type EvidenceVerdict

type EvidenceVerdict int

EvidenceVerdict is the P3a under-lock verdict. It consumes the EVIDENCE rows only (marker, receipt, marker_format match, recorded version vs floor, version vs MaxKnown). The populated column is deliberately NOT consulted: it is a P0 routing input, and P2's catch-up populates the Provenance tables by design.

const (
	EvidenceNormal EvidenceVerdict = iota + 1
	EvidenceRefuseUnbackedMark
	EvidenceNewerSchema
	EvidenceMarkerless
)

type IdentityColumn

type IdentityColumn struct{ Table, Column string }

IdentityColumn names one "table.column" pair in a conversion mirror.

type InjectPoint

type InjectPoint string

InjectPoint names an in-transaction checkpoint the test-only injection seam fires at. It is unexported surface: no production path constructs one.

type Inspection

type Inspection struct {
	StorePath string
	Format    provenance.IDFormat
	Populated bool
}

Inspection is the read-only identity-format inspection of one store. It never writes and never activates schema.

func Inspect

func Inspect(ctx context.Context, reader InspectionReader, storePath string) (Inspection, error)

Inspect reads the store's identity format without writing anything. storePath is used only to label errors; it is never opened here.

func (Inspection) Current

func (i Inspection) Current() bool

Current reports whether the store already carries the slash marker.

func (Inspection) Retired

func (i Inspection) Retired() bool

Retired reports whether the store must be converted before it can be opened.

type InspectionReader

type InspectionReader = provenance.IDFormatReader

InspectionReader is the read surface Inspect needs: an open database or the caller's transaction both satisfy it, so the same inspection runs before and under the conversion lock.

type Outcome

type Outcome int

Outcome is the read-only verdict of the cutover state machine. It is the ONE classification the guard, both pipeline frontends, the plain-migrate precheck and the guidance texts consume, so they can never disagree about whether a store is cut over.

const (
	// OutcomeNormal: marker + matching receipt + recorded version >= floor.
	OutcomeNormal Outcome = iota + 1
	// OutcomeCutover: no marker and no receipt, and the store has no Provenance
	// identity rows (or the file does not exist). The first open is a cutover.
	OutcomeCutover
	// OutcomeRefusePopulated: no marker and no receipt, but Provenance identity
	// rows exist. Only the operator command converts these.
	OutcomeRefusePopulated
	// OutcomeRefuseUnbackedMark: a marker outside a fenced, archived pasture
	// cutover (missing/mismatched receipt, receipt without marker, or a recorded
	// version below the receipt's floor).
	OutcomeRefuseUnbackedMark
	// OutcomeNewerSchema: the recorded audit version is above this build's
	// maximum; a read-only preflight refusal, evaluated before the rest.
	OutcomeNewerSchema
)

func (Outcome) String

func (o Outcome) String() string

type Preview

type Preview struct {
	Inspection
	// Columns counts, per "table.column", the distinct values that would convert.
	Columns map[string]int
	// EpochProjectionRows and AuditPayloadRows count structured documents that
	// declare identity fields that would convert.
	EpochProjectionRows int
	AuditPayloadRows    int
	// RecordedVersion is the audit schema version this run read.
	RecordedVersion int
	// PendingCatchUpSteps is how many audit steps the deferred catch-up phase
	// (P2) would apply before the conversion transaction runs.
	PendingCatchUpSteps int
	// EpochIDLowerBound is a LOWER-BOUND count of work the catch-up exposes: the
	// retired-shaped audit_events.epoch_id values a pre-v4 store still carries.
	// It is informational only — never a verdict — because the catch-up moves
	// them into context_edges where the conversion then translates them.
	EpochIDLowerBound int
	// Blockers names runtime-state tables that hold rows and would refuse the
	// conversion.
	Blockers []string
	// Preserved names "table.column" pairs the conversion deliberately preserves
	// byte-exact: opaque, equality-only labels that are outside the dependency's
	// owned identity set (see preservedDependencyColumns). Reporting them keeps
	// the dry run's plan and the real conversion in agreement — a column
	// excluded from conversion is named as preserved, not silently dropped.
	Preserved []string
}

Preview is the strictly read-only dry-run report. It writes nothing: no transaction, no DDL, no marker, no backup. Every count is what conversion WOULD change.

func DryRun

func DryRun(ctx context.Context, db *sql.DB, storePath string) (Preview, error)

DryRun runs the read-only dry run. It never writes and never begins a transaction, so the store's bytes (and its SHA-256) are unchanged by it.

func (Preview) WouldChange

func (p Preview) WouldChange() bool

WouldChange reports whether conversion has any work to do on this store.

type Receipt

type Receipt struct {
	MarkerFormat  string
	FloorVersion  int
	ArchivePath   string
	ArchiveSHA256 string
	CutoverAt     int64
}

Receipt is the pasture-owned singleton cutover record. It is evidence, not a lookup key: nothing resolves identities through it.

type ReceiptReader

type ReceiptReader interface {
	QueryContext(context.Context, string, ...any) (*sql.Rows, error)
	QueryRowContext(context.Context, string, ...any) *sql.Row
}

ReceiptReader is the read surface the state machine needs: an open database or the caller's transaction both satisfy it.

type Report

type Report struct {
	StorePath          string
	ArchivePath        string
	ArchiveSHA256      string
	AlreadyCurrent     bool
	ReceiptWritten     bool
	FloorVersion       int
	ChangedFields      int
	ChangedMutations   int
	ChangedAllocations int
	PastureRows        int
	Reclaimed          int
}

Report is what one conversion did. It is returned even for the no-op path (already current).

func Convert

func Convert(ctx context.Context, db *sql.DB, opts ConvertOptions) (Report, error)

Convert runs the P3 conversion transaction against an already-open shared handle. The handle MUST be the only writer: the caller (the CLI handler or the open path) is responsible for the stopped-writer requirement and for closing the handle before any reopen.

It never opens a second connection and never runs a normal open inside the conversion transaction.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL