migrate

package
v0.116.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 9, 2026 License: MIT Imports: 17 Imported by: 0

Documentation

Overview

Package migrate is the one-shot v0 → v1 importer described in docs/multi-device-storage.md section 5.

Design contract:

  • v0 directory is treated as immutable. This package opens every v0 file with O_RDONLY (enforced by readOnlyOpen) and never issues mkdir/unlink against a path under the v0 root.
  • Import is idempotent at the domain granularity. `migration_status` records each domain's phase so a kill -9 partway through a domain can be resumed by re-running `kojo --migrate`.
  • Manifest verification brackets the run. The pre-run manifest is written into `migration_in_progress.lock`; the same calculation is repeated at completion time. Mismatch (= v0 changed under us) fails the migration without producing `migration_complete.json`.

The bulk of per-domain logic lives in domain-specific Importer implementations under this package. This file orchestrates them.

Index

Constants

View Source
const (
	LockFileName     = "migration_in_progress.lock"
	CompleteFileName = "migration_complete.json"

	// MtimeSafetyWindow is the window during which a recent v0 file
	// modification is considered suspicious (5.5 step 2). Operators can
	// override via Options.MtimeSafetyWindow.
	MtimeSafetyWindow = 5 * time.Minute
)

LockFileName is dropped into the v1 directory while a migration is in flight. Removed (renamed to CompleteFileName) on success.

Variables

View Source
var (
	ErrV0Locked          = errors.New("migrate: v0 dir is locked (kojo v0 still running)")
	ErrV0RecentlyChanged = errors.New("migrate: v0 dir modified within the safety window")
	ErrAlreadyComplete   = errors.New("migrate: v1 dir already has migration_complete.json")
	ErrPartialV1         = errors.New("migrate: partial v1 dir exists; pass Restart=true or remove it manually")
	ErrManifestMismatch  = errors.New("migrate: v0 manifest changed during migration")
	ErrResumeMismatch    = errors.New("migrate: existing migration_in_progress.lock has a different v0 manifest; rerun with --migrate-restart")
	ErrNoImporters       = errors.New("migrate: no domain importers registered (Phase 1 build); run a build with importers wired")
	ErrV0EqualsV1        = errors.New("migrate: V0Dir and V1Dir resolve to the same canonical path; refusing to migrate")
)

Sentinel errors. Callers can branch on these via errors.Is.

Functions

func ChecksumOf

func ChecksumOf(ctx context.Context, st *store.Store, domain string) (string, error)

ChecksumOf returns the stored source_checksum for `domain`, or "" if unrecorded / NULL. Used by tests and audit tooling to verify that a completed import wrote the fingerprint.

func CountOf

func CountOf(ctx context.Context, st *store.Store, domain string) (int, error)

CountOf returns the stored imported_count for `domain`, or 0 if the row is absent (sql.ErrNoRows) or the column is NULL (the schema declares imported_count as nullable INTEGER, see 0001_initial.sql). Used by tests to verify a marker importer ran with the expected row count — direct SQL access from tests would couple them to the schema layout, which this accessor avoids.

func ManifestSHA256

func ManifestSHA256(root string) (string, error)

ManifestSHA256 walks `root` in deterministic order and returns a single hex-encoded SHA256 digest covering every regular file's relative path, size, and content hash. This is the input to the pre/post manifest comparison in section 5.5 (steps 3 and 10).

Determinism guarantees:

  • Walk order is sorted by relative path (case-sensitive byte order).
  • Symlinks are NOT followed (lstat). A symlink contributes its target string but no file content. This matches the read-only contract: we never traverse outside `root`.
  • The lock file (`kojo.lock`) is excluded so v0 binary starts/stops while we're idle do not invalidate the manifest.
  • Sockets / fifos / device files are excluded (they should not exist in a v0 config dir, and including them would make the manifest depend on transient OS state).

The output is a 64-char hex string, suitable for storing in LockFile.V0SHA256Manifest and CompleteFile.V0SHA256Manifest.

func MarkPhase

func MarkPhase(ctx context.Context, st *store.Store, domain, phase string, importedCount int, checksum string) error

MarkPhase records `phase` for `domain` in migration_status. Helper for importers; safe to call multiple times.

`checksum` is an optional content fingerprint for the domain's v0 source files. Importers compute it via importers.domainChecksum before marking the domain imported (the value reflects the directory walk performed at the start of the run, not work done during the row-copy loop). Pass "" to leave the column as it was on the prior row — useful when an importer only updates phase mid-flight without re-walking source files. On insert with an empty checksum, the column is stored as NULL.

finished_at is stamped for any terminal phase ('imported', 'cutover', 'complete') so operators can read elapsed durations from the table. 'pending' is treated as "in progress" and leaves finished_at NULL.

func PhaseOf

func PhaseOf(ctx context.Context, st *store.Store, domain string) (string, error)

PhaseOf returns the recorded phase for `domain`, or "" if unrecorded. An sql.ErrNoRows is treated as "not started" so callers can branch on the empty string without importing database/sql.

func Register

func Register(imp Importer)

Register adds an Importer to the run list. Importers are run in registration order, so packages with dependencies (e.g. agent_messages must run after agents) should register accordingly.

Types

type CompleteFile

type CompleteFile struct {
	V0Path           string `json:"v0_path"`
	V0SHA256Manifest string `json:"v0_sha256_manifest"`
	V1SchemaVersion  int    `json:"v1_schema_version"`
	CompletedAt      int64  `json:"completed_at"`
	MigratorVersion  string `json:"migrator_version"`
}

CompleteFile records a successful migration.

type Importer

type Importer interface {
	Domain() string
	Run(ctx context.Context, st *store.Store, opts Options) error
}

Importer migrates a single v0 domain into the v1 store. Each Domain() must be unique and is used both as the migration_status key and in human-readable progress logs.

Implementations MUST:

  • Be idempotent: re-running an importer after partial success must converge on the same final state. Use migration_status (`phase` column) to skip work that has already been recorded as complete.
  • Be read-only against opts.V0Dir. All file opens must go through readAllRO / readOnlyOpen.
  • Track work via store DB transactions; partial inserts must be visible only after the importer's own commit.

Concrete importers are registered from internal/migrate/importers in a single documented order. The orchestrator in migrate.go iterates this list in registration order.

type LockFile

type LockFile struct {
	V0Path           string `json:"v0_path"`
	V0SHA256Manifest string `json:"v0_sha256_manifest"`
	StartedAt        int64  `json:"started_at"`
	MigratorVersion  string `json:"migrator_version"`
}

LockFile is the on-disk representation of an in-progress migration.

type Options

type Options struct {
	V0Dir             string
	V1Dir             string
	MigratorVersion   string // e.g. "v1.0.0"; written into complete.json
	MtimeSafetyWindow time.Duration

	// SkipMtimeCheck, when true, bypasses the v0 mtime safety window
	// (5.5 step 2). The window exists to catch the "v0 binary still
	// active" footgun where mid-write files would copy half-finished
	// state — but operators who have stopped v0 and just touched the
	// dir manually (e.g. extracting from backup, ls -la for inspection)
	// hit the window even though v0 is dead. Use sparingly: a
	// mid-write at migration time produces silent corruption with no
	// recovery. Wired at the cmd/ layer as --migrate-force-recent-mtime.
	SkipMtimeCheck bool

	// Restart, when true, deletes a partial v1 dir (without
	// migration_complete.json) before starting fresh. Refuses to touch a
	// completed v1 dir (5.6).
	Restart bool

	// MigrateExternalCLI controls 5.5.1 symlink/projects.json updates.
	// Default true at the cmd/ layer; this package treats nil-vs-set
	// equivalently and uses the field as-is.
	MigrateExternalCLI bool

	// HomePeer is stamped on every blob_refs row created by the blobs
	// importer. Empty falls back to os.Hostname() inside the importer;
	// tests pass a fixed value so source_checksum and the row stamp stay
	// deterministic. Phase 4 peer_registry will rewrite these columns
	// via a one-shot UPDATE once peer ids are real.
	HomePeer string

	// Now is injectable for tests. Defaults to time.Now.
	Now func() time.Time

	// SkipV1Acquire, when true, suppresses migrate.Run's own
	// configdir.Acquire on V1Dir. The caller MUST have already
	// acquired the lock and MUST keep it held for the entire
	// Run call. Reserved for callers that want to coordinate the
	// lock externally; the normal flow lets Run take the lock.
	SkipV1Acquire bool

	// PreCompleteHook fires AFTER importers + post-import
	// manifest verify but BEFORE migration_complete.json is
	// published. Used by the cmd/ layer to drop the v0
	// credentials carry-forward into v1 at the right phase —
	// after Run's own canResumeV1 / wipe gate has accepted the
	// state, and inside Run's advisory-lock critical section,
	// so the carry-forward never trips canResumeV1's "credentials
	// without sentinel = partial v1" branch on the next retry.
	// A non-nil error returned by the hook fails Run; the
	// migration lockfile stays so an operator can retry.
	PreCompleteHook func() error
}

Options configures Run.

type Result

type Result struct {
	V0Manifest      string
	DomainsImported []string
	Warnings        []string
	CompletedAt     time.Time
}

Result summarizes a successful or failed Run for the caller (the cmd layer). Even on error, partial fields may be populated for diagnostics.

func Run

func Run(ctx context.Context, opts Options) (*Result, error)

Run performs the v0 → v1 migration. The function is intentionally long and linear so the state transitions match section 5.5 step-by-step.

Directories

Path Synopsis
Package externalcli implements docs/multi-device-storage.md §5.5.1's continuity layer for path-hashed external-CLI transcript stores (claude, codex).
Package externalcli implements docs/multi-device-storage.md §5.5.1's continuity layer for path-hashed external-CLI transcript stores (claude, codex).
Package importers contains the per-domain v0→v1 importers wired into the migrate orchestrator's Register() list.
Package importers contains the per-domain v0→v1 importers wired into the migrate orchestrator's Register() list.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL