repo

package
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 31, 2026 License: MIT Imports: 24 Imported by: 0

Documentation

Overview

Package repo This file defines the agentenv-specific archive ("bundle") format used by `agentenv export --format agentenv` / `agentenv import`. Unlike a flat rootfs tar (which carries a single node's filesystem for `docker import`), a bundle carries agentenv's own metadata so a whole environment — its commit-DAG, tags, and every node's rootfs — can move between agentenv instances and be restored with history and rollback points intact.

Layout inside the tar (optionally gzip-compressed): 

manifest.json        FIRST entry: the DAG/tags/HEAD (see Manifest)
nodes/<id>/...        one rootfs tree per node, hardlink-shared across nodes
                      (see image.TarInto's `seen` map) so a repo bundle stays
                      as compact as the on-disk store

A "snapshot" bundle is the same format with exactly one node whose Parent is cleared — a self-contained single environment without the surrounding history.

This file holds command execution inside the managed environment (one-shot Exec, interactive Shell / RunInteractive, background Spawn) plus the tracked- process table (Procs/Kill/killAll/AdoptProc) and the sandbox env allow-list.

This file holds disk-reclamation and DAG-thinning: GC removes orphan snapshots not referenced by the DAG, and Retain sparsifies old interior nodes on an exponential curve so the node count stays ~ keepRecent + log2(total).

Package repo is the orchestration layer: it ties dag (metadata) + btrfs (snapshots) + image (rootfs) + sandbox (execution) together and exposes the high-level operations init/exec/spawn/commit/checkout/log/gc.

A long-lived agent (see `agentenv serve`) keeps one *Repo for its whole session. Background processes started with Spawn are tracked in memory; on Checkout they are killed (the agent process itself, living in the control-root, is never touched). This realizes "agent survives, other processes reset".

On-disk layout (AGENTENV_ROOT, default /agentfs, must live on btrfs): 

<root>/
  nodes/<id>/   one read-only snapshot per immutable node
  work/current/ the live writable inner-env subvolume (commands run here)
  logs/<id>.log output of background (spawned) processes
  meta.json     commit-DAG metadata

This file holds the rootfs-seeding operations that create or grow the commit-DAG's contents from an external source: the one-time `init` (root node) and the incremental `init --sync` (child node). The bundle-based export/import lives in bundle.go; snapshot storage lives in the backend.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type CandidateResult 

type CandidateResult struct {
	Name     string // human label ("+", "B", "v1.2", ...)
	Cmd      string // shell command run inside the env to produce this branch
	Node     string // resulting branch tip (empty on build failure)
	TestExit int    // exit code of the test command on this branch (-1 if not run)
	Err      string // non-empty if the branch failed to build or test
}

CandidateResult is what one branch of a tournament produced.

type Capturer 

type Capturer struct {
	// contains filtered or unexported fields
}

Capturer auto-snapshots the inner-env whenever it changes and settles. Change detection is channel-agnostic and, where possible, event-driven:

  • An inotify Watcher signals on any change (shell command OR direct file edit), so the idle path does no filesystem walking.
  • A periodic Token() check is kept as a backstop (and as the only mechanism if the watcher can't be created).

This removes the need for the agent to ever call "commit" — the environment versions itself like a DVR.

func (*Capturer) Flush 

func (c *Capturer) Flush(label string)

Flush snapshots immediately if the env changed since the last node (used right after a shell command so the node lands promptly with the command's label).

func (*Capturer) SetLabel 

func (c *Capturer) SetLabel(s string)

SetLabel hints the label for the next snapshot (e.g. the command being run).

func (*Capturer) SetOnSnapshot 

func (c *Capturer) SetOnSnapshot(fn func(id, label string))

SetOnSnapshot registers a callback invoked after each auto-snapshot (UI feedback).

func (*Capturer) Stop 

func (c *Capturer) Stop()

Stop ends the loop and waits for it to exit.

type Change 

type Change struct {
	Kind byte // '+' added, '-' removed, 'M' modified
	Path string
}

Change is one path that differs between two nodes.

type Lock 

type Lock struct {
	// contains filtered or unexported fields
}

Lock holds an exclusive flock on the repo root so two mutating sessions cannot corrupt meta.json. Read-only commands skip it.

Why a struct instead of a `func() / *os.File` pair: the *os.File must stay reachable until release, but a `func()` closure that captures it doesn't keep it from being GC'd if the GC's escape analysis decides the closure itself is short-lived (it can be — callers `defer release()` then drop the variable). A returned struct is naturally kept by the caller's `defer lock.Release()`.

func AcquireLock 

func AcquireLock(root string) (*Lock, error)

AcquireLock takes the exclusive non-blocking flock and returns a Lock the caller must Release(). The kernel also drops the flock on process exit, but callers SHOULD route process termination through main (not os.Exit from random call sites) so deferred Release fires and so any in-flight save can finish.

func LockFromFD added in v0.3.1 

func LockFromFD(fd int) *Lock

LockFromFD wraps an already-open, flock-holding file descriptor (inherited across an execve during a hot upgrade) as a *Lock, WITHOUT taking the flock again. This is mandatory on the re-exec path: the flock is associated with the open file description, which survives execve as long as the fd stays open. Calling AcquireLock instead would open a SECOND description of .lock and LOCK_EX|LOCK_NB against it would EWOULDBLOCK — flock is mutually exclusive across descriptions even within the same process.

func (*Lock) FD added in v0.3.1 

func (l *Lock) FD() int

FD returns the underlying lock file descriptor, so a caller preparing an execve can clear its FD_CLOEXEC bit and pass the number to the new image. Returns -1 if the lock is nil/closed.

func (*Lock) Release 

func (l *Lock) Release()

Release drops the flock and closes the file. KeepAlive is belt-and-suspenders: it ensures the *os.File survives until this method runs even under aggressive GC reordering (so the kernel never drops the lock prematurely from the GC finalizer).

type Manifest added in v0.4.0 

type Manifest struct {
	FormatVersion int               `json:"format_version"`
	Kind          string            `json:"kind"` // bundleKindRepo | bundleKindSnapshot
	Head          string            `json:"head"`
	Tags          map[string]string `json:"tags,omitempty"`
	Nodes         []dag.Node        `json:"nodes"`
	CreatedAt     time.Time         `json:"created_at"`
}

Manifest is the JSON header of an agentenv bundle (the first tar entry). It is self-describing so `import` can auto-detect and reconstruct without out-of-band knowledge; a reader that does not find it treats the archive as a flat rootfs.

type Proc 

type Proc struct {
	PID     int
	Args    []string
	Started time.Time
	Log     string
	// contains filtered or unexported fields
}

Proc is a tracked background process running in the inner-env.

type Repo 

type Repo struct {
	// contains filtered or unexported fields
}

func Open 

func Open(root string, be *backend.Backend) (*Repo, error)

func (*Repo) AdoptProc added in v0.3.1 

func (r *Repo) AdoptProc(pid int, args []string, started time.Time, log string)

AdoptProc re-registers a background process that this process already owns as a child but has no *exec.Cmd handle for — the case after a hot-upgrade execve, where the new binary image inherits the old image's children (execve preserves the PID, so the kernel still lists them as this process's children) but loses all in-memory *exec.Cmd state.

Reaping: instead of cmd.Wait() we start a per-PID unix.Wait4 goroutine. Per-PID (never Wait4(-1)) is deliberate: a global reaper would race os/exec's own wait4(pid) for children started later via Spawn and steal their exit status. The two sets of PIDs are disjoint, so per-PID Wait4 for adopted procs coexists with cmd.Wait() for freshly-spawned ones.

If the PID is already gone (it exited in the execve window), we skip it.

func (*Repo) Backend 

func (r *Repo) Backend() *backend.Backend

Backend returns the active backend (for display).

func (*Repo) Checkout 

func (r *Repo) Checkout(ref string) error

Checkout rolls back / switches to any node (also accepts tag names or unique ID prefixes via ResolveRef). It kills the inner-env processes, rebuilds the working volume from that node, and points HEAD there. A later commit then forks under it, forming a tree. The agent process is untouched.

func (*Repo) CheckoutCount 

func (r *Repo) CheckoutCount() int

CheckoutCount returns how many checkouts (rollbacks) have happened — used by the supervisor to tell a rollback-kill from a normal agent exit.

func (*Repo) Commit 

func (r *Repo) Commit(message, command string) (*dag.Node, error)

Commit freezes the working volume as a child of HEAD and advances HEAD.

func (*Repo) Delete added in v0.2.0 

func (r *Repo) Delete(ref string) error

Delete removes a node from the DAG and deletes its on-disk snapshot. Its children are re-parented to its parent so the tree stays connected (deleting a middle node keeps its descendants). Refuses to delete the current HEAD (it would orphan the live working tree — checkout elsewhere first) or the last remaining node. Deleting a node's snapshot is safe even if children hardlink-share its files: the shared inodes stay alive as long as a child still references them (same reason GC is safe).

func (*Repo) Diff 

func (r *Repo) Diff(aRef, bRef string) ([]Change, error)

Diff lists the paths that differ between node aID and node bID (either may be "" to mean an empty tree). Accepts tag names or unique ID prefixes; ref resolution and DAG access happen UNDER the same opMu so a concurrent commit or retention pass can't change the resolution between resolve and use.

func (*Repo) Exec 

func (r *Repo) Exec(args []string, stdin io.Reader, stdout, stderr io.Writer) (int, error)

Exec runs a real command in the current working volume (foreground) and waits. It returns the command's exit code and an error. A command that runs to completion with a non-zero status is NOT an agentenv error: it returns (code, nil). A non-nil error means agentenv itself failed (could not set up the volume, start the process, etc.).

func (*Repo) Export added in v0.4.0 

func (r *Repo) Export(ref string, w io.Writer, format string, all, gz bool) error

Export writes a node's rootfs (format "rootfs", the default) or an agentenv bundle (format "agentenv") to w. For rootfs, ref selects the node (default HEAD) and the output is a flat tar suitable for `docker import` / `ADD`. For agentenv, all=true exports the whole repo (DAG + tags + every node's rootfs) and all=false exports a single self-contained snapshot of ref (default HEAD). gz gzip-compresses the stream. Export is read-only (reads immutable node dirs).

func (*Repo) GC 

func (r *Repo) GC() ([]string, error)

GC deletes node subvolumes on disk that are not referenced by the DAG (orphans), e.g. nodes dropped by retention. This is where sparsified snapshots are actually freed from disk.

func (*Repo) Head 

func (r *Repo) Head() string

Head returns the current HEAD node ID.

func (*Repo) ImportArchive added in v0.4.0 

func (r *Repo) ImportArchive(src, message string) (*dag.Node, error)

ImportArchive brings an external environment into this repo. It extracts src (a local path or http(s) URL, .tar/.tar.gz) to a temp dir and auto-detects:

  • agentenv repo bundle → restore the whole DAG/tags into an empty repo
  • agentenv snapshot bundle → add its single node under HEAD
  • plain rootfs tar → add the rootfs as one node under HEAD

The new HEAD's working rootfs is checked out on success.

func (*Repo) InitFrom 

func (r *Repo) InitFrom(src string) (*dag.Node, error)

InitFrom builds the root node by seeding the rootfs from an existing directory (e.g. "/" — the current container filesystem — or a workspace dir) instead of downloading a base image. This makes the managed environment BE the agent's real environment, so the agent can run inside it (system installs + files all captured) without changing its code — only its launch command.

func (*Repo) InitTarball 

func (r *Repo) InitTarball(src string) (*dag.Node, error)

InitTarball builds the root node by extracting a .tar / .tar.gz at src (filesystem path or http(s) URL) into the working rootfs. Use this for demo rootfs tarballs and air-gapped setups; for production, use InitFrom to seed from the agent's own running container.

func (*Repo) Kill 

func (r *Repo) Kill(pid int) error

Kill terminates one tracked background process (and its inner-env PID namespace).

func (*Repo) Leaves 

func (r *Repo) Leaves() []*dag.Node

Leaves returns the branch tips (nodes with no children), oldest first. Each tip is a distinct explored end-state — the unit of branch exploration.

func (*Repo) Lock added in v0.3.1 

func (r *Repo) Lock() *Lock

Lock returns the flock this session holds (nil if none was set).

func (*Repo) Nodes 

func (r *Repo) Nodes() []*dag.Node

Nodes returns all nodes oldest-first (for structured log output).

func (*Repo) ProcAlive 

func (r *Repo) ProcAlive(pid int) bool

ProcAlive reports whether a tracked background process is still running.

func (*Repo) Procs 

func (r *Repo) Procs() []*Proc

Procs returns a snapshot of the tracked background processes, sorted by PID.

func (*Repo) ResolveRef 

func (r *Repo) ResolveRef(ref string) string

ResolveRef turns a tag name OR a node ID (or unique ID prefix) into a node ID. It returns "" if nothing matches. Used by Checkout, Show, Diff, and the API so the agent/operator can refer to nodes by short, memorable names. ResolveRef resolves a ref (tag, full ID, or unique ID prefix) to a node ID, taking opMu around the lookup. Returns "" if not found / ambiguous / unsafe.

IMPORTANT: ResolveRef takes the lock and releases it before returning. Any caller that needs to act on the resolved ID atomically (Checkout / Show / Diff / SetTag — they all read the DAG further after resolving) MUST use resolveRefLocked while already holding opMu, otherwise a concurrent op can invalidate the resolution between the resolve and the act on it (TOCTOU).

func (*Repo) Retain 

func (r *Repo) Retain(keepRecent int) []string

Retain sparsifies the DAG: it keeps all "recent" and structural nodes and thins older interior nodes on an exponential (DVR ring) curve, so the node count stays ~ keepRecent + log2(total). Dropped nodes are removed from the DAG (their children are re-linked to the nearest surviving ancestor); their on-disk snapshots become orphans, reclaimed by GC. Returns the dropped node IDs.

Always kept:

  • the root and HEAD
  • branch points (>=2 children) and leaves (branch tips) — pruning these would lose reachable states
  • the newest keepRecent non-structural nodes

Then, among older non-structural nodes ordered newest->oldest by rank i, only ranks where i is 2^k-1 (0,1,3,7,15,31,...) are kept.

func (*Repo) Root added in v0.3.1 

func (r *Repo) Root() string

Root returns the repo root directory (AGENTENV_ROOT).

func (*Repo) RunInteractive added in v0.2.0 

func (r *Repo) RunInteractive(args []string) (int, error)

RunInteractive runs an interactive agent on a PTY (supervise's TTY mode) and registers it as the foreground process so a concurrent Checkout() can kill it. Returns the agent's exit code. The supervisor calls this in a loop: after a rollback kills the agent, CheckoutCount increases and the supervisor re-invokes RunInteractive to relaunch from the restored environment.

func (*Repo) SetLock added in v0.3.1 

func (r *Repo) SetLock(l *Lock)

SetLock records the flock this session holds, so long-running modes (daemon) can reach its fd — the hot-upgrade path clears FD_CLOEXEC on it and passes it to the re-exec'd image, which keeps the flock held with no release gap.

func (*Repo) SetPreserveProcs added in v0.2.0 

func (r *Repo) SetPreserveProcs(v bool)

SetPreserveProcs enables self-rollback mode: Checkout reverts the working rootfs in place WITHOUT killing running processes, so the agent that triggered the rollback survives and observes the reverted environment.

func (*Repo) SetTag 

func (r *Repo) SetTag(name, ref string) error

SetTag assigns name to a node ID (or another ref). Empty id removes the tag. Tag names cannot collide with node IDs (12-hex chars) — guarded here.

func (*Repo) Shell 

func (r *Repo) Shell(args []string) (int, error)

Shell runs an interactive shell inside the current working volume on a PTY, returning its exit code. File changes made in the shell are auto-captured by the Capturer running alongside it — no per-command wrapping or commit needed.

func (*Repo) Show 

func (r *Repo) Show(ref string) ([]Change, string, error)

Show returns what a node changed relative to its parent (like `git show --stat`), along with the parent ID ("" if it is the root). Accepts refs. Same lock discipline as Diff: resolve + read happen atomically.

func (*Repo) Spawn 

func (r *Repo) Spawn(args []string) (*Proc, error)

Spawn starts a background process in the current working volume, tracks it, and returns its handle. Output is written to logs/<id>.log. A reaper goroutine removes it from the table when it exits.

func (*Repo) StartCapturer 

func (r *Repo) StartCapturer() *Capturer

StartCapturer launches the background capture loop and records it on the repo so Checkout can coordinate (pause) with it.

func (*Repo) Status 

func (r *Repo) Status() Status

Status returns a runtime snapshot. Cheap: takes the lock briefly and stats the root tree once.

func (*Repo) SyncFrom added in v0.4.0 

func (r *Repo) SyncFrom(src, message string, del bool) (*dag.Node, error)

SyncFrom is the incremental counterpart of InitFrom (`init --sync`): instead of building a root node, it overlays new/changed files from src onto the CURRENT working rootfs and commits the result as a child of HEAD. This folds in files that landed on the host after the one-time init seed — e.g. an entrypoint step or `docker exec` that downloaded packages/config into the container's real "/" (which the sandboxed agent, running in work/current, never saw). With del=true it also removes files absent from src (mirror mode); the default is additive (overlay only), which is safe for the "downloaded new files" case.

func (*Repo) SyncTarball added in v0.4.0 

func (r *Repo) SyncTarball(src, message string) (*dag.Node, error)

SyncTarball is SyncFrom for a tarball source: it extracts the tar over the current working rootfs (overwrite-in-place) and commits a child of HEAD. Mirror-delete is not supported for tarball sources.

func (*Repo) Tags 

func (r *Repo) Tags() map[string]string

Tags returns a copy of the tag table (name → node ID), sorted by name.

func (*Repo) Tournament 

func (r *Repo) Tournament(baseRef, test string, candidates []struct{ Name, Cmd string }, keep bool) (*TournamentResult, error)

Tournament forks the environment N times from base, runs each candidate concurrently in its own isolated workspace, then runs test concurrently in each, and picks the first candidate (by input order, for determinism) whose test exits 0. The main work/current is NOT touched during the run; only the final HEAD reposition (always done at the end) affects it.

  • keep==true && there is a winner → HEAD lands on the winner's node.
  • otherwise → HEAD lands on the base.

Build and test phases are parallel; the only serialization point is a brief opMu hold per branch around assigning a node ID and adding to the DAG. The backend workspace primitives are concurrent-safe (each touches a distinct path), and the sandbox runner does too (each invocation forks its own child in its own namespaces).

func (*Repo) Tree 

func (r *Repo) Tree() string

Tree renders the whole commit-DAG, marking HEAD.

type Status 

type Status struct {
	Backend    string
	Root       string
	Head       string
	NodeCount  int
	LeafCount  int
	ProcCount  int
	DiskBytes  int64 // bytes on disk under root (real, hardlink-shared files counted once)
	PollMs     int
	DebounceMs int
	Ignore     []string // rootfs-relative paths currently excluded from snapshots
}

Status is a one-shot snapshot of the repo's runtime state (for `agentenv status`).

type TournamentResult 

type TournamentResult struct {
	Base       string            // resolved base node id
	Candidates []CandidateResult // one per candidate, in input order
	Winner     string            // candidate name whose test exit==0 (first such by input order)
	WinnerNode string            // the winning node id, "" if none
}

TournamentResult is the outcome of one tournament call.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.