vmm

package
v0.7.23 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 23, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Overview

Package vmm is the policy layer of the Firecracker warm-VMM pool described in plans/snapshot-clone-fast-boot.md Phase 4. It sits on top of the firecracker_vmm_pool table primitives in internal/store/store.go and exposes a typed API the runtime driver (PR 4-B) and metrics exporter (PR 4-C) consume.

What lives here vs. internal/store:

  • internal/store: schema, raw CRUD, partial unique index, the SELECT-then-UPDATE idempotent allocator under SQLite's single writer. The load-bearing transactional substrate. Touching this triggers pr-review.md §5 — the indexes are the contract.

  • this package: policy on top of those primitives. Per-template depth knobs, the Slot value type callers depend on (decoupled from store.FirecrackerVMMSlot so re-exports don't leak the store layer), and the future home for the refill goroutine (PR 4-B) and metrics aggregation (PR 4-C). No SQL of its own — the store is the boundary.

What does NOT live here in PR 4-A:

  • The concrete VMM spawner. spawner.go declares the seam (the Spawner and Handle interfaces); PR 4-B's runtime adapter (wrapping *firecracker.Driver) plugs in the real spawn-and-LoadSnapshot implementation. PR 4-A's tests inject a fake to exercise the state machine without touching real processes or root.

  • The refill background goroutine. This package's primitives — Acquire / Release / Stats — are the substrate the loop sits on; the loop itself lands in refill.go alongside PR 4-B's integration.

  • Any Driver.Create change. The pool is an inert substrate until PR 4-B wires it.

Import discipline: this package depends only on internal/store and the Go standard library. It MUST NOT import internal/runtime/..., internal/service, or any package above the store layer — those will import this one (via the runtime adapter in PR 4-B), and we keep the cycle wall up by being strict here.

Index

Constants

This section is empty.

Variables

View Source 
var ErrNoLoadedSlot = errors.New("vmm pool: no loaded slot for this template")

ErrNoLoadedSlot is the policy-layer mirror of store.ErrNoFreeFirecrackerVMMSlot. PR 4-B's runtime adapter translates this into "fall back to cold spawn" rather than treating it as an error state — an empty pool is the expected behavior between refill ticks.

Functions

This section is empty.

Types

type Pool 

type Pool struct {
	// contains filtered or unexported fields
}

Pool is the per-host warm-VMM pool's policy layer. The zero value is not usable; callers must go through New. Safe for concurrent use: per-template depth state is guarded by a small mutex, and all SQLite writes flow through store's single-writer connection.

func New 

func New(st *store.Store, logger *slog.Logger) *Pool

New returns a Pool. logger may be nil; the package logs nothing in PR 4-A but the spawn/refill paths in PR 4-B will use it heavily, so the field is wired now to avoid a constructor change later.

func (*Pool) Acquire 

func (p *Pool) Acquire(ctx context.Context, templateID, sandboxID string, now time.Time) (*Slot, error)

Acquire claims one 'loaded' slot for the (templateID, sandboxID) pair. Returns ErrNoLoadedSlot when no claimable slot exists for the template — PR 4-B's caller treats that as the cold-spawn fallback signal, not as a failed create.

Idempotency mirrors the store layer's: a second Acquire for the same sandbox returns the slot already claimed, not a new one. The partial unique index on sandbox_id is the load-bearing guarantee; the store's idempotency pre-check makes the happy path observable.

now is injected so tests can pin allocated_at. Pass time.Now() in production callers.

func (*Pool) AcquireWithHandle 

func (p *Pool) AcquireWithHandle(ctx context.Context, templateID, sandboxID string, now time.Time) (*Slot, SpawnedHandle, error)

AcquireWithHandle is Acquire plus the in-memory SpawnedHandle the refill loop registered after RecordLoaded. The runtime driver's pool-hit branch needs the handle to PATCH per-sandbox state onto the paused VMM before Resume; the SQLite row alone is insufficient because the host PID and process supervision live on the handle.

On hit: returns the slot row, the handle, and a nil error. The handle is removed from the pool's internal map at the same time — ownership transfers to the caller, whose Destroy path is now responsible for the final Shutdown.

On miss: returns (nil, nil, ErrNoLoadedSlot), same as Acquire.

Edge case — slot row exists but handle does not: the SQLite row says 'loaded' but the in-memory map has no entry. This happens after a daemon restart (the firecracker processes that were warm before the restart are orphans on the host, and the rows still say 'loaded' because the daemon never got to mark them released). The Acquire transaction succeeds at the row level but the handle is missing. We treat this as a miss + mark the row released so the GC sweep drops it (and the operator's reaper will clean up the orphan firecracker process via the next daemon-restart scan). Returns ErrNoLoadedSlot.

func (*Pool) Close 

func (p *Pool) Close(ctx context.Context, grace time.Duration) int

Close shuts down every still-registered warm VMM. Called by the daemon on graceful shutdown so the host doesn't get left with orphan firecracker processes. Best-effort: a per-handle Shutdown failure is logged and the loop continues — partial cleanup beats no cleanup. The grace duration is shared across all handles; pass something like 5s in production.

Returns the number of handles drained. The SQLite rows are not touched here — Release-then-Reap is the row lifecycle, and the next daemon start will release any leftover warm rows before the refill loop runs.

func (*Pool) DepthFor 

func (p *Pool) DepthFor(templateID string) int

DepthFor returns the configured warm-slot depth for templateID: per-template override if set, otherwise the default. Used by PR 4-B's refill tick to compute the spawn budget for this iteration.

func (*Pool) Get 

func (p *Pool) Get(ctx context.Context, sandboxID string) (*Slot, error)

Get returns the slot currently claimed by sandboxID, or nil if it has none. PR 4-B's destroy path uses this to find the VMM process whose Shutdown needs to be issued.

func (*Pool) ListNonReleased 

func (p *Pool) ListNonReleased(ctx context.Context, templateID string) ([]Slot, error)

ListNonReleased returns every slot for templateID that is in 'spawning', 'loaded', or 'allocated'. PR 4-B's refill tick uses this to compute the spawn budget: desired = DepthFor(templateID), current = len(ListNonReleased(templateID)), spawn the delta.

func (*Pool) ReapReleased 

func (p *Pool) ReapReleased(ctx context.Context, cutoff time.Time) (int, error)

ReapReleased deletes 'released' rows whose released_at is older than cutoff. Returns the number of rows actually deleted. PR 4-B's GC sweep calls this once it has confirmed the underlying firecracker process for each released row is gone — the order matters: row deletion is the LAST step, never the first, so a crash mid-sweep leaves the row pointing at the dead process for the next pass to find and clean up.

Per-row handle drain: when the in-memory map still has a handle for a row being reaped, it means the slot was released without ever being Acquire'd (e.g. a synthetic test, a future drain path, or a partial cleanup that released the row before the process stopped). Shutdown must succeed before delete so the row remains the durable breadcrumb for a retry if teardown fails.

func (*Pool) RecordFailed 

func (p *Pool) RecordFailed(ctx context.Context, slotID, errMsg string, now time.Time) error

RecordFailed is the spawner-failure entry point: the spawner could not produce a usable loaded VMM (process refused to boot, snapshot integrity mismatch, etc.). Drops the row straight to 'released' so the GC sweep cleans it up after the TTL, with errMsg preserved for operator triage.

func (*Pool) RecordLoaded 

func (p *Pool) RecordLoaded(ctx context.Context, slotID, apiSocket, runDir string, vsockCID uint32, now time.Time) error

RecordLoaded is the spawner-success entry point: the spawner has produced a paused, snapshot-loaded firecracker process whose API socket lives at apiSocket and whose chroot/runDir is runDir, and the guest snapshot bakes vsockCID. Promotes the row to 'loaded' so subsequent Acquire calls can pick it up.

func (*Pool) RecordSpawning 

func (p *Pool) RecordSpawning(ctx context.Context, slotID, templateID string, now time.Time) error

RecordSpawning is the refill-loop entry point: reserve a new row in 'spawning' for the given (slotID, templateID) so the spawner can proceed without racing a parallel refill tick on the same row id. Exposed in PR 4-A so PR 4-A's tests can drive the state machine, and so PR 4-B's refill goroutine has a typed boundary rather than reaching into the store layer.

slotID generation is the caller's job — PR 4-B's refill loop uses uuid-style "vmms-<16 hex>" mirroring generateTemplateID.

func (*Pool) Release 

func (p *Pool) Release(ctx context.Context, sandboxID string, now time.Time) error

Release moves the sandbox's slot to 'released' so the GC sweep (PR 4-B) tears the VMM process down. Idempotent in both the "sandbox never claimed a slot" and "slot already released" senses — the store-layer WHERE filters cover both cases.

now is injected for the same reason as Acquire.

func (*Pool) Run 

func (p *Pool) Run(ctx context.Context, cfg RefillConfig, lister TemplateLister, spawner Spawner)

Run starts the refill + GC loops. It blocks until ctx is cancelled, then returns. The Spawner and TemplateLister seams are caller- provided so production wires them to the firecracker adapter and the template store; tests inject fakes.

Returns immediately (without launching anything) when Spawner or Lister is nil — that's the "pool disabled" config in main.go: when SB_FIRECRACKER_VMM_POOL_ENABLED=false, main.go passes nil and Run is a no-op rather than panicking.

func (*Pool) SetDefaultDepth 

func (p *Pool) SetDefaultDepth(depth int)

SetDefaultDepth is the boot-time hook for the daemon-wide depth floor. PR 4-B's refill goroutine reads DepthFor(templateID) on every tick; when no per-template override is set the default is what it gets. Zero is allowed and means "do not warm this template" — makes the per-template opt-in trivial.

func (*Pool) SetDepth 

func (p *Pool) SetDepth(templateID string, depth int)

SetDepth records a per-template depth override. Idempotent: a repeat SetDepth with the same depth is a no-op. Passing depth=0 explicitly is the "stop warming this template" signal; the refill loop in PR 4-B will let existing loaded slots drain and not refill.

func (*Pool) Stats 

func (p *Pool) Stats(ctx context.Context, templateID string) (Stats, error)

Stats returns the per-template breakdown by status. Cheap — a single GROUP BY scan covered by idx_firecracker_vmm_pool_template_status.

type RefillConfig 

type RefillConfig struct {
	RefillInterval time.Duration
	GCInterval     time.Duration
	GCTTL          time.Duration
	SpawnTimeout   time.Duration
}

RefillConfig is the timing knobs the refill loop reads on construction. Both fields must be positive; Run panics with a clear message rather than silently picking a default — the daemon already does this validation at boot in main.go, and reaching this code with zero values would be a wiring bug worth surfacing.

SpawnTimeout bounds an individual Spawner.Spawn call. Cold-spawn + LoadSnapshot on a healthy host should land well under 1s; the default 30s in main.go is the operator-side ceiling that catches a hung firecracker without wedging the loop.

type Slot 

type Slot struct {
	ID          string
	TemplateID  string
	Status      Status
	SandboxID   string
	APISocket   string
	RunDir      string
	VsockCID    uint32
	CreatedAt   time.Time
	LoadedAt    time.Time
	AllocatedAt time.Time
	ReleasedAt  time.Time
	LastError   string
}

Slot is the policy-level view of one warm-VMM pool entry. Mirrors store.FirecrackerVMMSlot but lives in this package so the runtime driver and the future metrics exporter depend on a policy type, not on the store's row shape. The split is identical to the TAP pool's Slot / store.FirecrackerTapSlot pair.

SandboxID is non-empty only when Status == StatusAllocated. APISocket and RunDir are non-empty from StatusLoaded onward. The zero time on LoadedAt / AllocatedAt / ReleasedAt means "transition not yet happened" (the store layer maps SQL NULL to time.Time{} for us).

type SnapshotInputs 

type SnapshotInputs struct {
	TemplateID         string
	SnapshotMemoryPath string
	SnapshotStatePath  string
	SnapshotChecksum   string
	VsockCID           uint32
	HasOverlay         bool
}

SnapshotInputs is the per-template artifact reference the spawner needs to issue LoadSnapshot against. Mirrors the fields configureVMMForLoad in internal/runtime/firecracker/driver.go reads from a TemplateResolution; declared locally so this package keeps its zero-imports-from-runtime contract.

VsockCID is the GUEST CID baked into the snapshot state at template build time. The pool's Slot row will pick it up via RecordLoaded so the runtime driver dials the right CID when it claims the slot for a sandbox.

type SpawnedHandle 

type SpawnedHandle interface {
	APISocket() string
	RunDir() string
	Shutdown(ctx context.Context, grace time.Duration) error
	// Pid returns the host PID of the spawned firecracker process, or
	// 0 if the supervisor doesn't track one (test fakes, or a handle
	// whose process has exited). Phase 5 (PR 5-C) consumes this on the
	// Acquire side so the runtime can re-key the RSS sampler from
	// slotID to sandboxID without crossing the import-cycle wall to
	// inspect the underlying *firecracker.vmm directly.
	Pid() int
}

SpawnedHandle is the subset of a *firecracker.vmm the pool consumes after a successful Spawn. Interface form lets the refill goroutine hold the handle without dragging the firecracker package into this one. PR 4-B's runtime adapter satisfies it with the same VMMHandle shape internal/runtime/firecracker/seams.go already declares.

Shutdown is the only mutation the pool ever issues directly: when a slot moves from 'loaded' to 'released' (because the GC ages it out, or because the spawner attempted a refill that the daemon is now backing out of), the pool calls Shutdown to bring the VMM process down before deleting the row. Acquire's caller (PR 4-B's Driver.Create change) is responsible for the post-Resume lifecycle — the pool never resumes a slot itself.

type Spawner 

type Spawner interface {
	Spawn(ctx context.Context, slotID string, inputs SnapshotInputs) (SpawnedHandle, error)
}

Spawner is the seam the pool depends on for producing loaded-but-paused firecracker processes. PR 4-A ships the interface only; the concrete impl lands in PR 4-B's runtime adapter.

Spawn semantics:

  • On success: the returned handle's firecracker process MUST be in the 'Paused' state, with the snapshot at inputs.SnapshotMemoryPath / inputs.SnapshotStatePath loaded and EnableDiffSnapshots=true. The pool will subsequently call RecordLoaded with the handle's APISocket / RunDir / inputs.VsockCID.

  • On error: the spawner MUST leave no leaked process. The pool's caller then issues RecordFailed and the row goes to 'released' for the GC sweep to clean up the (now-empty) on-disk runDir.

  • ctx cancellation is the timeout mechanism. The refill goroutine enforces a per-spawn deadline; the spawner must honor it and return ctx.Err() rather than spinning.

type Stats 

type Stats struct {
	Total     int
	Spawning  int
	Loaded    int
	Allocated int
	Released  int
}

Stats is the per-template occupancy snapshot Pool.Stats returns. The fields mirror store.FirecrackerVMMPoolStats by name; re-declared in this package for the same store-decoupling reason as Slot.

type Status 

type Status string

Status is the typed wrapper over the store's bare-string status column. Local to this package so callers depend on the typed constants rather than free-form strings; the store layer's bare constants stay because they're directly used in SQL literals there. 

const (
	StatusSpawning  Status = Status(store.FirecrackerVMMSlotStatusSpawning)
	StatusLoaded    Status = Status(store.FirecrackerVMMSlotStatusLoaded)
	StatusAllocated Status = Status(store.FirecrackerVMMSlotStatusAllocated)
	StatusReleased  Status = Status(store.FirecrackerVMMSlotStatusReleased)
)

type TemplateLister 

type TemplateLister interface {
	// ListWarmableTemplates returns the active templates the refill
	// loop should consider on this tick. Order is unimportant; the
	// loop walks the slice and uses Pool.DepthFor(templateID) to
	// decide which templates to warm.
	ListWarmableTemplates(ctx context.Context) ([]TemplateWarmInput, error)
}

TemplateLister is the seam the refill loop uses to discover which templates are eligible for warming and where their snapshot artifacts live on disk. Declared as an interface so:

  • The vmm package keeps its "imports only store + stdlib" discipline. main.go injects an adapter that calls into the template service or store.

  • Tests inject a fake returning a fixed list, so the refill loop is exercisable on darwin without a real template pipeline.

Implementations MUST return only templates whose snapshot is ready and whose artifact paths exist on disk. A template in 'pending' or 'failed' should be filtered out before the loop sees it — the Spawner would just fail on it, and we'd rather not waste a tick.

type TemplateWarmInput 

type TemplateWarmInput struct {
	TemplateID         string
	SnapshotMemoryPath string
	SnapshotStatePath  string
	SnapshotChecksum   string
	VsockCID           uint32
	HasOverlay         bool
}

TemplateWarmInput is the per-template artifact the refill loop hands to the Spawner. Lifted from store.Template's snapshot fields so this package stays at arm's length from the template store row shape.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.