Documentation
¶
Overview ¶
Package devstack owns forge's parallel-dev-stack primitives: the raw git facts that distinguish one working tree from another, and a memoized port-block allocator. Together they let N dev stacks (one per git worktree) run in parallel against shared clusters without colliding — DECLARATIVELY: forge supplies the facts + the allocator, KCL composes them.
There is deliberately NO "instance" abstraction and NO user-visible index. forge exposes two raw git facts as KCL options and lets the KCL author decide which to key on:
option("worktree") -> the LINKED-worktree directory basename, or "" on
the PRIMARY checkout (any branch).
option("branch") -> the current git branch, sanitized DNS-safe, always.
and one resolved builtin that hides the port arithmetic entirely:
forge.allocate_port(base, key) -> base + block(key)*100
where block(key) is a small int forge assigns the first time it sees key and persists (see blocks.go). The default — primary checkout, key "" — renders byte-identically to a stack with no dev-stack parameterization.
Index ¶
- func ActiveDArgs() []string
- func AllocateBlock(projectDir, key string) (int, error)
- func AllocatePort(projectDir string, base int, key string) (int, error)
- func AllocatePortAvoidingForeign(projectDir string, base int, key string, isFree func(port int) bool) (int, error)
- func Branch(projectDir string) string
- func FirstAllocationNotice(key string, block int) string
- func ListStacks(projectDir string) ([]string, error)
- func LookupBlock(projectDir, key string) (block int, found bool, err error)
- func MaxStacksLimit() int
- func ReleaseKey(projectDir, key string) (int, error)
- func RepoAnchor(projectDir string) string
- func Sanitize(s string) string
- func SetActive(o Options)
- func SetMaxStacks(n int)
- func SharedProjectDir(projectDir string) string
- func WithFileLock(p string, fn func() error) error
- func Worktree(projectDir string) string
- type Block
- type Options
- type PrunedBlock
- type UnattributedBlock
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ActiveDArgs ¶
func ActiveDArgs() []string
ActiveDArgs is the convenience the render paths call: the `-D` bindings to push the active git facts into KCL (nil for the default stack).
func AllocateBlock ¶
AllocateBlock returns the stable block index for key, assigning the next free one (≥1) on first use. key == "" is block 0 (the default stack) unless a previous availability-aware allocation recorded otherwise (see AllocatePortAvoidingForeign), which is why the registry is consulted for it rather than short-circuited. Atomic under the registry lock.
A NEW block past the armed ceiling (see SetMaxStacks / checkCeiling) is refused. An EXISTING entry — one already in the registry, at whatever index it was assigned — always resolves, ceiling or not: the ceiling bounds what forge is willing to HAND OUT, never what it is willing to return for a block someone already holds.
func AllocatePort ¶
AllocatePort is the engine behind the forge.allocate_port(base, key) KCL builtin. It returns base + block(key)*100, where block(key) is the small integer forge assigns the FIRST time it sees key and MEMOIZES in the lock-guarded registry. The block index is INTERNAL — it never surfaces in KCL; KCL only ever sees the final port.
Semantics (the contract):
- key == "" ⇒ block 0 ⇒ returns base UNCHANGED, with no registry/lock touch (the byte-identical default-stack path).
- One block PER KEY: every allocate_port(*, key) call for the same key shares that key's block, so all of a stack's ports shift by the SAME offset.
- DETERMINISTIC: base + block*100, NO availability stepping. A port that must equal an externally-fixed value (a k3d pre-mapped host port; the host reliant's LISTEN port) must never step off a held port, so up and deploy — and the external mapping — always agree.
The registry read-modify-write happens entirely under the file lock, so a concurrent first-`up` of two worktrees cannot race two keys to the same block. Persistence makes the block stable across runs AND identical under both `forge env up` and `forge env deploy` (both call this through the same builtin), which is the permanent up-vs-deploy port fix.
func AllocatePortAvoidingForeign ¶ added in v0.1.45
func AllocatePortAvoidingForeign(projectDir string, base int, key string, isFree func(port int) bool) (int, error)
AllocatePortAvoidingForeign is AllocatePort with one addition: on the FIRST run of a given (base, key) it steps past a port some OTHER project is already holding, then memoizes the answer like any other block.
It applies to the DEFAULT key ONLY — see the named-key guard in the body for why probing a dev stack's port is actively wrong.
Why this exists. The block registry is per-repo, so the default key "" is block 0 in EVERY repo — and a second project on the same machine therefore asks for the identical base port. For the dev IdP that is fatal rather than inconvenient: forge (correctly) refuses to adopt an identity provider it did not start, because registering this project's application against another stack's IdP would succeed and mint tokens the wrong issuer signed. So the second project simply could not bring up sign-in.
Determinism is preserved where it actually matters. The reason this port cannot float is that the issuer bakes it into every token's `iss` claim and into a registered redirect URI, so it must not move BETWEEN RUNS — which is a constraint on the value AFTER it is chosen, not on how it is chosen the first time. Once assigned, the block is in the registry and every later run returns it unchanged even if the port is momentarily busy.
isFree is injected so the decision is testable without binding real ports.
func Branch ¶
Branch returns the current git branch for projectDir, sanitized DNS-safe, or "" outside a repo or on a detached HEAD. Unlike Worktree, Branch is reported for the primary checkout too — a consumer that WANTS to key on branch (e.g. a stack-per-branch workflow) can; one that wants the primary checkout to stay default keys on Worktree instead. The author chooses.
func FirstAllocationNotice ¶ added in v0.1.45
FirstAllocationNotice is the one-line diagnostic forge should print the FIRST time a key is assigned a port block — the moment AllocateBlock actually mutates .forge/blocks.json rather than just reading an existing entry.
Why this needs to exist at all: a caller like `forge env render` looks like a pure query, but rendering a new env for the first time allocates a REAL block through this same path — that silent write is exactly how the key "prod" appeared in control-plane's registry with nobody aware a write had happened. The write itself is legitimate and must stay (render has to produce the ports `up` will really use), so this only makes it visible, it does not gate it.
Call it at the exact point AllocateBlock decides a key has NO existing entry and is about to assign it the next free block — i.e. right where `block = nextFreeBlock(reg)` runs, before `writeRegistry` — with:
fmt.Fprintln(os.Stderr, FirstAllocationNotice(key, block))
func ListStacks ¶ added in v0.1.45
ListStacks returns just the DEV-STACK keys — the registered git worktrees — sorted by block index. This is the roster a per-stack config generator enumerates to emit one config block per running stack.
The DEFAULT stack (the primary checkout, key "") is NOT included: it is implicit block 0, never stored, and a generator always emits its config unconditionally. So this is strictly the named worktree stacks layered on top, and an empty result is the ordinary primary-checkout-only case.
func LookupBlock ¶ added in v0.1.45
LookupBlock reports the block already registered for key, and whether one is, WITHOUT claiming anything: no lock is taken (taking it creates .forge/blocks.lock), no entry is added, no label is reconciled. It is the engine behind a READ-ONLY render's allocate_port — `forge env render`, `forge env config`, `forge env status`, `forge env deploy --dry-run` — which must describe the stack a key already has, and must never be the thing that hands a new one out.
Reading without the lock is safe because writeRegistry replaces the file by atomic rename: a reader sees the whole old registry or the whole new one, never half of either.
The key is validated exactly as an allocation would validate it, so a malformed key is reported by the render that composed it rather than surfacing later at `forge env up`.
The default key "" is implicitly block 0 and is reported found=false until an availability-aware allocation recorded it; the caller resolves it to base, which is what AllocateBlock returns for it too.
func MaxStacksLimit ¶ added in v0.1.45
func MaxStacksLimit() int
MaxStacksLimit returns the armed ceiling (0 meaning unbounded/unarmed). A project's cluster port pre-map (e.g. k3d.yaml's hand-mapped host-port range) should be derived from this SAME number rather than hand- maintained separately — the duplication between an unbounded allocator and a hand-maintained pre-map comment is the root cause this ceiling exists to close (see DevStackConfig in internal/config).
func ReleaseKey ¶ added in v0.1.45
ReleaseKey reclaims the block held by exactly the named key, on the caller's explicit assertion that it is no longer in use. It is the manual counterpart to Prune: Prune acts on evidence forge recorded itself and will never touch an entry it cannot prove is dead, while this acts on a human's assertion about an entry forge has no evidence for.
Keeping the two separate is the point. Folding "release what I assert" into Prune would make a command people run to SEE what accumulated capable of moving a live stack's ports, which is the one mistake this whole area must not make. Here the caller names the key, so nothing is inferred and nothing is swept up alongside it.
The default key "" is refused: block 0 is the primary checkout's implicit block and releasing it is meaningless. A key with no entry is refused too, rather than silently succeeding, so a typo does not read as success.
Returns the released block index.
func RepoAnchor ¶ added in v0.1.45
RepoAnchor returns the directory machine-local, developer-scoped state lives under: the PRIMARY checkout's working tree, shared by every linked worktree of the same repo. Outside a git repo it returns projectDir unchanged, so a non-repo scaffold keeps that state local exactly as before.
Two consumers share this anchor, for the same underlying reason — both hold state that is gitignored (so `git worktree add` never materializes it) but machine-scoped rather than checkout-scoped:
- the port-block registry (blocks.go), because a block is a claim against a machine-wide resource; see the argument below.
- the dev secret store (internal/secrets), because a fresh worktree would otherwise start with NO values and fail every declared secret_ref, even though the developer set them all in the primary checkout an hour earlier.
WHY THE REGISTRY CANNOT BE PER-WORKTREE. A block is only meaningful as a claim against a machine-wide resource: block N means "host ports base+N*100 on this machine", pre-mapped once in the cluster's port map. A per-directory registry cannot express that claim, and it fails in both directions at once:
- Every fresh worktree starts with an EMPTY registry, so nextFreeBlock hands it block 1 — the block another worktree is already running on. Two stacks then render the identical host ports and the second one silently loses the bind, which is the exact collision allocate_port exists to prevent.
- The ceiling counts entries in that empty registry, so it also refuses legitimate NEW blocks while reporting the machine is "full". That is how an 8-stack ceiling refused the FIRST worktree on this machine: the registry it counted had zero entries in it.
Anchoring on the primary checkout makes the registry what the ceiling and the pre-map already assume it is — one roster per repo, per machine.
The anchor is git's own `--git-common-dir` (the primary's .git), whose PARENT is the primary working tree. That is the same authoritative distinction Worktree() uses, so the two can never disagree about which checkout is primary.
func Sanitize ¶
Sanitize reduces s to a DNS-safe label of bounded length: canonicalLabel plus the maxNameLen truncation. This is the form a git FACT takes, and the truncation is load-bearing there — see maxNameLen.
Note Sanitize is NOT usable as a well-formedness test, because truncation makes it disagree with a perfectly well-formed input purely by length. Callers testing shape must use canonicalLabel.
func SetActive ¶
func SetActive(o Options)
SetActive records the git facts for this process's subsequent renders. Call once, before the first render, on the up/deploy path only.
func SetMaxStacks ¶ added in v0.1.45
func SetMaxStacks(n int)
SetMaxStacks arms the ceiling for this process's subsequent NEW block allocations. n <= 0 disarms it (unbounded). Call once, before the first allocation, on the up/deploy path only — the same discipline SetActive already requires of its callers.
func SharedProjectDir ¶ added in v0.1.45
SharedProjectDir returns projectDir's counterpart in the repo's PRIMARY checkout, as an absolute path: the directory a machine-shared resource is driven from, whichever worktree forge runs in.
It is RepoAnchor plus projectDir's offset within its own working tree, so a forge project nested in a monorepo (`<repo>/services/api`) maps to `<primary>/services/api`, not to the primary's repo root. Outside a repo, or where the anchor falls back to projectDir (a bare repo), projectDir is its own shared directory.
Shared compose infrastructure (OnCompose.shared) and the files it mounts (fp.write_file(..., shared=True)) resolve here. Driving one compose project from N directories that share a basename is what made a linked worktree recreate every shared container: compose resolves relative bind mounts against the directory it runs from, so each checkout produced a different config for the SAME project.
func WithFileLock ¶ added in v0.1.46
WithFileLock serializes a registry transaction across processes. Readers of atomically replaced registries need no lock; every read-modify-write does.
func Worktree ¶
Worktree returns the LINKED-worktree directory basename for projectDir, or "" on the PRIMARY checkout (regardless of branch) and outside any git repo. This is the fact a consumer keys on when it wants the primary checkout to stay the DEFAULT stack on every branch — the everyday dev loop is then byte-identical to today, and only a `git worktree add`'ed checkout gets its own stack.
The parallelism unit here is the WORKTREE, never the branch: branches change constantly and the primary checkout must always render default.
Detection: a linked worktree's per-worktree git dir (`git rev-parse --absolute-git-dir` → …/.git/worktrees/<name>) differs from the repo's common dir (`--git-common-dir` → the primary's …/.git). The primary checkout has them equal. This is git's own authoritative distinction — far more robust than sniffing whether `.git` is a file vs a directory (which submodules and some tooling also make a file). The returned value is sanitized DNS-safe.
Types ¶
type Block ¶
type Block struct {
Key string
Index int
// Stack marks a DEV-STACK key (a registered git worktree) as opposed to
// a plain port-block key. See entry for why the two must not be
// confused.
Stack bool
// Origin is the worktree this key was DERIVED FROM ("prod-cp-obs" from
// worktree "cp-obs"), or "" when it is tied to no worktree and so can
// never be reclaimed. See entry.Origin.
Origin string
}
Block is one registry entry, for diagnostics.
func List ¶
List returns the WHOLE block registry sorted by block index — every key, both dev stacks and plain port-block allocations. This is the diagnostic view ("what has consumed a port block on this machine?").
A per-stack CONFIG GENERATOR must use ListStacks instead. Enumerating this one and treating each key as a running stack is precisely the bug that put a dev NATS account for a prod web port into version control.
func (Block) DisplayKey ¶ added in v0.1.45
DisplayKey is Key with the default stack's empty string rendered as something a reader can see.
type Options ¶
type Options struct {
Worktree string // option("worktree"): linked-worktree basename, "" on primary
Branch string // option("branch"): sanitized current branch, always
}
Options are the raw git facts pushed INTO KCL as options for this forge command. The zero value (both fields "") is the default stack — it emits NO -D args, so a plain render sees option("worktree") == None (KCL default "") and option("branch") == None, rendering byte-identically to a stack with no dev-stack parameterization.
func Active ¶
func Active() Options
Active returns the options set by SetActive (zero value when unset).
func Resolve ¶
Resolve gathers the git facts for projectDir. Cheap and side-effect-free (no files, no lock): the durable state lives in the block registry, which allocate_port touches lazily. Outside a repo both facts are "".
func (Options) DArgs ¶
DArgs returns the `-D key=value` KCL option bindings that push these git facts into the render — the extension of the existing namespace/image_tag option seam. An empty fact is OMITTED (not emitted as ""), so the default stack returns NO args and renders byte-identically. Values are QUOTED KCL string literals so an all-digit worktree/branch name stays str, never an int (the same coercion fix as image_tag).
type PrunedBlock ¶ added in v0.1.45
type PrunedBlock struct {
Key string
Index int
// Worktree is the gone worktree that made this entry reclaimable: the
// key itself for a dev-stack key, or the recorded origin for a key
// derived from one. Never empty — an entry tied to no worktree is never
// reclaimable.
Worktree string
}
PrunedBlock describes one registry entry Prune reclaimed (or, in dry-run mode, would reclaim).
func Prune ¶ added in v0.1.45
func Prune(projectDir string, dryRun bool) ([]PrunedBlock, error)
Prune reclaims registry entries whose WORKTREE no longer exists on disk, and returns what it reclaimed (or, when dryRun is true, what it WOULD reclaim, without writing anything).
An entry is reclaimable when it names a worktree and that worktree is gone. Two kinds of entry name one, and the second is the one this used to miss:
- A DEV-STACK key (Stack == true) — the key IS the worktree name.
- A DERIVED key (Origin != "") — the key was COMPOSED from a worktree, e.g. prod's web port keying on `"prod-" + option("worktree")`, so "prod-cp-obs" belongs to worktree "cp-obs" just as surely as a stack key does. Prune used to skip every non-stack key on the assumption that it was "not tied to any worktree", and for a derived key that assumption is simply false: delete the worktree and the block was held forever. Since the ceiling counts BLOCKS, enough of those make a new stack impossible to allocate at all — three stranded blocks out of eight is what blocked `forge env render prod` in control-plane.
Two things are NEVER touched, unconditionally, regardless of what git reports:
- A STANDALONE port-block key (Stack == false, Origin == "") — e.g. "prod", allocated from the primary checkout where there is no worktree to derive from. It is tied to no worktree, so it can never legitimately look dead, and reclaiming it would silently move a live stack's port. Note the discriminator is the RECORDED origin, never the key's shape: "prod" and "prod-cp-obs" are told apart by what allocation observed, not by how composed the name looks. That is the whole reason Origin is a field (see blocks.go's entry doc) — a standalone key can look every bit as composed as a derived one, and guessing wrong moves a live port.
- The default key "" (block 0). It is implicit and never stored by any normal write path, but Prune checks for it explicitly rather than relying on that invariant holding forever.
Liveness is determined by enumerating real worktrees via `git worktree list --porcelain` and comparing the entry's worktree against the SANITIZED basename of every surviving worktree root — the exact derivation Worktree() uses, so a key that genuinely still backs a live worktree is never mistaken for dead.
If that enumeration fails for ANY reason (git missing, projectDir is not a git checkout, the command errors), Prune reclaims NOTHING and returns the error. A failed enumeration gives no way to tell live from dead, and deleting a block that is actually still live would move a running stack's ports out from under it — the one mistake this command must never make.
The registry mutation happens under the same lock AllocateBlock uses (withLock), so a concurrent `forge env up` claiming a new block cannot race a prune.
type UnattributedBlock ¶ added in v0.1.45
type UnattributedBlock struct {
Key string
Index int
// LikelyLiveOrigin names a LIVE worktree that appears as a segment of
// Key, when one does. It is a strong signal the entry is derived and
// still in use — i.e. exactly what you must NOT release. Empty when no
// live worktree matches, which is the case worth a human's attention.
LikelyLiveOrigin string
}
UnattributedBlock is a block forge cannot classify: a non-stack entry with no recorded origin. It is never reclaimed — it is REPORTED, so a human can decide.
This exists because the conservative migration has a terminal case. An entry written before entry.Origin existed learns its origin from the next render that allocates the key from its own worktree — but if that worktree is ALREADY gone, no such render will ever happen, and the block is stranded not just until the next run but permanently. That is exactly the state control-plane was in: three pre-origin entries ("prod-cp204-fix-st", "prod-cp-litellm", "prod-cp-obs") whose worktrees had been deleted long before forge recorded origins at all.
Inferring their origin from their names is the thing the origin field exists to avoid, and the asymmetry has not changed: guessing wrong moves a live stack's ports. So forge does not guess. It reports what it cannot prove, with whatever evidence it does have, and leaves the decision — via ReleaseKey — to the person who knows which worktrees they deleted.
func Unattributed ¶ added in v0.1.45
func Unattributed(projectDir string) ([]UnattributedBlock, error)
Unattributed reports the blocks forge cannot classify — non-stack entries with no recorded origin — so a human can see what the conservative migration left behind instead of it silently occupying the ceiling forever. It never mutates anything.
A caller that wants to act on one uses ReleaseKey, which requires naming the key explicitly.
Source Files