hostinfra

package
v1.0.13 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 1, 2026 License: Apache-2.0 Imports: 28 Imported by: 0

Documentation

Overview

Package hostinfra runs a project's declared dev infrastructure as HOST PROCESSES, with no container runtime involved.

Why this exists

`forge env up` already runs the project's OWN code on the host — that is the whole point of the host dev loop. But the infrastructure underneath it (postgres, first and foremost) arrived as docker containers, so a dev loop that touched no container of its own still could not start without docker. On the small cloud VMs a lot of people actually develop on, that is a multi-hundred-megabyte resident cost paid before the project has done anything — and on some of them docker is not installable at all. A project whose code needs no container should need no container runtime.

So the DEFAULT flipped: a `forge.HostInfra` declaration in an environment's KCL means forge fetches and supervises the server itself. `forge.Compose` did not go anywhere — a project that wants containerized infra says so, and gets exactly what it used to. The escape hatch stopped being the only door.

Postgres

The postgres engine runs a REAL postgres binary, downloaded once and cached under the user cache dir, via github.com/fergusstrange/embedded-postgres. forge already depends on that library for generate-time schema introspection (pkg/pgtest), so the dev server and the shadow database used to project the ORM are the same postgres, behaving the same way. Migrations apply verbatim; there is no emulation layer to disagree with production.

What this package learned from pkg/pgtest, and where it deliberately differs

pkg/pgtest boots EPHEMERAL servers for tests and generate runs: it hands out scratch databases, reference-counts users across processes, and tears the server down when the last one leaves. Its hard-won configuration — mmap-backed shared memory (System V shmget exhausts macOS's SHMMNI limit and then every initdb fails), a reused binary cache path, a start timeout generous enough for the first run's archive extraction, and reaping the instances that a SIGKILLed process left holding IPC resources — is exactly as necessary here, and is reproduced below rather than re-learned.

Two things are the OPPOSITE of pgtest's, because a dev database is not a test database:

  • The DATA DIRECTORY PERSISTS, under the project's own .forge/hostinfra/. A developer's rows have to survive stopping the stack, the same way a compose volume's do. (This is also why DataPath is configured explicitly: embedded-postgres RemoveAll's its RuntimePath on every Start, and when DataPath is unset the data directory lives INSIDE RuntimePath — so an unconfigured instance silently wipes the database on every boot.)
  • The SERVER OUTLIVES THE PROCESS THAT STARTED IT. `forge env up` returns while the stack keeps serving, exactly as a `docker compose up -d` postgres does. There is no reference counting and no teardown-on-exit: the server is stopped explicitly, by `forge env down`.

Both differences point the same way — this is infrastructure with a lifecycle a human manages, not a fixture with a lifecycle a test manages.

Index

Constants

View Source
const DefaultDataDirRel = ".forge/hostinfra"

DefaultDataDirRel is where a HostInfra instance keeps its state when the declaration names no data_dir: under the project's own .forge/, which is already gitignored and already where forge keeps machine-local state (the port store, the dev-stack block registry, the IdP's PAT).

Project-local rather than a shared system directory ON PURPOSE. Two checkouts of the same project — the parallel-worktree case forge's allocate_port exists to serve — are two independent stacks with two independent databases, and a shared path would silently make them one.

View Source
const EngineNATS = "nats"

EngineNATS matches the `engine` discriminator on the KCL HostInfra schema.

View Source
const EnginePostgres = "postgres"

EnginePostgres is the first engine forge supervises natively (the others: zitadel.go, nats.go, temporal.go). It matches the `engine` discriminator on the KCL HostInfra schema.

View Source
const EngineTemporal = "temporal"

EngineTemporal matches the `engine` discriminator on the KCL HostInfra schema.

View Source
const EngineZitadel = "zitadel"

EngineZitadel is the second engine forge supervises natively. It matches the `engine` discriminator on the KCL HostInfra schema.

View Source
const NATSVersion = "v2.15.1"

NATSVersion is the pinned nats-server release.

View Source
const TemporalCLIVersion = "1.9.1"

TemporalCLIVersion is the pinned Temporal CLI release, whose dev server is what this engine runs (CLI 1.9.1 embeds Temporal Server 1.32.0).

View Source
const ZitadelVersion = "v4.19.5"

ZitadelVersion is the pinned Zitadel release.

KEEP THIS IN SYNC with the two other places the same version is pinned, or a dev stack and a deployed one disagree about what an IdP is:

  • internal/templates/project/docker-compose.yml.tmpl — the `idp` service's image tag, which is the containerized escape hatch for a project that opts back out of the host path.
  • control-plane's deploy/kcl/lib/infra.k — the in-cluster Zitadel Deployment.

Pinned MINOR.PATCH, never `latest`: an IdP that silently changes version under a dev stack turns "login broke today" into an archaeology exercise.

Variables

View Source
var ErrUnsupportedEngine = errors.New("hostinfra: unsupported engine")

ErrUnsupportedEngine is returned for an engine forge has no implementation for. The KCL schema check rejects these at render time, so reaching this is a forge bug or a hand-built Spec.

Functions

func PostgresMajor added in v1.0.0

func PostgresMajor(v string) string

PostgresMajor is the major of a declared or reported postgres version: "16" for "16", "16.9.0" or "16.9". A data directory records only its major (PG_VERSION), so that is the unit two versions are compared in.

func Start

func Start(ctx context.Context, projectDir string, spec Spec) error

Start brings the instance up, or confirms it is already up, and returns only once it is actually SERVING — not merely spawned. It is idempotent: running `forge env up` twice in a row does not start a second postgres, and does not fail the second time.

The already-up check dials the port and authenticates rather than looking for a pid file, because those answer different questions. A pid file says a process was started; a successful connection says the thing the app is about to dial will answer it. On a shared dev machine the port may also be held by something that is NOT this instance — another project's postgres, or a system one — and that case has to be reported, not adopted: a stack that silently attaches to the wrong database is the failure this whole path exists to make impossible (see internal/devpg for the incident).

func Stop

func Stop(projectDir string, spec Spec) error

Stop shuts the instance down cleanly and leaves the DATA in place.

Clean matters beyond tidiness: an orderly `pg_ctl stop` releases the SysV semaphore sets and the shared-memory interlock segment, while a SIGKILL leaks both. Leak enough of them and no postgres starts on this machine again until they are reclaimed by hand.

The data directory SURVIVES. `forge env down` is "stop the stack", not "throw away my rows" — the compose story it replaces kept its volume too. Deleting the data directory is the (explicit, manual) way to start clean.

func StopReport

func StopReport(projectDir string, spec Spec) (stopped bool, err error)

StopReport is Stop, and also reports whether a server was actually running — so `forge env down` can say what it stopped instead of claiming to have stopped something that was already down.

The server is located by its DATA DIRECTORY, not by the declared port. An instance running on a port the declaration has since moved off is still this instance, and still the one that must be stopped; finding it by port would leave it running and report success.

func ZitadelBinaryPath

func ZitadelBinaryPath() string

ZitadelBinaryPath is where forge caches the extracted zitadel binary, shared by every project on this machine.

The layout is deliberately parallel to the postgres cache (<UserCacheDir>/forge/embedded-postgres): one forge-owned directory per third-party server, VERSIONED, so two projects on different pins coexist and a bump never has to invalidate anything by hand.

<UserCacheDir>/forge/zitadel/v4.19.5/<goos>-<goarch>/zitadel

This path is a CONTRACT, not an implementation detail: an image that pre-caches the binary (control-plane's workspace image does) writes to exactly this path so a container start does not re-download 50 MB. Moving it silently would not break correctness — forge would just download again — but it would quietly undo that optimisation, so treat it as public.

Returns "" when the user cache dir cannot be determined, which the caller reports rather than guessing at a fallback location.

Types

type Spec

type Spec struct {
	// Name is the workload name (e.g. "postgres"). It names the instance in
	// logs and, by default, its data directory.
	Name string
	// Engine selects the implementation: EnginePostgres, EngineZitadel,
	// EngineNATS or EngineTemporal.
	Engine string
	// Port is the loopback port the server binds — the same number the app's
	// DSN dials, which is why the environment declares it once and hands it
	// to both.
	Port int
	// Database is created on first boot; User/Password are the credentials
	// the DSN carries.
	Database string
	User     string
	Password string
	// DataDir is where the cluster lives. Relative paths resolve against the
	// project root; empty means DefaultDataDirRel/<Name>.
	DataDir string
	// Version pins the engine major version. Empty means forge's default.
	Version string

	// IDPDatabase is the database Zitadel creates its schema in, on the
	// postgres server IDPDatabasePort names. Its OWN database — the
	// container path spent a second postgres container on the same
	// isolation.
	IDPDatabase string
	// IDPDatabasePort is where that postgres server listens. It is the
	// APP's host-native postgres, so this is the same number the app's DSN
	// carries, handed over from the one place the environment declares it.
	IDPDatabasePort int
	// IDPMasterKey is Zitadel's encryption key for the credentials it
	// stores. Exactly 32 characters, per Zitadel.
	IDPMasterKey string
	// IDPStepsFile is the declarative bootstrap passed as `--steps`
	// (idp-steps.yaml). Relative to the project root; this file IS the
	// instance's setup.
	IDPStepsFile string
	// IDPPATPath is where Zitadel writes the service-account personal
	// access token on first boot. It must match what the identity step of `forge env up`
	// reads — the two are one declaration.
	IDPPATPath string

	// NATSMonitorPort is the HTTP monitoring port (`--http_port`). Zero
	// means no monitoring endpoint.
	NATSMonitorPort int

	// TemporalNamespaces are created at launch; readiness waits for every
	// one of them. Empty means just Temporal's own "default".
	TemporalNamespaces []string
	// TemporalUIPort is the Web UI's port. Zero means headless.
	TemporalUIPort int
}

Spec is one declared host-infra instance: the rendered projection of a `forge.HostInfra` deploy block plus the workload name that carries it.

func (Spec) DSN

func (s Spec) DSN() string

DSN is the connection string for this instance's database.

func (Spec) PATPath added in v0.1.46

func (s Spec) PATPath(projectDir string) string

PATPath resolves the service-account token path against the project root. It must match what the identity step of `forge env up` reads (its --pat-file default, .forge/idp/pat.txt) — the two are the same declaration.

type Status

type Status struct {
	// Running reports whether a server this project owns is serving.
	//
	// Ownership is load-bearing and is why this cannot be a port probe.
	// On a shared dev box the declared port may be held by ANOTHER
	// project's postgres or a system one, and reporting that as "our
	// instance is up" is the same adoption mistake Start refuses to
	// make — it would show green for a database this project has never
	// written to.
	Running bool

	// Foreign reports the specific case of "something is listening on
	// the declared port, and it is not ours". Distinct from !Running
	// with nothing there: one means "start it", the other means "a
	// human must free the port", and collapsing them loses the only
	// actionable half.
	Foreign bool

	// Port is where the instance is ACTUALLY serving, which is not
	// necessarily the declared port: an instance started before the
	// declaration moved is still running on the old one. Zero when not
	// running, and -1 when it is alive but has not published a port yet
	// (mid-boot).
	Port int

	// Version is the engine build the instance is running, empty when
	// unknown. For postgres it is read from the cluster's own PG_VERSION
	// file — the server's record of what initdb created, rather than the
	// declaration's record of what was asked for. Those disagree exactly
	// when someone changed the declared version without recreating the
	// data directory, and that disagreement is the thing worth seeing.
	Version string

	// Ready reports whether the instance ANSWERS, as opposed to merely
	// having a live process. The two come apart during boot and stay
	// apart when a server wedges, and that gap is the whole difference
	// between "a process exists" and "the thing the app is about to dial
	// will respond" — the distinction Start's doc-comment already draws
	// for the write path.
	//
	// Meaningless when Running is false.
	Ready bool

	// DataDir is the resolved data directory, so a report can name the
	// thing on disk it is talking about.
	DataDir string
}

Status is a READ of one declared instance: is it running, where, and which build.

It exists as a separate verb rather than a return value of Start because the two answer different questions. Start reports whether a TRANSITION succeeded, and it may create a data directory, download a binary and run a bootstrap to get there. Status never does any of that, and a caller may run it against an instance that was never started — which is exactly what `forge env observe` needs, and exactly what calling Start to find out would get wrong (it would start it).

func StatusOf

func StatusOf(ctx context.Context, projectDir string, spec Spec) (Status, error)

StatusOf reports the current state of one declared instance, without starting, stopping or modifying anything.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL