Documentation
¶
Overview ¶
Package application implements the declarative app spec's service contract and the application controller: the reconcile.Controller that converges a real, store-backed desired service to a running container, replacing nginxdemo's hardcoded desired state with the real thing.
Deploy strategy: a container's name is derived deterministically from its image (ContainerName), so the level-triggered check "does the right container exist" needs no memory of past deploys beyond what's queryable from Docker directly. When the desired image changes, Reconcile creates a new, differently-named container alongside whatever's already running, waits for it to pass its readiness probe, then removes every other container for this service. Traffic switching itself (updating Caddy to point at the new container) is deliberately not this controller's job: this codebase's reconciler pattern is a reconcile loop per resource type, and ingress is its own resource type (not yet wired in). This controller's contract with that future ingress controller is simple: whichever container currently exists and is running for a service is the one meant to receive traffic.
Index ¶
- Constants
- Variables
- func ContainerName(serviceName, image, restartNonce string) string
- func ControllerName(serviceName string) string
- func DatabaseHost(dbName, zone string) string
- func NameImage(svc store.DesiredService) string
- func NetworkName(prefix, appID string) string
- func OwnsContainer(serviceName, name string) bool
- func ParseContainerName(name string) (service string, ok bool)
- func PrimaryAddr(state *docker.ContainerState) (string, error)
- func ReplicaContainerName(serviceName, image, restartNonce string, index int) string
- func SameNode(a, b, localNodeID string) bool
- func ServiceAlias(desired *store.DesiredService) string
- type AppIntegrationStore
- type AppLister
- type AppliedConfigRecorder
- type ConditionReader
- type Controller
- func (c *Controller) BuildJobSpec(ctx context.Context, desired *store.DesiredService, name string, ...) (JobSpec, error)
- func (c *Controller) JobSecretValues(ctx context.Context, desired *store.DesiredService) ([]string, error)
- func (c *Controller) Name() string
- func (c *Controller) Reconcile(ctx context.Context) (reconcile.Result, error)
- func (c *Controller) Teardown(ctx context.Context) error
- type DatabaseAttachmentStore
- type DatabaseMeshPorts
- type DatabaseScopeStore
- type DeleteFinalizer
- type DeployRecorder
- type DeploySafetySource
- type DrainTracker
- type EnvironmentEnvLister
- type ExitRecorder
- type ExternalDatabaseStore
- type GateRecorder
- type HookRunRecorder
- type JobSpec
- type LivenessTracker
- type MeshDatabaseConfig
- type NetworkCleanupController
- type NodeGPUChecker
- type Option
- func WithAppIntegrations(s AppIntegrationStore) Option
- func WithAppliedConfigRecorder(r AppliedConfigRecorder) Option
- func WithConditionReader(r ConditionReader) Option
- func WithDatabaseAttachments(s DatabaseAttachmentStore) Option
- func WithDatabaseScope(s DatabaseScopeStore) Option
- func WithDeployRecorder(r DeployRecorder) Option
- func WithDrainTracker(t *DrainTracker) Option
- func WithEgressReadyBudget(d time.Duration) Option
- func WithEgressReadyPollInterval(d time.Duration) Option
- func WithEnvironmentEnv(s EnvironmentEnvLister) Option
- func WithExitRecorder(r ExitRecorder) Option
- func WithGateRecorder(r GateRecorder) Option
- func WithGateTick(d time.Duration) Option
- func WithHTTPClient(c *http.Client) Option
- func WithHookRunRecorder(r HookRunRecorder) Option
- func WithHookTimeout(d time.Duration) Option
- func WithInstanceID(id string) Option
- func WithLivenessTracker(t *LivenessTracker) Option
- func WithMeshDNSAddr(addr string) Option
- func WithMeshDatabases(cfg MeshDatabaseConfig) Option
- func WithMeshPaths(r meshpath.Resolver) Option
- func WithMeshZone(zone string) Option
- func WithNetworkPrefix(prefix string) Option
- func WithNodeGPU(checker NodeGPUChecker) Option
- func WithOrganizationEnv(s OrganizationEnvStore) Option
- func WithPinnedPortRetry(d time.Duration) Option
- func WithPreviousReleaseHold(d time.Duration) Option
- func WithProbeAttemptRecorder(r ProbeAttemptRecorder) Option
- func WithProbeLimits(l probe.Limits) Option
- func WithProjectEnv(s ProjectEnvStore) Option
- func WithReadyBudget(d time.Duration) Option
- func WithRegistryCredentials(s RegistryCredentialStore) Option
- func WithRestartBackoff(b *RestartBackoff) Option
- func WithRollbackPinner(p RollbackPinner) Option
- func WithRolloutRecorder(r RolloutRecorder) Option
- func WithSecretResolver(r SecretResolver) Option
- func WithStabilityGate(defaults deploysafety.Effective, src DeploySafetySource) Option
- func WithStartupWatch(d time.Duration) Option
- func WithStorageTargets(s StorageTargetStore) Option
- func WithVaultResolver(r VaultResolver) Option
- func WithVaultSettings(s VaultSettingsStore) Option
- type OrganizationEnvStore
- type ProbeAttemptRecorder
- type ProjectEnvStore
- type RegistryCredentialStore
- type RestartBackoff
- type RollbackPinner
- type RolloutRecorder
- type RuntimeResolver
- type SecretResolver
- type ServiceStore
- type StorageTargetStore
- type TeardownStore
- type VaultResolver
- type VaultSettingsStore
Constants ¶
const (
// ExitLogTailLines is how many of an exited container's last lines are kept.
ExitLogTailLines = 200
)
Variables ¶
var ErrDatabaseOutOfScope = errors.New("database is out of network scope for this service")
ErrDatabaseOutOfScope is wrapped when a service references a database whose network scope excludes it.
var StorageEnvKeys = []string{
envKeyS3Endpoint,
envKeyS3Bucket,
envKeyS3Region,
envKeyS3AccessKey,
envKeyS3SecretKey,
}
StorageEnvKeys are every env var name resolveStorageEnv can inject, built from the same named constants that function assigns below so the two can never drift apart. The single source of truth for any caller needing to know these names ahead of time, e.g. internal/api's GET /api/v1/storage-env-keys, which lets the frontend warn an operator before attaching storage collides with one of their own env vars, instead of hardcoding a second copy of this list in TypeScript.
Functions ¶
func ContainerName ¶
ContainerName derives a stable, unique-per-image name so the level-triggered check "does the right container exist" needs no memory beyond what Docker itself can answer. Two deploys of the same image produce the same name (a genuine no-op redeploy correctly finds nothing to do); two different images always produce different names (so both can exist side by side during a cutover). Exported so the ingress controller (internal/reconcile/ingress) can derive the exact same name to find a service's currently active container, without reimplementing this hash logic a second time and risking the two drifting apart.
restartNonce (store.DesiredService.RestartNonce) is folded into the hash only when non-empty: an empty nonce, the state of every service that has never been restarted, must produce byte-identical output to before this parameter existed, so an upgrade never orphans an already-running container under a name it no longer recognizes. Once non-empty, changing it changes the name, which is what makes a restart (RestartService) indistinguishable from an image change to this controller's existing, already-tested blue-green/recreate cutover logic: no new branch needed, restarting an app is just another kind of desired-state change.
func ControllerName ¶ added in v0.2.0
ControllerName is Name's naming convention, usable without a *Controller.
func DatabaseHost ¶ added in v0.2.0
DatabaseHost returns the host name resolveDatabaseField resolves dbName to: its mesh DNS name (<dbName>.<zone>) when zone is non-empty, otherwise its Docker container name (database.ContainerName). Exported so internal/api can preview a connection's resolved host (GET /api/v1/apps/{name}/connections) without duplicating this package's own host-selection logic, the single source of truth resolveDatabaseField itself defers to.
func NameImage ¶ added in v0.2.0
func NameImage(svc store.DesiredService) string
NameImage is the image identity container names hash. A build's local image ID is folded in so rebuilding the same tag with new content gets a new container instead of silently keeping the old one.
func NetworkName ¶
NetworkName derives the per-app Docker network name every service belonging to appID shares: "<prefix>-app-<appID>". Exported so NetworkCleanupController (network_cleanup.go) can derive the exact same name a running app's containers are actually attached to, without reimplementing this naming rule a second time and risking the two drifting apart, the same reasoning ContainerName's own doc comment already gives for internal/reconcile/ingress's identical reuse of that function.
func OwnsContainer ¶ added in v0.2.0
OwnsContainer reports whether name is a container this service could have created for any image or replica index.
func ParseContainerName ¶ added in v0.2.0
ParseContainerName reports the service an app container name belongs to, accepting every shape the controller produces: ContainerName, replica suffixes and egress sidecars. ok is false for any other name.
func PrimaryAddr ¶ added in v0.2.0
func PrimaryAddr(state *docker.ContainerState) (string, error)
PrimaryAddr picks the address a readiness probe should hit: the sole port binding a service with exactly one declared port produces. Exported so internal/api's health-discovery endpoint (apps_health_discover.go) can resolve the same address without reimplementing this lookup.
func ReplicaContainerName ¶ added in v0.2.0
ReplicaContainerName returns the container name of replica index for the given image and restart nonce; index 0 equals ContainerName.
func SameNode ¶ added in v0.2.0
SameNode reports whether two node IDs name the same node, treating "" and the control plane's own node ID as one.
func ServiceAlias ¶ added in v0.2.0
func ServiceAlias(desired *store.DesiredService) string
ServiceAlias is the name sibling containers on desired's per-app network reach it by.
Types ¶
type AppIntegrationStore ¶ added in v0.2.0
type AppIntegrationStore interface {
ListAppIntegrationsForService(ctx context.Context, serviceName string) ([]store.AppIntegration, error)
}
AppIntegrationStore is the narrow surface this controller needs to resolve this service's attached internal/integrations catalog entries (resolveIntegrationEnv) into env vars, one list read. *store.DB satisfies this structurally.
type AppLister ¶
AppLister is the narrow surface NetworkCleanupController needs from internal/store: every currently-desired app. *store.DB satisfies this structurally.
type AppliedConfigRecorder ¶ added in v0.2.0
type AppliedConfigRecorder interface {
SaveAppliedConfig(ctx context.Context, c store.AppliedConfig) error
SetPreviousReleaseHeldUntil(ctx context.Context, name string, until time.Time) error
}
AppliedConfigRecorder persists what a container was created with and when the held previous release is due. *store.DB satisfies this structurally.
type ConditionReader ¶ added in v0.2.0
type ConditionReader interface {
GetConditions(ctx context.Context, controllerName string) ([]reconcile.Condition, error)
}
ConditionReader reads another controller's last stored conditions.
type Controller ¶
type Controller struct {
// contains filtered or unexported fields
}
Controller converges one named service's desired state (read fresh from ServiceStore on every Reconcile, never cached) to a running container.
func New ¶
func New(serviceName string, svcStore ServiceStore, runtime docker.Runtime, opts ...Option) *Controller
New builds a Controller for serviceName.
func (*Controller) BuildJobSpec ¶ added in v0.2.0
func (c *Controller) BuildJobSpec(ctx context.Context, desired *store.DesiredService, name string, command []string, res *docker.Resources) (JobSpec, error)
BuildJobSpec returns the container spec for running command once in a fresh container of desired's image: the same resolved env, volumes, network, DNS, hosts and registry auth the service itself gets, without its published ports, labels or restart behavior. command is the exact argv (the image entrypoint is replaced), the same contract as exec mode.
func (*Controller) JobSecretValues ¶ added in v0.2.0
func (c *Controller) JobSecretValues(ctx context.Context, desired *store.DesiredService) ([]string, error)
JobSecretValues resolves the service's env and returns only the values to redact, for exec-mode runs that build no container spec.
func (*Controller) Teardown ¶ added in v0.2.0
func (c *Controller) Teardown(ctx context.Context) error
Teardown stops and removes every container this controller owns, including any egress sidecar: removeStale's own ownsContainer-based matching never sees a sidecar (egressSidecarSuffix's own doc comment on why), so it needs its own explicit cleanup call here rather than being swept up for free. Callers must call it themselves right after deleting desired state: Reconcile treats ErrServiceNotFound as "not deployed yet," not "stop everything," so a deleted service is never reconciled again otherwise.
type DatabaseAttachmentStore ¶
type DatabaseAttachmentStore interface {
GetDesiredDatabase(ctx context.Context, name string) (*store.DesiredDatabase, error)
}
DatabaseAttachmentStore is the narrow surface this controller needs to resolve a { from: ... } env var (store.DesiredService.DatabaseEnv, store.DesiredService.DatabaseAttachment) against a real managed database: just the one read, the same shape StorageTargetStore/ RegistryCredentialStore already establish. *store.DB satisfies this structurally.
type DatabaseMeshPorts ¶ added in v0.2.0
type DatabaseMeshPorts interface {
EnsureDatabaseMeshPort(ctx context.Context, name string, minPort, maxPort int) (int, error)
}
DatabaseMeshPorts allocates a database's stable mesh port.
type DatabaseScopeStore ¶ added in v0.2.0
type DatabaseScopeStore interface {
GetDatabaseAccessSettings(ctx context.Context, database string) (store.DatabaseAccessSettings, error)
EnvironmentOfDatabase(ctx context.Context, databaseName string) (*store.EnvironmentRef, error)
}
DatabaseScopeStore is the surface the controller needs to honour a database's network scope. *store.DB satisfies it.
type DeleteFinalizer ¶ added in v0.2.0
type DeleteFinalizer struct {
// contains filtered or unexported fields
}
DeleteFinalizer removes the containers of deleted or moved apps. A tombstone is written before desired state is deleted and cleared only after the containers are confirmed gone, so an unreachable node or a failed stop is retried on every reconcile pass instead of orphaning containers.
func NewDeleteFinalizer ¶ added in v0.2.0
func NewDeleteFinalizer(s TeardownStore, resolve RuntimeResolver, logger *slog.Logger, opts ...Option) *DeleteFinalizer
NewDeleteFinalizer builds a DeleteFinalizer. opts are applied to the throwaway Controller used for container matching (instance ID, prefix).
func (*DeleteFinalizer) Begin ¶ added in v0.2.0
func (f *DeleteFinalizer) Begin(ctx context.Context, name, nodeID string) error
Begin writes the tombstone for name. Call before deleting desired state.
func (*DeleteFinalizer) Finalize ¶ added in v0.2.0
func (f *DeleteFinalizer) Finalize(ctx context.Context, name, nodeID string) error
Finalize tears name's containers down on nodeID and clears its tombstone. On failure the tombstone stays and the cause is recorded for retry.
func (*DeleteFinalizer) Name ¶ added in v0.2.0
func (f *DeleteFinalizer) Name() string
Name implements reconcile.Controller.
type DeployRecorder ¶
type DeployRecorder interface {
RecordDeploy(ctx context.Context, serviceName string, at time.Time) error
}
DeployRecorder is the narrow surface this controller needs to record the deploy-frequency metric. *telemetry.DB satisfies this structurally; not imported directly, same reasoning ServiceStore/ SecretResolver above already establish. RecordDeploy is only ever called on a real deploy cutover (justDeployed below, the "Deployed" reason), never on a reconcile tick that finds nothing to do, so a series of recorded samples over time IS deploy frequency, no separate rate computation needed downstream.
type DeploySafetySource ¶ added in v0.2.0
type DeploySafetySource interface {
GetDeploySafetySettings(ctx context.Context, serviceName string) (store.DeploySafetySettings, error)
}
DeploySafetySource returns an app's gate overrides. *store.DB satisfies it.
type DrainTracker ¶ added in v0.2.0
type DrainTracker struct {
// contains filtered or unexported fields
}
DrainTracker remembers each service's last cutover across passes, so the drain is measured from cutover. In memory: a restart falls back to the hold.
func NewDrainTracker ¶ added in v0.2.0
func NewDrainTracker() *DrainTracker
NewDrainTracker builds an empty tracker.
type EnvironmentEnvLister ¶
type EnvironmentEnvLister interface {
ListEnvironmentEnvVars(ctx context.Context, environmentID string) (map[string]string, error)
}
EnvironmentEnvLister is the narrow surface this controller needs to resolve store.DesiredService.EnvironmentID's shared env vars, the tier between ProjectEnvStore and this service's own Env (resolveEnv's own doc comment on precedence). Unlike OrganizationEnvStore, no join is needed: EnvironmentID already lives directly on DesiredService. *store.DB satisfies this structurally.
type ExitRecorder ¶
type ExitRecorder interface {
RecordDeployExit(ctx context.Context, e store.DeployExit) error
}
ExitRecorder persists a release container's exit and its last output. *store.DB satisfies it.
type ExternalDatabaseStore ¶ added in v0.2.0
type ExternalDatabaseStore interface {
GetExternalDatabase(ctx context.Context, name string) (*store.ExternalDatabase, error)
}
ExternalDatabaseStore is the one read needed to resolve a { from: ... } reference to a database this platform does not run. *store.DB satisfies it.
type GateRecorder ¶ added in v0.2.0
type GateRecorder interface {
RecordGateEvent(ctx context.Context, e store.DeployGateEvent) error
MarkRolloutFailureReason(ctx context.Context, serviceName, image, reason string) error
}
GateRecorder persists a release's gate timeline and the reason it failed. *store.DB satisfies it.
type HookRunRecorder ¶
HookRunRecorder is the narrow surface this controller needs to persist a pre/post-deploy hook's outcome (store.ServiceHooks, migrations/0083_service_hook_runs.sql), so GET /api/v1/apps/{name}/hook-runs has something to show. *store.DB satisfies this structurally. nil is valid: a hook still executes and still gates the deploy exactly the same way, its outcome (this pass) is just never persisted for later viewing, the same "optional persistence, mandatory behavior" split DeployRecorder above already makes for the deploy-frequency metric.
type JobSpec ¶ added in v0.2.0
type JobSpec struct {
Spec docker.ContainerSpec
SecretValues []string
}
JobSpec is a one-off container spec derived from a service, plus the resolved values a job's captured output must never contain.
type LivenessTracker ¶ added in v0.2.0
type LivenessTracker struct {
// contains filtered or unexported fields
}
LivenessTracker counts consecutive liveness failures per container, in memory only and never persisted. Persisting it would make the check depend on state carried across an interruption, which this codebase's reconcilers are not allowed to do (every pass re-derives what to do from observed state). The documented tradeoff: restarting the control plane resets every counter, so an already-failing app can take up to one extra threshold's worth of probes before it is restarted.
Keyed by container name, which already carries the service name, so one tracker is safely shared by every service's controller. It has to be: the controller set is rebuilt from desired state on every reconcile pass (cmd/levelrail's dynamicSource), so a controller's own state never survives the pass that created it. See WithLivenessTracker.
func NewLivenessTracker ¶ added in v0.2.0
func NewLivenessTracker() *LivenessTracker
NewLivenessTracker builds an empty tracker. Long-running callers want exactly one, shared across every controller they build.
type MeshDatabaseConfig ¶ added in v0.2.0
type MeshDatabaseConfig struct {
Endpoints meshpath.EndpointResolver
Ports DatabaseMeshPorts
Zone string
LocalNodeID string
MinPort int
MaxPort int
}
MeshDatabaseConfig wires cross-node database connections over the mesh.
type NetworkCleanupController ¶
type NetworkCleanupController struct {
// contains filtered or unexported fields
}
NetworkCleanupController reconciles the whole fleet's per-app Docker networks in a single pass, the same "all of them, not one resource at a time" shape internal/reconcile/ingress's own controller already establishes for an analogous whole-fleet concern (that package's own doc comment explains why: there's no per-network incremental update primitive to converge one at a time against). Every network this codebase creates is created by name (NetworkName's own doc comment) only when a service's Controller actually needs one (createAndStart); nothing ever proactively deletes one, so this controller is the only place that ever does: it diffs Docker's own observed networks under prefix against store.App rows that still exist, and removes any network whose app is gone.
Deliberately store-App-driven, not service-count-driven: a zero-service App (every service under it deleted individually, not via DeleteApp) still keeps its network, since a later deploy reusing that same app name is expected to find it again. Only an App row's own absence, from store.DeleteApp (internal/api's handleDeleteApp deletes it once removing a service leaves the App with no other members) or its cascade, means the network is actually orphaned.
func NewNetworkCleanupController ¶
func NewNetworkCleanupController(apps AppLister, runtime docker.Runtime, prefix, instanceID string) *NetworkCleanupController
NewNetworkCleanupController builds a NetworkCleanupController. prefix is the same value passed to every Controller's WithNetworkPrefix (typically brand.Brand.ShortName): both must agree, or this will never find the networks the per-service controllers actually create. instanceID is store.GetOrCreateInstanceID's result, the same value passed to every Controller's WithInstanceID; empty disables the instance-ownership check below, treating every matching-name network as this instance's own, this package's behavior before cross-instance safety existed.
func (*NetworkCleanupController) ForNode ¶ added in v0.2.0
func (c *NetworkCleanupController) ForNode(nodeID string) *NetworkCleanupController
ForNode names this controller after a remote node so one runs per node without sharing a reconcile status.
func (*NetworkCleanupController) Name ¶
func (c *NetworkCleanupController) Name() string
Name implements reconcile.Controller.
type NodeGPUChecker ¶ added in v0.2.0
NodeGPUChecker reports a node's GPU capability. nodeID "" is the local node.
type Option ¶
type Option func(*Controller)
Option configures optional Controller behavior.
func WithAppIntegrations ¶ added in v0.2.0
func WithAppIntegrations(s AppIntegrationStore) Option
WithAppIntegrations enables resolving this service's attached internal/integrations catalog entries as resolveEnv's own layer, at the same tier as WithProjectEnv/WithOrganizationEnv/WithEnvironmentEnv: applied below the service's own Env, so an operator's own same-named env var always wins over an integration's default. A service with no AppIntegrationStore configured just skips this layer entirely.
func WithAppliedConfigRecorder ¶ added in v0.2.0
func WithAppliedConfigRecorder(r AppliedConfigRecorder) Option
WithAppliedConfigRecorder records each new container's creation-time config, so the API can report pending changes. nil records nothing.
func WithConditionReader ¶ added in v0.2.0
func WithConditionReader(r ConditionReader) Option
WithConditionReader enables depends_on condition: service_healthy. Without it a healthy dependency degrades to start order, never to a deadlock.
func WithDatabaseAttachments ¶
func WithDatabaseAttachments(s DatabaseAttachmentStore) Option
WithDatabaseAttachments enables container creation to resolve store.DesiredService.DatabaseEnv/DatabaseAttachment into real connection env vars, the same "fail loudly if declared but unconfigured" shape WithStorageTargets already establishes: a service declaring a { from: ... } env var with no database store configured fails Reconcile rather than silently starting a container missing the value it needs.
func WithDatabaseScope ¶ added in v0.2.0
func WithDatabaseScope(s DatabaseScopeStore) Option
WithDatabaseScope makes the controller refuse to attach a service's network to a database whose scope excludes it, and detach one that is already attached. Without it every referenced database is attached, as before.
func WithDeployRecorder ¶
func WithDeployRecorder(r DeployRecorder) Option
WithDeployRecorder enables recording the deploy_count metric every time Reconcile actually performs a deploy cutover. Without one configured (the default), Reconcile behaves exactly as before, deploys just aren't measured.
func WithDrainTracker ¶ added in v0.2.0
func WithDrainTracker(t *DrainTracker) Option
WithDrainTracker measures the drain window from the real cutover time.
func WithEgressReadyBudget ¶ added in v0.2.0
WithEgressReadyBudget overrides how long waitEgressReady (egress.go) waits for a freshly created egress sidecar to confirm its enforcement rules are installed before giving up. Defaults to defaultEgressReadyBudget.
func WithEgressReadyPollInterval ¶ added in v0.2.0
WithEgressReadyPollInterval overrides how often waitEgressReady rechecks a not-yet-ready egress sidecar. Defaults to defaultEgressReadyPollInterval.
func WithEnvironmentEnv ¶
func WithEnvironmentEnv(s EnvironmentEnvLister) Option
WithEnvironmentEnv enables resolving store.DesiredService.EnvironmentID's shared env vars as resolveEnv's middle layer, applied above WithProjectEnv/WithOrganizationEnv and below the service's own Env. Same "purely an organizational label" reasoning as WithProjectEnv: a service with an EnvironmentID but no EnvironmentEnvLister configured does not fail Reconcile, environment vars are just silently skipped.
func WithExitRecorder ¶
func WithExitRecorder(r ExitRecorder) Option
WithExitRecorder captures the last output of a release container that exits before it can serve. nil captures nothing.
func WithGateRecorder ¶ added in v0.2.0
func WithGateRecorder(r GateRecorder) Option
WithGateRecorder records gate timeline events. nil records nothing.
func WithGateTick ¶ added in v0.2.0
WithGateTick overrides how often the stability gate re-checks a container; tests use it to avoid real multi-second waits.
func WithHTTPClient ¶
WithHTTPClient overrides the client used for readiness probes. Defaults to http.DefaultClient; tests override this to point at a fake transport rather than making real HTTP calls to a real client.
func WithHookRunRecorder ¶
func WithHookRunRecorder(r HookRunRecorder) Option
WithHookRunRecorder enables persisting every pre/post-deploy hook run's outcome. Without one configured (the default), configured hooks still execute and still gate the deploy exactly the same way; only the persisted-for-later-viewing record is skipped.
func WithHookTimeout ¶
WithHookTimeout overrides how long a single pre/post-deploy hook command may run before it's treated as failed. Defaults to defaultHookTimeout.
func WithInstanceID ¶ added in v0.2.0
WithInstanceID enables staleContainers' instance-ownership check: callers pass store.GetOrCreateInstanceID's result, the same "pass the resolved value in, don't import internal/store here" convention WithNetworkPrefix already establishes for brand.ShortName. Without one configured (the default, empty string), every container matching this service's own name pattern is treated as this service's own, exactly this package's behavior before instance labeling existed: an empty instanceID here isn't a degraded mode, it's what every caller that hasn't been updated for cross-instance safety yet still gets.
func WithLivenessTracker ¶ added in v0.2.0
func WithLivenessTracker(t *LivenessTracker) Option
WithLivenessTracker shares one liveness failure history across every controller a caller builds, which is what makes the failure threshold mean anything: without it each Controller keeps its own, and a caller that rebuilds its controllers every reconcile pass (cmd/levelrail) would never count past a single failure.
func WithMeshDNSAddr ¶
WithMeshDNSAddr points every container this controller creates at addr, a bare nameserver IP (never "ip:port": neither Docker's DNS HostConfig field nor a container's own resolv.conf supports a non-standard nameserver port, confirmed live while building this option, see cmd/levelrail/mesh.go's dockerNameserverPort doc comment), as an additional nameserver ahead of whatever Docker's own resolver would otherwise configure (docker.ContainerSpec.DNS). Without one configured (the default, empty string), Reconcile behaves exactly as before this field existed: no DNS override at all. addr is expected to already be a real, container-reachable address (cmd/levelrail/mesh.go's containerDNSAddr resolves Docker's own bridge gateway IP, only once the mesh DNS server is confirmed bound to port 53); this controller does not validate it.
func WithMeshDatabases ¶ added in v0.2.0
func WithMeshDatabases(cfg MeshDatabaseConfig) Option
WithMeshDatabases lets an app reach a managed database on another node at <db>.<zone>:<mesh port>, the name pinned in /etc/hosts to the database node's mesh address, so it resolves with no DNS server.
func WithMeshPaths ¶ added in v0.2.0
WithMeshPaths lets a service placed on a remote node publish its main port on that node's WireGuard mesh address, so this control plane's ingress can reach it. Without it, remote ports stay on loopback.
func WithMeshZone ¶ added in v0.2.0
WithMeshZone makes resolveDatabaseField resolve a database env var's host to that database's mesh DNS name (<dbName>.<zone>, the same name internal/reconcile/mesh's own refreshDNS already publishes for every database Placement) instead of its Docker container name. Without one configured (the default, empty string), host resolution is unchanged: the container name, reachable only via Docker's own embedded DNS on the database's own node. A zone with no matching WithMeshDNSAddr is still valid (the container's resolv.conf just wouldn't query it), but cmd/levelrail always wires both together from the same live mesh setup, never one without the other.
func WithNetworkPrefix ¶
WithNetworkPrefix sets the Docker network naming prefix (NetworkName's own doc comment) every per-app network this Controller creates uses: callers pass brand.Brand.ShortName, following the same "pass the resolved value in, don't import internal/brand here" convention internal/network.WithShortName already establishes, since brand.ShortName is runtime-configurable and this package (like internal/spec.ReservedLabelPrefix) has no business depending on that config directly. Without one configured (the default, empty string), defaultNetworkPrefix is used instead.
func WithNodeGPU ¶ added in v0.2.0
func WithNodeGPU(checker NodeGPUChecker) Option
WithNodeGPU makes a service that requests resources.gpu refuse to start on a node without a usable GPU.
func WithOrganizationEnv ¶
func WithOrganizationEnv(s OrganizationEnvStore) Option
WithOrganizationEnv enables resolving the organization that owns store.DesiredService.ProjectID's project into that organization's shared env vars, applied as resolveEnv's base layer beneath WithProjectEnv. Same "purely an organizational label" reasoning as WithProjectEnv: a ProjectID with no WithOrganizationEnv configured does not fail Reconcile, organization vars are just silently skipped.
func WithPinnedPortRetry ¶ added in v0.2.0
WithPinnedPortRetry sets how long a failed pinned-port handoff waits before the next attempt. Every attempt stops the serving release, so retrying a broken image each pass would repeat the outage.
func WithPreviousReleaseHold ¶ added in v0.2.0
WithPreviousReleaseHold keeps the previous release's running containers for d after a blue-green or rolling cutover, as an instant rollback target and for routes still draining. 0 (the default) removes them at cutover.
func WithProbeAttemptRecorder ¶ added in v0.2.0
func WithProbeAttemptRecorder(r ProbeAttemptRecorder) Option
WithProbeAttemptRecorder enables persisting every individual readiness-probe attempt waitReady makes during a deploy's cutover. Without one configured (the default, nil), probing behaves exactly as before: attempts are just never persisted. See probe.WithOnAttempt's own doc comment for why this can never affect WaitReady's retry or pass/fail decision, only what gets recorded alongside it.
func WithProbeLimits ¶ added in v0.2.0
WithProbeLimits sets redirect, exec-output and default-timing bounds for readiness and liveness probes. Defaults to probe's own defaults.
func WithProjectEnv ¶
func WithProjectEnv(s ProjectEnvStore) Option
WithProjectEnv enables resolving store.DesiredService.ProjectID's shared env vars as resolveEnv's base layer. Unlike WithSecretResolver/ WithStorageTargets, a service with a ProjectID but no ProjectEnvStore configured does not fail Reconcile: a project is purely an organizational label (internal/api/projects.go's own doc comment), so assigning one must never be able to break an otherwise-healthy service's deploys, project vars are just silently skipped.
func WithReadyBudget ¶
WithReadyBudget overrides how long Reconcile waits for a freshly started container to pass its readiness probe before giving up. Defaults to 60s. A service's own store.ServiceHealth.ReadyTimeout, when set, takes precedence over this construction-time value; see effectiveReadyBudget.
func WithRegistryCredentials ¶
func WithRegistryCredentials(s RegistryCredentialStore) Option
WithRegistryCredentials enables resolving store.DesiredService.RegistryCredentialID into real pull credentials, the same "fail loudly if declared but unconfigured" shape WithStorageTargets establishes: an unauthenticated pull silently succeeding for a service that explicitly asked for a private-registry credential would be a worse failure mode than Reconcile erroring.
func WithRestartBackoff ¶ added in v0.2.0
func WithRestartBackoff(b *RestartBackoff) Option
WithRestartBackoff shares one crash restart backoff across every controller a caller builds, for the same reason as WithLivenessTracker.
func WithRollbackPinner ¶ added in v0.2.0
func WithRollbackPinner(p RollbackPinner) Option
WithRollbackPinner pins every release at cutover. nil pins nothing.
func WithRolloutRecorder ¶ added in v0.2.0
func WithRolloutRecorder(r RolloutRecorder) Option
WithRolloutRecorder records, per reconcile, whether the running containers match the deployed image. nil records nothing.
func WithSecretResolver ¶
func WithSecretResolver(r SecretResolver) Option
WithSecretResolver enables container creation to fill in secret-backed env vars (store.DesiredService.SecretEnv). Without one configured (the default), a service that declares any secret-backed env var fails Reconcile loudly rather than silently starting a container missing a variable it needs, the same "fail loudly" choice internal/deploy.Pipeline makes at save time.
func WithStabilityGate ¶ added in v0.2.0
func WithStabilityGate(defaults deploysafety.Effective, src DeploySafetySource) Option
WithStabilityGate holds a new container back from traffic until it passes N readiness checks in a row and a stable window with no restart or OOM kill.
func WithStartupWatch ¶
WithStartupWatch makes a release with no readiness probe and no stable window still prove it keeps running for d before it takes traffic.
func WithStorageTargets ¶
func WithStorageTargets(s StorageTargetStore) Option
WithStorageTargets enables container creation to resolve store.DesiredService.StorageTargetID into S3_* env vars (resolveEnv's own doc comment). Without one configured (the default), a service that declares a StorageTargetID fails Reconcile loudly rather than silently starting a container missing the credentials it needs, the same "fail loudly" shape WithSecretResolver's own absence already produces for SecretEnv.
func WithVaultResolver ¶ added in v0.2.0
func WithVaultResolver(r VaultResolver) Option
WithVaultResolver enables container creation to read a VaultEnv entry's live value from an external Vault instance, the same "fail loudly if declared but unconfigured" shape WithVaultSettings establishes.
func WithVaultSettings ¶ added in v0.2.0
func WithVaultSettings(s VaultSettingsStore) Option
WithVaultSettings enables container creation to resolve store.DesiredService.VaultEnv against the platform's own Vault connection config, the same "fail loudly if declared but unconfigured" shape WithStorageTargets already establishes. Both this and WithVaultResolver are needed for a VaultEnv entry to resolve; either missing is treated as "not configured" (see resolveEnv).
type OrganizationEnvStore ¶
type OrganizationEnvStore interface {
ListOrganizationEnvVarsForProject(ctx context.Context, projectID string) (map[string]string, error)
}
OrganizationEnvStore is the narrow surface this controller needs to resolve store.DesiredService.ProjectID's organization's shared env vars, one tier below ProjectEnvStore (resolveEnv's own doc comment on precedence). *store.DB satisfies this structurally: its ListOrganizationEnvVarsForProject hides the projects.org_id join so this package never needs to know a project carries one.
type ProbeAttemptRecorder ¶ added in v0.2.0
type ProbeAttemptRecorder interface {
RecordProbeAttempt(ctx context.Context, a store.ProbeAttempt) error
}
ProbeAttemptRecorder persists individual readiness-probe attempts made during a deploy's cutover (migrations/0284_probe_attempts.sql), the per-attempt detail (status code, latency) a reconcile condition's own single Reason/Message summary never captures. *store.DB satisfies this structurally.
type ProjectEnvStore ¶
type ProjectEnvStore interface {
ListProjectEnvVars(ctx context.Context, projectID string) (map[string]string, error)
}
ProjectEnvStore is the narrow surface this controller needs to resolve store.DesiredService.ProjectID into that project's shared env vars (resolveEnv's own doc comment on precedence). *store.DB satisfies this structurally, the same type that already backs internal/api's own project env endpoints.
type RegistryCredentialStore ¶
type RegistryCredentialStore interface {
GetRegistryCredential(ctx context.Context, id string) (store.RegistryCredential, error)
}
RegistryCredentialStore is the narrow surface this controller needs to resolve store.DesiredService.RegistryCredentialID into a real username, the same "just the one read" shape StorageTargetStore establishes just above. *store.DB satisfies this structurally.
type RestartBackoff ¶ added in v0.2.0
type RestartBackoff struct {
// contains filtered or unexported fields
}
RestartBackoff delays restarting a container that keeps exiting, so a crashlooping app cannot spin the reconciler (every die event triggers a pass) and the Docker daemon. In memory only and level-triggered: each pass re-derives whether a restart is allowed yet from the clock. One instance is shared by every controller, see WithRestartBackoff.
func NewRestartBackoff ¶ added in v0.2.0
func NewRestartBackoff(base, ceiling, resetAfter time.Duration) *RestartBackoff
NewRestartBackoff builds a tracker whose delay doubles from base per consecutive restart up to max. A container that stays up for resetAfter starts over. A zero base disables backoff.
type RollbackPinner ¶ added in v0.2.0
type RollbackPinner interface {
PinServing(ctx context.Context, desired store.DesiredService, imageID string) (pinRef string, err error)
}
RollbackPinner keeps a release that just took traffic available as a rollback target, so image cleanup can never remove it.
type RolloutRecorder ¶ added in v0.2.0
type RolloutRecorder interface {
RecordRollout(ctx context.Context, serviceName, image, state, runningImageID string) error
RecordRolloutFailure(ctx context.Context, serviceName, image, reason string) error
}
RolloutRecorder stamps what is actually serving onto deploy history. *store.DB satisfies this structurally.
type RuntimeResolver ¶ added in v0.2.0
RuntimeResolver maps a node ID to the runtime that reaches it.
type SecretResolver ¶
type SecretResolver interface {
Exists(ctx context.Context, serviceName, envKey string) (bool, error)
Resolve(ctx context.Context, serviceName, envKey string) (string, error)
}
SecretResolver is the narrow surface this controller needs from internal/secrets.Manager, so tests can fake it without a real master key or database. *secrets.Manager satisfies this structurally. Resolve's plaintext return value is used exactly once, merged into a container's env map immediately before docker.Runtime.Create, and never persisted anywhere by this controller.
type ServiceStore ¶
type ServiceStore interface {
GetDesiredService(ctx context.Context, name string) (*store.DesiredService, error)
// ListAppStreamsForService returns every raw TCP stream
// (migrations/0279_app_streams.sql) targeting this service, read
// fresh on every container creation: createAndStart publishes one
// extra container port per entry, alongside desired.Port.
ListAppStreamsForService(ctx context.Context, serviceName string) ([]store.AppStream, error)
}
ServiceStore is the narrow surface this controller needs from internal/store, so tests can fake it without a real database. *store.DB satisfies this.
type StorageTargetStore ¶
type StorageTargetStore interface {
GetBackupTarget(ctx context.Context, id string) (store.BackupTarget, error)
}
StorageTargetStore is the narrow surface this controller needs to resolve store.DesiredService.StorageTargetID into a real bucket connection: just the one read, GetBackupTarget, the same "narrow interface at the boundary" shape ServiceStore/SecretResolver already establish. *store.DB satisfies this structurally, the same type that already backs internal/api's own BackupTargetStore for GET/POST/DELETE /api/v1/backup-targets.
type TeardownStore ¶ added in v0.2.0
type TeardownStore interface {
ServiceStore
AddPendingTeardown(ctx context.Context, name, nodeID string) error
DeletePendingTeardown(ctx context.Context, name string) error
RecordPendingTeardownFailure(ctx context.Context, name, cause string) error
ListPendingTeardowns(ctx context.Context) ([]store.PendingTeardown, error)
GetNode(ctx context.Context, id string) (*store.Node, error)
}
TeardownStore is the persistence DeleteFinalizer needs. *store.DB satisfies it.
type VaultResolver ¶ added in v0.2.0
type VaultResolver interface {
Resolve(ctx context.Context, cfg store.VaultSettings, credential, path, key string) (string, error)
}
VaultResolver is the narrow surface this controller needs from internal/vault.Resolver: a live read against an external Vault instance, never persisted anywhere by this controller, the same discipline SecretResolver.Resolve already holds for envelope-encrypted values.
type VaultSettingsStore ¶ added in v0.2.0
type VaultSettingsStore interface {
GetVaultSettings(ctx context.Context) (store.VaultSettings, error)
}
VaultSettingsStore is the narrow surface this controller needs to resolve a { vault: ... } env var (store.DesiredService.VaultEnv) against the platform's own Vault connection config: just the one read, the same shape StorageTargetStore/DatabaseAttachmentStore already establish. *store.DB satisfies this structurally.
Source Files
¶
- alias.go
- applied_config.go
- container_name_parse.go
- controller.go
- database_scope.go
- delete_finalizer.go
- depends_on.go
- drain.go
- egress.go
- exit_capture.go
- external_database.go
- gpu.go
- image_truth.go
- job_spec.go
- liveness.go
- mesh_bind.go
- mesh_database.go
- network_cleanup.go
- pinned_port.go
- probe_attempts.go
- probe_target.go
- replica_names.go
- restart_backoff.go
- rollback_pin.go
- stability_gate.go