Documentation
¶
Overview ¶
Package deploy runs the deployment pipeline: everything after the plan boundary (design 05 §3, steps 8-16).
Steps 1-7 belong to the planner and create nothing. Everything here has side effects, which is why the split exists: a plan-time failure means nothing was touched, and a failure in this package means something was.
Index ¶
- Constants
- func BundlePlanShape(s *spec.AppSpec, image string, perWorkload map[string]string, ...) (api.BundlePlan, error)
- func DefaultConcurrency() int
- func EnvFingerprint(s *spec.AppSpec, versions map[string]int) string
- func PlanFingerprint(p api.BundlePlan) string
- type BuildConcurrency
- type DetectionBuildFinder
- type ImageCheck
- type LogStore
- type PolicyReader
- type Queue
- type Reconciles
- type Revisions
- type Runner
- func (r *Runner) BuildImages(ctx context.Context, s *spec.AppSpec, checkout *source.Checkout, ...) (string, map[string]string, error)
- func (r *Runner) BuiltImageAuth(ctx context.Context, image string) *api.RegistryAuth
- func (r *Runner) CheckImage(ctx context.Context, app state.App) (ImageCheck, error)
- func (r *Runner) Environments(ctx context.Context, s *spec.AppSpec) (map[string]map[string]secret.Value, error)
- func (r *Runner) PrepareRevision(ctx context.Context, rev state.Revision, by string) (state.Revision, error)
- func (r *Runner) Run(ctx context.Context, dep state.Deployment, rev state.Revision) error
- func (r *Runner) ServiceShapes(ctx context.Context, s *spec.AppSpec) (api.BundlePlan, error)
- func (r *Runner) UpgradeImage(ctx context.Context, app state.App, digest, by string) (state.Revision, error)
- func (r *Runner) WithBuildConcurrency(limit BuildConcurrency) *Runner
- func (r *Runner) WithBuildRegistry(reg imageregistry.Provider) *Runner
- func (r *Runner) WithDetectionBuilds(b DetectionBuildFinder) *Runner
- func (r *Runner) WithImages(images *oci.Images) *Runner
- func (r *Runner) WithSecurity(s Security) *Runner
- func (r *Runner) WithServices(services *state.Services, secrets *state.Secrets) *Runner
- func (r *Runner) WithSources(sources source.Sources) *Runner
- type Secrets
- type Security
- type Sink
Constants ¶
const LogRetention = 5 * time.Minute
LogRetention is how long a deploy's log stays in memory once nothing holds it: the deploy has finished (or never started writing here) and no one is following it. Long enough for a console that opens a deploy just after it ends to see the whole log; short enough that a busy install does not keep every deploy's output for the life of the process.
const ServiceSecretPrefix = "pando.service."
ServiceSecretPrefix namespaces the connection strings Pando generates.
A prefix rather than the bare slot key so a provisioned DATABASE_URL cannot collide with a secret someone set by hand under the same name — and so that looking at an app's secrets makes it obvious which ones Pando owns and will overwrite.
Variables ¶
This section is empty.
Functions ¶
func BundlePlanShape ¶
func BundlePlanShape(s *spec.AppSpec, image string, perWorkload map[string]string, rules api.EgressRules) (api.BundlePlan, error)
bundlePlan resolves the spec into what the runtime is asked to apply. BundlePlanShape builds the bundle plan without resolving any environment.
The reconciler compares what should be running against what is, on every tick for every app. That comparison needs the shape — workloads, images, ports, mounts, volumes — and must never be a reason to decrypt a secret, so this stops short of environment and R-193's fingerprint covers the rest.
Egress is the rules the app's last successful deploy ran with, which the deployment records: rules take effect at a deploy and a running app is not changed underneath it (R-183, O-10), so correcting drift restores what was deployed rather than what policy says now.
func DefaultConcurrency ¶
func DefaultConcurrency() int
DefaultConcurrency is how many deploys one replica runs at once when configuration does not say [P]: one per CPU, and at least two. A deploy is a clone, a build and a wait for health; the build is the expensive part and BuildKit spreads one build across cores, so more than one per core mostly makes every build slower.
func EnvFingerprint ¶
EnvFingerprint summarizes the environment an app was applied with.
This is the whole mechanism behind R-193 — "a rotated secret reaches the app" — and it exists because `Observe` returns no environment and deliberately never will. Reading environment back would require every runtime adapter to handle secret-bearing data, which is precisely what WorkloadPlan.Env's one-way flow is designed to prevent. So the comparison happens state-side: what was applied, against what would be applied now.
**It hashes secret versions, never secret values.** A fingerprint is stored in a column, shown in logs when it changes, and survives in backups. Hashing the value would put an oracle for every secret in the app into all three — an attacker with the column and a guess could confirm the guess. Hashing (key, version) answers the only question being asked, which is whether the thing changed, and answers nothing else.
func PlanFingerprint ¶
func PlanFingerprint(p api.BundlePlan) string
PlanFingerprint summarizes a resolved bundle plan's environment.
Used where the spec is not at hand but the plan is. It takes the resolved values, so it hashes them — which is safe only because it is compared in-process and never stored. EnvFingerprint is the one that persists.
Types ¶
type BuildConcurrency ¶ added in v0.5.0
BuildConcurrency bounds how many of an app's images are built at once.
func DeployConcurrency ¶ added in v0.5.0
func DeployConcurrency(p PolicyReader) BuildConcurrency
DeployConcurrency is host policy's max_concurrent_deploys, as a build limit: one app's builds take no more of the machine than that many deploys.
type DetectionBuildFinder ¶ added in v0.5.0
type DetectionBuildFinder interface {
// Built is the app's detection build, nil when there is none.
Built(ctx context.Context, appID string) (*detect.Built, error)
}
DetectionBuildFinder finds what detection built for an app.
type ImageCheck ¶ added in v0.14.0
type ImageCheck struct {
Image string `json:"image"`
// Current is the digest the pinned revision runs; empty when it names none.
Current string `json:"current_digest"`
// Latest is the digest the tag names now.
Latest string `json:"latest_digest"`
Available bool `json:"update_available"`
// Revision is the pinned revision that was checked.
Revision int `json:"revision"`
}
ImageCheck is what a check of an image app's tag found (R-143a).
type LogStore ¶
type LogStore struct {
// contains filtered or unexported fields
}
LogStore holds build and deploy output for streaming.
In memory and bounded. Logs on disk under a size cap are R-222/R-223's concern and belong to the reconciler's garbage collection; this is the live tail a console watches while a deploy runs.
A stream is dropped LogRetention after it was last held (see stream.idle). Eviction is lazy and amortized: Writer and Follow sweep the map at most once per quarter of the retention, and Has reports an expired stream as gone even before a sweep removes it. No goroutine is involved, so there is nothing to stop.
func NewLogStore ¶
func NewLogStore() *LogStore
func (*LogStore) Follow ¶
Follow returns the lines so far plus a channel of new ones.
The backlog is returned with the channel rather than replayed through it, so a client that connects late still sees the whole build instead of joining mid-stream.
type PolicyReader ¶ added in v0.5.0
PolicyReader is host policy, narrowed to the one call DeployConcurrency needs.
type Queue ¶
type Queue struct {
Runner *Runner
Deployments *state.Deployments
Revisions Revisions
// Limit is how many deploys this replica runs at once. DefaultConcurrency
// when zero. Concurrency, when set, is read instead.
Limit int
// Concurrency reads the limit from host policy's max_concurrent_deploys
// each time the queue looks for work (issue #93): zero for the default, or
// an error. On an error the last limit read stays in force rather than
// the queue stopping, or running without one.
Concurrency func(ctx context.Context) (int, error)
// Poll is how often to look for deploys another replica queued. Two
// seconds when zero.
Poll time.Duration
Logger *zap.Logger
// contains filtered or unexported fields
}
Queue runs queued deployments on this replica (issue #72, O-32).
A deployment is created pending and unclaimed (state.Deployments.Create). Every replica's queue claims what it has room for, so a deploy runs on whichever replica is free rather than the one whose request created it, no replica runs more than Limit at once, and a deploy queued when every replica was busy — or when the one that took the request restarted — runs when one has room.
Deploys of one app are still never concurrent: an app with a deploy in flight, queued included, refuses the next one (design 05 §5).
func (*Queue) Serve ¶
Serve runs queued deploys until ctx ends. Deploys still running then are stopped and put back in the queue for another replica.
type Reconciles ¶
type Reconciles interface {
SetAppliedEnvFingerprint(ctx context.Context, appID, fingerprint string) error
}
Reconciles records what an app was last applied with.
type Revisions ¶
type Revisions interface {
RevisionByID(ctx context.Context, specID string) (state.Revision, bool, error)
}
Revisions reads the revision a queued deployment is for.
type Runner ¶
type Runner struct {
// ProxyUpstream is where routing adapters must send traffic (R-023). It is
// Pando's proxy, always, and it is passed to every Ensure so that no adapter
// has to work it out.
ProxyUpstream string
// contains filtered or unexported fields
}
Runner executes deployments.
func (*Runner) BuildImages ¶ added in v0.5.0
func (r *Runner) BuildImages(ctx context.Context, s *spec.AppSpec, checkout *source.Checkout, sink io.Writer, tag string) (string, map[string]string, error)
BuildImages builds every image s needs from checkout, exactly as a deploy builds them — the same builder, the same Dockerfile or plan, the same egress rules (R-118) and isolation floor, and no runtime socket in the build (R-112) — and delivers them to the runtime the same way. tag names the build, as a deployment's ID names a deploy's (R-146).
Detection's build (R-311). Returns the app's image, or a compose app's images by workload.
One difference from a deploy, when detection asks for it (detect.WithSteer): the builder may steer a plan it made from convention when a build of it fails in a way it recognizes, and build again. The plan it built from in the end replaces s.Build.GeneratedFiles, so what detection proposes is what was built, and what a deploy replays (R-020).
func (*Runner) BuiltImageAuth ¶ added in v0.5.0
BuiltImageAuth is the credential an image Pando built is pulled with: the install registry's, for an image pushed there, and nil for any other.
func (*Runner) CheckImage ¶ added in v0.14.0
CheckImage reports whether the tag the app's pinned revision names now points at a different build. It writes nothing (R-143a), so a newer saved revision does not stop it; only the upgrade would bury one.
func (*Runner) Environments ¶
func (r *Runner) Environments(ctx context.Context, s *spec.AppSpec) (map[string]map[string]secret.Value, error)
Environments is the environment each of an app's workloads runs with, keyed by workload name: its own workloads' and its provisioned services'.
For the reconciler, at the moment it is about to apply a correction (R-148). Its comparison is shape, and environment is deliberately absent from that (R-193) — but a shape is not something to start. A workload it re-created from the shape came up with no environment at all: a provisioned Redis whose password arrives in REDIS_PASSWORD exited on every start, and an app lost every variable and secret it was deployed with.
Resolved the way a deploy resolves them, with one difference: a slot with no recorded instance is skipped, as in ServiceShapes. Restoring what exists is the reconciler's job; creating a database is a deploy's.
func (*Runner) PrepareRevision ¶
func (r *Runner) PrepareRevision(ctx context.Context, rev state.Revision, by string) (state.Revision, error)
PrepareRevision pins the commit a deploy will build.
Resolving a ref is an explicit act that produces a revision, never something a deploy does on its own (design 01 §2.1, R-120). A revision that already names a commit is returned unchanged, so redeploying it builds the same code even after the branch has moved — which is the whole point of pinning.
func (*Runner) Run ¶
Run executes a deployment to completion.
The app's state moves to deploying when the deploy is started and to running or degraded at the end. A deploy that fails before the runtime is touched — a build failure, a refused scan — puts back the state it found, because nothing about the running app changed (R-146). The deployment is failed; the app is not. One that fails later leaves the app failed or degraded (design 05 §1.2).
func (*Runner) ServiceShapes ¶
ServiceShapes is what an app's provisioned services should look like, for the reconciler's drift comparison.
Without it the reconciler's idea of "what should be running" contains the app's workloads and not the database standing next to them, and two things go wrong on every tick: a provisioned service that was killed is never restored, because nothing wants it (R-148), and the one that is running is reported as a workload the spec does not declare — an app told every fifteen seconds that its own database is a stranger.
Reads nothing sensitive and writes nothing. Provision is called without the stored connection string, which changes only the generated password, and environment is dropped here anyway: the reconciler's comparison is shape, and it must never be a reason to decrypt a secret (R-193). A slot with no recorded instance is skipped rather than provisioned — this runs every fifteen seconds for every app and is not where a database gets created.
func (*Runner) UpgradeImage ¶ added in v0.14.0
func (r *Runner) UpgradeImage(ctx context.Context, app state.App, digest, by string) (state.Revision, error)
UpgradeImage writes a revision that runs digest, the build a check found (R-143a). Deploying it is the caller's next step, through the ordinary path.
func (*Runner) WithBuildConcurrency ¶ added in v0.5.0
func (r *Runner) WithBuildConcurrency(limit BuildConcurrency) *Runner
WithBuildConcurrency sets how many of a compose app's services build at once: the install's deploy concurrency, so one app's builds take no more of the machine than that many deploys would. Without it, they build one at a time.
func (*Runner) WithBuildRegistry ¶
func (r *Runner) WithBuildRegistry(reg imageregistry.Provider) *Runner
WithBuildRegistry sets the install's image registry (issue #72, PR 5). Without it, a build reaches only a runtime that imports it.
Asked on every build and pull rather than held, because the registry and its credential can be changed from the console while Pando runs.
func (*Runner) WithDetectionBuilds ¶ added in v0.5.0
func (r *Runner) WithDetectionBuilds(b DetectionBuildFinder) *Runner
WithDetectionBuilds lets a deploy run the image detection built when its spec builds the same thing from the same source. Without it every deploy builds.
func (*Runner) WithImages ¶
WithImages sets how an image app's image is read and pulled: its digest is resolved and pinned before a deploy, and its registry credential resolved for the pull (issue #41). Without it an image is pulled anonymously by tag.
func (*Runner) WithSecurity ¶
WithSecurity enables the security score (R-310 – R-314).
Optional in the same way WithServices is: a Runner without it deploys exactly as before. An installation with no scanner configured has no scores and no threshold to enforce, which is the shipped posture (R-317).
func (*Runner) WithServices ¶
WithServices enables provisioned slots (R-131).
Optional rather than a constructor argument because a Runner without it is still correct: an app with no provisioned slot never notices, and one with a provisioned slot is refused at plan time rather than deployed half-wired.
type Secrets ¶
type Secrets interface {
// Resolve returns an app's secret values by key. Called once per deploy, at
// step 11 — after the plan boundary, because reading a secret is a side
// effect and an audited one.
Resolve(ctx context.Context, appID string) (map[string]secret.Value, error)
// Versions returns each secret's version without decrypting anything. It
// feeds the environment fingerprint (R-193), which is compared on every
// reconciliation and must never be a reason to read a value.
Versions(ctx context.Context, appID string) (map[string]int, error)
}
Secrets resolves an app's stored secrets into values.
type Security ¶
type Security interface {
Scan(ctx context.Context, req api.ScanRequest, principal audit.Event) (state.Scan, error)
// Reuse returns a successful scan of the app's source and image, attached
// to specID, when there is one: a deploy of an unchanged source and image
// uses that scan. A scan of the source alone does not cover an image
// (R-311).
Reuse(ctx context.Context, appID, specID, source, image string) (state.Scan, bool, error)
Allows(ctx context.Context, appID, specID string) (security.Standing, error)
Configured() (string, bool)
}
Security is what a deploy needs from the security service, narrowed so the deploy path cannot reach for anything else.
Source Files