Documentation
¶
Overview ¶
Package reconciler runs the loop that converges observed state toward pinned specs.
The dividing line: the reconciler may create and start things; it may not destroy anything a human may have wanted (R-148, R-028). A failed app stays failed — R-151 is true because there is no code path here that touches the failed state, not because a flag is checked. See design 05.
Package reconciler runs the loop that converges observed state toward pinned specs.
The dividing line: the reconciler may create and start things; it may not destroy anything a human may have wanted (R-148, R-028). A failed app stays failed — R-151 is true because there is no code path here that touches the failed state, not because a flag is checked. See design 05.
Index ¶
- Constants
- Variables
- func EnvHash(s *spec.AppSpec, versions map[string]int) string
- func PlanShape(s *spec.AppSpec, image string, perWorkload map[string]state.WorkloadImage, ...) api.BundlePlan
- func TriggerOf(ad spec.AutoDeploy) spec.AutoDeployTrigger
- type AuditEvent
- type AuditWriter
- type Auditor
- type AutoDeploy
- type BackupRunner
- type BuildCaches
- type Deployer
- type Desired
- type Difference
- type Drift
- type Environments
- type GC
- type Holds
- type Inputs
- type Notifier
- type OwnerNotifier
- type PolicyLoader
- type PolicyStore
- type Reconciler
- type RefResolver
- type Registry
- type RegistryImages
- type RollingBackups
- type RouteSync
- type SecurityScores
- type SecurityState
- type ServiceShapes
Constants ¶
const ( BranchPollInterval = 5 * time.Minute ReleasePollInterval = 15 * time.Minute )
Auto-deploy polling intervals. [P], design 05 §5.
const ( DeliveryPoll = "poll" DeliveryWebhook = "webhook" )
How a check came about: the poll, or a webhook from the git host (R-142).
const ( DriftWorkloadMissing = "workload missing" DriftWorkloadStopped = "workload stopped" DriftWrongImage = "wrong image" DriftStaleEnvironment = "stale environment" DriftVolumeMissing = "volume missing" DriftVolumeLostData = "volume that held data is missing" DriftUnknownWorkload = "unrecognized workload" DriftRouteMissing = "route missing" )
Kinds of difference.
const ( // Interval between ticks. Short enough that a killed container comes back // while someone is still looking at the page, long enough that a hundred // apps is not a hundred Observe calls a second. Interval = 15 * time.Second // Concurrency is how many apps are reconciled at once. Concurrency = 8 // DefaultFailureThreshold and DefaultFailureWindow are R-150's give-up rule. DefaultFailureThreshold = 10 DefaultFailureWindow = 30 * time.Minute // DefaultLeaseDuration is how long a claim on an app lasts unless the // pass holding it extends it, which it does every third of this. It is // how long an app held by a replica that died waits to be visited again. DefaultLeaseDuration = 2 * time.Minute // DefaultMinRevisit is how recently visited an app must be for a pass to // leave it for the next one. Half an interval: with several replicas // ticking out of phase, an app one of them has just looked at is not // looked at again by the next a few seconds later, and on one replica a // pass that finishes within half an interval still visits every app on // every tick. DefaultMinRevisit = Interval / 2 )
Tunables, all [P] from design 05 §2.
const ( // DefaultSettledRevisit is how long a settled app on a runtime whose // events are followed waits for its next visit: the sweep that catches // anything an event did not report. Twenty times the fast cadence, which // takes 20,000 settled apps from about 1,300 Observe calls a second to // about 67. DefaultSettledRevisit = 5 * time.Minute )
Tunables, [P] (design 05 §2.3).
const GCInterval = time.Hour
GCInterval is how often garbage collection runs. [P], design 05 §6.
const MinProductionCap = 30 * time.Second
MinProductionCap is the smallest last-step backoff that is not obviously a test setting.
Not enforced — a floor would make the schedule untestable end to end, which is the problem this configurability exists to solve. It is the threshold for saying so loudly at startup instead: an install retrying a broken app every two seconds forever is a real way to melt a host, and it should not be something an operator can do without being told.
const RestartSettleWindow = 30 * time.Second
RestartSettleWindow is how long a workload that has restarted must stay up before it counts as recovered rather than looping. [P].
const RouteSyncInterval = time.Minute
RouteSyncInterval is how often a control host checks its routes [P].
const RouteSyncRefresh = time.Hour
RouteSyncRefresh is how long a route this host wrote is trusted to be on its edge before it is written again [P].
const TeardownBatch = 20
TeardownBatch bounds how many bundles one pass destroys.
Bounded because this is destructive and runs unattended. An install upgrading with two hundred long-deleted apps should reclaim them over a few hours rather than issue two hundred destroys in a burst against a runtime that is also serving live traffic.
Variables ¶
var DefaultBackoff = []time.Duration{0, 5 * time.Second, 15 * time.Second, 60 * time.Second, 5 * time.Minute}
DefaultBackoff is R-149, capped at five minutes.
Indexed by consecutive failures, so the first correction is immediate: a container killed once should come back now, not in five seconds. The cap matters more than the curve — an app that cannot start must not be retried forever at speed, and must still be retried.
Functions ¶
func PlanShape ¶
func PlanShape(s *spec.AppSpec, image string, perWorkload map[string]state.WorkloadImage, rules api.EgressRules) api.BundlePlan
PlanShape builds what should be running, without resolving any secret.
The reconciler compares shape — which workloads, which images, which ports and mounts — and that comparison runs every fifteen seconds for every app. It must never be a reason to decrypt anything, so environment is deliberately absent here and drift in it is detected by fingerprint instead (R-193).
Secrets are resolved only when there is something to apply, which is the uncommon case.
func TriggerOf ¶
func TriggerOf(ad spec.AutoDeploy) spec.AutoDeployTrigger
TriggerOf is the trigger an app's settings use; empty is the branch.
Types ¶
type AuditEvent ¶
AuditEvent is what the reconciler records.
type AuditWriter ¶
AuditWriter appends to the audit log.
type Auditor ¶
type Auditor interface {
Write(ctx context.Context, e AuditEvent) error
}
Auditor writes the events a reconciliation produces.
type AutoDeploy ¶
type AutoDeploy struct {
Apps *state.Apps
Deployments *state.Deployments
Checks *state.AutoDeployChecks
Resolver RefResolver
// Deployer starts a deploy as a person's would (approval.Service).
Deployer Deployer
// Audit records each auto-deploy, with the commit it moves from and to,
// before anything is created (R-227, R-228).
Audit AuditWriter
Logger *zap.Logger
// Concurrency is how many apps are checked at once. Eight when zero
// [P]: each check is a `git ls-remote` against somebody's git host, and
// in series a poll over a few thousand apps outlasted its own interval.
Concurrency int
// Policy is host policy as it is now. An app whose deploys need approval
// does not auto-deploy (R-158); nil reads as no policy, so nothing is
// skipped for it.
Policy PolicyLoader
}
AutoDeploy watches tracked refs and deploys what they point at (R-141).
A separate scheduled job, deliberately not part of the reconciler. It never modifies a running app: it creates a spec revision carrying the new commit and deploys it through the same service a person's deploy goes through, so every plan-time check, the capacity hold and the audit event are the same. The reconciler converges what is pinned; this decides what should be pinned. Conflating them would make an unrelated drift correction able to ship new code.
Off by default (R-141). Nothing here runs for an app that did not ask.
func (*AutoDeploy) CheckApp ¶
func (a *AutoDeploy) CheckApp(ctx context.Context, appID, delivery string) error
CheckApp checks one app now, as a poll would: what a webhook from the git host asks for (R-142). The same path as the poll, so a webhook can start nothing a poll could not — it only makes the check sooner.
func (*AutoDeploy) Poll ¶
func (a *AutoDeploy) Poll(ctx context.Context)
Poll checks every app that auto-deploys, whichever trigger it uses.
func (*AutoDeploy) PollTrigger ¶
func (a *AutoDeploy) PollTrigger(ctx context.Context, trigger spec.AutoDeployTrigger)
PollTrigger checks the apps that auto-deploy on trigger, and no others: what one tick of Run does.
func (*AutoDeploy) Run ¶
func (a *AutoDeploy) Run(ctx context.Context)
Run polls until the context is canceled: apps following a branch every BranchPollInterval, apps following releases every ReleasePollInterval. Releases are rarer than pushes, and a release check lists every tag.
type BackupRunner ¶
type BackupRunner interface {
CreateForApp(ctx context.Context, id string, req backup.AppCreateRequest) (backup.Created, error)
Discard(ctx context.Context, adapterRef, objectName string) error
}
BackupRunner takes and prunes per-app backups.
An interface rather than *backup.Service so the collector is testable without a destination, and so this package cannot reach anything else the service exposes — restore in particular, which the janitor has no business calling.
type BuildCaches ¶
BuildCaches has the builder that built an app forget its cache.
type Deployer ¶
type Deployer interface {
Deploy(ctx context.Context, p authz.Principal, app state.App, rev state.Revision, trigger string) (state.Deployment, error)
}
Deployer starts a deploy of rev on behalf of p.
type Desired ¶
Desired sets an app's desired state, which is how an app is stopped and started (design 09 §4.2: never a delete, never `failed`).
type Difference ¶
Difference is one thing that does not match.
type Drift ¶
type Drift struct {
// Reconcilable are differences the reconciler may correct. Every one of
// them is a thing to create or start.
Reconcilable []Difference
// ReportOnly are differences it must not. Every one of them would mean
// destroying or overwriting something a person may have put there.
ReportOnly []Difference
}
Drift is the difference between what a spec says and what is running.
func Classify ¶
func Classify(want api.BundlePlan, observed api.ObservedBundle, in Inputs) Drift
Classify compares what should be running against what is.
The dividing line, and the rule to hold when new cases come up (design 05 §2.1): **the reconciler may create and start things; it may not destroy anything a human may have wanted.**
Everything in Reconcilable is a creation or a start. Everything in ReportOnly is something where the only way to "fix" it would be to throw away state or a container that someone put there deliberately — and being wrong about that is unrecoverable, while being wrong about reporting it costs a notification.
func (Drift) Actionable ¶
Actionable reports whether there is anything the reconciler may fix.
type Environments ¶
type Environments interface {
Environments(ctx context.Context, s *spec.AppSpec) (map[string]map[string]secret.Value, error)
}
Environments resolves the environment each workload runs with, keyed by workload name. Asked only when a correction is about to be applied, which is the uncommon case: comparing never decrypts anything (R-193).
type GC ¶
type GC struct {
Apps *state.Apps
Logger *zap.Logger
// Registry resolves the adapters that hold a deleted app's bundle. Nil
// disables teardown rather than failing: an install with no runtime
// configured has nothing to tear down.
Registry Registry
// Auditor records teardowns. Destroying a bundle is destruction, and
// R-227's rule does not have an exception for the janitor.
Auditor Auditor
// Backups gates reclaiming a deleted app's storage on a backup holding
// its data. Nil disables that rather than failing. Taking and expiring
// rolling backups is RollingBackups, a job of its own (issue #72).
Backups *state.Backups
// The security pass (R-315, R-316). All four are nil on an installation
// with no scanner, where nothing is scored and the pass does nothing.
Security SecurityScores
SecurityState SecurityState
PolicyStore PolicyStore
Desired Desired
Notifier OwnerNotifier
// Clock is here so retention is testable without waiting a day.
Clock clock.Clock
// Interval overrides GCInterval. Zero means the default.
//
// Configurable for the same reason the retry backoff is: the acceptance
// test for bundle teardown would otherwise have to wait an hour to watch a
// janitor run. Nothing here is urgent, so a short interval is wasteful
// rather than dangerous — which is why this one gets no startup warning.
Interval time.Duration
// TeardownNow starts a teardown pass between ticks. Deleting an app sends
// on it, so the app's containers stop when it is deleted rather than at the
// next pass — which was up to an hour later, while a force-deleted app kept
// running and holding its share of the host (issue #55). Nil means ticks
// only.
//
// Served by a loop of its own, so a delete is acted on while the rest of
// a pass — a security pass over every app, reclaiming storage — is still
// running, rather than after it (issue #72).
TeardownNow <-chan struct{}
// TeardownEvery is how often teardown runs when nothing asks: a delete
// whose signal reached another replica, or one whose runtime was away.
// Ten seconds when zero.
TeardownEvery time.Duration
// BuildCaches and DiscardUpload remove a deleted app's build cache and
// uploaded source at teardown. Nil skips each.
BuildCaches BuildCaches
DiscardUpload func(ctx context.Context, appID string) error
// DiscardCredential removes a deleted app's registry credential (issue
// #41). Apps soft-delete, so the row's cascade never fires; a credential
// for an app nobody can deploy is a key kept for no one. Nil skips it.
DiscardCredential func(ctx context.Context, appID string) error
// RegistryImages deletes a deleted app's builds from the install's image
// registry (issue #72 PR 5, R-224). Nil when the install has none. Best
// effort: a failure is recorded on the teardown's audit event and does
// not hold the teardown.
RegistryImages RegistryImages
// contains filtered or unexported fields
}
GC reclaims what retention says should be gone.
A separate job from the reconciler, on a much slower clock, because nothing here is urgent and all of it is destructive. The reconciler's rule — never destroy anything a human may have wanted — applies here too, and is the reason each of these deletions has an explicit exemption attached.
func (*GC) Collect ¶
Collect runs one pass.
Spec revision pruning only, for now. Log retention (R-222–R-224) is not here because there is nothing for it to act on: Pando does not hold app logs, it streams them from the runtime, and the bytes live wherever that runtime put them. Enforcing a per-app cap means the log driver's own options at container creation; enforcing the aggregate would mean recreating every container when one app turns chatty, which is destruction on a schedule and not something the reconciler may do. Recorded as O-16 rather than guessed at.
Rolling backups and their expiry (R-211) are RollingBackups, a job of its own, and teardown also runs on a loop of its own in Run (issue #72); Collect tears down too, so one call is a whole pass.
func (*GC) TearDownDeleted ¶
tearDownDeletedBundles destroys the bundles of apps that have been deleted.
Nothing used to do this. Deleting an app archived the row and left its containers running on a private network nobody reclaimed — and Pando takes one network per app against a Docker pool that holds about thirty, so an install that adds and removes apps eventually cannot start one. The failure arrives as a message about subnets, on an unrelated deploy, long after the deletion that caused it.
Here rather than in the delete handler because convergence is the reconciler's job (design 05): a delete that destroyed inline would leave a bundle standing forever if the runtime happened to be unreachable at that moment, and nothing would ever come back for it.
Volumes are destroyed only when the delete said so. R-204 has a delete settle the app's storage — keep a final backup or discard it — and a delete of an app with storage is refused until it does; both answers end with the volumes gone, the second after a backup holds the data. That decision is recorded on the app (TeardownTarget.DiscardStorage) and is the only thing that sets KeepVolumes false here: this is the one place a bug would silently destroy data, so nothing is inferred. TearDownDeleted tears down the bundles of deleted apps now, outside the GC's own loop: the replica that took a delete calls it whether or not it leads (issue #72). teardownMu keeps it from overlapping this replica's own pass, and teardown is idempotent, so meeting the leader's pass on one app is harmless.
type Holds ¶ added in v0.17.0
type Holds interface {
Held(ctx context.Context, appID string) (state.Hold, error)
// HostHeld is what holds every app on host, a name the runtime gave
// (api.ObservedBundle.Host): whether it is lost or fenced.
HostHeld(ctx context.Context, host string) (state.Hold, error)
}
Holds reads what keeps the reconciler off an app in a cluster (R-148, R-153, R-419, R-421). *state.Standbys is one.
R-148 restores what diverges from the spec, and three things in a cluster look like divergence and are not:
- an app whose host is marked lost: its standby is started elsewhere, and restarting the copy on the lost host would be a second copy (R-153);
- an app whose host is fenced: the host stopped its apps because it is cut off from most of the cluster, and starting them again would undo the fence (R-421);
- an app whose standby is being promoted: the promotion is starting it, and a second Apply beside it would start it twice.
Each is reported as the app being unobservable, never as a failure, so it counts toward nothing (R-151). A hold that cannot be read is not a hold (heldCheck). An install of one host asks nothing (consultHolds). That holds for every app on such a host, with a standby or without: an app's home is its standby row's, and for any app the host the runtime found it on (HostHeld).
type Inputs ¶
type Inputs struct {
// ExpectedDigests is what each workload ran, by name. A workload with no
// entry is not checked: unknown is not drift.
//
// Per workload because an app's parts can be built separately. One digest
// compared against every workload made every service but one look wrong
// forever, and "correcting" that replaced them with the primary's image.
ExpectedDigests map[string]string
// ExpectedDigest is the image the last successful deployment ran, which is
// the primary workload's. Kept for the single-image case and for the
// report text; the per-workload map decides.
ExpectedDigest string
// AppliedEnvHash and CurrentEnvHash stand in for environment observation
// (R-193). Either being empty means the comparison cannot be made, and an
// unanswerable question is not drift.
AppliedEnvHash string
CurrentEnvHash string
// VolumesThatHeldData is the difference between a volume Pando may recreate
// and one it must only report.
VolumesThatHeldData map[string]bool
RouteExpected bool
RoutePresent bool
}
Inputs are the facts Classify needs that are not in the bundle plan.
type Notifier ¶
type Notifier interface {
Notify(ctx context.Context, n api.Notification) error
}
Notifier tells someone an app needs attention.
type OwnerNotifier ¶
OwnerNotifier tells an app's owner what is about to happen to it (R-315).
type PolicyLoader ¶
PolicyLoader reads host policy, per poll rather than cached (R-274): an administrator who starts requiring approval stops the next poll.
type PolicyStore ¶
PolicyStore reads host policy.
type Reconciler ¶
type Reconciler struct {
Apps *state.Apps
Reconciles *state.Reconciles
Secrets *state.Secrets
// Services adds provisioned services to what should be running (R-131).
// Nil on an install with no provisioner, where every app's shape is already
// complete.
Services ServiceShapes
// Environments supplies what each workload runs with when a correction
// is applied (R-148). Nil applies the shape as it is, which is complete
// only for an app whose workloads take no environment.
Environments Environments
Volumes *state.Volumes
Registry Registry
Auditor Auditor
Notifier Notifier
Logger *zap.Logger
Clock clock.Clock
// BuiltImageAuth is the install registry's credential for an image Pando
// pushed there, nil for any other (issue #72, PR 5). A workload restored
// on a runtime that pulls needs it for the pull, as the deploy did. Nil
// means no install registry. Nil too for an image s names: the
// credential is for the app's own builds (R-025).
BuiltImageAuth func(ctx context.Context, s *spec.AppSpec, ref string) *api.RegistryAuth
// Backoff, FailureThreshold and FailureWindow override R-149 and R-150's
// defaults. Zero values mean the defaults, so a caller that does not care
// sets nothing.
//
// Configurable because the acceptance test for R-151 — a crash-looping app
// reaches failed and stays there — has to wait out the real schedule, and
// at the production numbers that is forty minutes of a forty-three minute
// suite. The test is asserting the state machine, not the durations, and it
// was paying for the durations.
//
// The state machine is what stays under test either way: compressing the
// schedule changes how long each step waits and nothing about which step
// comes next.
Backoff []time.Duration
FailureThreshold int
FailureWindow time.Duration
// MinRevisit is how recently visited an app may be and still be left
// for a later pass (DefaultMinRevisit in production). Zero visits every
// app on every pass, which is what a test calling Tick in a row wants.
MinRevisit time.Duration
// LeaseDuration overrides DefaultLeaseDuration. Zero means the default.
LeaseDuration time.Duration
// SettledRevisit is how long an app found as it should be is left before
// it is looked at again, when its runtime reports events (O-52, events.go).
// Zero means DefaultSettledRevisit.
SettledRevisit time.Duration
// Runtimes names the runtime adapters whose events Run follows (O-52).
// Asked again on every tick, so a runtime added in the console is
// followed without a restart. Nil follows none, and every app is visited
// on every pass, as before events.
Runtimes func() []string
// Holds keeps the reconciler off an app whose host is lost or fenced, or
// whose standby is being promoted (holds.go). Nil holds nothing, as on
// an install with one host.
Holds Holds
// Hosts counts the cluster's hosts (hosts.Service.Count). While it is
// one, nothing can hold an app — no other host to be lost, fenced or
// promoted to — and Holds is not consulted (holds.go). Nil consults Holds
// whenever it is set.
Hosts func(ctx context.Context) (int, error)
// LocalFenced reports whether this replica's own host is fenced (R-421).
// While it is, Tick does nothing: a host cut off from most of the
// cluster starts nothing. Nil is never.
LocalFenced func() bool
// ProxyUpstream is where routes point. Every route points at Pando's proxy
// and never at a workload (R-023) — the reconciler re-ensuring a route must
// not be the one place that forgets.
ProxyUpstream string
// contains filtered or unexported fields
}
Reconciler converges running apps toward their pinned specs.
func (*Reconciler) Complete ¶ added in v0.17.0
func (r *Reconciler) Complete(ctx context.Context, want *api.BundlePlan, s *spec.AppSpec) error
Complete adds what a plan needs to be started and not only compared: the pull credential and each workload's environment.
func (*Reconciler) Run ¶
func (r *Reconciler) Run(ctx context.Context)
Run ticks until the context is canceled.
Every replica runs this, and every replica follows every runtime's events (O-52, events.go): the lease gives no app an owning replica, so the replica that hears about an app is not necessarily the one that will visit it. An event makes the app due in the database, where any replica's next claim finds it, and wakes this replica's loop so it does not wait for the tick.
func (*Reconciler) Shape ¶ added in v0.17.0
func (r *Reconciler) Shape(ctx context.Context, appID string) (api.BundlePlan, *spec.AppSpec, error)
Shape is what the reconciler compares one app against now — its pinned spec's shape and its provisioned services, without environment — for something other than the loop that must start it the same way: its warm standby, made and promoted on another host (R-419). Complete adds the rest. Building it here keeps one definition of what an app is started with.
func (*Reconciler) Tick ¶
func (r *Reconciler) Tick(ctx context.Context)
Tick reconciles every app that is due.
Apps are claimed in small batches under a lease (state.Lease), least recently visited first, until none is left that was last visited before the pass began. The batches keep the number held but not yet started small, and the lease is what keeps two passes — on this replica or another — off one app. Nothing is held on a database connection while an app is reconciled, however long its Apply takes.
type RefResolver ¶
type RefResolver interface {
Resolve(ctx context.Context, src spec.Source, ad spec.AutoDeploy) (source.Tracked, error)
}
RefResolver finds what an app's auto-deploy trigger points at now.
Listing a remote's refs rather than cloning: this runs every few minutes for every app that auto-deploys, and cloning each time to learn a SHA that has usually not changed would be most of Pando's network traffic.
type Registry ¶
type Registry interface {
Runtime(ref string) (api.RuntimeAdapter, bool)
Routing(ref string) (api.RoutingAdapter, bool)
}
Registry resolves adapters by reference.
type RegistryImages ¶
RegistryImages removes a deleted app's images from the install's registry, by manifest. The layers are freed by the registry's own scheduled garbage collection (O-38).
type RollingBackups ¶
type RollingBackups struct {
Apps *state.Apps
Backups *state.Backups
Backup BackupRunner
BundleSource *state.BundleSource
Auditor Auditor
Logger *zap.Logger
Clock clock.Clock
// Every is how often to look for apps due a backup. Five minutes when
// zero [P]: a backup is due once a day, so this is how late one can be.
Every time.Duration
// Concurrency is how many backups run at once. Two when zero [P]: a
// backup copies a volume, and the disk it reads is the one the apps use.
Concurrency int
// RetryAfter is how long after an attempt that was skipped or failed the
// app is tried again. An hour when zero, which is what the attempt's
// remedy tells a person ("within the hour").
RetryAfter time.Duration
// Batch is the most apps one pass takes. 200 when zero.
Batch int
}
RollingBackups takes rolling backups for apps that are due one, and removes backups whose retention has passed (R-211). A leader job of its own (issue #72), not part of the GC's hourly pass: that walked every app with storage in series, so one slow destination held up every other app's backup and the rest of the GC with it.
Due-driven: each pass asks the database for a page of apps due a backup and takes them a few at a time, so how long a pass takes depends on how many are due, not on how many apps there are.
func (*RollingBackups) Pass ¶
func (b *RollingBackups) Pass(ctx context.Context)
Pass takes every due backup it is given a page of, then expires old ones.
func (*RollingBackups) Run ¶
func (b *RollingBackups) Run(ctx context.Context)
Run takes backups until ctx ends.
type RouteSync ¶ added in v0.17.0
type RouteSync struct {
Apps interface {
LiveApps(ctx context.Context, f planner.InventoryFilter) ([]planner.InventoryApp, error)
}
Registry Registry
// ProxyUpstream is where every route points: this host's Pando.
ProxyUpstream string
// ControlHosts counts the hosts holding the control role
// (hosts.Service.ControlCount). With one, every edge is that host's and
// its routes are the reconciler's alone, so the pass is skipped, however
// many app hosts there are. Nil runs it always.
ControlHosts func(ctx context.Context) (int, error)
// Exclusive runs pass unless another replica on this host is running
// one (state.Cluster.OnePerHost), and reports whether it ran: every
// replica of a host writes to one edge, so one pass at a time is enough.
// Nil runs it always.
Exclusive func(ctx context.Context, pass func(context.Context)) (bool, error)
Interval time.Duration
Clock clock.Clock
Logger *zap.Logger
// contains filtered or unexported fields
}
RouteSync writes every app's route on this host, from the database (R-417).
The reconciler writes an app's route on whichever control host holds its lease, which is the only edge that learns it; another control host's edge would send the app's hostname to its catch-all, without the app's own certificate. Each control host runs this, so its edge is generated from the pinned specs rather than shared as a directory. Every route points at Pando's proxy (R-023): a route this pass writes is the one the reconciler would, and one it leaves behind for a deleted app reaches Pando, which no longer knows the hostname.
A pass writes only the routes that differ from what this host last wrote (written), and one not written for RouteSyncRefresh, in case its edge lost it: rewriting every route every minute reached its adapter — a file Traefik reloads, a provider's API — for nothing.
type SecurityScores ¶
type SecurityScores interface {
Configured() (string, bool)
Place(ctx context.Context, scores map[string]security.Scores) (map[string]security.Placed, error)
}
SecurityScores is what this pass needs from the security service.
Place, the batched path, rather than a Report per app: a report is a policy load, a scan read and a state read for each app, which over twenty thousand apps is sixty thousand queries an hour for an answer one query and one policy load give (issue #72). The scores come from LiveSecurityState.
type SecurityState ¶
type SecurityState interface {
LiveSecurityState(ctx context.Context, t state.SecurityThreshold) ([]state.SecurityState, error)
MarkInsecure(ctx context.Context, appID string, at time.Time) error
ClearInsecure(ctx context.Context, appID string) error
SetStoppedForSecurity(ctx context.Context, appID string, stopped bool) error
}
SecurityState is the store half.
type ServiceShapes ¶
type ServiceShapes interface {
ServiceShapes(ctx context.Context, s *spec.AppSpec) (api.BundlePlan, error)
}
ServiceShapes adds an app's provisioned services to what should be running.
An interface rather than the Runner so a Reconciler can be built without a deploy Runner, which the unit tests do. Nil means an install with no provisioner, where every app's shape is already complete.