Documentation
¶
Overview ¶
Package deploysafety decides whether a new release may take traffic and whether a live one must be rolled back, and holds the shared reason strings.
Index ¶
Constants ¶
const ( PhaseReadiness = "readiness" PhaseStability = "stability" PhaseCutover = "cutover" PhaseDrain = "drain" PhaseWatch = "watch" PhaseRollback = "rollback" PhasePin = "pin" )
Phases of a deploy's gate timeline.
const ( StatusStarted = "started" StatusPassed = "passed" StatusFailed = "failed" StatusInfo = "info" )
Statuses a gate event can carry.
const ( ReasonReadinessFailed = "ReadinessFailed" ReasonOOMKilledDuringReadiness = "OOMKilledDuringReadiness" ReasonExitedDuringReadiness = "ExitedDuringReadiness" ReasonOOMKilledDuringWindow = "OOMKilledDuringStableWindow" ReasonExitedDuringWindow = "ExitedDuringStableWindow" ReasonRestartedDuringWindow = "RestartedDuringStableWindow" ReasonGateTimeout = "StabilityGateTimeout" ReasonWatchCrashloop = "PostDeployCrashloop" ReasonWatchOOMKilled = "PostDeployOOMKilled" ReasonWatchExited = "PostDeployExited" ReasonWatchErrorRate = "PostDeployErrorRate" ReasonAutoRolledBack = "AutoRolledBack" ReasonNoRollbackTarget = "NoRollbackTarget" )
Reasons a deploy fails its gate or a live release is rolled back. These land in deploy_attempts.reason, gate events and reconcile conditions.
const ( EnvSuccessThreshold = "APP_DEPLOY_SUCCESS_THRESHOLD" EnvStableWindow = "APP_DEPLOY_STABLE_WINDOW" EnvDrainTimeout = "APP_DEPLOY_DRAIN_TIMEOUT" EnvWatchWindow = "APP_DEPLOY_WATCH_WINDOW" EnvWatchMaxRestarts = "APP_DEPLOY_WATCH_MAX_RESTARTS" EnvWatchErrorRate = "APP_DEPLOY_WATCH_ERROR_RATE" EnvWatchMinRequests = "APP_DEPLOY_WATCH_MIN_REQUESTS" EnvAutoRollback = "APP_DEPLOY_AUTO_ROLLBACK" EnvRollbackPins = "APP_DEPLOY_ROLLBACK_PINS" EnvWatchInterval = "APP_DEPLOY_WATCH_INTERVAL" )
Environment variables that set the platform-wide defaults.
const ( StagePending = "pending" StageGating = "gating" StageGateFailed = "gate_failed" StageWatching = "watching" StageCleared = "cleared" StageWatchFailed = "watch_failed" StageRolledBack = "rolled_back" )
Stages a release moves through, derived from its gate timeline.
const MaxOverrideWindow = 30 * time.Minute
MaxOverrideWindow caps per-app durations so one app cannot stall the sequential reconcile loop or hold a watch open indefinitely.
Variables ¶
var GateFailureReasons = []string{ ReasonReadinessFailed, ReasonOOMKilledDuringReadiness, ReasonExitedDuringReadiness, ReasonOOMKilledDuringWindow, ReasonExitedDuringWindow, ReasonRestartedDuringWindow, ReasonGateTimeout, }
GateFailureReasons is every reason that means a release never took traffic.
var WatchTripReasons = []string{ReasonWatchCrashloop, ReasonWatchOOMKilled, ReasonWatchExited, ReasonWatchErrorRate}
WatchTripReasons is every reason the post-deploy watch can trip on.
Functions ¶
func Stage ¶
func Stage(events []store.DeployGateEvent) string
Stage folds a timeline (oldest first) into the stage the release is in now. A gate failure followed by a passing retry is gating again.
func StageFailed ¶
StageFailed reports whether stage means the release did not hold up.
func StageTerminal ¶
StageTerminal reports whether nothing more will happen to a release in stage without an operator acting.
func Validate ¶
func Validate(s store.DeploySafetySettings) error
Validate rejects overrides outside the ranges the database enforces.
Types ¶
type Effective ¶
type Effective struct {
// SuccessThreshold is how many consecutive readiness passes a new
// container needs before it may take traffic.
SuccessThreshold int `json:"success_threshold"`
// StableWindow is how long a new container must stay running, without
// a restart or OOM kill, before it may take traffic.
StableWindow time.Duration `json:"stable_window"`
// DrainTimeout is the minimum time the previous release keeps serving
// after cutover, so in-flight requests finish before it stops.
DrainTimeout time.Duration `json:"drain_timeout"`
WatchWindow time.Duration `json:"watch_window"`
WatchMaxRestarts int `json:"watch_max_restarts"`
WatchErrorRate float64 `json:"watch_error_rate"`
WatchMinRequests int `json:"watch_min_requests"`
AutoRollback bool `json:"auto_rollback"`
// RollbackPins is how many past releases per app stay pinned.
RollbackPins int `json:"rollback_pins"`
WatchInterval time.Duration `json:"watch_interval"`
}
Effective is a fully resolved set of gate and watch settings.
func BuiltinDefaults ¶
func BuiltinDefaults() Effective
BuiltinDefaults are the defaults with no environment set.
func DefaultsFromEnv ¶
DefaultsFromEnv reads the APP_DEPLOY_* defaults over BuiltinDefaults. An invalid value keeps the builtin and is reported in warnings.
type GateObservation ¶
type GateObservation struct {
Running bool
OOMKilled bool
ExitCode int
// RestartCount and StartedAt are compared against the values seen
// when the gate began; a zero StartedAt means the runtime cannot tell.
RestartCount int
BaselineRestartCount int
StartedAt time.Time
BaselineStartedAt time.Time
ConsecutivePasses int
SinceStart time.Duration
BudgetExhausted bool
}
GateObservation is what the reconciler saw of a new container on one tick of its stability gate.
type Verdict ¶
type Verdict string
Verdict is the outcome of one evaluation of a gate or watch rule.
const ( VerdictPending Verdict = "pending" VerdictPass Verdict = "pass" VerdictFail Verdict = "fail" )
Verdicts.
func EvaluateGate ¶
func EvaluateGate(o GateObservation, cfg Effective) (Verdict, string)
EvaluateGate decides whether a new container may take traffic yet.
func EvaluateWatch ¶
func EvaluateWatch(o WatchObservation, cfg Effective) (Verdict, string, string)
EvaluateWatch decides whether a live release must be rolled back, has cleared its watch window, or is still being watched.
Source Files