Documentation
¶
Overview ¶
Package appsleep stops opted-in apps that have had no requests for a while and wakes them on the next request, so idle apps cost no memory or CPU.
Index ¶
- Constants
- Variables
- func EffectiveIgnorePaths(svc *store.DesiredService, custom []string) []string
- func NetworkActivity(samples []NetSample, floorPerMin int64, maxGap time.Duration) time.Time
- func PinnedPorts(svc *store.DesiredService) []int
- func RealRequests(total float64, routes []telemetry.RouteRow, ignore []string) float64
- func Record(ctx context.Context, es EventStore, logger *slog.Logger, e store.SleepEvent) int64
- func Sleep(ctx context.Context, sleep interface{ ... }, apps Apps, nudger Nudger, ...) error
- func Wake(ctx context.Context, sleep interface{ ... }, apps Apps, nudger Nudger, ...) (bool, error)
- func WokeEvent(name, reason, actor string, slept, now time.Time) store.SleepEvent
- type ActiveOps
- type Apps
- type Breakdowns
- type Config
- type ConnCounter
- type DecideInput
- type Decision
- type Eligibility
- type EventStore
- type Facts
- type Fix
- type Fleet
- type FleetStore
- type Item
- type Metrics
- type NetSample
- type Nudger
- type PortSyncer
- type PortWakeManager
- func (m *PortWakeManager) Open(_ context.Context, name string, ports []int) int
- func (m *PortWakeManager) Run(ctx context.Context) error
- func (m *PortWakeManager) Status(name string) PortWakeStatus
- func (m *PortWakeManager) Supported(svc *store.DesiredService) (bool, string)
- func (m *PortWakeManager) Sync(ctx context.Context)
- type PortWakeStatus
- type PortWakeStore
- type PortWaker
- type Scheduler
- type Signals
- type SleepStore
- type Snapshot
- type Traffic
Constants ¶
const ( ReasonIdle = "idle" ReasonRecentUse = "recent_activity" ReasonConnections = "open_connections" ReasonOperation = "operation_in_flight" ReasonCooldown = "wake_cooldown" ReasonUnobserved = "window_not_observed" )
Idle decision reasons, shown in logs and used by tests.
const ( // EnvNetworkFloor is the container network traffic, in bytes per minute // (rx plus tx), below which an app still counts as idle. EnvNetworkFloor = "APP_SLEEP_NETWORK_FLOOR_BYTES_PER_MIN" // EnvWakeCooldown is how long after a wake an app cannot be put back to sleep. EnvWakeCooldown = "APP_SLEEP_WAKE_COOLDOWN" // EnvPortWakeTimeout bounds how long a held connection waits for the app. EnvPortWakeTimeout = "APP_SLEEP_PORT_WAKE_TIMEOUT" // EnvPortWakeRetry is how often a missing port wake listener is retried. EnvPortWakeRetry = "APP_SLEEP_PORT_WAKE_RETRY" // EnvPortWakeCollect is how long the listener keeps accepting after the // first connection, so a burst of clients is held rather than refused. EnvPortWakeCollect = "APP_SLEEP_PORT_WAKE_COLLECT" // EnvPortWakeMinPort is the lowest host port port wake will ever bind. EnvPortWakeMinPort = "APP_SLEEP_PORT_WAKE_MIN_PORT" // EnvIdleMin, EnvIdleMax and EnvIdleDefault bound and default the idle // time, in minutes. EnvIdleMin = "APP_SLEEP_IDLE_MIN_MINUTES" EnvIdleMax = "APP_SLEEP_IDLE_MAX_MINUTES" EnvIdleDefault = "APP_SLEEP_IDLE_DEFAULT_MINUTES" // EnvHoldMin and EnvHoldMax are the floor and cap of how long a request is // held while its app wakes; EnvHoldDefault applies with no wake history and // EnvHoldHeadroom multiplies the measured p95 wake. EnvHoldMin = "APP_SLEEP_HOLD_MIN" EnvHoldMax = "APP_SLEEP_HOLD_MAX" EnvHoldDefault = "APP_SLEEP_HOLD_DEFAULT" EnvHoldHeadroom = "APP_SLEEP_HOLD_HEADROOM" // EnvWakeProbeInterval is the readiness probe interval while an app wakes. EnvWakeProbeInterval = "APP_SLEEP_WAKE_PROBE_INTERVAL" )
Environment variables that tune sleep detection and port wake.
const ( DefaultNetworkFloor int64 = 4096 DefaultWakeCooldown = 5 * time.Minute DefaultPortWakeTimeout = 30 * time.Second DefaultPortWakeRetry = 2 * time.Second DefaultPortWakeCollect = 250 * time.Millisecond DefaultPortWakeMinPort = 1024 DefaultIdleMin = 5 DefaultIdleMax = 7 * 24 * 60 DefaultIdleDefault = 30 DefaultHoldMin = 10 * time.Second DefaultHoldMax = 120 * time.Second DefaultHoldDefault = 30 * time.Second DefaultHoldHeadroom = 1.5 DefaultWakeProbeInterval = 250 * time.Millisecond )
Defaults for the variables above. 4096 bytes per minute sits above keepalives and probes (a few hundred bytes) and below real use.
const ( CodePinnedPort = "pinned_port" CodePortInUse = "port_wake_port_in_use" CodeDatabase = "database" CodeNodeUnschedulable = "node_unschedulable" CodeCanaryTarget = "canary_target" CodeNoDomain = "no_domain" CodeDirectTraffic = "direct_traffic_only" CodeBackgroundWork = "background_work_invisible" CodeTrial = "trial_deployment" CodeReplicas = "multiple_replicas" CodeDependency = "dependency_of_other_apps" CodeStatefulImage = "stateful_image" CodeLastWakeFailed = "last_wake_failed" )
Blocker and warning codes returned by Evaluate.
const ( FixEnablePortWake = "enable_port_wake" FixRemovePinnedPort = "remove_pinned_port" FixSetHoldRequests = "set_hold_requests" FixAttachDomain = "attach_domain" )
Fix kinds a suggested fix can carry.
const ( SourceHealthProbe = "health probe" SourceBot = "bot" )
Probe and bot labels for request wakes that look automated.
const DefaultInterval = time.Minute
DefaultInterval is how often idle apps are looked for.
Variables ¶
var DefaultIgnorePaths = []string{"/health", "/healthz", "/ready", "/readyz", "/livez", "/ping", "/status", "/up"}
DefaultIgnorePaths are request routes that never count as traffic: health checks, readiness endpoints and uptime or status page probes.
Functions ¶
func EffectiveIgnorePaths ¶
func EffectiveIgnorePaths(svc *store.DesiredService, custom []string) []string
EffectiveIgnorePaths merges the defaults, the app's readiness path and the operator's list, normalized the way request routes are stored.
func NetworkActivity ¶
NetworkActivity returns the newest time the traffic rate between two consecutive samples exceeded floorPerMin bytes per minute. Counter resets and gaps over maxGap are skipped; samples are oldest first.
func PinnedPorts ¶
func PinnedPorts(svc *store.DesiredService) []int
PinnedPorts lists the host ports svc pins.
func RealRequests ¶
RealRequests subtracts the requests on ignored routes from total. A route breakdown that is missing (nil routes while total > 0) cannot discount anything, so every request counts.
func Record ¶
func Record(ctx context.Context, es EventStore, logger *slog.Logger, e store.SleepEvent) int64
Record stores e, and for slept and woke also an app timeline entry. A failure is logged and never returned: the state change already happened. It returns the history row's ID, 0 when nothing was stored.
func Sleep ¶
func Sleep(ctx context.Context, sleep interface { SetAppSleeping(ctx context.Context, serviceName string, sleeping bool, now time.Time) error }, apps Apps, nudger Nudger, name string, now time.Time) error
Sleep stops name now and marks it asleep, rolling the flag back if the stop fails.
func Wake ¶
func Wake(ctx context.Context, sleep interface { SetAppSleeping(ctx context.Context, serviceName string, sleeping bool, now time.Time) error GetAppSleep(ctx context.Context, serviceName string) (store.AppSleep, error) }, apps Apps, nudger Nudger, name string, now time.Time) (bool, error)
Wake starts a sleeping app and resets its idle clock. It reports whether the app was actually asleep, so repeated wake requests are harmless.
Types ¶
type Apps ¶
type Apps interface {
GetDesiredService(ctx context.Context, name string) (*store.DesiredService, error)
UpdateServiceSuspended(ctx context.Context, name string, suspended bool) error
}
Apps is the slice of the app store the scheduler needs.
type Breakdowns ¶
type Breakdowns interface {
QueryBreakdown(ctx context.Context, resourceID string, from, to time.Time) (telemetry.Breakdown, error)
}
Breakdowns reads per-route request counts, used to discount probe routes.
type Config ¶
type Config struct {
NetworkFloorBytesPerMin int64
WakeCooldown time.Duration
PortWakeTimeout time.Duration
PortWakeRetry time.Duration
PortWakeCollect time.Duration
PortWakeMinPort int
IdleMin int
IdleMax int
IdleDefault int
HoldMin time.Duration
HoldMax time.Duration
HoldDefault time.Duration
HoldHeadroom float64
WakeProbeInterval time.Duration
}
Config holds the environment-driven sleep thresholds.
func ConfigFromEnv ¶
ConfigFromEnv reads the Env* variables, using the defaults for anything missing or malformed.
type ConnCounter ¶
ConnCounter counts established connections to an app's pinned host ports.
type DecideInput ¶
type DecideInput struct {
Now time.Time
IdleMinutes int
// Since is when the app was enabled or last woke.
Since time.Time
// ObservedFrom is when this process started watching; earlier time is unknown.
ObservedFrom time.Time
Cooldown time.Duration
Signals Signals
}
DecideInput is everything Decide needs.
type Decision ¶
type Decision struct {
Sleep bool
Reason string
// LastActivity is the newest sign of use, or the baseline when there is none.
LastActivity time.Time
IdleFor time.Duration
// WillSleepAt is when the app would sleep if nothing else happens, zero when held.
WillSleepAt time.Time
}
Decision is the outcome of one idle check.
func Decide ¶
func Decide(in DecideInput) Decision
Decide says whether an app is idle for its whole window. Idle means no real request, no network traffic above the floor and no open connection since the newest of: enabling, the last wake, and when this process began watching.
type Eligibility ¶
Eligibility is the result of Evaluate. Blockers refuse enabling, warnings do not.
func Evaluate ¶
func Evaluate(f Facts) Eligibility
Evaluate lists why app cannot sleep and what may surprise the operator.
func (Eligibility) Eligible ¶
func (e Eligibility) Eligible() bool
Eligible reports whether sleep can be enabled.
type EventStore ¶
type EventStore interface {
AddSleepEvent(ctx context.Context, e store.SleepEvent) (int64, error)
AddAppEvent(ctx context.Context, e store.AppEvent) error
}
EventStore persists sleep history and mirrors slept and woke into the app timeline.
type Facts ¶
type Facts struct {
App string
// IdleMinutes is the current setting, 0 when off; fixes keep it.
IdleMinutes int
// PinnedPorts are the host ports the app pins; empty means none.
PinnedPorts []int
// PortWakeSupported says Levelrail can hold those ports while the app sleeps.
PortWakeSupported bool
// PortWakeUnsupportedReason explains why not, shown when it matters.
PortWakeUnsupportedReason string
// PortWakeEnabled is the per-app port_wake setting.
PortWakeEnabled bool
// PortWakeError is the last bind failure for the pinned ports, if any.
PortWakeError string
// IsManagedDatabase is true when the app name is a managed database.
IsManagedDatabase bool
// StatefulImage is true for a well known database image.
StatefulImage bool
// NodeUnschedulable is true when the app's node is cordoned or drained.
NodeUnschedulable bool
CanaryTarget bool
// DependedOnBy lists sibling apps that start after, and reach, this one.
DependedOnBy []string
HoldRequests bool
HasDomain bool
HasStreams bool
IsTrial bool
Replicas int
// LastWakeError is the last failed wake, empty when the last wake worked.
LastWakeError string
}
Facts is everything Evaluate needs, gathered by the caller so the decision stays a pure function.
type Fix ¶
type Fix struct {
Kind string `json:"kind"`
Label string `json:"label"`
Method string `json:"method"`
Path string `json:"path"`
Body map[string]any `json:"body,omitempty"`
}
Fix is a one-click remedy: the API call that applies it, or a link when there is nothing to call.
type Fleet ¶
type Fleet struct {
Services map[string]store.DesiredService
// contains filtered or unexported fields
}
Fleet is a snapshot of the facts Evaluate needs, loaded once so listing many apps costs a fixed number of queries.
type FleetStore ¶
type FleetStore interface {
ListDesiredServices(ctx context.Context) ([]store.DesiredService, error)
ListNodes(ctx context.Context) ([]store.Node, error)
ListCanaryReleases(ctx context.Context) ([]store.CanaryRelease, error)
ListDesiredDatabases(ctx context.Context) ([]store.DesiredDatabase, error)
CountAppStreamsByService(ctx context.Context) (map[string]int, error)
}
FleetStore is what Load needs, each call one query for the whole fleet.
type Item ¶
type Item struct {
Code string `json:"code"`
Title string `json:"title"`
Detail string `json:"detail"`
Fix *Fix `json:"fix,omitempty"`
}
Item is one reason an app cannot sleep (blocker) or may sleep badly (warning).
type Metrics ¶
type Metrics interface {
QueryMany(ctx context.Context, resourceIDs, metrics []string, from, to time.Time) ([]telemetry.Sample, error)
}
Metrics reads stored samples for the network activity check.
type NetSample ¶
NetSample is one cumulative rx plus tx byte reading.
func MergeNetwork ¶
MergeNetwork sums rx and tx samples taken at the same instants into one series.
type PortSyncer ¶
PortSyncer makes the port wake listeners match desired state.
type PortWakeManager ¶
type PortWakeManager struct {
Store PortWakeStore
Apps Apps
Nudger Nudger
Events EventStore
Logger *slog.Logger
Cfg Config
Now func() time.Time
// IsLocal says whether nodeID is the node this process runs on; nil treats "" as local.
IsLocal func(nodeID string) bool
// Ready adds an application level readiness check on top of the port answering.
Ready func(ctx context.Context, name string, since time.Time) bool
// Listen and Dial are replaceable for tests.
Listen func(network, addr string) (net.Listener, error)
Dial func(ctx context.Context, network, addr string) (net.Conn, error)
// contains filtered or unexported fields
}
PortWakeManager holds a sleeping app's pinned ports and wakes it on the first connection. The listener is released before the start so Docker can publish the port; Sync rebuilds listeners from stored state, so restarts lose nothing.
func (*PortWakeManager) Open ¶
Open counts connections to name that this process is proxying plus the established ones the kernel reports on the pinned ports.
func (*PortWakeManager) Run ¶
func (m *PortWakeManager) Run(ctx context.Context) error
Run syncs every retry interval until ctx is done.
func (*PortWakeManager) Status ¶
func (m *PortWakeManager) Status(name string) PortWakeStatus
Status reports the listener state for name.
func (*PortWakeManager) Supported ¶
func (m *PortWakeManager) Supported(svc *store.DesiredService) (bool, string)
Supported reports whether svc's pinned ports can be held while it sleeps.
func (*PortWakeManager) Sync ¶
func (m *PortWakeManager) Sync(ctx context.Context)
Sync makes the listeners match the sleeping apps that use port wake.
type PortWakeStatus ¶
type PortWakeStatus struct {
Active bool
Ports []int
// Error is the last bind failure, empty when the listeners are fine.
Error string
}
PortWakeStatus is what the manager knows about one app's pinned ports.
type PortWakeStore ¶
type PortWakeStore interface {
SleepStore
GetAppSleep(ctx context.Context, serviceName string) (store.AppSleep, error)
SetAppSleepWakeError(ctx context.Context, serviceName, msg string, now time.Time) error
}
PortWakeStore is the sleep settings surface port wake reads and writes.
type PortWaker ¶
type PortWaker interface {
// Supported says whether port wake can hold svc's pinned ports, and if not why.
Supported(svc *store.DesiredService) (bool, string)
// Status reports the live listener state for name.
Status(name string) PortWakeStatus
}
PortWaker is the port wake surface eligibility and the API read.
type Scheduler ¶
type Scheduler struct {
Sleep SleepStore
Apps Apps
Traffic Traffic
Nudger Nudger
// Events records slept and woke history; nil skips it.
Events EventStore
Logger *slog.Logger
Now func() time.Time
// Optional signals. Without one, that signal is not consulted.
Metrics Metrics
Breakdowns Breakdowns
Conns ConnCounter
Ops ActiveOps
Fleet FleetStore
PortWake PortWaker
PortSync PortSyncer
Cfg Config
// ObservedFrom starts the observation window; zero means the first tick.
ObservedFrom time.Time
// contains filtered or unexported fields
}
Scheduler puts idle opted-in apps to sleep.
type Signals ¶
type Signals struct {
// LastRequest is the newest real (not ignored) ingress request, zero if none.
LastRequest time.Time
// LastNetwork is the newest moment container traffic was above the floor.
LastNetwork time.Time
// OpenConnections counts established connections to the app right now.
OpenConnections int
// OperationInFlight is true while a deploy, build or rollback runs.
OperationInFlight bool
}
Signals is what was seen for one app in the idle window.
type SleepStore ¶
type SleepStore interface {
ListAppSleep(ctx context.Context) ([]store.AppSleep, error)
SetAppSleeping(ctx context.Context, serviceName string, sleeping bool, now time.Time) error
}
SleepStore is the sleep settings table.