autoscale

package
v0.17.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 6, 2026 License: Apache-2.0 Imports: 6 Imported by: 0

README

Autoscale

The autoscale package holds the primitives shared by Spirit's phase-level thread-count controllers. Two phases currently scale their worker pools at runtime — the copier's read/write pools (issue #831) and the checksum's reader pool (#1087) — and both apply the same control law to different pools. Defining that law once means the two cannot silently drift apart.

Autoscaling is experimental and opt-in via --enable-experimental-autoscaling. Nothing here runs unless it is set.

The zone law

Each tick, a controller classifies the throttler's continuous utilization signal (0 = idle, 1.0 = exactly where the binary hard-stop flips) with Classify:

Utilization Action Move
< LowWatermark (0.4) Grow +1 thread
[LowWatermark, HighWatermark) Hold nothing
[HighWatermark, PanicThreshold) (0.7–1.0) Shed −1 thread
>= PanicThreshold (1.0) Halve halve the pool

The shape is "gentle in the normal regime, abrupt only in emergencies". The full derivation — why additive steps rather than classic AIMD, why the dead band has hysteresis, and why the resting point depends on which side the band is approached from — lives on copier.autoScaler, where it was first worked out. This package holds only the mechanism.

MinVCPUs (4) is part of the law rather than of any phase: the signal's denominator is the instance vCPU count, so below it one thread is half or a third of the whole scale and no dead band is wide enough to rest in. The migration runner enforces it once at setup by disabling autoscaling for the whole migration.

What the controllers share

  • Gate turns one tick's signals (Inputs) into a Plan. It owns the precedence between the zones, a caller-supplied veto, and the cooldown bookkeeping — the part most likely to drift if each phase kept its own copy, and the hardest to notice when it does. Precedence, highest first: Halve, then the veto, then Shed, Grow, and recovery inside the dead band.

    Decide is pure; the caller reports back with Applied or Idle. That split exists because a controller may legitimately decline a permitted plan — the copier does not grow a balanced pipeline — and must not burn a cooldown for a move it never made.

    Increases and decreases hold independent cooldowns. A decrease also arms the increase cooldown (so a shed is not immediately undone by a signal that has not yet reflected the cut), but not the reverse: a fresh overload must be answerable at once, even right after the increase that likely caused it.

  • Ceiling resolves a scalable pool's upper bound: the start value when scaling is off, twice it when on. This bounds threads, not connections — the connection pool is --max-connections and does not grow to meet a ceiling, so workers scaled past it queue on checkout instead of each being guaranteed a connection.

  • ReadBounds derives the read side's starting size and ceiling from the instance vCPU count — max(2, ceil((vCPUs - VCPUReserve) / 4)) up to ceil(vCPUs / 2), so a 4xlarge reads with 4 workers and may grow to 8, a 24xlarge with 24 growing to 48. This is where the asymmetry with the write side lives: write threads mostly sit parked on a redo-log flush, so a count above the vCPU count is not oversubscription (and the redo-aware signal excludes those waiters), whereas a read thread scanning an in-buffer-pool table is pure CPU and does compete with the application for cores. The read side therefore starts at about a quarter of the instance and earns its way up through the band.

    Unlike Ceiling, the read ceiling is a share of the instance rather than a multiple of the start. That is because of the checksum: its snapshot transactions must all take their read view at one instant, so the entire pool is created serially under the table lock whether or not scaling reaches it. The ceiling is spent up front, in lock time, which is why it stops at half the box. (For most real instance sizes — any multiple of 4 above MinVCPUs — the two formulas happen to agree, but they are not the same rule and should not be collapsed.)

    Both bounds come from the instance rather than from --threads. When autoscaling engages, the migration runner ignores --threads and --write-threads entirely: a controller told to find the right size should not also be told where to stop, and those flags are usually left at their defaults. docs/migrate.md has the sizing worked out per instance type.

  • FlushBounds derives the change feed's drain shape from the instance — a (concurrency, batch size) pair rather than a start-and-ceiling, because the flush is not steered by the utilization band. It has its own AIMD controller keyed on lock contention, so the instance only sets where that controller starts: max(MinFlushConcurrency, WriteStart(vCPUs)) capped at MaxFlushConcurrency, paired with whatever batch size holds FlushRowsInFlight rows in flight.

    The product being constant is the whole point. A larger instance buys more concurrent REPLACE statements, each holding proportionally fewer row locks — not more rows in flight at once. That is what makes widening past the historical concurrency of 8 safe rather than a throughput-for-deadlocks trade: a flush batch takes a next-key lock per row per UNIQUE secondary index, so two batches collide when any of their rows land in adjacent slots of any such index, and the chance of that is set by how many slots each statement claims, not by how many siblings it has. 32 × 250 and 8 × 1000 push the same rows and the former holds a quarter of the locks per statement. TestReplaceContendsOnlyOnUniqueIndexes in pkg/applier establishes the premise against a real server: rows adjacent in the primary key do not contend (a REPLACE's clustered-index conflict is with an exact PK, so under READ COMMITTED it is a record lock with no gap), rows adjacent in a UNIQUE secondary index do.

    MaxFlushConcurrency is not an independent judgement: it is exactly FlushRowsInFlight / MinFlushBatchSize, the widest flush that can still hold the invariant. MinFlushConcurrency is the historical change.DefaultFlushConcurrency, so every instance below 4xlarge receives precisely the pre-derivation pair and this mechanism is a no-op there. Deriving downwards was never the goal — the contention controller already narrows a flush that is actually colliding, and it does so from evidence rather than from a core count.

    FlushRowsInFlight is a bare 8000 because this package cannot name change.DefaultFlushConcurrency × change.DefaultBatchSize (pkg/change imports this one). TestFlushBoundsPreservesChangeDefaults in pkg/migration — which can see both — pins the agreement.

  • ClientCeiling bounds every derivation above by spirit's own CPU: ClientThreadsPerCore (16) × GOMAXPROCS, so a container CPU limit is respected rather than the machine size. It is a ceiling only — callers min() with the target-derived size and never scale up to it, since a fast client does not justify more workers than the target can absorb. It applies to the growth ceilings as well as the starts: capping a start while letting growth walk past it would just re-arrive at an unrunnable count, one step every 15s.

    This is the one place a derivation looks at something other than the instance, and it is here because the premise the others rest on — that a worker is mostly waiting on the server — fails quietly when spirit is the small side. A write worker also builds its INSERT client-side, a datum conversion and a string format per value; measured against a 96-vCPU target that was ~60% of a worker's cycle even on a 16-core host. Spirit on a 4-core pod derives 94 write threads, progresses about 7 threads' worth, and spends the rest on queueing — while the target's CPU and commit latency both read idle, so no server-side signal can report it.

    16 per core is permissive on purpose, and the ratio has to sit well above the healthy operating point rather than near it. That same 16-core host ran ~99 write workers — 6.2 per core — without saturating local CPU, so a cap of 8 per core would have clipped its write pool's room to grow (188 → 128) while every signal read healthy. 16 leaves ~2.6x headroom over the measured point and still cuts the 4-core pod by a third. A 16-core host clears every derivation up to 128 vCPUs, growth included; the largest size in the table (192 vCPUs, write ceiling 380) needs 24 cores to be fully unconstrained.

  • RunTicker and Emit are the loop and the best-effort gauge send every controller repeats. A controller must never stall or fail a migration because a metrics sink is unavailable, so Emit bounds the send and logs failures at Debug.

  • Limiter is a concurrency gate whose limit can change while work is in flight. errgroup.SetLimit cannot: the errgroup contract forbids changing the limit while any goroutine in the group is active, so a phase that wants to be resized mid-pass needs its own gate. Shrinking never interrupts in-flight work — for the checksum a cancelled chunk is wasted I/O that has to be redone — the reduction is absorbed by subsequent releases instead.

What stays with each phase

Everything genuinely phase-specific: how big a step is, which pool it lands on, and what may veto one.

  • Copier (pkg/copier) has two pools fed by one signal, so utilization alone cannot say which to grow. The applier queue between them arbitrates: starved → readers are the bottleneck, full → writers are. A balanced pipeline holds. The write side additionally refuses to grow when the redo-aware Aurora signal has no commit-latency backstop (throttler.ResolveMaxWriteThreads).
  • Checksum (pkg/checksum) has one pool and no arbitration, plus a veto the utilization signal cannot see: the change feed's post-flush residual. A feed losing ground means the checksum's reads are winning a race against writes that actually have to finish, so a worker is shed — on stock MySQL as well as Aurora, where there is no continuous signal at all.

See Also

  • pkg/throttler — the source of the continuous signal (GradualThrottler) and of the binary hard-stop underneath all of this
  • pkg/copier — the write/read autoscaler and the law's derivation
  • pkg/checksum — the checksum controller and its backlog veto
  • docs/migrate.md — operator-facing documentation for --enable-experimental-autoscaling

Documentation

Overview

Package autoscale holds the primitives shared by spirit's phase-level thread-count controllers.

Everything here is needed by more than one phase and belongs to none of them:

  • The utilization zone law: the watermark/cooldown constants, Classify, and MinVCPUs (the instance size below which the signal is too coarse for the law to work at all). The copier's write/read autoscaler and the checksum controller apply the same law to different pools; defining the thresholds once means they cannot silently drift apart.
  • Gate (in controller.go), which turns one tick's signals into a Plan. It owns the precedence between the zones, a caller-supplied veto, and the cooldown bookkeeping — the part most likely to drift if each phase kept its own copy, and the hardest to notice when it does.
  • The plumbing every controller repeats: Ceiling for a scalable pool's upper bound, RunTicker for the tick loop, Emit for best-effort gauges.
  • Limiter, a concurrency gate whose limit can change while work is in flight. errgroup.SetLimit cannot: the errgroup contract forbids modifying the limit while any goroutine in the group is active, so a phase that wants to be resized mid-pass needs its own gate.

What stays with each phase is what is genuinely phase-specific: how big a step is, which pool it lands on, and what may veto one. The copier apportions a step between its read and write pools using the applier queue; the checksum has one pool and a change-feed backlog veto.

The law's rationale — why additive steps with a multiplicative panic backoff, and why the dead band has hysteresis — is documented at length on the copier's autoScaler, which was where it was first derived (issue #831). This package deliberately holds only the mechanism, not that history.

Index

Constants

View Source 
const (
	// LowWatermark is the effective setpoint: below it there is headroom, so a
	// pool may add a thread (subject to cooldown).
	LowWatermark = 0.4
	// HighWatermark starts the additive back-off. The dead band between the
	// watermarks must be wider than the utilization step of a single thread,
	// otherwise one +1 can vault across the band and ping-pong with the -1
	// path.
	HighWatermark = 0.7
	// PanicThreshold is where back-off turns multiplicative. At 1.0 the
	// smoothed signal has reached the throttle point, where the binary
	// hard-stop is typically already firing on raw samples — so halving is
	// about resuming gently, not about stopping the bleeding.
	PanicThreshold = 1.0
	// CooldownTicks is how many ticks a direction holds after a change before
	// it may fire again, giving the change time to register in the signal.
	// Increases and decreases hold independent cooldowns.
	CooldownTicks = 2
	// MinVCPUs is the smallest instance size (in vCPUs) on which a controller is
	// allowed to engage at all. It is a property of the law above rather than of
	// any one phase: the utilization signal's denominator is the vCPU count, so
	// below this one thread is half or a third of the whole scale and no dead band
	// is wide enough to rest in — the controller can only oscillate. Observed in
	// staging on r6g.large (2 vCPUs): the write-thread count ping-ponged 1↔2
	// indefinitely (issue #831). At 4+ vCPUs the worst-case per-thread step (0.25)
	// fits inside the dead band.
	//
	// The migration runner enforces it once, at setup, by disabling autoscaling
	// for the whole migration; the controllers themselves never see a small
	// instance.
	MinVCPUs = 4

	// VCPUReserve is how many vCPUs a pool sized from the instance leaves free,
	// so spirit never nominally claims the whole server: the other pool, the
	// server's own background work, and the application all need room. Both the
	// write pool (max(1, vCPUs-VCPUReserve)) and the read-side starting point
	// (see ReadBounds) subtract it.
	VCPUReserve = 2

	// MinReadStartThreads is the floor on a read-side pool's starting size.
	// ReadBounds' divisor drives small instances down to 1, which would make the
	// copy single-threaded until the controller has ramped for 15s a step; two
	// threads is the smallest start that still overlaps read and apply work from
	// the first chunk.
	MinReadStartThreads = 2

	// FlushRowsInFlight is how many buffered rows one change-feed drain has
	// outstanding across all of its concurrent REPLACE statements. FlushBounds
	// holds it constant across instance sizes, trading batch size for
	// concurrency rather than adding rows.
	//
	// Its value is the historical change.DefaultFlushConcurrency ×
	// change.DefaultBatchSize, so an instance small enough to hit the floors
	// below gets exactly what it got before this derivation existed. This
	// package cannot import pkg/change (change imports it), so the agreement is
	// pinned by TestFlushBoundsPreservesChangeDefaults over in pkg/migration,
	// which can see both.
	FlushRowsInFlight = 8000

	// MinFlushConcurrency is the floor on a derived flush width, equal to the
	// historical change.DefaultFlushConcurrency. Deriving downwards was never
	// the goal: the AIMD controller already narrows a flush that is actually
	// contending, and it does so from evidence rather than from a core count.
	MinFlushConcurrency = 8

	// MinFlushBatchSize is the floor on a derived batch size. Below roughly this
	// many rows a REPLACE spends more of its life on the round trip than on the
	// rows it carries, so splitting further stops buying a smaller lock
	// footprint and starts buying only statements. It is deliberately well above
	// the AIMD controller's own floor (change.minAdaptiveBatchSize, 50), which
	// is a distress value reached only after four contention steps and not a
	// sane starting point.
	MinFlushBatchSize = 250

	// MaxFlushConcurrency caps the derived flush width. It is not an independent
	// judgement — it is exactly where FlushRowsInFlight meets MinFlushBatchSize,
	// i.e. the widest flush that can still hold the rows-in-flight invariant.
	// Past this point more concurrency would mean more rows in flight, which is
	// the trade FlushBounds exists to avoid making.
	MaxFlushConcurrency = FlushRowsInFlight / MinFlushBatchSize
)
View Source 
const ClientThreadsPerCore = 16

ClientThreadsPerCore bounds how many workers spirit will run per core of its *own* host, independent of how large the target is.

Every derivation in this file sizes pools from the target instance, on the premise that a worker is mostly waiting on the server. That premise fails quietly when spirit itself is small: a worker also builds its INSERT statement client-side (a datum conversion and a string format per value), and that part is pure local CPU. Measured on a 96-vCPU target, roughly 60% of a write worker's cycle was client-side even on a 16-core host — so the useful worker count is closer to a small multiple of local cores than to the target's vCPU count.

16 is deliberately permissive rather than tuned. The failure it exists to prevent is the order-of-magnitude one: spirit on a 4-core pod deriving 94 write threads from a 96-vCPU target, where ~7 threads' worth of work actually progressed and the other 87 only added queueing and latency. That is ~23 threads per core; capping it at 64 still cuts it by a third, while a 16-core host (256) clears every derivation for targets up to 128 vCPUs, growth ceilings included. The growth ceiling is what binds, not the start: the largest size in the docs' table (192 vCPUs -> 190 write threads growing to 380) needs 24 cores to be fully unconstrained. TestClientCeiling pins both.

The ratio has to sit well above the healthy operating point, not near it. Measured on 16 cores: ~99 write workers, 6.2 per core, with local CPU not saturated — so a cap of 8 per core would begin binding on a host that was still keeping up, and would clip the write pool's room to grow (188 -> 128) while every signal read healthy. 16 leaves ~2.6x headroom over the observed point. A ratio tight enough to be a real sizing input would need the client-side share measured live, which is a feedback loop rather than a constant.

View Source 
const Tick = 5 * time.Second

Tick is how often a controller should re-evaluate. Aligned with the throttler poll interval — sampling faster than the signal updates just adds noise. Callers copy this into their own var so tests can shorten it without racing other packages.

Variables

This section is empty.

Functions

func CeilDiv 

func CeilDiv(n, d int) int

CeilDiv returns ceil(n/d) for positive integers — used for multiplicative halving so that, e.g., 3 backs off to 2 rather than 1.

func Ceiling 

func Ceiling(start int, enabled bool) int

Ceiling resolves the upper bound of a scalable pool: the start value when scaling is disabled (so the pool cannot move), twice it when enabled.

It is here rather than in either phase because three copies of "2 ×" invite drift, and because the migration runner resolves the same ceilings itself to hand back to the phases. Callers with an extra rule of their own wrap this rather than reimplementing it (see throttler.ResolveMaxWriteThreads, which additionally refuses to grow the write pool when the redo-aware signal has no commit-latency backstop).

This is a thread ceiling and not a connection budget. Threads scaled past the size of the sql.DB pool they share queue on checkout and buy no parallelism; the pool is --max-connections and does not grow to meet them.

The floor of 1: a zero or negative start would hand a phase that always runs at least one worker a ceiling it already exceeds, which its controller has no way to steer down from.

func ClientCeiling 

func ClientCeiling() int

ClientCeiling returns the largest worker count this process can usefully run, from GOMAXPROCS rather than the machine's core count so that a container CPU limit is respected (Go 1.25+ derives GOMAXPROCS from the cgroup quota).

It is a ceiling only: callers take min() with the target-derived size and never scale *up* to it. A fast client does not justify more workers than the target can absorb.

func Emit 

func Emit(ctx context.Context, sink metrics.Sink, logger *slog.Logger, values ...metrics.MetricValue)

Emit sends one tick's gauges, or does nothing when sink is nil. The send is bounded by metrics.SinkTimeout and a failure is logged at Debug: a controller must never stall or fail a migration because a metrics sink is unavailable.

Callers compose their own value list, because which gauges are meaningful differs per phase — notably, a phase with no continuous signal omits utilization rather than reporting a hard zero, which on a dashboard would read as "completely idle" instead of "not measured".

func FlushBatchSize added in v0.17.0 

func FlushBatchSize(concurrency int) int

FlushBatchSize returns the batch size that pairs with the given flush concurrency to hold FlushRowsInFlight rows in flight, floored at MinFlushBatchSize. FlushBounds returns both halves together; this exists for the caller that has already clipped the concurrency FlushBounds derived (the migration runner applies ClientCeiling to it) and needs the batch size re-paired to the number it is actually going to use. Re-pairing rather than keeping the original batch size is the point: a narrower flush wants *larger* batches to push the same rows, and the collision cost of that is the cost the pre-derivation code already paid.

func FlushBounds added in v0.17.0 

func FlushBounds(vCPUs int) (concurrency, batchSize int)

FlushBounds returns the concurrency and batch size a change-feed flush should use on an instance of the given vCPU count. Unlike the read and write pools these are not autoscaled at runtime — the flush path has its own AIMD controller, driven by lock contention rather than by CPU — so this is a starting point only, and the AIMD penalty shifts both terms down from here.

The two returned values are not independent. Their product is the number of rows a drain has in flight, and it is held at FlushRowsInFlight regardless of instance size: a larger instance buys *more statements*, not more rows at once. That is the whole point, and it is why this can be widened past the historical concurrency of 8 without re-opening the deadlocks that made the AIMD controller necessary in the first place.

The reason the trade is free is that a REPLACE's collision risk scales with its own lock footprint, not with how many siblings it has. A flush batch takes a next-key lock per row per UNIQUE secondary index, so two batches collide when any of their rows land in adjacent slots of any such index; the chance of that is set by how many slots each statement claims. Concurrency only sets how many claims are outstanding at once. So 32x250 and 8x1000 push the same rows per unit time, but the wide-and-narrow form holds a quarter of the locks per statement and is correspondingly less likely to collide — strictly safer than what it replaces, not a risk traded for throughput. (TestReplaceContendsOnlyOnUniqueIndexes in pkg/applier establishes the premise: PK-adjacent rows do not contend, UNIQUE-secondary-adjacent rows do.)

Callers must have already established that the instance is at least MinVCPUs; below that they should not call this at all and the change package's defaults apply. Small instances get today's values anyway, because the concurrency floor is the historical default.

func ReadBounds 

func ReadBounds(vCPUs int) (start, ceiling int)

ReadBounds returns the starting size and ceiling for a read-side pool — the copier's read workers and the checksum's workers — on an instance of the given vCPU count: start at about a quarter of the instance, grow to at most half of it. Callers must have already established that the instance is at least MinVCPUs; below that no controller engages at all.

This is deliberately not the write side's shape (start at vCPUs-VCPUReserve, grow to 2x that), because the two pools are limited by different things. Write threads spend most of their life parked on a redo-log flush, so a count above the vCPU count is not oversubscription — it is what keeps the log busy, and it is why the redo-aware load signal excludes those waiters. A read thread scanning a table that is already in the buffer pool is pure CPU, so the same count really does compete with the application for cores: oversubscribing here is how a checksum ends up degrading the workload it was supposed to be invisible to.

The ceiling is a fixed share of the instance rather than a multiple of the start because for the checksum it is not a hypothesis, it is a cost paid up front: the snapshot transactions must all take their read view at the same instant, so the whole pool is created serially under the table lock whether or not scaling ever reaches it. Half the box is the most that is worth holding a cutover-class lock to reserve, and it leaves the other half to the workload spirit is supposed to be invisible to.

At exactly MinVCPUs the two bounds meet (start 2, ceiling 2), so the read side can shed but not grow. That is the intended reading of a 4-vCPU instance: two readers is already half of it.

func RunTicker 

func RunTicker(ctx context.Context, interval time.Duration, fn func(context.Context))

RunTicker drives fn every interval until ctx is cancelled. Controllers keep their tick step as a separate method so tests can drive it directly without real time; this is only the loop around it.

func WriteStart 

func WriteStart(vCPUs int) int

WriteStart returns the starting size for the apply (write) pool on an instance of the given vCPU count: the whole instance less VCPUReserve, floored at one.

There is no matching WriteCeiling here because the write side's upper bound is not purely a function of the instance — it also depends on which load signal the throttler picked and whether the commit-latency backstop is armed. That rule lives with the throttler, in ResolveMaxWriteThreads.

Types

type Action 

type Action int

Action is the decision the zone law reaches for one sample of the utilization signal. 

const (
	// Hold means the signal is inside the dead band: change nothing.
	Hold Action = iota
	// Grow means there is headroom to add one thread.
	Grow
	// Shed means soft overload: remove one thread.
	Shed
	// Halve means the signal has reached the throttle point: multiplicative
	// backoff.
	Halve
)

func Classify 

func Classify(util float64) Action

Classify maps one utilization sample onto the zone law: 

util < LowWatermark                    Grow
[LowWatermark, HighWatermark)          Hold
[HighWatermark, PanicThreshold)        Shed
util >= PanicThreshold                 Halve

It is deliberately pure and cooldown-free: callers own the cooldown state, because which cooldown gates which action differs between the write-only, dual-pool, and checksum controllers.

type Gate 

type Gate struct {
	// contains filtered or unexported fields
}

Gate owns the cooldown bookkeeping every controller needs and turns Inputs into a Plan. It is a value type; the zero value is ready to use.

Increases and decreases hold independent cooldowns. That asymmetry is load-bearing: a fresh overload must be able to shed immediately even right after an increase — which likely caused it — while consecutive decreases still space themselves out far enough for the signal to reflect the previous cut.

func (*Gate) Applied 

func (g *Gate) Applied(p Plan)

Applied arms the cooldowns for a plan the controller carried out. Any decrease arms the up cooldown too, so a shed is not immediately undone by a growth signal that has not yet had time to reflect the cut.

func (*Gate) Decide 

func (g *Gate) Decide(in Inputs) Plan

Decide resolves one tick. It is pure: the caller records the outcome with Applied or Idle, because whether a permitted plan was actually carried out is the controller's business (the copier declines to grow a balanced pipeline even when the zone law allows it, and must not burn a cooldown for a move it did not make).

Precedence, highest first:

  1. Halve. Both this and a veto shed, and this sheds faster, so it has to come first — otherwise a tick spent inside the veto's cooldown would consume the panic response and downgrade the backoff to one thread per cooldown at exactly the moment the server is most overloaded.
  2. The veto, which therefore outranks Shed, Grow and Hold. Utilization can show plenty of headroom while a caller-visible prerequisite falls behind.
  3. The zone law: Shed, then Grow, then recovery inside the dead band.

func (*Gate) Idle 

func (g *Gate) Idle()

Idle records a tick on which nothing changed — the dead band, a declined plan, or a cooldown still running — and decays the cooldowns toward zero.

type Inputs 

type Inputs struct {
	// Zone is the utilization verdict, from Classify. A controller with no
	// continuous signal passes Hold, which leaves recovery available but rules
	// out growth.
	Zone Action
	// Veto, when set, sheds a thread and outranks the zone law's Shed, Grow and
	// Hold — but not Halve. It is for pressure the utilization signal cannot
	// see: the checksum's change-feed backlog is a hard prerequisite for
	// finishing a migration while contributing almost nothing to server load.
	Veto bool
	// GrowthBlocked suppresses PlanGrow and PlanRecover without shedding
	// anything. Two cases need it: a veto that has already been paid for this
	// window (otherwise the intervening ticks would grow straight back into the
	// shed and the two signals would cancel out), and a veto signal that has
	// gone stale, where freezing is right but shedding on no evidence is not.
	GrowthBlocked bool
	// CanRecover enables PlanRecover — typically "the live count is below the
	// configured start".
	CanRecover bool
}

Inputs is one tick's worth of signals for Gate.Decide.

type Limiter 

type Limiter struct {
	// contains filtered or unexported fields
}

Limiter is a counting semaphore whose limit may change while permits are held. It is the resizable stand-in for errgroup.SetLimit, which may not be modified while goroutines in the group are active.

Lowering the limit never interrupts work in flight: the reduction is absorbed by subsequent Releases, so callers holding a permit always run to completion. That property is what makes shedding cheap for phases whose unit of work is expensive to redo — a cancelled chunk is wasted I/O, a parked worker is not.

The zero value is not usable; call NewLimiter.

func NewLimiter 

func NewLimiter(limit int) *Limiter

NewLimiter returns a Limiter admitting at most limit concurrent holders. A limit below 1 is raised to 1 — a limiter that admits nobody would deadlock its caller rather than throttle it.

func (*Limiter) Acquire 

func (l *Limiter) Acquire(ctx context.Context) error

Acquire blocks until a permit is available or ctx is done, returning ctx.Err() in the latter case. On success the caller must call Release exactly once.

func (*Limiter) InFlight 

func (l *Limiter) InFlight() int

InFlight reports how many permits are currently held. It can exceed Limit transiently, right after a reduction that in-flight holders have not yet absorbed.

func (*Limiter) Limit 

func (l *Limiter) Limit() int

Limit reports the current permit ceiling.

func (*Limiter) Release 

func (l *Limiter) Release()

Release returns a permit. It must be called exactly once per successful Acquire.

func (*Limiter) SetLimit 

func (l *Limiter) SetLimit(n int)

SetLimit changes the number of permits. Raising it wakes waiters; lowering it takes effect as holders release, never by interrupting them. A limit below 1 is raised to 1, matching NewLimiter.

type Plan 

type Plan int

Plan is what a controller is permitted to do on one tick: the zone law's verdict after cooldowns and any caller-supplied veto have been applied.

It says what kind of move is allowed, not how big it is or which pool it lands on. Those stay with the controller: the copier apportions a step between its read and write pools using the applier queue, while the checksum has a single pool and no arbitration to do. 

const (
	// PlanNone means nothing may change this tick — the dead band, or a
	// cooldown still running, or growth suppressed with nothing to shed.
	PlanNone Plan = iota
	// PlanHalve is the multiplicative backoff of the panic zone.
	PlanHalve
	// PlanShed is the additive decrease of the soft-overload zone.
	PlanShed
	// PlanShedVeto is an additive decrease demanded by the caller's veto rather
	// than by utilization. It is distinguished from PlanShed so the controller
	// can log the real reason, and so a controller that must record having paid
	// for the veto can tell the two apart.
	PlanShedVeto
	// PlanGrow is the additive increase of the headroom zone.
	PlanGrow
	// PlanRecover is an additive increase back toward the configured start
	// value, offered inside the dead band. It exists for phases that shed for a
	// reason unrelated to utilization and would otherwise sit on the reduced
	// count for the rest of the run — a signal saying "no longer overloaded" is
	// enough to undo such a shed, whereas going *past* the operator's configured
	// count requires positive headroom (PlanGrow).
	PlanRecover
)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.