Documentation
¶
Index ¶
Constants ¶
const ( BatchBackoffMax = 1 * time.Second BatchBackoffMin = 50 * time.Millisecond )
Maintainers will sleep a brief period of time between batches to give the database some breathing room.
const ( LogPrefixRanSuccessfully = ": Ran successfully" LogPrefixRunLoopStarted = ": Run loop started" LogPrefixRunLoopStopped = ": Run loop stopped" // TimeoutDefault is a reasonable timeout for any large maintenance-related // queries. Some maintainers may opt to switch to their own timeout, but // this one should generally be used unless there's a good reason to have a // specific version. TimeoutDefault = 30 * time.Second )
const ( CancelledJobRetentionPeriodDefault = 24 * time.Hour CompletedJobRetentionPeriodDefault = 24 * time.Hour DiscardedJobRetentionPeriodDefault = 7 * 24 * time.Hour JobCleanerIntervalDefault = 30 * time.Second JobCleanerTimeoutDefault = 30 * time.Second )
Constants related to JobCleaner.
const ( // BatchSizeDefault is the default batch size of most maintenance services. // // Bulk maintenance tasks like job removal operate in batches so that even // in the event of an enormous backlog of work to do, transactions stay // relatively short and aren't at risk of cancellation. This number is the // batch size, or the number of rows that are handled at a time. // // The specific value is somewhat arbitrary as large enough to make good // progress, but not so large as to make the operation overstay its welcome. // For now it's not configurable because we can likely pick a number that's // suitable for almost everyone. // // In case database degradation is detected, most maintenance services will // back off to use the smaller batch size BatchSizeReduced. BatchSizeDefault = 10_000 // BatchSizeReduced is the reduced batch size of most maintenance services. // // Services start out with a batch size of BatchSizeDefault, but as they // detect a degraded database may switch to BatchSizeReduced instead so that // they're trying to do less work per operation. BatchSizeReduced = 1_000 )
Variables ¶
This section is empty.
Functions ¶
func ReducedBatchSizeBreaker ¶
func ReducedBatchSizeBreaker(batchSizes BatchSizes) *circuitbreaker.CircuitBreaker
ReducedBatchSizeBreaker returns a reduced batch circuit breaker suitable for use in most maintenance services. After being tripped three consecutive times inside a ten minute window it switches to a reduced batch size. A success at any point between failures will reset it, but after the circuit breaker has tripped, it stays tripped for the life time of the program.
Types ¶
type BatchSizes ¶
type BatchSizes struct {
// Default is the maximum number of jobs to transition at once from
// "scheduled" to "available" during periodic scheduling checks.
Default int
// Reduced is a considerably smaller batch size that the service
// uses after encountering 3 consecutive timeouts in a row. The idea behind
// this is that if it appears the database is degraded, then we start doing
// less work in the hope that it can better succeed.
Reduced int
}
BatchSizes containing batch size information for maintenance services. It's mean to be embedded on each service's configuration struct so as to provide a common way of organizing and initializing batch sizes for improved succinctness.
func (BatchSizes) MustValidate ¶
func (b BatchSizes) MustValidate() BatchSizes
MustValidate validates the struct and panics in case a value is invalid.
func (BatchSizes) WithDefaults ¶
func (b BatchSizes) WithDefaults() BatchSizes
WithDefaults returns the struct with any configuration overrides that were already set, but sets defaults for any that were zero values.
type QueueMaintainerServiceBase ¶
type QueueMaintainerServiceBase struct {
baseservice.BaseService
// contains filtered or unexported fields
}
QueueMaintainerServiceBase is a struct that should be embedded on all queue maintainer services. Its main use is to provide a StaggerStart function that should be called on service start to avoid thundering herd problems.
func (*QueueMaintainerServiceBase) StaggerStart ¶
func (s *QueueMaintainerServiceBase) StaggerStart(ctx context.Context)
StaggerStart is called when queue maintainer services start. It jitters by sleeping for a short random period so services don't all perform their first run at exactly the same time.
func (*QueueMaintainerServiceBase) StaggerStartupDisable ¶
func (s *QueueMaintainerServiceBase) StaggerStartupDisable(disabled bool)
StaggerStartupDisable sets whether the short staggered sleep on start up is disabled. This is useful in tests where the extra sleep involved in a staggered start up is not helpful for test run time.
func (*QueueMaintainerServiceBase) StaggerStartupIsDisabled ¶
func (s *QueueMaintainerServiceBase) StaggerStartupIsDisabled() bool
Source Files