Documentation
¶
Overview ¶
Package leasing runs scan batches across processes. A batch (kept in a catalog.BatchStore) is leased by one run of it at a time, which beats while it runs: it goes on when the console that started it is closed, every replica shows it, and when its owner stops beating, as on a restart, another process takes it over and does what is left. Every write of a run moves one item from the state it expects, on the condition that the run still owns the unfinished batch (catalog.BatchStore.MoveItem), so runs never overwrite each other. An item that fails is tried again after a backoff, a few times.
Index ¶
- Variables
- type Options
- type Resolver
- type Runner
- func (r *Runner) Close()
- func (r *Runner) Owner() string
- func (r *Runner) Running(id string) bool
- func (r *Runner) Start(ctx context.Context, b catalog.Batch, maxOpen int, work Work) (catalog.Batch, bool, error)
- func (r *Runner) Stop(id string)
- func (r *Runner) TakeOver(ctx context.Context, resolve Resolver)
- func (r *Runner) Watch(resolve Resolver)
- type Work
Constants ¶
This section is empty.
Variables ¶
var ErrGone = errors.New("leasing: the datasource is no longer configured")
ErrGone is the error of a Resolver for a datasource no longer configured: its batches end (their items left canceled) instead of being taken over again and again, counting against the open batches for good.
Functions ¶
This section is empty.
Types ¶
type Options ¶
type Options struct {
// Parallel bounds the items of one batch run at once (2).
Parallel int
// Beat is how often a run records that it still runs a batch (5s); a
// batch whose owner missed Stale (30s) is taken over.
Beat, Stale time.Duration
// Attempts bounds the runs of an item (3); Backoff is the wait before
// the second (5s), doubling for each one after.
Attempts int
Backoff time.Duration
// MoveTries bounds the writes of one move when the store fails (4); the
// first retry waits MoveBackoff (100ms), doubling after.
MoveTries int
MoveBackoff time.Duration
// Now is the clock (time.Now); Bus runs the goroutines (nil runs them
// plain).
Now func() time.Time
Bus *bus.Bus
}
Options tune a Runner; a zero value takes its default.
type Runner ¶
type Runner struct {
// contains filtered or unexported fields
}
Runner runs the batches this process owns.
func New ¶
func New(store catalog.BatchStore, opts Options) *Runner
New is a Runner of the batches in store.
func (*Runner) Close ¶
func (r *Runner) Close()
Close stops the batches this process runs (they stay unfinished, for a takeover) and waits for them.
func (*Runner) Start ¶
func (r *Runner) Start(ctx context.Context, b catalog.Batch, maxOpen int, work Work) (catalog.Batch, bool, error)
Start creates b, its ID, owner and times set here, and runs it; false with the batch started before by b's RequestKey instead, which is not run again. catalog.ErrTooManyBatches when b's datasource has maxOpen unfinished batches.
func (*Runner) TakeOver ¶
TakeOver runs the unfinished batches whose owner stopped beating: what they did not do yet is done here, and the items they were doing again. resolve says how to work a datasource as configured now. The items of a batch are read once it is claimed, not those of every batch others run.