Documentation
¶
Overview ¶
Package leasing runs scan batches across processes. A batch (kept in a catalog.BatchStore) is leased by one run of it at a time, which beats while it runs: it goes on when the console that started it is closed, every replica shows it, and when its owner stops beating, as on a restart, another process takes it over and does what is left. Every write of a run moves one item from the state it expects, on the condition that the run still owns the unfinished batch (catalog.BatchStore.MoveItem), so runs never overwrite each other. An item that fails is tried again after a backoff, a few times.
Index ¶
- type Options
- type Resolver
- type Runner
- func (r *Runner) Close()
- func (r *Runner) Owner() string
- func (r *Runner) Running(id string) bool
- func (r *Runner) Start(ctx context.Context, b catalog.Batch, maxOpen int, work Work) (catalog.Batch, bool, error)
- func (r *Runner) Stop(id string)
- func (r *Runner) TakeOver(ctx context.Context, resolve Resolver)
- func (r *Runner) Watch(resolve Resolver)
- type Work
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Options ¶
type Options struct {
// Parallel bounds the items of one batch run at once (2).
Parallel int
// Beat is how often a run records that it still runs a batch (5s); a
// batch whose owner missed Stale (30s) is taken over.
Beat, Stale time.Duration
// Attempts bounds the runs of an item (3); Backoff is the wait before
// the second (5s), doubling for each one after.
Attempts int
Backoff time.Duration
// MoveTries bounds the writes of one move when the store fails (4); the
// first retry waits MoveBackoff (100ms), doubling after.
MoveTries int
MoveBackoff time.Duration
// Now is the clock (time.Now); Bus runs the goroutines (nil runs them
// plain).
Now func() time.Time
Bus *bus.Bus
}
Options tune a Runner; a zero value takes its default.
type Runner ¶
type Runner struct {
// contains filtered or unexported fields
}
Runner runs the batches this process owns.
func New ¶
func New(store catalog.BatchStore, opts Options) *Runner
New is a Runner of the batches in store.
func (*Runner) Close ¶
func (r *Runner) Close()
Close stops the batches this process runs (they stay unfinished, for a takeover) and waits for them.
func (*Runner) Start ¶
func (r *Runner) Start(ctx context.Context, b catalog.Batch, maxOpen int, work Work) (catalog.Batch, bool, error)
Start creates b, its ID, owner and times set here, and runs it; false with the batch started before by b's RequestKey instead, which is not run again. catalog.ErrTooManyBatches when b's datasource has maxOpen unfinished batches.
Source Files