Documentation
¶
Overview ¶
Package operation recovers server mutations that were interrupted by a panel crash or restart. A disconnected Docker request has an unknown outcome: the operation journal keeps the server locked until startup has observed the container, instead of allowing a blind retry.
Index ¶
- Variables
- func Digest(kind string, server *db.Server) string
- func RetryReview(ctx context.Context, store ReviewStore, observer Observer, log *logger.Logger, ...)
- func RunJournaled(ctx context.Context, store Journal, log *logger.Logger, server *db.Server, ...) error
- func StorageKey(requestID, kind string) string
- func ValidateRequestID(id string) error
- type Journal
- type Observed
- type Observer
- type ReviewStore
- type Store
- type Summary
Constants ¶
This section is empty.
Variables ¶
var ErrAlreadyFailed = errors.New("matching lifecycle operation already failed")
ErrAlreadyFailed is returned when the same request id already failed. A new request id is required to try again.
var ErrInProgress = errors.New("matching lifecycle operation is already in progress")
ErrInProgress is returned when the same request id is already running.
Functions ¶
func RetryReview ¶
func RetryReview(ctx context.Context, store ReviewStore, observer Observer, log *logger.Logger, ids []string, every time.Duration)
RetryReview re-observes operations whose container state was unknown at startup. It stops when every id has been released or ctx is cancelled. Operations that are not in the list are left alone, so an in-flight mutation is never finalized by this loop.
func RunJournaled ¶
func RunJournaled(ctx context.Context, store Journal, log *logger.Logger, server *db.Server, kind, requestID string, mutate func() error) error
RunJournaled persists the intent of a server mutation before it is attempted, advances the journal through the attempt, and finalizes it. requestID is a client idempotency key: a repeat of a completed key does not run mutate again, and a repeat of a key that is still running returns ErrInProgress. An empty requestID is unique to this call.
If the process dies between the external effect and finalization, the entry stays active and startup recovery observes the container before releasing the per-server lock (see RecoverInterrupted). The active lock is the shared coordinator: no two lifecycle mutations of one world can interleave. A mutation error finalizes as failed so the user can retry without restarting the panel.
func StorageKey ¶
StorageKey is the journal key for a client request id and mutation kind. An empty request id is unique to this call, so it is not idempotent.
func ValidateRequestID ¶
ValidateRequestID rejects client request ids that do not fit the journal.
Types ¶
type Journal ¶
type Journal interface {
Store
BeginServerOperation(ctx context.Context, serverID, nodeID, containerID, key, kind, digest string) (*db.ServerOperation, bool, error)
}
Journal is the operation journal subset that RunJournaled needs. *db.Store satisfies it.
type Observer ¶
type Observer interface {
Observe(ctx context.Context, serverID, containerID string) (Observed, error)
}
Observer resolves the current container state for a server. Implementations must return an error when the state cannot be established (for example the node is unreachable); recovery treats that as "leave the lock in place".
func NewDockerObserver ¶
NewDockerObserver adapts a Docker client to recovery's observation interface.
type ReviewStore ¶
type ReviewStore interface {
GetServerOperation(ctx context.Context, id string) (*db.ServerOperation, error)
AdvanceServerOperation(ctx context.Context, id, from, to, detail string) error
}
ReviewStore is the journal subset used to retry locks that could not be observed at startup. *db.Store satisfies it.
type Store ¶
type Store interface {
ListActiveServerOperations(ctx context.Context) ([]db.ServerOperation, error)
AdvanceServerOperation(ctx context.Context, id, from, to, detail string) error
}
Store is the journal subset that recovery needs. *db.Store satisfies it.
type Summary ¶
type Summary struct {
Scanned int
Recovered int
NeedsReview int
// ReviewIDs are the active operations whose container state could not be
// established. A later pass may observe them once the daemon is reachable.
ReviewIDs []string
}
Summary reports what startup recovery did.
func RecoverInterrupted ¶
func RecoverInterrupted(ctx context.Context, store Store, observer Observer, log *logger.Logger) (Summary, error)
RecoverInterrupted observes every operation that still owns a server lock and finalizes it when the container state is known. Operations whose state cannot be observed keep their lock and are reported as requiring review; their server stays blocked until an operator or a later startup resolves them.