Documentation
¶
Overview ¶
Package indexcoord provides Experimental whole-set mutation authority shared by embeddable index engines, CLI adapters, and destructive maintenance.
Index ¶
- Constants
- Variables
- func AuthorityRoot(segmentSetRoot string) (string, error)
- type Lease
- type LeaseAttribution
- type LeaseReport
- type LeaseVerdict
- type ReclaimReport
- type RecoverStalledOptions
- type RecoverStalledOutcome
- type RecoverStalledResult
- type StalledPlanClass
- type StalledPlanOptions
- type StalledRecoveryPlan
Constants ¶
const ( LeaseHeld = LeaseVerdict(indexsubstrate.LeaseHeld) LeaseUnheld = LeaseVerdict(indexsubstrate.LeaseUnheld) LeaseMissing = LeaseVerdict(indexsubstrate.LeaseMissing) LeaseInvalid = LeaseVerdict(indexsubstrate.LeaseInvalid) )
const DefaultStalledHeartbeatGrace = 3 * ManagedHeartbeatInterval
DefaultStalledHeartbeatGrace is 3× the heartbeat interval. Callers may raise it; they may not lower it below MinStalledHeartbeatGrace.
const ManagedHeartbeatInterval = 30 * time.Second
ManagedHeartbeatInterval is the product heartbeat cadence. Grace floors are defined relative to this interval so a caller cannot shrink stop authority to an instant.
const MinStalledHeartbeatGrace = 2 * ManagedHeartbeatInterval
MinStalledHeartbeatGrace is the floor (2× heartbeat interval).
Variables ¶
var ( // ErrHeld reports contention with another whole-set operation. ErrHeld = indexsubstrate.ErrSetAuthorityHeld // ErrLost reports that the caller no longer owns the canonical lock binding. ErrLost = indexsubstrate.ErrSetAuthorityLost // ErrScope reports that a lease was presented for a different set or root. ErrScope = indexsubstrate.ErrSetAuthorityScope )
var ErrLeaseHeld = indexsubstrate.ErrSetAuthorityHeld
ErrLeaseHeld reports that a reclaim target is held by a live process and was refused. It aliases the substrate held-authority sentinel.
Functions ¶
func AuthorityRoot ¶
AuthorityRoot returns the stable authority directory for a segment-set root.
Types ¶
type Lease ¶
type Lease struct {
// contains filtered or unexported fields
}
Lease is an unforgeable whole-set authority token. Its backing OS lock lives outside the set-specific segment root and therefore survives root quarantine.
func Acquire ¶
Acquire takes authority for indexSetID at the stable sibling of segmentSetRoot. The call is non-blocking and returns ErrHeld on contention.
func (*Lease) AssertHeld ¶
func (*Lease) AssertHeldFor ¶
func (*Lease) Release ¶
Release drops whole-set authority and removes the lease artifact this holder created, so a completed operation leaves nothing for recovery to collect. It is idempotent.
An error means the lock was released but its artifact could not be removed — for example because the pathname no longer names the held lock, which Release refuses to delete rather than removing whatever occupies the name. What survives is observable through the lease surface either way, but the verdict depends on the artifact: an intact original reports unheld and is reclaimable, while a swapped-in replacement reports invalid and is never reaped automatically. Untrappable termination bypasses Release entirely and leaves the unheld kind.
type LeaseAttribution ¶
type LeaseAttribution struct {
// Matched is true when a job record's identity matches the lease holder.
Matched bool
// JobID is the matched job's ID (empty when unmatched).
JobID string
// PID is the matched job's recorded process ID (0 when unmatched).
PID int
// ProcessAlive is the liveness of PID, evaluated only when Matched.
ProcessAlive bool
// JobState is the matched job's last recorded state.
JobState string
}
LeaseAttribution is the best-effort operator context joined from the job registry. It answers "who left this lease, and is that process still alive" — never "is the lease held". The held/unheld verdict comes solely from the lock probe; a matched-and-alive PID does not imply held, and an unmatched or dead-PID lease is never treated as unheld on that basis.
type LeaseReport ¶
type LeaseReport struct {
IndexSetID string
Path string
Verdict LeaseVerdict
Holder string
AcquiredAt time.Time
ModTime time.Time
Detail string
Attribution LeaseAttribution
}
LeaseReport is a probed set-authority lease plus its joined attribution.
func EnumerateLeases ¶
func EnumerateLeases(authorityRoot string, jobs []jobregistry.JobRecord) ([]LeaseReport, error)
EnumerateLeases probes every set-authority lease under authorityRoot and joins each to the supplied job snapshot for attribution. A missing authority root yields an empty slice. Enumeration is fully non-mutating.
func ProbeLease ¶
func ProbeLease(authorityRoot, indexSetID string, jobs []jobregistry.JobRecord) (LeaseReport, error)
ProbeLease probes a single lease and joins attribution.
The wrapper never downgrades a state it received: whatever verdict the library reached is passed through unchanged, including alongside an error. It follows the library's refusal contract exactly — an artifact-grounded refusal carries LeaseInvalid; an infrastructure failure (unusable authority root, unexpected lock error) returns an error claiming no artifact state. Check the error before trusting a verdict.
type LeaseVerdict ¶
type LeaseVerdict string
LeaseVerdict is the non-mutating lock-state verdict for a set-authority lease.
type ReclaimReport ¶
type ReclaimReport struct {
IndexSetID string
Path string
Verdict LeaseVerdict
Reclaimed bool
Holder string
}
ReclaimReport is the outcome of a reclaim attempt for one lease.
func ReclaimUnheldLease ¶
func ReclaimUnheldLease(authorityRoot, indexSetID string) (ReclaimReport, error)
ReclaimUnheldLease removes a provably-unheld set-authority lease. A held lease is refused with ErrLeaseHeld and never touched; a missing lease is a no-op success (idempotent). The held/unheld decision is made solely by the lock probe — never by attribution.
It follows the library's refusal contract unchanged: an artifact-grounded refusal carries LeaseInvalid with its error; an infrastructure failure returns an error claiming no artifact state. Reclaimed is false on every error path.
type RecoverStalledOptions ¶
type RecoverStalledOptions struct {
AuthorityRoot string
Confirm bool // false = dry-run (byte-preserving plan only)
WaitTimeout time.Duration
PollInterval time.Duration
HeartbeatGrace time.Duration
// Now is ignored when Confirm is true. Mutation authorization always uses
// the library's production clock so callers cannot forge stall age. Dry-run
// / plan-only may still use Now for inspection (via PlanManagedStalledRecovery).
Now time.Time
// Owner is optional; empty mints a new recovery owner id.
Owner string
// Deadline, when non-zero, is the absolute monotonic attempt deadline
// (D-R13-03). Zero means derive from WaitTimeout at attempt entry.
// Tests may set an already-expired deadline to prove refuse paths.
Deadline time.Time
}
RecoverStalledOptions configures a single-job recover-stalled mutation.
type RecoverStalledOutcome ¶
type RecoverStalledOutcome string
RecoverStalledOutcome is a distinct non-boolean result class for mutation.
const ( OutcomeDryRun RecoverStalledOutcome = "dry-run" OutcomeRefused RecoverStalledOutcome = "refused" OutcomeSignalled RecoverStalledOutcome = "signalled-stopped" OutcomeReapedOnly RecoverStalledOutcome = "reaped-unheld" OutcomeAlreadyStopped RecoverStalledOutcome = "already-stopped" OutcomeRecoveryFailed RecoverStalledOutcome = "recovery-failed" OutcomeLeaseStillHeld RecoverStalledOutcome = "lease-still-held" OutcomeInvalidResidue RecoverStalledOutcome = "invalid-residue" OutcomeNoop RecoverStalledOutcome = "noop" )
type RecoverStalledResult ¶
type RecoverStalledResult struct {
JobID string `json:"job_id"`
Outcome RecoverStalledOutcome `json:"outcome"`
Detail string `json:"detail,omitempty"`
Plan StalledRecoveryPlan `json:"plan"`
DryRun bool `json:"dry_run"`
Owner string `json:"recovery_owner,omitempty"`
Signalled bool `json:"signalled"`
ForcedKill bool `json:"forced_kill"`
Reclaimed bool `json:"reclaimed"`
JobState string `json:"job_state,omitempty"`
LeaseAfter LeaseVerdict `json:"lease_verdict_after,omitempty"`
}
RecoverStalledResult is the library result of recover-stalled.
func RecoverManagedStalled ¶
func RecoverManagedStalled(store *jobregistry.Store, jobID string, opts RecoverStalledOptions) (RecoverStalledResult, error)
RecoverManagedStalled is the recover-stalled mutation: single managed job, confirm-gated, identity-bound signal, observe death, then W2 unheld reclaim. It never calls Store.Stop as the authority and never unlinks a held lease.
type StalledPlanClass ¶
type StalledPlanClass string
StalledPlanClass is a typed recovery-plan outcome. It is never a boolean: each class names a distinct authorization posture for a later mutating recover.
const ( // PlanHealthy: managed job is live under a matching identity with a fresh // heartbeat (or is still in a pre-lease phase with a fresh claim heartbeat). PlanHealthy StalledPlanClass = "healthy" // PlanSuspectHeartbeatOverdue: live matching identity, held lease, heartbeat // past grace — a candidate for a later confirm-gated recover. plan-stalled never // signals on this class. PlanSuspectHeartbeatOverdue StalledPlanClass = "suspect-heartbeat-overdue" // PlanTerminalContradiction: running/stopping record whose process is dead. // No signal; only re-probe + possible unheld reclaim in a later wave. PlanTerminalContradiction StalledPlanClass = "terminal-contradiction" // PlanIdentityMismatch: live PID but birth token does not match the record. PlanIdentityMismatch StalledPlanClass = "identity-mismatch" // PlanForegroundOrUnmatched: holder is not a durable managed job identity. PlanForegroundOrUnmatched StalledPlanClass = "foreground-or-unmatched" // PlanLeaseNotHeld: process identity may match, but the set-authority lease // is not held (nothing for process-stop to unlock). PlanLeaseNotHeld StalledPlanClass = "lease-not-held" // PlanInvalid: wrong type, unreadable record, or invalid lease residue. PlanInvalid StalledPlanClass = "invalid" // PlanIndeterminate: identity unproven, heartbeat writes failing, missing // index set correlation, or other incomplete evidence. Refuse, never signal. PlanIndeterminate StalledPlanClass = "indeterminate" )
type StalledPlanOptions ¶
type StalledPlanOptions struct {
// AuthorityRoot is the set-authority directory used for the lease probe.
// Required when the job has an IndexSetID; empty skips lease correlation and
// yields indeterminate once identity otherwise looks stalled.
AuthorityRoot string
// Now is the observation clock. Zero means time.Now().UTC().
Now time.Time
// HeartbeatGrace is how long a live matching process may go without a
// heartbeat before becoming suspect. Zero uses DefaultStalledHeartbeatGrace.
// Values below MinStalledHeartbeatGrace are raised to the floor.
HeartbeatGrace time.Duration
}
StalledPlanOptions configures a read-only recovery plan.
type StalledRecoveryPlan ¶
type StalledRecoveryPlan struct {
JobID string `json:"job_id"`
Class StalledPlanClass `json:"class"`
Detail string `json:"detail,omitempty"`
JobState string `json:"job_state,omitempty"`
JobType string `json:"job_type,omitempty"`
PID int `json:"pid,omitempty"`
ProcessAlive bool `json:"process_alive"`
RecordedIdentity string `json:"recorded_identity,omitempty"`
ObservedIdentity string `json:"observed_identity,omitempty"`
IndexSetID string `json:"index_set_id,omitempty"`
LeaseVerdict LeaseVerdict `json:"lease_verdict,omitempty"`
LeaseHolder string `json:"lease_holder,omitempty"`
HeartbeatAge string `json:"heartbeat_age,omitempty"`
HeartbeatGrace string `json:"heartbeat_grace,omitempty"`
HeartbeatWriteErr string `json:"heartbeat_persist_error,omitempty"`
// SignalCandidate is true only when every gate for a later recover-stalled recover is
// satisfied (suspect-heartbeat-overdue). plan-stalled never acts on it.
SignalCandidate bool `json:"signal_candidate"`
// MayReapUnheld is true for terminal-contradiction when the lease probe is
// already unheld/missing — process-stop is not required.
MayReapUnheld bool `json:"may_reap_unheld"`
// Auth* is the exact plan-time record snapshot for fence CAS. Recover must
// pass these unchanged into BeginStalledRecovery — never re-read them after
// plan. Omitted from JSON (operator surface uses formatted fields above).
AuthSnapshotOK bool `json:"-"`
AuthState string `json:"-"`
AuthPID int `json:"-"`
AuthStartMS uint64 `json:"-"`
AuthBootID string `json:"-"`
AuthTokenVersion int `json:"-"`
AuthStartTicks uint64 `json:"-"`
AuthStartSec int64 `json:"-"`
AuthStartUsec int64 `json:"-"`
AuthFiletime uint64 `json:"-"`
AuthIndexSetID string `json:"-"`
AuthLastHeartbeat *time.Time `json:"-"`
AuthHeartbeatPersistError string `json:"-"`
AuthRecoveryOwner string `json:"-"`
AuthRecoveryIntent string `json:"-"`
}
StalledRecoveryPlan is the read-only judgment for one managed job. It never signals, reaps, or rewrites registry state.
func PlanManagedStalledRecovery ¶
func PlanManagedStalledRecovery(store *jobregistry.Store, jobID string, opts StalledPlanOptions) (StalledRecoveryPlan, error)
PlanManagedStalledRecovery classifies one job for stalled recovery without mutating registry or lease state. It is the plan-stalled microscope: typed outcomes only, zero signals.