Documentation
¶
Overview ¶
Package s3wal is the S3-manifest-WAL RefStore experiment (issue #133, docs/plans/s3wal-plan.md): one JSON manifest object per base repo group, advanced by conditional PUT (If-Match / If-None-Match: *). Phase 1a ships only the conditional-object CAS primitive (this file) and the boot-time store probe (probe.go); manifest schema and the RefStore driver are Phase 1b.
Instance lease (plan §2.11): the best-effort dual-serve guard for the zero-infra serve mode (issue #133 Phase 3). One CAS-heartbeat object, LeaseKey, detects the accidental two-writers topology — two serve processes pointed at one bucket — and turns it into a refused startup or a fail-stop, instead of the silent TTL-clock weirdness and cache staleness §2.4 describes.
Correctness does NOT depend on this lease: every state change is CAS-serialized by the manifest/meta objects regardless. It is purely a detection device. Freshness is judged on the STORE's clock alone — the lease object's Last-Modified against the HTTP Date witness (§2.4-3) — never by comparing two hosts' wall clocks: a starter whose clock runs ≥ TTL ahead of the holder's would otherwise classify a freshly renewed lease as expired and steal it, creating exactly the dual-serve topology the guard exists to detect (PR #168). The lease body records the writer's TTL so a starting process judges freshness by the HOLDER's contract, not its own (possibly reconfigured) one.
Manifest schema v1 (plan §2.2) plus the §2.4 manifest logical clock and the §2.9 key-alphabet validators. One JSON manifest object holds the ENTIRE authority state of one base repo group {base, base+ephemeral, base+import}: refs, packs, retained tombstones, generations, quota ledger, reservations (incl. adoption claims and absence covers), LFS registry, delete leases, purge fence, and sync-claim token — so every audited multi-entity transaction is a single-object CAS.
MemCAS is the in-memory ConditionalStore fake: the fast (no Docker) substrate for the driver's recovery-predicate unit tests and the Phase 2 DST fault plan. It reproduces the store semantics the driver depends on — conditional create/advance with content-derived ETags (identical bytes ⇒ identical ETag, so the §5-R4 ETag-ABA hazard is reproducible), byte- ordered listing — plus a small, explicit fault surface: each conditional PUT can be directed to fail ambiguously WITH or WITHOUT the write applying (the crucial §2.3 window), to fail before execution, or to exhaust contention retries; reads can serve the previous version once (stale read). It is deliberately not a general mock framework.
ConditionalStore is the seam between the manifest driver (package ref's s3wal store) and its CAS substrate: the real conditional S3 client (*Client) in production, the fault-injecting in-memory fake (*MemCAS) in unit/DST tests. Byte-ordered List is part of the contract — fleet scans (§2.9) depend on it.
Index ¶
- Constants
- Variables
- func BaseFromManifestKey(key string) (string, bool)
- func ClkNext(localNow, prev time.Time) time.Time
- func DateOK(localNow, date time.Time, ttl time.Duration) bool
- func DateSkewBound(ttl time.Duration) time.Duration
- func DecodeSeq(body []byte) (int64, bool)
- func DriftOK(wallDrift, ttl time.Duration) bool
- func Elapsed(localNow, clk, stamp time.Time) time.Duration
- func ManifestKey(base string) string
- func NewOpID() string
- func Probe(ctx context.Context, c *Client) error
- func SkewOK(localNow, clk time.Time, wallDrift, ttl time.Duration) bool
- func ValidBaseRepoKey(base string) bool
- type AmbiguousError
- type Client
- func (c *Client) Advance(ctx context.Context, key string, body []byte, etag string) (string, error)
- func (c *Client) Close()
- func (c *Client) Create(ctx context.Context, key string, body []byte) (string, error)
- func (c *Client) Delete(ctx context.Context, key string) error
- func (c *Client) Get(ctx context.Context, key string) (Object, error)
- func (c *Client) GetIfChanged(ctx context.Context, key, etag string) (Object, error)
- func (c *Client) List(ctx context.Context, prefix, startAfter string, limit int) ([]string, error)
- func (c *Client) StoreDate() (date, observedAt time.Time, ok bool)
- type ConditionalStore
- type DeleteLock
- type InstanceLease
- type LFSObject
- type LFSState
- type LeaseConfig
- type LeaseHeldError
- type Manifest
- type MemCAS
- func (m *MemCAS) Advance(_ context.Context, key string, body []byte, etag string) (string, error)
- func (m *MemCAS) Create(_ context.Context, key string, body []byte) (string, error)
- func (m *MemCAS) Delete(_ context.Context, key string) error
- func (m *MemCAS) Get(_ context.Context, key string) (Object, error)
- func (m *MemCAS) GetIfChanged(ctx context.Context, key, etag string) (Object, error)
- func (m *MemCAS) List(_ context.Context, prefix, startAfter string, limit int) ([]string, error)
- func (m *MemCAS) StoreDate() (date, observedAt time.Time, ok bool)
- type MemCASHooks
- type Member
- type Object
- type PackRecord
- type ProbeError
- type PutFault
- type RefRecord
- type Reservation
- type RetainedPack
- type SyncClaim
Constants ¶
const ( ProbeRejected = "rejected" ProbeIgnored = "ignored" ProbeMishandled = "mishandled" ProbeInconclusive = "inconclusive" )
Probe failure classes: how a probe step failed. The distinction matters operationally — the first three are DEFINITIVE store behavior ("ignored" is the silent-corruption class: the store accepted a write the condition forbids; "rejected" is loud-but-wrong: a real rejection response that is not the required precondition outcome, such as a 403 or the wrong conditional sentinel; "mishandled" is any other definitive divergence, such as success where failure was required or an unchanged ETag after an advance) and mean the store is non-compliant. "inconclusive" means the probe could not obtain a definitive store verdict at all (ambiguous post-send failure where the write may have landed, a transport/not-executed failure, or a failed scratch cleanup): the store is NOT proven non-compliant, but a boot probe must be conclusive (§2.6), so serving is refused pending a retry.
const DateTolerance = 2 * time.Second
DateTolerance is the resolution floor of the §2.4-3 store-Date witness: HTTP Date carries 1s resolution, so agreement bounds tighter than ±2s cannot distinguish skew from truncation noise.
const DefaultLeaseTTL = 2 * time.Minute
DefaultLeaseTTL is the §2.11 protocol default; the derived cadence (TTL/4 = 30 s) leaves three missed renewals before a foreign starter may steal.
const LeaseKey = "_wal/instance-lease"
LeaseKey is the dual-serve guard object (§2.1 key table).
const ManifestVersion = 1
ManifestVersion is the only schema version this binary reads or writes.
const OpsRingSize = 64
OpsRingSize bounds the manifest's recent-mutation op-ID ring (plan §2.3, K=64): ambiguous retries of NON-idempotent ops test membership here instead of blindly re-applying. Ring eviction cannot forge a "not applied" verdict for a live entry because the op-ID is ALSO recorded on the touched reservation entry itself (Reservation.OpID), cleared with the entry.
const ProbeKey = "_wal/probe"
ProbeKey is the scratch-key PREFIX the boot-time probe exercises. Plan §2.1 names the fixed key `_wal/probe`; the code refines that with a per-invocation `/<128-bit-hex-nonce>` suffix because two instances probing one bucket simultaneously on a shared fixed key can interleave — one deleting/recreating it between the other's steps — forging ErrExists/ErrLostRace outcomes that falsely reject a compliant store and block replica startup. The prefix lives outside `_wal/repo/` so probe writes can never collide with a manifest.
const RepoPrefix = "_wal/repo/"
RepoPrefix is the bucket prefix manifests live under. Keys carry NO file extension: a suffix like ".json" would corrupt listing order between prefix-related repo IDs ("abc-1.json" sorts before "abc.json" although abc < abc-1) — plan §2.1/§2.9.
Variables ¶
var ( // ErrNotExist reports a Get on a key that does not exist (404/NoSuchKey). ErrNotExist = errors.New("s3wal: object does not exist") // ErrNotModified reports a GetIfChanged whose ETag still matches (304). ErrNotModified = errors.New("s3wal: object not modified") // ErrExists reports a Create that lost to an existing object: the // duplicate create was rejected with 412 and did not execute. ErrExists = errors.New("s3wal: object already exists") // ErrLostRace reports an Advance rejected with 412 PreconditionFailed: // someone else advanced the object first and our write did NOT happen. // The caller re-reads fresh state and re-evaluates (Phase 1b). ErrLostRace = errors.New("s3wal: conditional write lost the race") // ErrNotExecuted reports a request that provably never reached the // store (dial failure, ctx expired while queued) — the PG notExecuted // twin, never recovered as a win. ErrNotExecuted = errors.New("s3wal: request not executed") // ErrCASContention reports 409 ConditionalRequestConflict retries // exhausted — surfaced, never silently dropped. ErrCASContention = errors.New("s3wal: conditional-write contention retries exhausted") )
Typed sentinels for the plan §2.3 error taxonomy. Callers branch with errors.Is; every sentinel below the ambiguity line means the conditional write did NOT execute, so retrying can never double-apply.
Functions ¶
func BaseFromManifestKey ¶
BaseFromManifestKey inverts ManifestKey for bucket-listing results.
func ClkNext ¶
ClkNext derives the write-time manifest clock: max(localNow, prev+1ms), monotone across restarts and immune to backward local jumps (§2.4).
func DateOK ¶
DateOK is the §2.4-3 store-clock cross-check for destructive/steal verdicts: false — demote to report-only, log loudly — when the local clock disagrees with the store's Date witness by more than DateSkewBound. This is the only arm that catches an AT-BOOT-wrong local clock (SkewOK's drift arm anchors at process start; its manifest arm reads an old clk as a quiet repo). Deliberately scoped to destructive verdicts rather than asserted on every CAS as the plan's crash-and-burn sketch suggested: mild skew is routine and a per-operation assert would DoS every read under it, while destruction is where wrongness is irreversible. Freshness of the witness is the caller's obligation (ref.S3WAL requires an observation younger than its per-verdict bound and fails closed without one).
func DateSkewBound ¶
DateSkewBound is the §2.4-3 agreement bound for the store-Date witness: min(15min, ttl/4), the same scaling as SkewOK's arms. A degenerate bound below DateTolerance (ttl < 4·tolerance = 8s — the conformance suite's sub-second windows) returns 0 = arm DISABLED: at that scale legitimate second-boundary truncation is indistinguishable from skew, and every production window (delete locks ~2.75m, claims 30m, graces ≥1h) sits far above it.
func DecodeSeq ¶
DecodeSeq is the Client.DecodeSeq hook for manifest bodies: it arms the §2.3 read-side stale-read assert (and the write-side watermark raise) on every conditional manifest operation.
func DriftOK ¶
DriftOK is the wall-vs-monotonic drift arm of SkewOK, also consulted alone by LIVENESS checks (which lean live under a jump): a manifest clock merely AHEAD already leans live through the plain elapsed arithmetic, so liveness needs only this arm. A degenerate bound of 0 (ttl < 4ns — the conformance suite's "window already elapsed" idiom) disables the arm: at that scale measurement noise and jumps are indistinguishable, and ttl <= 0 is the explicit waiver spelling upstream.
func Elapsed ¶
Elapsed computes the §2.4 elapsed-time rule for TTL/grace comparisons: min(localNow, clkCandidate) − stamp with clkCandidate = max(localNow, clk+1ms) — a backward local-clock jump (localNow < clk) charges only the local clock's smaller reading, so destruction is only ever DELAYED by clock trouble, never accelerated. Because clkCandidate >= localNow by construction, the reduction always equals localNow − stamp: a FORWARD local-clock jump inflates this reading, and it is SkewOK — not this helper — that fails destructive verdicts closed against it.
func ManifestKey ¶
ManifestKey builds the extension-free manifest object key for one BASE repo, asserting the §2.9 member-contiguity alphabet at key-build time: every byte must sort ABOVE '+' (0x2B) so a base's `+`-suffixed member keys stay contiguous in bucket listing order, and no base may itself contain '+' or non-ASCII/control bytes. Production repo IDs are server-generated [0-9a-z-]; the validator deliberately enforces only the weaker ordering invariant so conformance tests can exercise raw test-derived names (slashes, underscores, uppercase — all > 0x2B).
func NewOpID ¶
func NewOpID() string
NewOpID mints a fresh 128-bit op-ID for non-idempotent mutations (§2.3).
func Probe ¶
Probe proves the store's conditional-write fidelity before serving (plan §2.6): a store that ignores the conditional headers corrupts silently — the worst failure mode — so any deviation refuses to serve with an explicit diagnosis. The five steps: create must succeed; a duplicate create must be rejected; a matched advance must succeed with a new ETag; a stale-ETag advance must be rejected; two racing advances from one snapshot must produce exactly one winner.
func SkewOK ¶
SkewOK is the §2.4 fail-closed skew guard for destructive/steal checks: false — expiry verdicts demote to report-only (log loudly, never expire early) — on either of two clock-trouble evidences, both bounded by min(15min, ttl/4):
- the manifest clock runs AHEAD of the local clock by more than the bound: some writer's clock (ours or a previous one) is grossly wrong. This comparison stays one-sided: a manifest clock is last-WRITE time, so a merely OLD clock (quiet repo) is normal aging, not skew — §2.4's literal |localNow − clk| reading would demote every quiet-repo expiry forever (a deliberate, documented deviation from that wording).
- the process wall clock has drifted from its monotonic clock by more than the bound (wallDrift, either sign): a local-clock jump during this process's lifetime — exactly the window in which a forward jump could otherwise expire the leases of live in-process holders (guarded deletes, adoption claims, reservations) early.
Jumps within the bound are acceptable: the conservative grace floors (§2.4-4) absorb them as noise. Wall jumps predating this process are invisible to both arms — an AT-BOOT-wrong clock reports ~zero drift and an old manifest clock reads as a quiet repo — so destructive verdicts additionally cross-check the §2.4-3 store-Date witness (DateOK, wired in ref.S3WAL), the absolute clock reading no local trouble can forge.
func ValidBaseRepoKey ¶
ValidBaseRepoKey reports whether base satisfies the §2.9 ordering alphabet (see ManifestKey).
Types ¶
type AmbiguousError ¶
AmbiguousError reports a conditional write whose fate is unknown: the request was (or may have been) sent, and the transport failed before a definitive store verdict arrived (timeout, connection loss, 5xx). The write MAY have landed; recovery predicates are Phase 1b — Phase 1a only surfaces the class. When classification is uncertain we default to ambiguous (fail-closed direction per plan §2.3).
func (*AmbiguousError) Error ¶
func (e *AmbiguousError) Error() string
func (*AmbiguousError) Unwrap ¶
func (e *AmbiguousError) Unwrap() error
type Client ¶
type Client struct {
// DecodeSeq optionally decodes a payload sequence number from a fetched
// body. When set, Get/GetIfChanged enforce the §2.3 read-side defensive
// assert: a per-key seq regression means the store served a stale read —
// fail loudly. Acknowledged Create/Advance writes also RAISE the per-key
// watermark to the seq of the body they wrote (see raiseSeq), so a read
// older than this session's own accepted write trips the assert too.
// Phase 1b plugs in the manifest decoder; Phase 1a tests and benches
// decode their {seq, nonce} payloads.
//
// The assert is SESSION-scoped: it compares reads made through this
// Client, which processes responses sequentially per caller. Two
// goroutines sharing one Client can legitimately process responses out
// of store-serialization order (an older read's response arriving after
// a newer read's — benign reordering, not store staleness), so give
// each concurrent reader of a hot key its own Client. Phase 1b must
// keep this in mind when sharing a Client across request handlers.
DecodeSeq func(body []byte) (seq int64, ok bool)
// contains filtered or unexported fields
}
Client performs conditional single-object operations over the raw vendored s3.Client (never gocloud — plan §5-R1). It carries no manifest or RefStore logic.
func Dial ¶
Dial opens an s3wal store URL and returns the conditional CAS client for its bucket, with the manifest seq watermark armed:
s3wal://bucket?endpoint=http://127.0.0.1:8333®ion=us-east-1
The URL grammar mirrors blob.OpenS3 (bucket = host, endpoint/region = query, credentials from the standard AWS env chain, path-style always on for self-hosted stores); the legacy s3ForcePathStyle query key is accepted and ignored for symmetry. The optional expected-bucket-owner enforcement comes from envExpectedBucketOwner — the same knob as internal/blob, and a malformed value is boot-fatal here too. Dial performs no network I/O — probe and reachability run at InitSchema (ADR-12: Open never runs DDL).
func New ¶
New wraps an s3.Client for conditional object operations on one bucket. It always derives its own no-retry client from the input client's options: the SDK silently re-sending a conditional PUT after an ambiguous failure can turn our own landed write into a spurious 412 — misclassified as a lost race. Taxonomy fidelity requires exactly one wire attempt per classified attempt, so auto-retry is disabled by construction, regardless of how the caller built the input client.
func (*Client) Advance ¶
Advance overwrites key with If-Match: etag — the CAS advance. A 412 rejection returns ErrLostRace (not executed; re-read and re-evaluate).
func (*Client) Close ¶
func (c *Client) Close()
Close releases pooled transport resources when this Client owns them (Dial-built clients); Clients wrapped around caller-owned s3.Clients close nothing.
func (*Client) Create ¶
Create writes key with If-None-Match: * — it succeeds only when the key does not exist. A 412 rejection returns ErrExists (not executed).
func (*Client) Delete ¶
Delete removes key unconditionally. The protocol never guards a DELETE with a precondition (§2.7); Phase 1a uses this only for probe scratch cleanup and test teardown.
func (*Client) GetIfChanged ¶
GetIfChanged fetches key only when its ETag differs from etag (the cheap generation-poll path, §2.3): an unchanged object returns ErrNotModified from the store's 304 without a body transfer.
func (*Client) List ¶
List pages keys under prefix in S3's lexicographic (byte) order via ListObjectsV2 — the §2.9 fleet-scan primitive. startAfter is exclusive.
type ConditionalStore ¶
type ConditionalStore interface {
// Get fetches key; ErrNotExist reports a missing object.
Get(ctx context.Context, key string) (Object, error)
// GetIfChanged fetches key unless its ETag still matches (304 ⇒
// ErrNotModified) — the cheap generation-poll path.
GetIfChanged(ctx context.Context, key, etag string) (Object, error)
// Create writes key iff it does not exist (If-None-Match: *).
Create(ctx context.Context, key string, body []byte) (string, error)
// Advance overwrites key iff etag is current (If-Match).
Advance(ctx context.Context, key string, body []byte, etag string) (string, error)
// Delete removes key unconditionally (§2.7: never conditional).
Delete(ctx context.Context, key string) error
// List returns up to limit keys under prefix, strictly byte-ordered,
// strictly greater than startAfter ("" = from the beginning).
List(ctx context.Context, prefix, startAfter string, limit int) ([]string, error)
// StoreDate returns the store wall clock freshest observed on this
// client's responses (the HTTP Date header, 1s resolution), the
// local monotonic-anchored instant of the observation, and whether
// any observation exists — the §2.4-3 absolute clock witness that
// destructive TTL verdicts cross-check against the local clock.
StoreDate() (date, observedAt time.Time, ok bool)
}
ConditionalStore is the conditional-object surface the manifest driver consumes (§2.3 taxonomy: every method returns the casobj.go sentinels / *AmbiguousError classes).
type DeleteLock ¶
type DeleteLock struct {
Owner string `json:"owner"`
OpID string `json:"op_id"`
CreatedAt time.Time `json:"created_at"`
}
DeleteLock is one §2.5 W6 delete lease.
type InstanceLease ¶
type InstanceLease struct {
// contains filtered or unexported fields
}
InstanceLease is one held lease: acquired by AcquireInstanceLease, renewed by Heartbeat, released by Release.
func AcquireInstanceLease ¶
func AcquireInstanceLease(ctx context.Context, cs ConditionalStore, cfg LeaseConfig) (*InstanceLease, error)
AcquireInstanceLease claims LeaseKey for a fresh holder nonce: create when absent, steal when the object's STORE-clock age has passed the RECORDED TTL (plus the Date-resolution margin), refuse with *LeaseHeldError while fresh. Exactly one contender wins any create/steal race — both paths are conditional writes.
func (*InstanceLease) Heartbeat ¶
func (l *InstanceLease) Heartbeat(ctx context.Context) error
Heartbeat renews the lease every cadence until Release or ctx cancellation (both return nil — orderly shutdown). A non-nil return is LEASE LOSS: the caller must fail-stop the process (crash, not limp — §2.11); continuing to serve would be exactly the dual-serve topology the lease exists to detect.
func (*InstanceLease) Holder ¶
func (l *InstanceLease) Holder() string
Holder exposes the lease's holder nonce for logs and tests.
func (*InstanceLease) Release ¶
func (l *InstanceLease) Release(ctx context.Context)
Release stops the heartbeat and deletes the lease object when it is still ours (owner-guarded read-then-delete: §2.7 forbids conditional DELETE, and the residual TOCTOU — a thief overwriting between our read and delete — can only shorten the thief's own detection window, never affect correctness). Best-effort: any failure just leaves the lease to expire on its TTL. Idempotent.
type LFSObject ¶
type LFSObject struct {
Bytes int64 `json:"bytes"`
CreatedAt time.Time `json:"created_at"`
UnreferencedSince *time.Time `json:"unreferenced_since,omitempty"`
}
LFSObject is one lfs_objects row.
type LFSState ¶
type LFSState struct {
Count int64 `json:"count"`
Objects map[string]*LFSObject `json:"objects,omitempty"`
Mismatches map[string]string `json:"mismatches,omitempty"`
AdoptionGen int64 `json:"adoption_gen"`
}
LFSState is the group's LFS registry: maintained count, object rows, mismatch markers, and the adoption generation.
type LeaseConfig ¶
type LeaseConfig struct {
TTL time.Duration // 0 = DefaultLeaseTTL
Cadence time.Duration // 0 = TTL/4 (30 s at the default TTL)
Now func() time.Time // nil = time.Now (unit tests inject clocks)
}
LeaseConfig tunes the lease protocol; the zero value is production.
type LeaseHeldError ¶
LeaseHeldError is the dual-serve refusal: a fresh foreign lease exists, so another serve process is (or very recently was) running against this bucket. errors.As-able so serve can surface it verbatim. Age is the store-clock age of the lease object (storeAge), not a wall-clock delta.
func (*LeaseHeldError) Error ¶
func (e *LeaseHeldError) Error() string
type Manifest ¶
type Manifest struct {
V int `json:"v"`
// Seq is strictly monotonic, bumped by every CAS. Together with the
// fresh Writer nonce it guarantees distinct bytes per logical version
// ⇒ distinct ETag (kills ETag-ABA, §5-R4).
Seq int64 `json:"seq"`
// Writer is the 128-bit random nonce of the LAST writer, fresh per
// write — the ambiguous-PUT recovery identity check (§2.3).
Writer string `json:"writer"`
// Ops is the bounded ring (oldest first) of recent mutation op-IDs.
Ops []string `json:"ops,omitempty"`
// Clk is the manifest logical clock: every CAS writes
// max(localNow, clk_prev + 1ms) — monotone across restarts and
// backward local-clock jumps (§2.4). Every stamp stored in the
// manifest uses this write-time value.
Clk time.Time `json:"clk"`
// PurgedAt non-nil is the terminal purge fence + tombstone stamp: the
// manifest object is NEVER deleted (burned-name invariant).
PurgedAt *time.Time `json:"purged_at,omitempty"`
// Members holds per-repo state, keyed by full repo key (base,
// base+ephemeral, base+import).
Members map[string]*Member `json:"members,omitempty"`
// UsageBytes is the group's pack_usage counter.
UsageBytes int64 `json:"usage_bytes"`
// Reservations is the pack_reservations table: in-flight admission
// rows keyed by blob key, plus /adopt-claim entries and absence
// covers.
Reservations map[string]*Reservation `json:"reservations,omitempty"`
// DeleteLocks holds the short-TTL delete leases (§2.5 W6).
DeleteLocks map[string]*DeleteLock `json:"delete_locks,omitempty"`
// LFS is the group's LFS registry (keyed by the BASE repo — ADR-35).
LFS LFSState `json:"lfs"`
// SyncClaim is the upstream-sync claim token (§2.8). Phase 1b only
// READS it (the CASRef fence); the write-through seam is Phase 3.
SyncClaim *SyncClaim `json:"sync_claim,omitempty"`
}
Manifest is the §2.2 v1 schema — the whole repo group's authority state.
func DecodeManifest ¶
DecodeManifest parses and validates one fetched manifest body. Corruption is a crash-and-burn error, never a silent empty state: a manifest that exists but cannot be trusted must stop the writer.
func NewManifest ¶
func NewManifest() *Manifest
NewManifest returns the empty (never-written) manifest state: Seq 0, so the first EncodeAdvance writes seq 1 through the If-None-Match: * create path. A missing manifest object and this value are interchangeable.
func (*Manifest) EncodeAdvance ¶
EncodeAdvance encodes the NEXT version of m: seq+1, a fresh 128-bit writer nonce (distinct bytes per write ⇒ distinct ETag — §5-R4), and the given write-time clock, which the caller derived via ClkNext from the SAME snapshot so in-body stamps and the manifest clock agree. m is updated in place so the caller can read the minted Writer nonce for ambiguous-PUT recovery.
func (*Manifest) EnsureMember ¶
EnsureMember returns the repo's member state, creating it on first mutation.
func (*Manifest) OpApplied ¶
OpApplied reports whether opID is in the ops ring OR recorded on a live reservation entry — the §2.3 membership test for ambiguous retries of non-idempotent ops. Absence is NOT proof of non-application (the ring is bounded); callers fail closed when this and the semantic predicate are both inconclusive.
type MemCAS ¶
type MemCAS struct {
Hooks MemCASHooks
// Now, when set, is the fake STORE wall clock served as the §2.4-3
// Date witness and stamped as every written version's LastModified:
// like the real store, every operation records an
// observation (StoreDate). Nil serves the real clock — correct for
// every real-clock test, while fake-clock tests wire Now to their
// local fake (agreeing clocks) or to a second clock to simulate
// boot skew in either direction.
Now func() time.Time
// contains filtered or unexported fields
}
MemCAS implements ConditionalStore in memory. The zero value is not usable; call NewMemCAS.
func (*MemCAS) GetIfChanged ¶
GetIfChanged fetches key unless etag is still current.
type MemCASHooks ¶
type MemCASHooks struct {
// PutFault classifies one conditional PUT attempt (op is "create" or
// "advance"). It runs under the store lock: it must not call back into
// the MemCAS.
PutFault func(op, key string) PutFault
// StaleGet, when it returns true, serves the key's PREVIOUS version
// (body and ETag) for this read — the stale-read fidelity violation
// the seq watermark must catch. Reads of keys with no previous
// version serve normally.
StaleGet func(key string) bool
// AfterPut, when set, runs after every conditional PUT attempt
// (op is "create" or "advance"), OUTSIDE the store lock and
// regardless of the attempt's outcome: recovery tests use it to
// interleave a competing writer between a faulted PUT and the
// driver's recovery read. Unlike PutFault, it MAY call back into
// the MemCAS.
AfterPut func(op, key string)
}
MemCASHooks is the per-op fault surface. Nil members mean no faults.
type Member ¶
type Member struct {
Gen int64 `json:"gen"`
Refs map[string]RefRecord `json:"refs,omitempty"`
Packs []PackRecord `json:"packs,omitempty"`
Retained map[string]RetainedPack `json:"retained,omitempty"`
}
Member is one repo's refs/packs/retained state plus its mutation generation (driver-bumped per changed row, replacing the SQL triggers).
type Object ¶
type Object struct {
Body []byte
ETag string
// LastModified is the STORE's write instant for this version
// (Last-Modified header, 1s resolution). Paired with the Date witness
// (StoreDate) it lets freshness be judged entirely on the store's
// clock — the instance lease depends on it, because comparing two
// hosts' wall clocks is exactly what cross-machine skew breaks. Zero
// when the store omitted the header; consumers fail closed.
LastModified time.Time
}
Object is one fetched conditional object: full body plus the store ETag, handled verbatim as returned (quoted) and never compared to MD5.
type PackRecord ¶
type PackRecord struct {
ID string `json:"id"`
Key string `json:"key"`
Bytes int64 `json:"bytes"`
CreatedAt time.Time `json:"created_at"`
}
PackRecord is one registered pack row (bytes persisted — absence scans and ListFleetPacks need them).
type ProbeError ¶
type ProbeError struct {
Step int // 1-5, the §2.6 step that deviated
Surface string // the conditional surface under test
Class string // ProbeRejected | ProbeIgnored | ProbeMishandled | ProbeInconclusive
Diagnosis string
}
ProbeError is the refuse-to-serve verdict from a failed conditional-write probe: which step failed, on which surface, classified and diagnosed.
func (*ProbeError) Error ¶
func (e *ProbeError) Error() string
type PutFault ¶
type PutFault int
PutFault directs one MemCAS conditional PUT attempt.
const ( // FaultNone applies normal conditional-write semantics. FaultNone PutFault = iota // FaultAmbiguousApplied applies the write, then reports *AmbiguousError // — the "response lost after commit" window. FaultAmbiguousApplied // FaultAmbiguousDropped drops the write and reports *AmbiguousError — // the "request lost before commit" window. FaultAmbiguousDropped // FaultNotExecuted drops the write and reports ErrNotExecuted — the // provably-never-sent class. FaultNotExecuted // FaultContention drops the write and reports ErrCASContention — the // exhausted-409-retries class. FaultContention )
type RefRecord ¶
RefRecord is one ref row. CreatedAt is set on create, preserved across SHA updates, and reset by delete+recreate — the branches[].created_at contract.
type Reservation ¶
type Reservation struct {
Owner string `json:"owner"`
Bytes int64 `json:"bytes"`
Attempts int64 `json:"attempts"`
Gen string `json:"gen"`
CreatedAt time.Time `json:"created_at"`
OpID string `json:"op_id,omitempty"`
}
Reservation is one pack_reservations row. OpID records the last non-idempotent mutation that touched this entry (§2.3 OPS recovery); it is cleared with the entry, so ops-ring eviction cannot forge a "not applied" verdict while the entry lives.
type RetainedPack ¶
type RetainedPack struct {
Bytes int64 `json:"bytes"`
SupersededAt time.Time `json:"superseded_at"`
}
RetainedPack is one superseded-blob tombstone (pack_retained row).
Directories
¶
| Path | Synopsis |
|---|---|
|
Package s3waltest carries the shared s3wal test fixtures: the pinned SeaweedFS/MinIO testcontainers, raw path-style S3 client construction, bucket bootstrap/cleanup helpers, and the Phase 1a {seq, nonce} payload codec.
|
Package s3waltest carries the shared s3wal test fixtures: the pinned SeaweedFS/MinIO testcontainers, raw path-style S3 client construction, bucket bootstrap/cleanup helpers, and the Phase 1a {seq, nonce} payload codec. |