Documentation
¶
Overview ¶
Package partition compiles canonical, hashed, lane-assigned enumeration plans and defines the lane identity that names one unit of that plan.
Experimental: the plan encoding and the lane-identity contract will churn until a consumer persists them across runs.
The package is deliberately neutral. One plan governs enumeration, the identity carried alongside the records enumeration emits, resume, and the verification a mutating consumer performs before it writes. Owning the plan inside any one of those consumers would make a downstream stage the authority over an upstream one and force an import in the wrong direction; a shared contract package lets producers and consumers depend on the plan without depending on each other.
Index ¶
- Constants
- type Authority
- func (a *Authority) AuthorizeLane(claim LaneRef) error
- func (a *Authority) BaseIdentity() string
- func (a *Authority) ConfigFingerprint() string
- func (a *Authority) Coverage() Coverage
- func (a *Authority) Digest() string
- func (a *Authority) Lane(ordinal int) (Lane, error)
- func (a *Authority) LaneCount() int
- func (a *Authority) LaneRef(ordinal int) (LaneRef, error)
- func (a *Authority) Lanes() []Lane
- func (a *Authority) Plan() *Plan
- func (a *Authority) Version() int
- type Coverage
- type Lane
- type LaneRef
- type Plan
- type PlanRequest
- type SubsumedPrefix
Constants ¶
const LaneCeiling = 64
LaneCeiling is the highest lane count a plan may resolve to. It is a refusal, not a clamp: an operator asking for more lanes than the engine will run should be told so rather than silently getting fewer, because the lane count they believe is running is what they will reason about when reading throughput.
const PlanVersion = 1
PlanVersion is the schema version of the canonical plan encoding that PlanHash is taken over. Any change to the hashed payload shape must bump this, because a resume comparing plan digests across versions would otherwise read an encoding change as a plan change.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Authority ¶
type Authority struct {
// contains filtered or unexported fields
}
Authority is a validated, immutable enumeration plan that a run holds while it executes.
Plan and Authority are deliberately different things. Plan is data: it is JSON-tagged, exported, and mutable, because it is what gets persisted, read back, and handed around. Its digest field is whatever the last writer put there. Checking a claim against a Plan therefore proves only that the claim agrees with an unverified value — a plan literal carrying a made-up digest, or no digest at all, would authorize claims built from itself.
An Authority is the same information after that value has been proven: the digest is recomputed from the canonical payload and compared, every compiler invariant is rechecked, and the contents are deep-copied into unexported fields so that mutating the Plan afterwards cannot change what the run enforces. Issuing and checking lane identity is available only here, so there is no unvalidated path to either.
func CompileAuthority ¶
func CompileAuthority(req PlanRequest) (*Authority, error)
CompileAuthority compiles a plan and returns the run authority for it. It is the ordinary entry point: a caller that is planning work wants the authority, and CompilePlan exists for callers that need the persistable data.
func NewAuthority ¶
NewAuthority validates a plan and returns the run authority for it.
It fails closed: any invariant the compiler guarantees but the supplied plan does not satisfy is a refusal, because a plan that reached this point without satisfying them was not produced by the compiler, and the run cannot tell whether that is corruption, hand-editing, or tampering.
func (*Authority) AuthorizeLane ¶
AuthorizeLane checks a claimed lane reference against the plan this run holds.
A claim arriving alongside data — records read from a stream, a resumed checkpoint, an API request — is untrusted input: it says which lane it believes it belongs to, and this answers whether that is true of the plan the run is actually executing. It refuses rather than warns, because the caller's next step is a mutation the claim would otherwise scope.
Authorization is not completion. A verified reference says a record belongs to a lane of this plan; it says nothing about whether that lane's enumerator reached the end of its input or whether its work is durably finished. Only the run-owned coordinator may record those.
func (*Authority) BaseIdentity ¶
BaseIdentity reports the source this plan enumerates. It is resume identity, not a rate-limit authority.
func (*Authority) ConfigFingerprint ¶
ConfigFingerprint reports the caller digest of behavior-affecting configuration this plan was compiled under.
func (*Authority) Coverage ¶
Coverage reports what the plan claims about the scope it was compiled from.
func (*Authority) Lane ¶
Lane returns the lane with the given ordinal, refusing one this plan does not contain.
func (*Authority) LaneRef ¶
LaneRef returns the identity of the lane with the given ordinal.
It refuses an ordinal this plan does not contain rather than returning a reference to a lane that does not exist: a fabricated reference would compare equal to itself and could otherwise travel as though it named real work.
func (*Authority) Lanes ¶
Lanes returns a copy of the lane assignment. It copies rather than exposing the slice so a caller ranging over lanes cannot alter what the run enforces.
type Coverage ¶
type Coverage string
Coverage is the caller's declaration of what the plan claims about the scope it was compiled from. The compiler cannot derive this: it receives prefixes, not the scope configuration they came from (see the note on CompilePlan), so completeness is an assertion the caller makes and the plan records.
const ( // CoverageComplete asserts the prefixes union to the entire intended scope. CoverageComplete Coverage = "complete" // CoveragePartial asserts the plan deliberately covers only part of the // scope. Recorded so a downstream consumer never mistakes a partial sweep // for an authoritative one. CoveragePartial Coverage = "intentionally_partial" )
type Lane ¶
Lane is one enumeration unit: an ordinal plus the prefixes it lists. Ordinal is derived from assignment order rather than completion order, so lane identity does not vary with scheduling.
An ordinal alone is not an identity — see LaneRef.
type LaneRef ¶
type LaneRef struct {
PlanVersion int `json:"plan_version"`
PlanDigest string `json:"plan_digest"`
Ordinal int `json:"ordinal"`
}
LaneRef is the identity of one lane: the plan encoding version, the plan digest, and the lane ordinal within that plan.
All three are required. An ordinal alone is not stable identity — lane 3 names entirely different work under a different plan — so a claim carrying only an ordinal cannot be checked against anything. The version is carried because two encodings of the same planned work produce different digests, and a consumer needs to say which mismatch it is looking at.
A LaneRef is issued and checked by an Authority, never by a Plan. A Plan is data: its digest field is whatever the last writer put there, so comparing a claim against it proves only that the claim agrees with an unverified value.
type Plan ¶
type Plan struct {
Version int `json:"version"`
Coverage Coverage `json:"coverage"`
BaseIdentity string `json:"base_identity"`
ConfigFingerprint string `json:"config_fingerprint"`
Lanes []Lane `json:"lanes"`
// PlanHash is the digest of the canonical payload. It is an integrity
// fingerprint, not an authentication tag: anyone who can construct the same
// inputs can construct the same digest. A consumer verifies a claimed lane
// against a plan it was handed by the run that owns it — never against a
// plan reconstructed from the claim itself. See AuthorizeLane.
PlanHash string `json:"plan_hash"`
// Subsumed records prefixes removed because another retained prefix already
// covers them. Normalization here is union-preserving — dropping a/b/ when
// a/ is present enumerates exactly the same objects — but it does reduce the
// number of partitionable units, so an operator who wrote three prefixes and
// got one lane needs to be able to see why. Reported rather than refused
// precisely because coverage is unchanged.
Subsumed []SubsumedPrefix `json:"subsumed,omitempty"`
}
Plan is a canonical, hashed, lane-assigned enumeration plan. It is compiled and hashed before enumeration begins so the digest describes the work that was planned, not the work that happened to complete.
func CompilePlan ¶
func CompilePlan(req PlanRequest) (*Plan, error)
CompilePlan normalizes the requested prefixes, refuses inputs whose partition semantics are not decidable, assigns stable lane ordinals, and hashes the result.
Prefix containment uses literal byte-prefix semantics, not path semantics, because that is what an object-store LIST prefix means: listing "a" returns "ab/x" as well as "a/x". Treating containment as path-aware would leave a genuinely redundant prefix in the plan and enumerate those objects twice.
type PlanRequest ¶
type PlanRequest struct {
Prefixes []string
Coverage Coverage
// BaseIdentity names the source provider and base URI this plan enumerates.
// It is part of the digest because a plan is only the same plan when it runs
// against the same source: identical prefixes under a different provider or
// bucket describe entirely different work, and a resume must not match across
// that boundary.
//
// This is resume identity. It is deliberately not a rate-limit authority,
// which may be broader than one bucket, and it must never carry credentials.
BaseIdentity string
// ConfigFingerprint is the caller digest of behavior-affecting configuration.
// The compiler cannot compute it — the adapter owns what is
// behavior-affecting — but the plan must carry it so a resume refuses a run
// whose semantics changed under it.
ConfigFingerprint string
// MaxLanes bounds the lane count. Zero means one lane per retained prefix.
MaxLanes int
}
PlanRequest is the compiler's input.
Prefixes are explicit. The compiler deliberately does not accept a scope configuration: scope compilation reaches a schema dependency that the embeddable engine's dependency boundary denies, so it belongs to the calling adapter. The adapter compiles configuration into prefixes; this package receives facts.
type SubsumedPrefix ¶
SubsumedPrefix is one dropped prefix and the retained prefix that covers it.