projection

package
v0.0.21 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 18, 2026 License: MIT Imports: 17 Imported by: 0

Documentation

Index

Constants

View Source
const PruneCap = 4096

PruneCap bounds how many orphan payload blobs ONE BATCH of the operator prune deletes. The prune verb runs batches so that each batch is its own short write transaction: a store that is being written while the prune runs keeps its hook writers moving between batches. The number is a row count and not a latency claim, because the latency of a delete depends on the store's page shape; it is sized well under the read path's fifteen-command drain of the measured legacy backlog so one batch stays a small hold on the write lock, and the verb loops until the backlog is gone or the operator's budget is met.

View Source
const ReclaimCap = 1024

ReclaimCap bounds how many orphan payload blobs one projection rebuild reclaims. It is 1024 because the rebuild runs on a READ command and not under the hook-invocation tier: the bound sizes a read command's added latency, not a host budget. At 1024 the 14,490 legacy orphans measured on a live store drain in about fifteen read commands. It is a constant and not configuration on purpose: the reclaim is a safety valve for a state the write order creates deliberately, not a retention policy.

Variables

This section is empty.

Functions

func CompareOccurrencesToEvidence added in v0.0.21

func CompareOccurrencesToEvidence(ctx context.Context, tx *sql.Tx, reader DefinitionResolver) error

CompareOccurrencesToEvidence verifies that the disposable occurrence and binding rows committed in tx equal the journal truth read through reader — the caller's transaction view. It is the conversion's semantic comparison: it catches a replay that inserted rows the evidence does not contain, dropped a row the evidence does contain, or wrote a different actor, envelope, digest or binding bytes. It never writes, never reclaims and never owns the transaction.

func QueryFingerprint

func QueryFingerprint(q model.OccurrenceQuery) (model.QueryFingerprint, error)

QueryFingerprint returns the normalized semantic filter identity used by cursors.

func RebuildOccurrencesInTx added in v0.0.21

func RebuildOccurrencesInTx(ctx context.Context, tx *sql.Tx, reader DefinitionResolver) error

RebuildOccurrencesInTx recreates the disposable occurrence projection from converted journal truth inside the CALLER's transaction, reading that truth through reader. It is the offline identity-format conversion's same-transaction replay: the reader is bound to the caller's SQL transaction, so every evidence row, effective actor and digest it reads reflects the converted rows that have not been committed yet.

RECLAIM IS STRUCTURALLY ABSENT, not disabled. This entry has no clock, no window and no savepoint, so no caller can reclaim by passing a zero window or a default: there is no reclaim call to reach. The migration must preserve the complete raw blob inventory, so reclaim does not belong in this path at all. The ordinary RebuildOccurrences wrapper keeps its reclaim, its snapshot timing and its outcome unchanged, and both entries share the decode/write functions above.

It never calls Begin, Commit or Rollback: transaction ownership stays with the migration, which rolls the whole thing back on any error.

Before the projection is touched, interpreted and consultation integrity is verified on the same transaction view: every v3 record resolves its complete semantic envelope and proves its pairing, and every consultation proves its exact-payload link. A migration marker must never publish over damaged semantic evidence.

func ReclaimFailureLine added in v0.0.9

func ReclaimFailureLine(failure error) string

ReclaimFailureLine renders the ONE diagnostic line a failed reclaim earns. The rebuild does not write it; the store that ran the rebuild does, on the diagnostic sink it was constructed with, so a test can pin the phrases against the text the operator reads.

Types

type DefinitionResolver added in v0.0.21

type DefinitionResolver interface {
	EvidencePageReader
	LookupCommitted(provenance.OperationID) (provenance.CommittedResult, error)
}

DefinitionResolver is the owner read capability replay validation needs. Evidence alone cannot authenticate an operation's actual committed result slots, so v2 envelope validation requires the committed-result lookup alongside the evidence query. It is satisfied by the ordinary journal and by provenance.TransactionEvidenceReader on the caller's transaction.

type EvidencePageReader added in v0.0.21

type EvidencePageReader interface {
	QueryEvidence(query provenance.EvidenceQuery) (provenance.EvidencePage, error)
}

EvidencePageReader is the narrow evidence surface the non-reclaiming replay needs. It is satisfied BOTH by the ordinary pooled journal (provenance.FactQueryAPI) and by provenance.TransactionEvidenceReader, which reads only the caller's active SQL transaction. The migration binds the transaction reader so every converted evidence row, actor join and digest it replays comes from the SAME uncommitted view, never a second connection.

type LinkReader

type LinkReader struct {
	Facts provenance.FactQueryAPI
}

LinkReader reads committed occurrence lineage links from journal truth. Like the occurrence Reader it is a disposable projection over the journal: it holds no state, reads under one pinned snapshot, and validates every row's digest before decoding (the F18 reader discipline). It is strictly read-side and never writes — materialization is the lineage command's gated write.

func (r LinkReader) Links() ([]model.LinkRecord, error)

Links returns every committed link record under a single consistent snapshot, sorted by committed journal identity. Each row is digest-validated against its canonical payload before decoding, so a tampered or noncanonical link row is refused rather than returned. The read is bounded page-by-page (provenance.MaxFactPageSize) and the pinned snapshot is re-asserted on every page so a concurrent write cannot yield a torn view.

type Reader

type Reader struct {
	DB    *sql.DB
	Facts provenance.FactQueryAPI
	// Defs is the owner read capability v3 envelope validation needs:
	// evidence queries plus the committed-result lookup on one snapshot. It
	// is wired by tasks.NewLifecycleReader. A v3 record read with nil Defs
	// fails closed rather than returning an unvalidated envelope.
	Defs receipt.DefinitionReader
}

func (Reader) Occurrences

func (Reader) Payload

func (r Reader) Payload(ctx context.Context, ref digest.Digest) ([]byte, error)

func (Reader) Records

type RebuildOptions added in v0.0.9

type RebuildOptions struct {
	Clock   receipt.Clock
	Window  time.Duration
	Reclaim ReclaimPolicy
}

RebuildOptions carries what the rebuild's reclaim needs beyond the store: the clock that supplies the snapshot instant, the longest window any writer may hold between its blob write and its journal append (the WorkflowResult tier of the injected profile), and the retention policy that widens the reclaim for the operator prune verb. All are required and all come from the store that owns the rebuild, never from a caller above it. There is no silent default for a bound: a zero window would reclaim in-flight blobs. The rebuild takes NO output stream; it RETURNS what the reclaim did, and the layer that owns a diagnostic stream prints the one failure line from that outcome.

type ReclaimOutcome added in v0.0.9

type ReclaimOutcome struct {
	Reclaimed      []digest.Digest
	ReclaimedBytes int64
	Failure        error
}

ReclaimOutcome is what the reclaim inside one rebuild did: the digests it deleted, how many bytes those rows held, and the failure it reported if it could not run. A failure never fails the rebuild; the rebuild returns it here and the caller that owns a diagnostic stream prints ReclaimFailureLine once.

func RebuildOccurrences

func RebuildOccurrences(ctx context.Context, journal provenance.Journal, db *sql.DB, options RebuildOptions) (ReclaimOutcome, error)

RebuildOccurrences derives the complete occurrence projection from journal evidence in JournalID order, and reclaims orphan payload blobs inside the same write transaction. The projection is disposable: journal truth is read first, then one transaction replaces all projection rows, and after the rows are re-inserted, where the projection is authoritative, the bounded reclaim runs under a savepoint (see reclaimOrphanPayloads for the safety argument). A reclaim failure never fails the rebuild: it is returned in the outcome, and the caller that owns a diagnostic stream prints it. Every read command reaches this through the tasks wrapper; there is no rebuild without the reclaim.

func (ReclaimOutcome) Count added in v0.0.9

func (o ReclaimOutcome) Count() int

Count is how many payload blobs this rebuild reclaimed.

type ReclaimPolicy added in v0.0.21

type ReclaimPolicy struct {
	MaxAge     time.Duration
	Cap        int
	KeepNewest bool
}

ReclaimPolicy bounds the orphan payload blobs one rebuild may delete. Its zero value is the READ-COMMAND policy: delete up to ReclaimCap orphans older than the writer window, and nothing else. The operator prune verb is the one caller that sets it, and each field is a deliberate widening of the read path's bound:

  • MaxAge is the operator's RETENTION age. An orphan is eligible only when it is older than max(Window, MaxAge), so a value below the writer window cannot shorten the in-flight safety floor; zero leaves the floor as the only age bound, which is the read path's behavior.
  • Cap bounds the rows this rebuild deletes. Zero uses ReclaimCap.
  • KeepNewest refuses to delete the row at the greatest written_at. The read path does not need it: a read command is bounded small, runs unattended, and its reclaim is a safety valve. The operator verb asks for a bulk delete, so it states the one exclusion an operator would assume.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL