storage

package
v0.3.0-20260708152012-... Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 8, 2026 License: Apache-2.0 Imports: 4 Imported by: 0

README

Storage

Pluggable persistence interfaces for SubmitQueue entities (requests, batches, dependents, logs, etc.). Implementations live under extension/storage/<impl>/.

Optimistic locking contract

Entities that support concurrent mutation carry an int32 Version field. Updates are conditional on the version: the write only succeeds if the persisted version matches the caller's expected version. On mismatch, the implementation returns storage.ErrVersionMismatch.

Version arithmetic is owned by the controller, not the store. Update methods take both oldVersion (the where-clause guard) and newVersion (the value to write):

UpdateState(ctx, id, oldVersion, newVersion int32, newState entity.RequestState) error

The store performs a pure conditional write — it does not compute oldVersion + 1 internally. This keeps the in-memory entity and the persisted row in sync without the storage layer mutating values the caller didn't supply.

Caller pattern
newVersion := entity.Version + 1
if err := store.UpdateState(ctx, entity.ID, entity.Version, newVersion, newState); err != nil {
    return err // entity.Version unchanged on failure — safe to retry
}
entity.Version = newVersion // only after the write succeeded

The post-success assignment matters whenever the entity is read again later in the same flow. Pre-incrementing in memory before the call is a bug pattern: if the call fails and the caller swallows the error, the in-memory version is now ahead of the database and subsequent updates will fail with ErrVersionMismatch for non-obvious reasons.

Documentation

Index

Constants

This section is empty.

Variables

View Source
var ErrAlreadyExists = errors.New("record already exists")

ErrAlreadyExists is returned by storage implementations when attempting to create a record with an ID that already exists.

View Source
var ErrNotFound = errors.New("record not found")

ErrNotFound is returned by storage implementations when the requested record is not found in the database.

View Source
var ErrVersionMismatch = errors.New("version mismatch")

ErrVersionMismatch is returned by storage implementations when the expected entity version does not match the current version of the object. This is used to implement an optimistic locking mechanism, allowing multiple clients to update the same entity concurrently and either retry or implement idempotent operations.

Functions

func IsNotFound

func IsNotFound(err error) bool

IsNotFound returns true if any error in the error chain is a ErrNotFound.

func WrapNotFound

func WrapNotFound(err error) error

WrapNotFound wraps ErrNotFound with the original error from the storage implementation.

Types

type BatchDependentStore

type BatchDependentStore interface {
	// Get retrieves the batch dependent by batch ID.
	// If the batch contains no dependents, the returned BatchDependent will have an empty Dependents list.
	// Returns ErrNotFound if the batch itself is not found, which should never happen in steady-state system and
	// therefore does not need a special handling.
	Get(ctx context.Context, batchID string) (entity.BatchDependent, error)

	// Create creates a new batch dependent.
	// Returns ErrAlreadyExists if the entry already exists for the given batch ID.
	Create(ctx context.Context, batchDependent entity.BatchDependent) error

	// UpdateDependents updates the dependents of a batch dependent and the version to newVersion
	// if the current persisted version matches oldVersion. If versions do not match, returns ErrVersionMismatch.
	// Version arithmetic is owned by the caller; the store performs a pure conditional write.
	UpdateDependents(ctx context.Context, batchID string, oldVersion, newVersion int32, dependents []string) error
}

BatchDependentStore is an interface that defines methods for managing batch dependent information in the database.

A BatchDependent is a reverse index ("batches that depend on me") paired one-to-one with a Batch. The batch-creation flow always calls Create here before creating the Batch itself, so every active Batch is guaranteed to have a corresponding BatchDependent row. Lookups via Get are only performed for batch IDs returned from the active-batch set, meaning a missing row indicates data corruption or out-of-band manipulation rather than a normal "not found" outcome. ErrNotFound is therefore part of the contract for completeness but is not expected to be returned in steady-state operation.

type BatchStore

type BatchStore interface {
	// Get retrieves a batch by ID. Returns ErrNotFound if the batch is not found.
	Get(ctx context.Context, id string) (entity.Batch, error)

	// Create creates a new batch. The batch must have a unique ID already assigned.
	// Returns ErrAlreadyExists if a batch with the same ID already exists.
	Create(ctx context.Context, batch entity.Batch) error

	// UpdateState updates the state of a batch to newState and the version to newVersion
	// if the current persisted version matches oldVersion. If versions do not match, returns ErrVersionMismatch.
	// Version arithmetic is owned by the caller; the store performs a pure conditional write.
	UpdateState(ctx context.Context, id string, oldVersion, newVersion int32, newState entity.BatchState) error

	// UpdateScoreAndState atomically updates the score and state of a batch and the version to newVersion
	// if the current persisted version matches oldVersion. If versions do not match, returns ErrVersionMismatch.
	// Version arithmetic is owned by the caller; the store performs a pure conditional write.
	UpdateScoreAndState(ctx context.Context, id string, oldVersion, newVersion int32, score float64, newState entity.BatchState) error

	// GetByQueueAndStates retrieves all batches that belong to the given queue and are in the given states.
	GetByQueueAndStates(ctx context.Context, queue string, states []entity.BatchState) ([]entity.Batch, error)
}

BatchStore is an interface that defines methods for managing batches in the database.

type BuildStore

type BuildStore interface {
	// Get retrieves a build by ID. Returns ErrNotFound if the build is not found.
	Get(ctx context.Context, id string) (entity.Build, error)

	// Create creates a new build. The build must have a unique ID already assigned.
	// Returns ErrAlreadyExists if a build with the same ID already exists.
	Create(ctx context.Context, build entity.Build) error

	// UpdateStatus updates the status of a build.
	UpdateStatus(ctx context.Context, id string, newStatus entity.BuildStatus) error
}

BuildStore is an interface that defines methods for managing builds in the database.

type ChangeStore

type ChangeStore interface {
	// Create persists a single ChangeRecord (identity + Details) in one write. A
	// primary-key conflict on (Queue, URI, RequestID) is silently ignored, which makes
	// the call idempotent under queue redeliveries of the same request (first write
	// wins). Records belonging to different requests do not conflict on the PK —
	// cross-request overlap is detected by GetByURI, not by Create.
	Create(ctx context.Context, record entity.ChangeRecord) error

	// GetByURI returns every ChangeRecord for the given (queue, uri). Multiple
	// requests can have claimed the same URI over time, so the slice may have
	// any number of entries; an empty slice means no claim has ever been
	// recorded for this URI in this queue.
	//
	// The store does not filter by request_id or by the owning request's
	// state — callers that want to skip self filter by RequestID, and callers
	// that want only live owners consult RequestStore for liveness.
	GetByURI(ctx context.Context, queue string, uri string) ([]entity.ChangeRecord, error)
}

ChangeStore manages per-URI claim records for in-flight land requests. Each row records that a specific URI was claimed by a specific request, scoped to a queue. The (Queue, URI, RequestID) triple is the immutable identity of a record, and a record's Details are captured at claim time and never updated — records are immutable.

The interface is intentionally per-record / per-URI so that any backend (SQL, DynamoDB, Bigtable, …) can implement it without needing batch-atomicity or multi-key query support. Callers loop when they have multiple URIs to claim or check; the typical request has a small number of URIs (a single PR or a short stack), so the loop overhead is negligible.

type RequestLogStore

type RequestLogStore interface {
	// Insert appends a new request log record. Timestamps should be generated by the caller and not modified by the implementation.
	Insert(ctx context.Context, log entity.RequestLog) error

	// List retrieves all request log records for a given request ID, ordered by timestamp ascending.
	// Returns ErrNotFound if no records exist for the given request ID.
	List(ctx context.Context, requestID string) ([]entity.RequestLog, error)
}

RequestLogStore is an interface that defines methods for managing request log records in an append-only database. Request logs are used to reconcile request statuses with eventual consistency into a separate database from RequestStore.

type RequestStore

type RequestStore interface {
	// Get retrieves a land request by ID. Returns ErrNotFound if the request is not found.
	Get(ctx context.Context, id string) (entity.Request, error)

	// Create creates a new land request. The request must have a unique ID already assigned.
	// Returns ErrAlreadyExists if a request with the same ID already exists.
	Create(ctx context.Context, request entity.Request) error

	// UpdateState updates the state of a land request to newState and the version to newVersion
	// if the current persisted version matches oldVersion. If versions do not match, returns ErrVersionMismatch.
	// Version arithmetic is owned by the caller; the store performs a pure conditional write.
	UpdateState(ctx context.Context, id string, oldVersion, newVersion int32, newState entity.RequestState) error
}

RequestStore is an interface that defines methods for managing land requests in the database.

type SpeculationTreeStore

type SpeculationTreeStore interface {
	// Get retrieves the speculation tree by batch ID.
	// Returns ErrNotFound if the speculation tree is not found.
	Get(ctx context.Context, batchID string) (entity.SpeculationTree, error)

	// Create creates a new speculation tree.
	// Returns ErrAlreadyExists if the entry already exists.
	Create(ctx context.Context, speculationTree entity.SpeculationTree) error

	// UpdateSpeculations updates the speculations of a speculation tree.
	// Returns ErrNotFound if the speculation tree is not found.
	UpdateSpeculations(ctx context.Context, batchID string, speculations []entity.SpeculationInfo) error
}

SpeculationTreeStore is an interface that defines methods for managing speculation trees in the database.

type Storage

type Storage interface {
	// GetRequestStore returns the RequestStore instance.
	GetRequestStore() RequestStore

	// GetChangeStore returns the ChangeStore instance.
	GetChangeStore() ChangeStore

	// GetBatchStore returns the BatchStore instance.
	GetBatchStore() BatchStore

	// GetBatchDependentStore returns the BatchDependentStore instance.
	GetBatchDependentStore() BatchDependentStore

	// GetBuildStore returns the BuildStore instance.
	GetBuildStore() BuildStore

	// GetSpeculationTreeStore returns the SpeculationTreeStore instance.
	GetSpeculationTreeStore() SpeculationTreeStore

	// GetRequestLogStore returns the RequestLogStore instance.
	GetRequestLogStore() RequestLogStore

	// Close closes the storage and all underlying connections. Should only be called once at the end of the program.
	Close() error
}

Storage is a factory interface that aggregates all entity stores into a single injectable dependency.

Directories

Path Synopsis
Package mock is a generated GoMock package.
Package mock is a generated GoMock package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL