Documentation
¶
Overview ¶
Package kv holds the shared bbolt store primitives used by both the core subflux store (internal/store) and the auth store (internal/authstore).
It is a leaf package: it depends only on the standard library and go.etcd.io/bbolt and MUST NOT import internal/store or internal/authstore. The dependency DAG is:
api <- kv <- {store, authstore} <- {main, cli}
The primitives are:
- Key encoders (Be64, Join/Split, TimeIndexKey) that produce keys whose byte-lexicographic order matches their semantic order, so bbolt's ordered cursors and Seek give chronological / insertion-order scans and correct prefix queries.
- A versioned-JSON value codec (Encode/Decode). Values are JSON; keys are binary. The schema version is a small scalar the caller stores in its own meta key (the core store uses core_schema_version, the auth store uses auth_schema_version); kv provides the codec mechanism and the meta scalar helpers (GetUint64/PutUint64), the callers own the keys. DecodeOrHandle centralises the fail-closed vs tolerant-skip policy so a derived-bucket read can skip-with-warning while an auth/lock/uniqueness read fails closed.
- The surrogate-id helper NextID, wrapping bbolt's Bucket.NextSequence.
- The generic secondary-index maintenance helpers PutIndexed and DeleteIndexed, which keep declared IndexSpec index buckets and declared CounterSpec meta counters consistent with their primary in the same all-or-nothing bbolt transaction.
Index ¶
- Constants
- func Be64(v uint64) []byte
- func Decode(data []byte, v any) error
- func DecodeBe64(b []byte) (uint64, bool)
- func DecodeOrHandle(mode DecodeMode, bucket string, key, data []byte, v any) (skip bool, err error)
- func DeleteIndexed[T any](tx *bolt.Tx, primaryBucket string, key []byte, indexes []IndexSpec[T], ...) (existed bool, err error)
- func Encode(v any) ([]byte, error)
- func GetUint64(b *bolt.Bucket, key []byte) (uint64, bool)
- func Join(parts ...string) []byte
- func NextID(b *bolt.Bucket) (id uint64, key []byte, err error)
- func PutIndexed[T any](tx *bolt.Tx, primaryBucket string, key []byte, value *T, ...) error
- func PutUint64(b *bolt.Bucket, key []byte, v uint64) error
- func ReadCounter(b *bolt.Bucket, key []byte) int64
- func Split(key []byte) []string
- func SplitTimeIndexKey(key []byte) (unixNano int64, primary []byte, ok bool)
- func TimeIndexKey(t time.Time, primary []byte) []byte
- type CounterSpec
- type DecodeMode
- type IndexSpec
Constants ¶
const Sep byte = 0x00
Sep is the single NUL byte used to join the components of a composite text key. Key components are NUL-free UTF-8 text, so the separator unambiguously delimits component boundaries and keeps prefix scans correct.
Variables ¶
This section is empty.
Functions ¶
func Be64 ¶
Be64 encodes v as an 8-byte big-endian value. Big-endian preserves numeric ordering under byte-lexicographic comparison, so be64-encoded keys sort by their numeric value: surrogate ids sort in insertion order and unixnano timestamps sort chronologically. A forward cursor therefore walks oldest to newest and a Seek jumps to a numeric cutoff exactly.
func Decode ¶
Decode deserialises a JSON bucket value into v, returning an error on malformed input. Callers decide what to do with that error: a read over a derived core bucket skips the record with a warning (the next scan rebuilds it), while an auth, lock, or uniqueness read fails closed. DecodeOrHandle centralises that decision.
func DecodeBe64 ¶
DecodeBe64 decodes an 8-byte big-endian value produced by Be64. It returns (value, true) for an 8-byte input and (0, false) otherwise.
func DecodeOrHandle ¶
DecodeOrHandle decodes data into v and applies the decode-failure policy for mode. On success it returns (false, nil). On a decode error it returns (true, nil) after logging a warning in TolerantSkip mode, or (false, err) in FailClosed mode. Callers walking a cursor use the skip flag to continue to the next record.
func DeleteIndexed ¶
func DeleteIndexed[T any](tx *bolt.Tx, primaryBucket string, key []byte, indexes []IndexSpec[T], counters []CounterSpec) (existed bool, err error)
DeleteIndexed removes key from primaryBucket and the declared index entries, and decrements the declared counters, all in the supplied transaction. It reads the prior value to derive the exact index entries to delete. It returns existed=false (and makes no changes) when the key is absent, so a delete of a missing key is a no-op and the operation is idempotent.
As in PutIndexed, the prior value is decoded only when index entries must be derived from it: with no declared indexes a corrupt prior value cannot block its own deletion (the self-heal path for unindexed derived buckets), while indexed buckets keep failing closed so stale index entries are never silently orphaned.
func Encode ¶
Encode serialises a bucket value as JSON. Bucket values are JSON (stdlib-only, debuggable, and additively forward-compatible without a migration script); bucket keys remain binary.
func GetUint64 ¶
GetUint64 reads an 8-byte big-endian scalar (a counter or a schema version) from bucket b. It returns (value, true) when the key holds an 8-byte value and (0, false) otherwise.
func Join ¶
Join builds a composite key by joining the components with a single Sep (0x00) byte. Components must be NUL-free UTF-8 text so the separator unambiguously delimits boundaries; this keeps prefix scans correct (a scan for "mt\x00mid\x00" never matches a value that merely shares a prefix of the next component). Pass at least one component for a clean Split round-trip.
func NextID ¶
NextID allocates the next surrogate id for bucket b via bbolt's monotonic Bucket.NextSequence and returns it both as the raw uint64 and its Be64 key encoding. The be64 encoding makes the id sort in insertion order, so a surrogate-keyed bucket walks oldest to newest and an "id DESC" tiebreak is recoverable. Must be called inside a read-write transaction.
func PutIndexed ¶
func PutIndexed[T any](tx *bolt.Tx, primaryBucket string, key []byte, value *T, indexes []IndexSpec[T], counters []CounterSpec) error
PutIndexed writes value at key in primaryBucket and maintains the declared indexes and counters, all in the supplied read-write transaction. It is the single chokepoint for the secondary-index maintenance invariant: it reads the prior value (if any) to delete its stale index entries, then writes the new primary value, then adds the fresh index entries (including any projected index value). Counters are incremented only on a genuine insert (not on an update of an existing key). Because bbolt transactions are all-or-nothing, an index can never diverge from its primary.
The value is JSON-encoded via Encode; T must be JSON-serialisable. All referenced buckets must already exist (the core store owns bucket bootstrap).
func PutUint64 ¶
PutUint64 writes v as an 8-byte big-endian scalar at key in bucket b. It is used for maintained counters and for the caller-owned schema-version meta keys (core_schema_version, auth_schema_version).
func ReadCounter ¶
ReadCounter returns the maintained counter at key in bucket b as a signed int64 (counters are non-negative), or 0 when absent.
func Split ¶
Split parses a composite key built by Join back into its components, splitting on the Sep byte. It is the inverse of Join for keys built from one or more components.
func SplitTimeIndexKey ¶
SplitTimeIndexKey parses a key built by TimeIndexKey into its unixnano timestamp and primary key. It returns ok=false if the key is too short or is missing the separator. The returned primary slice aliases key; copy it if it must outlive the enclosing bbolt transaction.
func TimeIndexKey ¶
TimeIndexKey builds a time-ordered secondary-index key of the form be64(unixnano) 0x00 primary. The big-endian timestamp makes a forward cursor walk oldest to newest and a Seek to be64(cutoff) land exactly at a time cutoff; the primary key is appended (after a Sep) so the index entry is unique per primary and dereferences back to it. The primary component may itself contain NUL bytes (e.g. a composite key) because the fixed 8-byte timestamp prefix makes the boundary unambiguous.
The timestamp is t.UnixNano() reinterpreted as uint64; ordering is correct for the post-epoch timestamps this store deals with.
Types ¶
type CounterSpec ¶
type CounterSpec struct {
// Bucket holds the counter (typically the meta bucket).
Bucket string
// Key is the counter's key within Bucket.
Key []byte
// Delta is added on insert and subtracted on delete; zero means 1.
Delta int64
}
CounterSpec declares a maintained integer counter held in a meta bucket. PutIndexed adds Delta on a genuine insert and DeleteIndexed subtracts it on a delete, in the same transaction as the primary write, so the counter answers O(1) count queries (e.g. Stats, TotalSubtitleFiles) that bbolt cannot serve from a live count. A zero Delta is treated as 1. Counters never go below zero.
type DecodeMode ¶
type DecodeMode int
DecodeMode selects how a cursor-walking read handles a record it cannot decode.
const ( // FailClosed treats an undecodable record as a fatal error for the read. // Use it for auth buckets and for lock- or uniqueness-bearing reads, where // silently skipping a row could drop a security-relevant record (treat any // affected lock as held rather than absent). FailClosed DecodeMode = iota // TolerantSkip logs a warning and skips an undecodable record so the scan // continues. Use it for derived core buckets, which the next full scan // rebuilds. TolerantSkip )
type IndexSpec ¶
type IndexSpec[T any] struct { // Key derives the index entry key from the primary key and decoded value. Key func(primaryKey []byte, value *T) []byte // Value, when non-nil, derives the projected index entry value. Value func(primaryKey []byte, value *T) []byte // Bucket is the name of the secondary-index bucket. Bucket string }
IndexSpec declares one secondary index that PutIndexed and DeleteIndexed keep consistent with a primary record. T is the decoded primary value type.
The helper derives the index entry key from the primary key and value via Key. When Value is non-nil, its result is stored as the index entry's value: a small projection of query-relevant fields that lets a hot-path read answer from the index walk alone, without dereferencing the primary record. When Value is nil, the index is existence-only (an empty value is stored).