Help us shape the future of Go! Take the 2026 Go Developer Survey

dict

package
v1.99.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 20, 2026 License: GPL-3.0 Imports: 7 Imported by: 0

Documentation

Overview

Package dict is yarilo's pluggable key-value store abstraction.

It provides a single contract — Dict + Tx + Iterator — that every feature needing durable per-user or per-mailbox state speaks: METADATA (RFC 5464), quota counters, ACL state, sieve script indices, future replication cursors. Concrete storage choices (a local JSON file for standalone, Redis for shared cluster state, PostgreSQL for operators who already run one) live behind drivers and are selected via config, not code.

Driver authors implement the Dict + Tx + Iterator interfaces and register themselves via Register("name", initFunc). Callers obtain a Dict via Open(Config{...}); the driver name in Config.Driver selects which factory runs.

Index

Constants

View Source
const (
	PathPrivate = "priv/"
	PathShared  = "shared/"
)

Path prefixes reserved at the application level so that per-user (priv/) and per-shared-resource (shared/) namespaces can co-exist in a single dict instance. Drivers do not interpret these prefixes; the convention is enforced by callers.

Variables

View Source
var ErrClosed = errors.New("dict: closed")

ErrClosed is returned from Dict methods invoked after Close. Drivers SHOULD wrap their own context with this sentinel.

View Source
var ErrUnknownDriver = errors.New("dict: unknown driver")

ErrUnknownDriver is returned by Open when Config.Driver names a driver that was never Registered. Most likely cause: missing blank-import of pkg/dict/drivers/all (or the specific driver package) in the binary's main package.

Functions

func Drivers

func Drivers() []string

Drivers returns the sorted list of currently-registered driver names. Used by yarilo-admin to print the available choices when the user passes an unknown driver name.

func Escape

func Escape(s string) string

Escape converts a string into a value safe to embed in a dict key path component. It percent-encodes '/' and '%' so that downstream path-splitting on '/' is unambiguous.

func PathMatches

func PathMatches(path, key string, recurse, exactKey bool) bool

PathMatches reports whether key falls under path according to the dict iteration semantics: when recurse is true, any key starting with path matches; otherwise key must lie directly under path with no further '/' beyond it. When exactKey is true, only key == path matches. Drivers reuse this in their Iterate implementations.

func Register

func Register(name string, init InitFunc)

Register adds a driver factory under name. It panics on duplicate registration so that import-cycle / two-drivers-same-name bugs surface at binary startup rather than first use. Driver packages call this from their init().

func Unescape

func Unescape(s string) string

Unescape is the inverse of Escape. Invalid percent sequences are passed through verbatim.

Types

type CommitResult

type CommitResult int

CommitResult is the outcome of Tx.Commit. A driver returns CommitOK on success, CommitNotFound when atomic-inc targeted a missing key, CommitFailed on a hard error, and CommitWriteUncertain when a remote write could not be confirmed (e.g. network timeout). Callers MAY treat CommitWriteUncertain as success-with-warning depending on their durability requirements.

const (
	CommitOK             CommitResult = 1
	CommitNotFound       CommitResult = 0
	CommitFailed         CommitResult = -1
	CommitWriteUncertain CommitResult = -2
)

type Config

type Config struct {
	// Driver names the registered driver to instantiate.
	Driver string `koanf:"driver"`
	// Settings is the driver-specific configuration tree. Keys and
	// types are defined by each driver's documentation.
	Settings map[string]any `koanf:"settings"`
}

Config selects and configures a dict driver. The Driver field names the registered driver (e.g. "file", "redis"); the driver- specific configuration lives in Settings, decoded by the driver's InitFunc from a generic map.

Keeping all driver-specific knobs under a single map[string]any (rather than a typed sub-struct per driver) lets pkg/config stay driver-agnostic — adding a new driver does not require editing config.go. Driver authors define the schema in their own package and decode at Open time.

type Dict

type Dict interface {
	// Lookup returns every value bound to key, plus found=true when at
	// least one value exists. The values slice is nil when found=false.
	// Drivers that store at most one value per key return a 1-element
	// slice. Honour ctx for cancellation.
	Lookup(ctx context.Context, set *OpSettings, key string) (values [][]byte, found bool, err error)

	// Iterate returns an Iterator over keys under path. The flags
	// bitmask controls recursion, sorting, value-skipping and exact-key
	// mode. The iterator MUST be Closed even when iteration stops early.
	Iterate(ctx context.Context, set *OpSettings, path string, flags IterFlag) (Iterator, error)

	// Begin opens a new transaction context. Mutations are buffered in
	// the Tx and applied atomically on Commit. Drivers that lack native
	// atomic multi-key writes simulate them via locking or write-ahead.
	Begin(ctx context.Context, set *OpSettings) (Tx, error)

	// ExpireScan asks the driver to drop expired entries. Drivers
	// without TTL support return nil (a no-op). Long-running drivers
	// (SQL) MAY do this in chunks bounded by ctx.
	ExpireScan(ctx context.Context) error

	// Wait blocks until every async operation kicked off by this Dict
	// has finished. Sync drivers (file, memory, fail) return nil
	// immediately. Reserved for the future redis async-pipeline path.
	Wait(ctx context.Context) error

	// Close releases all resources (open files, network connections,
	// background goroutines). Idempotent.
	Close() error

	// Name returns the driver name this Dict was opened with. Useful
	// for diagnostics and logging.
	Name() string
}

Dict is the read/write contract every driver implements.

All read methods (Lookup, Iterate) honour context cancellation; drivers that block on I/O MUST return ctx.Err() promptly. Mutations flow through Begin → Tx; there is no direct Set/Unset on Dict by design — the transaction-only mutation surface keeps drivers' write paths consistent. Close releases all driver resources; subsequent calls on a closed Dict return ErrClosed.

func Open

func Open(cfg Config) (Dict, error)

Open instantiates the driver named cfg.Driver with the supplied settings. Returns ErrUnknownDriver if the driver was not Registered.

type Factory added in v1.77.0

type Factory struct {
	// contains filtered or unexported fields
}

Factory opens per-user Dict instances by varexpanding %u/%h/%n/%d in settings. Callers open a Dict for each user operation and close it when done.

For the file driver, each user gets a separate JSON file; the path is derived by expanding %h in the "path" setting. For Redis, the prefix is derived by expanding %u in the "prefix" setting so that each user occupies a separate keyspace — configure prefix as "myapp:sieve:%u:".

func FixedFactory added in v1.77.0

func FixedFactory(d Dict) *Factory

FixedFactory returns a Factory that always yields the same pre-existing Dict. Close on the returned Dict is a no-op. Intended for unit tests.

func NewFactory added in v1.77.0

func NewFactory(cfg Config) *Factory

NewFactory creates a Factory from a driver config. Settings strings may contain %-variables (%u, %h, %n, %d, %%); they are expanded at Open time with the caller-supplied username and homeDir.

func (*Factory) Open added in v1.77.0

func (f *Factory) Open(username, homeDir string) (Dict, error)

Open opens a Dict for the given user. The caller MUST Close the returned Dict when done; for the file driver this flushes writes, for Redis it releases the connection pool.

type InitFunc

type InitFunc func(cfg Config) (Dict, error)

InitFunc constructs a Dict from a Config. Drivers register one of these via Register at init() time.

type IterFlag

type IterFlag uint32

IterFlag is a bitmask of options for Dict.Iterate.

const (
	// IterRecurse descends into sub-hierarchies. Without it, iterating
	// "foo/" returns "foo/a" but not "foo/a/b".
	IterRecurse IterFlag = 1 << iota
	// IterSortByKey returns rows in ascending key order.
	IterSortByKey
	// IterSortByValue returns rows in ascending value order.
	// Drivers that cannot sort by value MAY fall back to insertion order.
	IterSortByValue
	// IterNoValue skips loading values — useful for cheap key listings.
	IterNoValue
	// IterExactKey returns every value bound to the exact key (multi-value
	// dicts). Equivalent to Lookup but iterator-shaped.
	IterExactKey
)

type Iterator

type Iterator interface {
	// Next advances to the next row. Returns false when iteration ends
	// (Err may carry an error in that case) or when ctx was cancelled.
	Next() bool
	// Key returns the current row's key. Valid only after a true Next().
	Key() string
	// Values returns every value for the current key. Single-value
	// drivers return a 1-element slice. Nil when IterNoValue was set.
	Values() [][]byte
	// Err returns the first iteration error, or nil if iteration ended
	// cleanly. Always check after the loop.
	Err() error
	// Close releases resources. Idempotent. Safe to call before Next()
	// returns false (early-exit iteration).
	Close() error
}

Iterator yields key/value pairs from Dict.Iterate. Drivers MUST support repeated Next() until it returns false; after that, Err() reports any I/O error. Close() releases iterator resources and is safe to call multiple times.

type MemoryTx

type MemoryTx struct {
	Ops []TxOp
}

MemoryTx is a reusable transaction buffer for drivers that lack native atomic multi-key writes (file, memory). Drivers embed one of these in their concrete Tx type and call Apply at Commit time with a per-driver callback that writes the buffered ops.

func (*MemoryTx) AtomicInc

func (t *MemoryTx) AtomicInc(key string, delta int64)

AtomicInc buffers an AtomicInc operation.

func (*MemoryTx) Reset

func (t *MemoryTx) Reset()

Reset drops all buffered ops so the MemoryTx can be reused.

func (*MemoryTx) Set

func (t *MemoryTx) Set(key string, value []byte)

Set buffers a Set operation.

func (*MemoryTx) Unset

func (t *MemoryTx) Unset(key string)

Unset buffers an Unset operation.

type OpSettings

type OpSettings struct {
	// Username of the operation's owner. Required by drivers that
	// scope keys per-user (e.g. file path "%h/.metadata" relies on it).
	Username string
	// HomeDir of the user, if known. Used for "%h" expansion in the
	// file driver's path; left empty when the caller does not have it.
	HomeDir string
	// ExpireSecs, when non-zero, asks the driver to add a TTL of this
	// many seconds to every key written by the transaction or set call.
	// Drivers without TTL support silently ignore it.
	ExpireSecs uint32
	// NoSlownessWarning suppresses the driver's slow-commit warning log.
	// Set for transactions whose latency does not matter (e.g. best-
	// effort metadata writes).
	NoSlownessWarning bool
	// HideLogValues redacts values from any debug/trace logs the driver
	// might emit. Set when values contain sensitive data.
	HideLogValues bool
}

OpSettings is the per-call context passed to every dict operation. It carries the username and home dir for path expansion (file driver), per-call TTL (drivers that support it), and a few hints used by remote drivers (no-slowness-warning, hide-log-values).

type Tx

type Tx interface {
	// Set assigns value to key. Overwrites any prior value.
	// For multi-value drivers (SQL), appends to the value list when
	// the driver-specific "append" flag is enabled; otherwise replaces.
	Set(key string, value []byte) error
	// Unset removes key (and all its values for multi-value drivers).
	// Unsetting a missing key is not an error.
	Unset(key string) error
	// AtomicInc atomically adds delta (positive or negative) to the
	// integer value of key. If key is missing at commit time, Commit
	// returns CommitNotFound. Values are stored as decimal strings.
	AtomicInc(key string, delta int64) error
	// Commit flushes the buffered mutations. Returns CommitOK on
	// success; see CommitResult docs for the other values.
	Commit() (CommitResult, error)
	// Rollback discards every buffered mutation. Idempotent.
	Rollback() error
}

Tx buffers mutations until Commit. Drivers MAY apply writes optimistically (e.g. SQL transactions) or defer them entirely until commit time (e.g. memory driver). Rollback discards every pending mutation. Set/Unset/AtomicInc are NEVER visible to other Dict callers before Commit.

type TxOp

type TxOp struct {
	Kind  TxOpKind
	Key   string
	Value []byte
	Delta int64
}

TxOp is one buffered transaction operation. Set carries Value; AtomicInc carries Delta; Unset carries neither.

func (TxOp) FormatOp

func (o TxOp) FormatOp() string

FormatOp returns a short human-readable description of an op, for use in error messages and CLI dumps.

type TxOpKind

type TxOpKind int

TxOpKind is one buffered operation in a MemoryTx.

const (
	OpSet TxOpKind = iota
	OpUnset
	OpAtomicInc
)

Directories

Path Synopsis
Package dicttest is a contract test suite that any pkg/dict driver can run against.
Package dicttest is a contract test suite that any pkg/dict driver can run against.
drivers
all
Package all blank-imports every in-tree dict driver so a single import enables all of them in the importing binary.
Package all blank-imports every in-tree dict driver so a single import enables all of them in the importing binary.
Package fail is a dict driver that fails every operation.
Package fail is a dict driver that fails every operation.
Package file is a JSON-backed local-filesystem dict driver.
Package file is a JSON-backed local-filesystem dict driver.
Package memory is an in-process dict driver backed by a sync.Map.
Package memory is an in-process dict driver backed by a sync.Map.
Package redis is a Redis-backed dict driver.
Package redis is a Redis-backed dict driver.
Package sql is a database/sql-backed dict driver.
Package sql is a database/sql-backed dict driver.
Package varexpand performs %-variable substitution for templated path / prefix strings used by dict drivers.
Package varexpand performs %-variable substitution for templated path / prefix strings used by dict drivers.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL