storage

package
v0.2.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 6, 2026 License: MIT Imports: 24 Imported by: 0

Documentation

Overview

Package storage provides shared database connections for all features. This abstraction allows multiple features (audit logging, IAM, guardrails) to share a single database connection.

Index

Constants

View Source
const (
	TypeSQLite     = "sqlite"
	TypePostgreSQL = "postgresql"
	TypeMongoDB    = "mongodb"
)

Type constants for storage backends

View Source
const BackgroundIndexRetry = time.Minute

BackgroundIndexRetry is how often a builder checks again while another replica holds its lock, or after a build failed.

View Source
const DefaultMongoDatabase = "gomodel"

DefaultMongoDatabase is the database used when neither the explicit Database config nor the connection string names one.

View Source
const LegacySQLitePath = "data/gomodel.db"

LegacySQLitePath is the historical default database location, relative to the working directory. It stays the default whenever a ./data directory exists (existing deployments, the Docker image), so upgrades never move anyone's database.

Variables

This section is empty.

Functions

func DefaultSQLitePath

func DefaultSQLitePath() string

DefaultSQLitePath returns the database path used when none is configured: LegacySQLitePath when a ./data directory already exists, otherwise the OS-conventional per-user data directory (see platformdir.DataFile).

func MongoSwapSubfields added in v0.1.100

func MongoSwapSubfields(field string, current, next map[string]string) (filter bson.M, update mongo.Pipeline)

MongoSwapSubfields builds a conditional update of string values inside the subdocument at field: the filter expression matches only while every key in current still holds its value there, and the update pipeline sets each of those keys to its value in next. Keys are addressed with $getField and $setField rather than dotted paths, so names containing "." or starting with "$" work too (MongoDB 5.0+). Compare key by key rather than the whole subdocument: a stored subdocument's key order is not stable.

func MongoUnexpiredFilter

func MongoUnexpiredFilter(id string, now time.Time) bson.M

MongoUnexpiredFilter matches the document with the given id whose expiry is unset or still in the future.

func ResolveBackend

func ResolveBackend[T any](
	store Storage,
	sqlite func(*sql.DB) (T, error),
	postgresql func(*pgxpool.Pool) (T, error),
	mongodb func(*mongo.Database) (T, error),
) (T, error)

ResolveBackend dispatches to the callback matching the concrete storage backend.

func ResolveSQLBackend added in v0.1.60

func ResolveSQLBackend[T any](
	ctx context.Context,
	store Storage,
	sqlStore func(sqlx.DB) (T, error),
	mongodb func(*mongo.Database) (T, error),
) (T, error)

ResolveSQLBackend dispatches a storage backend to one of two constructors: a single SQL one serving both SQLite and PostgreSQL through sqlx.DB, and a MongoDB one.

Every SQL store is served through it; ResolveBackend is the dispatch underneath.

func RunCleanupLoop

func RunCleanupLoop(stop <-chan struct{}, interval time.Duration, cleanupFn func())

RunCleanupLoop runs a cleanup function immediately and then at the given interval until the stop channel is closed. Store backends use it for periodic retention sweeps.

func UnixOrZero

func UnixOrZero(t time.Time) int64

UnixOrZero converts a time into a unix-seconds retention column value, mapping the zero time to the 0 "never expires" sentinel.

func UnixTime

func UnixTime(sec int64) time.Time

UnixTime converts a unix-seconds retention column into a time, mapping the 0 "never expires" sentinel to the zero time.

func WriterID added in v0.2.2

func WriterID() string

WriterID returns the identifier this process stamps on the audit and usage rows it writes. Another replica's live log view uses it to tell the rows it wrote itself, which it already streamed, from the rows of its peers.

Types

type BackgroundIndex added in v0.2.2

type BackgroundIndex struct {
	// Name is the index name, as to_regclass resolves it.
	Name string
	// Create builds the index: CREATE INDEX CONCURRENTLY IF NOT EXISTS ...
	Create string
	// Label names the index in log messages, e.g. "usage auth key".
	Label string
}

BackgroundIndex is a PostgreSQL index added to a table that may already hold years of history. It is built CONCURRENTLY off the startup path, so the table keeps accepting writes while the build scans it.

type BackgroundIndexBuilder added in v0.2.2

type BackgroundIndexBuilder struct {
	Pool *pgxpool.Pool
	// LockKey is the advisory lock key. Builders of one table share a key,
	// since PostgreSQL runs one concurrent build per table at a time.
	LockKey int64
	// Retry is the pause between attempts; BackgroundIndexRetry when zero.
	Retry time.Duration
}

BackgroundIndexBuilder builds BackgroundIndexes one at a time, and only while it holds a session advisory lock, so of all the replicas sharing a database one builds and the rest wait for it.

The lock is taken without waiting. A blocking wait would hold a snapshot that the other replica's concurrent build has to wait out, deadlocking the two. It also means an invalid index the lock holder finds is a failed or abandoned build, never one in progress, so it is safe to drop and rebuild.

func (BackgroundIndexBuilder) Build added in v0.2.2

func (b BackgroundIndexBuilder) Build(ctx context.Context, indexes ...BackgroundIndex) bool

Build tries each index that is not valid yet, one at a time and in order, and repeats every Retry until all of them are valid. An index that keeps failing does not hold up the ones after it. It reports whether all are valid, which is false only when ctx ends first.

func (BackgroundIndexBuilder) Start added in v0.2.2

func (b BackgroundIndexBuilder) Start(indexes ...BackgroundIndex) (stop func())

Start builds the indexes in the background until each is valid. The returned stop cancels the build and waits for it to return; it is safe to call more than once.

func (BackgroundIndexBuilder) TryBuild added in v0.2.2

TryBuild makes one attempt and reports whether the index is valid when it returns. It does nothing but check when another session holds the lock.

type Config

type Config struct {
	// Type specifies the storage backend: "sqlite", "postgresql", or "mongodb"
	Type string

	// SQLite configuration
	SQLite SQLiteConfig

	// PostgreSQL configuration
	PostgreSQL PostgreSQLConfig

	// MongoDB configuration
	MongoDB MongoDBConfig
}

Config holds storage configuration

type HealthChecker

type HealthChecker interface {
	Ping(ctx context.Context) error
}

HealthChecker is implemented by storage backends that can verify connectivity to the underlying database. All concrete backends satisfy it; readiness checks type-assert against this interface.

type MongoDBConfig

type MongoDBConfig struct {
	// URL is the connection string (e.g., mongodb://localhost:27017)
	URL string
	// Database is the database name (default: gomodel)
	Database string
}

MongoDBConfig holds MongoDB-specific configuration

type MongoDBStorage

type MongoDBStorage interface {
	Storage
	Database() *mongo.Database
}

MongoDBStorage exposes a MongoDB database handle.

func NewMongoDB

func NewMongoDB(ctx context.Context, cfg MongoDBConfig) (MongoDBStorage, error)

NewMongoDB creates a new MongoDB storage connection.

type PostgreSQLConfig

type PostgreSQLConfig struct {
	// URL is the connection string (e.g., postgres://user:pass@localhost/dbname)
	URL string
	// MaxConns is the maximum connection pool size (default: 10)
	MaxConns int
}

PostgreSQLConfig holds PostgreSQL-specific configuration

type PostgreSQLStorage

type PostgreSQLStorage interface {
	Storage
	Pool() *pgxpool.Pool
}

PostgreSQLStorage exposes a PostgreSQL connection pool.

func NewPostgreSQL

func NewPostgreSQL(ctx context.Context, cfg PostgreSQLConfig) (PostgreSQLStorage, error)

NewPostgreSQL creates a new PostgreSQL storage connection. It creates a connection pool for efficient connection reuse.

type RecentWrite added in v0.2.2

type RecentWrite struct {
	ID        string
	WrittenAt int64
}

RecentWrite names one persisted row and when it was written: the writer's clock at the batch write that stored it, in unix microseconds.

type RowScanner

type RowScanner interface {
	Scan(dest ...any) error
}

RowScanner is the single-row result shape shared by database/sql and pgx.

type SQLiteConfig

type SQLiteConfig struct {
	// Path is the database file path (default: DefaultSQLitePath())
	Path string
}

SQLiteConfig holds SQLite-specific configuration

type SQLiteStorage

type SQLiteStorage interface {
	Storage
	DB() *sql.DB
}

SQLiteStorage exposes a SQLite database handle.

func NewSQLite

func NewSQLite(cfg SQLiteConfig) (SQLiteStorage, error)

NewSQLite creates a new SQLite storage connection. It enables WAL mode for better concurrent read/write performance.

type Storage

type Storage interface {
	Close() error
}

Storage manages the lifecycle of a shared storage backend.

func New

func New(ctx context.Context, cfg Config) (Storage, error)

New creates a new Storage based on the configuration. It validates the configuration and establishes the database connection.

type WriteStamp added in v0.2.2

type WriteStamp struct {
	WriterID  string `bson:"writer_id,omitempty"`
	WrittenAt int64  `bson:"written_at,omitempty"`
}

WriteStamp marks the batch write that persisted an audit or usage entry: the writing process (WriterID) and its clock in unix microseconds. Entries carry a pointer to it so an unstamped entry costs one word; the MongoDB stores inline it as the writer_id and written_at fields.

func (*WriteStamp) SQLValues added in v0.2.2

func (s *WriteStamp) SQLValues() (writerID, writtenAt any)

SQLValues returns the writer_id and written_at column values, both NULL for an unstamped entry so it stays out of the partial written_at index.

Directories

Path Synopsis
Package mongotest runs a store's test suite against MongoDB.
Package mongotest runs a store's test suite against MongoDB.
Package sqlutil provides small SQL query-building and value-conversion helpers shared by the SQL-backed stores and readers.
Package sqlutil provides small SQL query-building and value-conversion helpers shared by the SQL-backed stores and readers.
Package sqlx lets one store implementation serve both SQL backends.
Package sqlx lets one store implementation serve both SQL backends.
sqlxtest
Package sqlxtest runs a store's test suite against every SQL dialect.
Package sqlxtest runs a store's test suite against every SQL dialect.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL