Documentation
¶
Overview ¶
Package db provides a multi-layer database abstraction supporting: - User-level SQLite with sqlite-vec for personal data and vector search - Organization-level SQLite for shared tenant data - Hanzo Datastore (DATASTORE_URL) for deep analytics and parallel queries
Architecture:
┌─────────────────────────────────────────────────────────────┐ │ Query Layer │ ├─────────────────────────────────────────────────────────────┤ │ User SQLite │ Org SQLite │ Hanzo Datastore │ │ (per-user) │ (per-org) │ (DATASTORE_URL) │ │ + sqlite-vec │ + sqlite-vec │ (parallel queries) │ │ Fast queries │ Shared data │ Deep analytics │ └─────────────────────────────────────────────────────────────┘
At-rest encryption for the per-tenant SQLite stores.
Commerce keeps each tenant's data (balances, transactions, usage — MONEY) in its own SQLite file (users/<id>/data.db, orgs/<id>/data.db). This file wires those files to the Hanzo encrypted SQLite driver (github.com/hanzoai/sqlite): SQLCipher page-level AES-256 at rest, keyed per-tenant, with the master key sourced from KMS.
ENVELOPE MODEL (identical to Hanzo IAM's object/orgdb.go, so there is exactly ONE at-rest encryption scheme across the platform):
- A master key (32 bytes) is supplied via COMMERCE_KMS_MASTER_KEY, sourced from KMS at deploy time — never hardcoded, never in git.
- Each tenant file gets its OWN random Data Encryption Key (DEK). SQLCipher encrypts the pages with the DEK; it never changes for the life of the file.
- The DEK is wrapped (AES-256-GCM) under a per-tenant KEK = HKDF(masterKey, principal) and stored in a sidecar (<data.db>.dek). The raw DEK is never written to disk.
- Rotating the master key only rewraps the sidecar — the DEK, and therefore every encrypted page, is untouched (O(1), cannot brick a file).
Posture is decided once, in ResolveMasterKey():
- unset → unencrypted per-tenant files (dev / CI).
- set + live codec → per-tenant SQLCipher encryption (production).
- set + no live codec → hard error. Commerce's dual concurrent-read/ serialized-write pool needs the LIVE libsqlcipher codec; it cannot use the single-writer codec envelope that a non-libsqlcipher build falls back to. So a build without libsqlcipher is refused. (Such a build would still ENCRYPT via the envelope — it does not write plaintext — but it cannot serve commerce's dual pool, so opening it here would be wrong.)
Plaintext → encrypted at-rest migration for the per-tenant SQLite stores.
Once COMMERCE_KMS_MASTER_KEY is configured, the daemon opens each tenant file enveloped and REFUSES a db that has no DEK sidecar (fail-closed — see resolveDEK). Any pre-existing PLAINTEXT tenant file must therefore be converted first. EncryptDataDir does that, without losing a row:
read plaintext → write a fresh encrypted copy → verify per-table content parity → atomically swap the encrypted copy into place → shred the plaintext (no tenant money data survives a verified migration in the clear at rest).
It is idempotent: a tenant that already has a sidecar is skipped, so it is safe to run repeatedly (and as a one-shot Job before flipping the key on the deployment). The CLI is cmd/commerce-encrypt-dbs.
Package db provides database abstractions. This file contains the Model base type for entities using the new db.DB interface.
Index ¶
- Variables
- func ResolveMasterKey() ([]byte, error)
- type AfterCreateHook
- type AfterDeleteHook
- type AfterUpdateHook
- type BeforeCreateHook
- type BeforeDeleteHook
- type BeforeUpdateHook
- type Config
- type Cursor
- type DB
- type Datastore
- type DatastoreBatch
- type DatastoreConfig
- type DatastoreRows
- type Entity
- type IsolationLevel
- type Iterator
- type Key
- type Kind
- type Layer
- type Manager
- type MigrateReport
- type Model
- func (m *Model) Create(ctx context.Context) error
- func (m *Model) DB() DB
- func (m *Model) Delete(ctx context.Context) error
- func (m *Model) Entity() Kind
- func (m *Model) Exists(ctx context.Context) (bool, error)
- func (m *Model) Get(ctx context.Context) error
- func (m *Model) GetByID(ctx context.Context, id string) error
- func (m *Model) GetID() string
- func (m *Model) GetNamespace() string
- func (m *Model) Init(database DB, entity Kind)
- func (m *Model) IsCreated() bool
- func (m *Model) IsLoaded() bool
- func (m *Model) JSON() ([]byte, error)
- func (m *Model) JSONString() string
- func (m *Model) Key() Key
- func (m *Model) Kind() string
- func (m *Model) MarkLoaded()
- func (m *Model) MustCreate(ctx context.Context)
- func (m *Model) MustDelete(ctx context.Context)
- func (m *Model) MustGet(ctx context.Context)
- func (m *Model) MustGetByID(ctx context.Context, id string)
- func (m *Model) MustPut(ctx context.Context)
- func (m *Model) MustUpdate(ctx context.Context)
- func (m *Model) Put(ctx context.Context) error
- func (m *Model) Query() Query
- func (m *Model) RunInTransaction(ctx context.Context, fn func(tx Transaction) error) error
- func (m *Model) SetDB(database DB)
- func (m *Model) SetEntity(entity Kind)
- func (m *Model) SetID(id string)
- func (m *Model) SetKey(key Key) error
- func (m *Model) SetKeyFromString(id string) error
- func (m *Model) SetNamespace(ns string)
- func (m *Model) SoftDelete(ctx context.Context) error
- func (m *Model) Update(ctx context.Context) error
- type NoOpDatastore
- func (n *NoOpDatastore) AsyncInsert(ctx context.Context, query string, wait bool, args ...interface{}) error
- func (n *NoOpDatastore) Close() error
- func (n *NoOpDatastore) Exec(ctx context.Context, query string, args ...interface{}) error
- func (n *NoOpDatastore) PrepareBatch(ctx context.Context, query string) (DatastoreBatch, error)
- func (n *NoOpDatastore) Query(ctx context.Context, query string, args ...interface{}) (DatastoreRows, error)
- func (n *NoOpDatastore) Select(ctx context.Context, dest interface{}, query string, args ...interface{}) error
- type PostgresDB
- func (db *PostgresDB) AllocateIDs(kind string, parent Key, n int) ([]Key, error)
- func (db *PostgresDB) Close() error
- func (db *PostgresDB) Delete(ctx context.Context, key Key) error
- func (db *PostgresDB) DeleteMulti(ctx context.Context, keys []Key) error
- func (db *PostgresDB) Get(ctx context.Context, key Key, dst interface{}) error
- func (db *PostgresDB) GetMulti(ctx context.Context, keys []Key, dst interface{}) error
- func (db *PostgresDB) NewIncompleteKey(kind string, parent Key) Key
- func (db *PostgresDB) NewKey(kind string, stringID string, intID int64, parent Key) Key
- func (db *PostgresDB) Put(ctx context.Context, key Key, src interface{}) (Key, error)
- func (db *PostgresDB) PutMulti(ctx context.Context, keys []Key, src interface{}) ([]Key, error)
- func (db *PostgresDB) PutVector(ctx context.Context, kind string, id string, vector []float32, ...) error
- func (db *PostgresDB) Query(kind string) Query
- func (db *PostgresDB) RunInTransaction(ctx context.Context, fn func(tx Transaction) error, opts *TransactionOptions) error
- func (db *PostgresDB) TenantID() string
- func (db *PostgresDB) TenantType() string
- func (db *PostgresDB) VectorSearch(ctx context.Context, opts *VectorSearchOptions) ([]VectorResult, error)
- type PostgresDBConfig
- type Query
- type SQLiteConfig
- type SQLiteDB
- func (ns SQLiteDB) AllocateIDs(kind string, parent Key, n int) ([]Key, error)
- func (db *SQLiteDB) Close() error
- func (db *SQLiteDB) Delete(ctx context.Context, key Key) error
- func (db *SQLiteDB) DeleteMulti(ctx context.Context, keys []Key) error
- func (db *SQLiteDB) Encrypted() bool
- func (db *SQLiteDB) Get(ctx context.Context, key Key, dst any) error
- func (db *SQLiteDB) GetMulti(ctx context.Context, keys []Key, dst any) error
- func (ns SQLiteDB) NewIncompleteKey(kind string, parent Key) Key
- func (ns SQLiteDB) NewKey(kind string, stringID string, intID int64, parent Key) Key
- func (db *SQLiteDB) Put(ctx context.Context, key Key, src any) (Key, error)
- func (db *SQLiteDB) PutMulti(ctx context.Context, keys []Key, src any) ([]Key, error)
- func (db *SQLiteDB) PutVector(ctx context.Context, kind string, id string, vector []float32, ...) error
- func (db *SQLiteDB) Query(kind string) Query
- func (db *SQLiteDB) RunInTransaction(ctx context.Context, fn func(tx Transaction) error, opts *TransactionOptions) error
- func (db *SQLiteDB) TenantID() string
- func (db *SQLiteDB) TenantType() string
- func (db *SQLiteDB) VectorSearch(ctx context.Context, opts *VectorSearchOptions) ([]VectorResult, error)
- type SQLiteDBConfig
- type SyncConfig
- type Syncable
- type Syncer
- type TenantFailure
- type Transaction
- type TransactionOptions
- type Validator
- type VectorResult
- type VectorSearchOptions
Constants ¶
This section is empty.
Variables ¶
var ( // ErrNoSuchEntity is returned when an entity is not found ErrNoSuchEntity = errors.New("db: no such entity") // ErrInvalidKey is returned when a key is invalid ErrInvalidKey = errors.New("db: invalid key") // ErrInvalidEntityType is returned when an entity type is invalid ErrInvalidEntityType = errors.New("db: invalid entity type") // ErrConcurrentModification is returned when optimistic locking fails ErrConcurrentModification = errors.New("db: concurrent modification") // ErrDatabaseClosed is returned when operating on a closed database ErrDatabaseClosed = errors.New("db: database closed") )
var ( // ErrEntityNotFound is returned when Get operations fail to find an entity ErrEntityNotFound = ErrNoSuchEntity // ErrValidationFailed is returned when entity validation fails ErrValidationFailed = errors.New("db: validation failed") )
Functions ¶
func ResolveMasterKey ¶ added in v1.46.25
ResolveMasterKey reads COMMERCE_KMS_MASTER_KEY and validates it. It returns:
- (nil, nil) when unset on a pure-Go dev/CI build → unencrypted dev mode.
- (key, nil) when set, 64 hex chars, AND this build ACTUALLY encrypts (cgo + libsqlcipher linked and proven at runtime).
- (nil, error) when set but malformed; OR set on a build that cannot really encrypt; OR UNSET on a production (codec-linked) build — we refuse to run rather than silently write plaintext money data.
The codec check is the crux, and it gates on CodecLinked, NOT EncryptionAvailable. EncryptionAvailable() is always true now (every build encrypts — via the live libsqlcipher codec OR the pure-Go codec envelope), so it no longer distinguishes a commerce-capable build. Commerce needs the LIVE codec specifically: its dual read/write pool (encDriverDSN opens a concurrent read pool + a serialized write pool on the same file) cannot use the single-writer envelope. CodecLinked() runs a one-time runtime probe (open a keyed temp db, assert real ciphertext), so a build without the live codec is refused here rather than mis-opened. A refused build is NOT one that would write plaintext — it would encrypt via the envelope — it simply cannot serve commerce's dual pool.
This is the ONE place the master key is sourced, so every tenant store shares an identical posture decision.
Types ¶
type AfterCreateHook ¶
type AfterCreateHook interface {
AfterCreate() error
}
AfterCreateHook is called after entity creation
type AfterDeleteHook ¶
type AfterDeleteHook interface {
AfterDelete() error
}
AfterDeleteHook is called after entity deletion
type AfterUpdateHook ¶
type AfterUpdateHook interface {
AfterUpdate(prev interface{}) error
}
AfterUpdateHook is called after entity update
type BeforeCreateHook ¶
type BeforeCreateHook interface {
BeforeCreate() error
}
BeforeCreateHook is called before entity creation
type BeforeDeleteHook ¶
type BeforeDeleteHook interface {
BeforeDelete() error
}
BeforeDeleteHook is called before entity deletion
type BeforeUpdateHook ¶
type BeforeUpdateHook interface {
BeforeUpdate(prev interface{}) error
}
BeforeUpdateHook is called before entity update
type Config ¶
type Config struct {
// DataDir is the base directory for data storage
DataDir string
// UserDataDir is the directory for per-user SQLite databases
// Defaults to DataDir/users
UserDataDir string
// OrgDataDir is the directory for per-org SQLite databases
// Defaults to DataDir/orgs
OrgDataDir string
// MasterKey is the 32-byte at-rest encryption master key, supplied by the
// EMBEDDER. When non-nil it is used verbatim and COMMERCE_KMS_MASTER_KEY is
// never read.
//
// It exists so a host that already holds a master key does not have to mint a
// SECOND one and hand it over through the environment. cloud embeds commerce
// in its own process and already resolves a 32-byte KEK
// (CLOUD_KMS_MASTER_KEY_REF); making commerce reach independently for an env
// var meant two keys, two KMS paths and two sync objects for one process —
// and when the second was never provisioned, commerce refused to boot, Mount
// never published its handler, and every balance read in the fleet failed.
// The embedder passing what it already has is one key, one source, and a
// dependency the type system can see.
//
// nil ⇒ unchanged: ResolveMasterKey reads the env, which remains the
// standalone deployment's path.
MasterKey []byte
// MaxOpenTenants bounds how many per-tenant stores are open at once; the
// least recently used idle one is closed to stay under it.
// Defaults to defaultMaxOpenTenants.
MaxOpenTenants int
// TenantIdleTTL closes a per-tenant store untouched for this long, even
// below MaxOpenTenants. Defaults to defaultTenantIdleTTL; negative disables.
TenantIdleTTL time.Duration
// DatastoreDSN is the connection string for Hanzo Datastore (DATASTORE_URL)
DatastoreDSN string
// EnableDatastore enables the Hanzo Datastore layer
EnableDatastore bool
// EnableVectorSearch enables sqlite-vec for vector embeddings
EnableVectorSearch bool
// VectorDimensions is the default dimension for vector embeddings
VectorDimensions int
// SQLite configuration
SQLite SQLiteConfig
// Datastore configuration (Hanzo Datastore)
Datastore DatastoreConfig
// IsDev enables development mode logging
IsDev bool
}
Config holds database configuration options
type Cursor ¶
type Cursor interface {
String() string
}
Cursor represents a position in a result set
type DB ¶
type DB interface {
// Core operations
Get(ctx context.Context, key Key, dst interface{}) error
Put(ctx context.Context, key Key, src interface{}) (Key, error)
Delete(ctx context.Context, key Key) error
// Batch operations
GetMulti(ctx context.Context, keys []Key, dst interface{}) error
PutMulti(ctx context.Context, keys []Key, src interface{}) ([]Key, error)
DeleteMulti(ctx context.Context, keys []Key) error
// Query
Query(kind string) Query
// Vector search (sqlite-vec)
VectorSearch(ctx context.Context, opts *VectorSearchOptions) ([]VectorResult, error)
PutVector(ctx context.Context, kind string, id string, vector []float32, metadata map[string]interface{}) error
// Key management
NewKey(kind string, stringID string, intID int64, parent Key) Key
NewIncompleteKey(kind string, parent Key) Key
AllocateIDs(kind string, parent Key, n int) ([]Key, error)
// Transactions
RunInTransaction(ctx context.Context, fn func(tx Transaction) error, opts *TransactionOptions) error
// Lifecycle
Close() error
// Tenant info
TenantID() string
TenantType() string
}
DB is the main database interface for user/org SQLite databases
type Datastore ¶
type Datastore interface {
// Query executes datastore queries
Query(ctx context.Context, query string, args ...interface{}) (DatastoreRows, error)
// Select scans results into a destination slice
Select(ctx context.Context, dest interface{}, query string, args ...interface{}) error
// Exec executes a non-query statement
Exec(ctx context.Context, query string, args ...interface{}) error
// Batch insert for high-throughput data ingestion
PrepareBatch(ctx context.Context, query string) (DatastoreBatch, error)
// AsyncInsert for fire-and-forget event logging
AsyncInsert(ctx context.Context, query string, wait bool, args ...interface{}) error
// Close closes the datastore connection
Close() error
}
Datastore is the interface for Hanzo Datastore (ClickHouse) analytics queries
func NewDatastore ¶
NewDatastore creates a new Hanzo Datastore connection This connects to ClickHouse via hanzo/datastore-go for deep analytics
type DatastoreBatch ¶
type DatastoreBatch interface {
Append(v ...interface{}) error
AppendStruct(v interface{}) error
Flush() error
Send() error
Abort() error
Rows() int
Close() error
}
DatastoreBatch for bulk inserts into Hanzo Datastore
type DatastoreConfig ¶
type DatastoreConfig struct {
// MaxOpenConns for parallel queries
MaxOpenConns int
// MaxIdleConns for connection pooling
MaxIdleConns int
// ConnMaxLifetime for connection recycling
ConnMaxLifetime time.Duration
// Compression method (lz4, zstd, etc.)
Compression string
// QueryTimeout for datastore queries
QueryTimeout time.Duration
}
DatastoreConfig holds Hanzo Datastore (ClickHouse) configuration
type DatastoreRows ¶
type DatastoreRows interface {
Next() bool
Scan(dest ...interface{}) error
ScanStruct(dest interface{}) error
Columns() []string
Close() error
Err() error
}
DatastoreRows represents datastore query results
type Entity ¶
type Entity interface {
// Kind returns the entity kind/table name
Kind() string
}
Entity is the interface that all model entities should implement
type IsolationLevel ¶
type IsolationLevel int
IsolationLevel represents transaction isolation levels
const ( IsolationDefault IsolationLevel = iota IsolationReadUncommitted IsolationReadCommitted IsolationRepeatableRead IsolationSerializable )
type Key ¶
type Key interface {
// Kind returns the entity kind/table name
Kind() string
// StringID returns the string identifier (if any)
StringID() string
// IntID returns the integer identifier (if any)
IntID() int64
// Parent returns the parent key (for hierarchical keys)
Parent() Key
// Namespace returns the namespace/tenant
Namespace() string
// Incomplete returns true if this key needs an ID assigned
Incomplete() bool
// Encode returns an encoded string representation
Encode() string
// Equal checks if two keys are the same
Equal(other Key) bool
}
Key represents a unique identifier for an entity
type Kind ¶
type Kind interface {
Kind() string
}
Kind interface for entities that have a kind/table name
type Layer ¶
type Layer int
Layer represents which database layer to use
const ( // LayerUser uses the user-specific SQLite database LayerUser Layer = iota // LayerOrg uses the organization-level SQLite database LayerOrg // LayerDatastore uses the Hanzo Datastore (ClickHouse) for analytics LayerDatastore // LayerAll queries all layers (for cross-cutting queries) LayerAll )
type Manager ¶
type Manager struct {
// contains filtered or unexported fields
}
Manager is the main entry point for database operations. It manages multiple database layers and provides unified access.
func NewManager ¶
NewManager creates a new database manager
func (*Manager) Encrypted ¶ added in v1.46.25
Encrypted reports whether per-tenant SQLite files are SQLCipher-encrypted at rest (i.e. COMMERCE_KMS_MASTER_KEY was supplied and this build can encrypt).
type MigrateReport ¶ added in v1.46.25
type MigrateReport struct {
Encrypted []string // tenant db paths converted this run
Skipped []string // already-encrypted (sidecar present) or empty
Rows int // total rows copied
Failed []TenantFailure
}
MigrateReport summarizes an EncryptDataDir run.
func EncryptDataDir ¶ added in v1.46.25
func EncryptDataDir(dataDir string, masterKey []byte, dryRun bool) (*MigrateReport, error)
EncryptDataDir walks the users/ and orgs/ trees under dataDir and encrypts every plaintext tenant data.db in place. masterKey is the 32-byte KMS master key (same value the daemon runs with). When dryRun is true it reports what WOULD be migrated without writing anything.
func (*MigrateReport) Err ¶ added in v1.49.29
func (r *MigrateReport) Err() error
Err reports the run's overall verdict: nil when every tenant converted or was already encrypted. Callers should treat a non-nil Err as "retry the named tenants", not "the fleet is broken" — the successes are real and durable.
type Model ¶
type Model struct {
// Parent key for hierarchical data
Parent Key `json:"-"`
// Common fields
ID string `json:"id,omitempty"`
CreatedAt time.Time `json:"createdAt,omitempty"`
UpdatedAt time.Time `json:"updatedAt,omitempty"`
Deleted bool `json:"deleted,omitempty"`
// Version for optimistic locking
Version int64 `json:"version,omitempty"`
// Namespace/tenant for multi-tenancy
Namespace_ string `json:"-"`
// Flag for mock mode (testing)
Mock bool `json:"-"`
// Flag for string-based keys vs auto-generated
UseStringKey bool `json:"-"`
// contains filtered or unexported fields
}
Model is a base type that provides common functionality for entities using the new db.DB interface. Embed this in your entity structs.
func (*Model) GetNamespace ¶
Namespace returns the namespace for this entity
func (*Model) JSONString ¶
JSONString returns the JSON string representation
func (*Model) MustCreate ¶
MustCreate creates the entity or panics
func (*Model) MustDelete ¶
MustDelete deletes the entity or panics
func (*Model) MustGetByID ¶
MustGetByID retrieves by ID or panics
func (*Model) MustUpdate ¶
MustUpdate updates the entity or panics
func (*Model) RunInTransaction ¶
RunInTransaction executes a function within a transaction
func (*Model) SetKeyFromString ¶
SetKeyFromString sets the key from a string ID
func (*Model) SetNamespace ¶
SetNamespace sets the namespace
func (*Model) SoftDelete ¶
SoftDelete marks the entity as deleted without removing it
type NoOpDatastore ¶
type NoOpDatastore struct{}
NoOpDatastore is a no-op implementation when datastore is disabled
func (*NoOpDatastore) AsyncInsert ¶
func (*NoOpDatastore) Close ¶
func (n *NoOpDatastore) Close() error
func (*NoOpDatastore) Exec ¶
func (n *NoOpDatastore) Exec(ctx context.Context, query string, args ...interface{}) error
func (*NoOpDatastore) PrepareBatch ¶
func (n *NoOpDatastore) PrepareBatch(ctx context.Context, query string) (DatastoreBatch, error)
func (*NoOpDatastore) Query ¶
func (n *NoOpDatastore) Query(ctx context.Context, query string, args ...interface{}) (DatastoreRows, error)
type PostgresDB ¶
type PostgresDB struct {
// contains filtered or unexported fields
}
PostgresDB implements the DB interface using PostgreSQL
func NewPostgresDB ¶
func NewPostgresDB(cfg *PostgresDBConfig) (*PostgresDB, error)
NewPostgresDB creates a new PostgreSQL database connection
func (*PostgresDB) AllocateIDs ¶
AllocateIDs pre-allocates entity IDs
func (*PostgresDB) Close ¶
func (db *PostgresDB) Close() error
Close closes the database connection and prepared statements
func (*PostgresDB) Delete ¶
func (db *PostgresDB) Delete(ctx context.Context, key Key) error
Delete removes an entity (soft delete), scoped to the ctx tenant.
func (*PostgresDB) DeleteMulti ¶
func (db *PostgresDB) DeleteMulti(ctx context.Context, keys []Key) error
DeleteMulti removes multiple entities
func (*PostgresDB) Get ¶
func (db *PostgresDB) Get(ctx context.Context, key Key, dst interface{}) error
Get retrieves an entity by key, scoped to the ctx tenant. A row written by another tenant with the SAME id+kind is invisible here (Red CRIT-2).
func (*PostgresDB) GetMulti ¶
func (db *PostgresDB) GetMulti(ctx context.Context, keys []Key, dst interface{}) error
GetMulti retrieves multiple entities
func (*PostgresDB) NewIncompleteKey ¶
func (db *PostgresDB) NewIncompleteKey(kind string, parent Key) Key
NewIncompleteKey creates a key that will be assigned an ID on Put
func (*PostgresDB) PutVector ¶
func (db *PostgresDB) PutVector(ctx context.Context, kind string, id string, vector []float32, metadata map[string]interface{}) error
PutVector stores a vector embedding
func (*PostgresDB) Query ¶
func (db *PostgresDB) Query(kind string) Query
Query returns a new query for the given kind
func (*PostgresDB) RunInTransaction ¶
func (db *PostgresDB) RunInTransaction(ctx context.Context, fn func(tx Transaction) error, opts *TransactionOptions) error
RunInTransaction executes a function within a transaction
func (*PostgresDB) TenantID ¶
func (db *PostgresDB) TenantID() string
TenantID returns the tenant identifier
func (*PostgresDB) TenantType ¶
func (db *PostgresDB) TenantType() string
TenantType returns "user" or "org"
func (*PostgresDB) VectorSearch ¶
func (db *PostgresDB) VectorSearch(ctx context.Context, opts *VectorSearchOptions) ([]VectorResult, error)
VectorSearch performs similarity search using pgvector
type PostgresDBConfig ¶
type PostgresDBConfig struct {
// DSN is the connection string
// Format: postgres://user:pass@host:port/dbname?sslmode=disable
DSN string
// Config for connection pool
MaxOpenConns int
MaxIdleConns int
ConnMaxLifetime time.Duration
ConnMaxIdleTime time.Duration
// QueryTimeout for queries
QueryTimeout time.Duration
// TenantID for multi-tenant isolation
TenantID string
// TenantType is "user" or "org"
TenantType string
// Schema for tenant isolation (optional)
// If set, uses PostgreSQL schemas for multi-tenancy
Schema string
// EnableVectorSearch enables pgvector extension
EnableVectorSearch bool
// VectorDimensions for embeddings
VectorDimensions int
}
PostgresDBConfig holds configuration for a PostgreSQL database
type Query ¶
type Query interface {
// Filtering
Filter(filterStr string, value interface{}) Query
FilterField(fieldPath string, op string, value interface{}) Query
// Ordering
Order(fieldPath string) Query
OrderDesc(fieldPath string) Query
// Pagination
Limit(limit int) Query
Offset(offset int) Query
// Projection
Project(fieldNames ...string) Query
Distinct() Query
// Ancestor queries (for hierarchical data)
Ancestor(ancestor Key) Query
// Execution
GetAll(ctx context.Context, dst interface{}) ([]Key, error)
First(ctx context.Context, dst interface{}) (Key, error)
Count(ctx context.Context) (int, error)
Keys(ctx context.Context) ([]Key, error)
Run(ctx context.Context) Iterator
// Cursors for pagination
Start(cursor Cursor) Query
End(cursor Cursor) Query
}
Query provides a fluent interface for querying entities
type SQLiteConfig ¶
type SQLiteConfig struct {
// MaxOpenConns for concurrent reads
MaxOpenConns int
// MaxIdleConns for connection pooling
MaxIdleConns int
// BusyTimeout in milliseconds before giving up on locked DB
BusyTimeout int
// JournalMode (WAL recommended for concurrency)
JournalMode string
// Synchronous mode (NORMAL for balance of safety/speed)
Synchronous string
// CacheSize in KB (negative = KB, positive = pages)
CacheSize int
// QueryTimeout for SELECT queries
QueryTimeout time.Duration
}
SQLiteConfig holds SQLite-specific configuration
type SQLiteDB ¶
type SQLiteDB struct {
// contains filtered or unexported fields
}
SQLiteDB implements the DB interface using SQLite
func NewSQLiteDB ¶
func NewSQLiteDB(cfg *SQLiteDBConfig) (*SQLiteDB, error)
NewSQLiteDB creates a new SQLite database connection
func (SQLiteDB) AllocateIDs ¶
AllocateIDs pre-allocates entity IDs
func (*SQLiteDB) DeleteMulti ¶
DeleteMulti removes multiple entities
func (*SQLiteDB) Encrypted ¶ added in v1.46.25
Encrypted reports whether this tenant file is SQLCipher-encrypted at rest.
func (SQLiteDB) NewIncompleteKey ¶
NewIncompleteKey creates a key that will be assigned an ID on Put
func (*SQLiteDB) PutVector ¶
func (db *SQLiteDB) PutVector(ctx context.Context, kind string, id string, vector []float32, metadata map[string]any) error
PutVector stores a vector embedding
func (*SQLiteDB) RunInTransaction ¶
func (db *SQLiteDB) RunInTransaction(ctx context.Context, fn func(tx Transaction) error, opts *TransactionOptions) error
RunInTransaction executes a function within a transaction
func (*SQLiteDB) TenantType ¶
TenantType returns "user" or "org"
func (*SQLiteDB) VectorSearch ¶
func (db *SQLiteDB) VectorSearch(ctx context.Context, opts *VectorSearchOptions) ([]VectorResult, error)
VectorSearch performs similarity search using sqlite-vec
type SQLiteDBConfig ¶
type SQLiteDBConfig struct {
// Path to the SQLite database file
Path string
// Config for SQLite options
Config SQLiteConfig
// EnableVectorSearch enables sqlite-vec extension
EnableVectorSearch bool
// VectorDimensions for embeddings
VectorDimensions int
// TenantID for this database (userID or orgID)
TenantID string
// TenantType is "user" or "org"
TenantType string
// MasterKey is the 32-byte KMS-sourced master key. When non-nil the file is
// opened SQLCipher-encrypted at rest (per-tenant DEK wrapped under a KEK
// derived from this key). Nil means unencrypted (dev/CI). Set once by the
// Manager from COMMERCE_KMS_MASTER_KEY — see encryption.go.
MasterKey []byte
}
SQLiteDBConfig holds configuration for a SQLite database
type SyncConfig ¶
type SyncConfig struct {
// Enabled turns on datastore sync
Enabled bool
// BatchSize is the number of records to batch before sending
BatchSize int
// FlushInterval is how often to flush partial batches
FlushInterval time.Duration
// Kinds specifies which entity kinds to sync (empty = all)
Kinds []string
// AsyncInsert uses ClickHouse async insert for fire-and-forget
AsyncInsert bool
}
SyncConfig configures how data is synced to Hanzo Datastore
func DefaultSyncConfig ¶
func DefaultSyncConfig() *SyncConfig
DefaultSyncConfig returns default sync configuration
type Syncable ¶
type Syncable interface {
Entity
// SyncToDatastore returns true if this entity should be synced
SyncToDatastore() bool
}
Syncable entities can be synced to Hanzo Datastore
type Syncer ¶
type Syncer struct {
// contains filtered or unexported fields
}
Syncer handles syncing data from SQLite to Hanzo Datastore
func NewSyncer ¶
func NewSyncer(config *SyncConfig, datastore Datastore) *Syncer
NewSyncer creates a new datastore syncer
type TenantFailure ¶ added in v1.49.29
TenantFailure is one tenant the run could not convert. The source file is untouched in every failure path (encryptTenantFile only cuts over after parity passes), so a failure costs that tenant nothing but another attempt.
type Transaction ¶
type Transaction interface {
Get(key Key, dst interface{}) error
Put(key Key, src interface{}) (Key, error)
Delete(key Key) error
Query(kind string) Query
}
Transaction represents a database transaction
type TransactionOptions ¶
type TransactionOptions struct {
// ReadOnly indicates this is a read-only transaction
ReadOnly bool
// MaxAttempts for retries on conflict
MaxAttempts int
// Isolation level (for SQL databases)
Isolation IsolationLevel
}
TransactionOptions configures transaction behavior
type Validator ¶
type Validator interface {
Validate() error
}
Validator interface for entities that support validation
type VectorResult ¶
type VectorResult struct {
// ID is the entity identifier
ID string
// Score is the similarity score (0-1, higher is more similar)
Score float32
// Metadata is additional data stored with the vector
Metadata map[string]interface{}
}
VectorResult represents a vector search result
type VectorSearchOptions ¶
type VectorSearchOptions struct {
// Kind is the entity type to search
Kind string
// Vector is the query vector
Vector []float32
// Limit is the maximum number of results
Limit int
// MinScore filters results below this similarity score
MinScore float32
// Filters are additional SQL conditions
Filters map[string]interface{}
}
VectorSearchOptions configures vector similarity search