config

package
v0.30.38 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 7, 2026 License: Apache-2.0 Imports: 6 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func LoadFromEnv

func LoadFromEnv(cfg *Config)

LoadFromEnv loads configuration from environment variables. All environment variables use the XOLU_ prefix.

Types

type APIKeyGrant

type APIKeyGrant = authconfig.APIKeyGrant

Config holds application configuration APIKeyGrant binds a single API key to the tenants it may act on under TenantAuthMode "scoped". Exactly one of Tenants or Admin should be set.

The definition moved to pkg/authconfig in the T-19 auth extraction; this alias preserves every existing reference.

type Config

type Config struct {
	// Server configuration
	Host string
	Port int

	// Storage configuration
	StorageType string // "sqlite" (only supported backend since v0.9.9)
	BaseDir     string
	SchemaDir   string
	Schema      string

	// Cache configuration
	CacheType   string // "memory" or "redis"
	CacheTTL    int    // seconds
	RedisHost   string
	RedisPort   int
	CacheSize   int
	CacheShards int // Number of shards for in-memory cache (0 = default 16; must be power of 2)

	// API v2 configuration
	// APIV2Enabled gates the entire /api/v2 surface. False by default.
	// Meta subsystem (S3)
	// MetaMaxValueBytes is the maximum size of a single metadata value in bytes.
	MetaMaxValueBytes int // default 65536 (64 KB)
	// MetaGCEnabled enables the metadata TTL sweeper (default true when v2 enabled).
	MetaGCEnabled bool
	// MetaGCIntervalSecs is how often the metadata GC sweep runs (default 300).
	MetaGCIntervalSecs int
	// DxpGCEnabled enables the dxp_txn deadline sweeper (default true when
	// v2 enabled) — marks instances stuck 'active' past their own deadline
	// (a crash or unrecovered panic mid-dispatch, T-100) as 'expired'.
	DxpGCEnabled bool
	// DxpGCIntervalSecs is how often the dxp sweep runs (default 60 — dxp's
	// own phase_ttl deadlines run in seconds-to-minutes, shorter than
	// meta's TTLs, so a stuck instance is worth noticing sooner).
	DxpGCIntervalSecs int
	// DxpTxnRetentionSecs is how long a dxp_txn instance is kept after
	// reaching a terminal state (committed/released/expired) before the
	// same dxp-gc sweep purges it — direct instruction (2026-07-31):
	// "keep tombstones for a configurable period... defaults to 48
	// hours before they're gone." Measured from created_at: dispatch is
	// synchronous, so creation and termination are the same instant for
	// every ordinary instance; only T-100's own sweep-caught crash
	// residue terminates later than it was created, and retention from
	// created_at is still the honest, simpler choice there — no
	// separate terminal_at column exists, and this is a coarse cleanup
	// window, not a tight SLA. Default 172800 (48h).
	DxpTxnRetentionSecs int
	// Set XOLU_API_V2_ENABLED=true to enable experimental v2 functionality.
	APIV2Enabled bool

	// Graph configuration
	GraphEnabled         bool
	GraphMode            string // "flat" (default) or "disabled"
	AsyncJobRetentionTTL int    // seconds; how long completed async job records are kept (default 86400)
	OQLQueryCacheTTL     int    // seconds; HTTP-layer OQL result cache; 0 = disabled (default 30)
	GraphQueryCacheTTL   int    // seconds; HTTP-layer Sulpher result cache; 0 = disabled (default 30)
	GraphMaxVisitedNodes int    // Max nodes visited during traversal (0 = default 10000)
	GraphMaxResults      int    // Max result paths returned (0 = no limit)
	GraphCycleDetection  string // "warn", "error", "ignore"
	GraphCycleCheckLimit int    // BFS budget for cycle detection (0 = use default 512)

	// Full-text search
	FullTextEnabled bool

	// Commit endpoint behaviour.
	//
	// When StrictCommit is true (the default), POST /commit runs the same
	// schema validation and graph cycle prechecks as the normal write
	// endpoints (save, create, patch) before executing the storage
	// transaction. Set to false only when the caller is trusted
	// infrastructure that manages its own invariants and the extra
	// validation overhead is undesirable.
	//
	// 	// has been deprecated for production use; enforcing this at the HTTP
	// layer prevents silent correctness problems.
	StrictCommit bool

	// Query configuration
	MaxQueryDepth   int
	MaxEmbedDepth   int
	RefEmbedDepth   int
	DefaultPageSize int

	// Query guardrails — limits that prevent runaway queries from becoming
	// outages. All limits are enforced server-side; client timeouts alone
	// are not sufficient because they abandon work without freeing resources.
	QueryTimeout          int // Seconds; max execution time for OQL/Sulpher queries (0 = use default 30)
	QueryMaxRows          int // Max rows returned by a single query (0 = use default 10000)
	QueryMaxScanRows      int // Max rows scanned before aborting (0 = use default 100000)
	QueryMaxResponseBytes int // Max JSON response size in bytes (0 = use default 10MB)

	// Entity configuration
	PatchNullBehavior string // "store" or "delete"
	MaxEntitySize     int    // bytes

	// Cascade delete configuration
	CascadingDelete     bool
	MaxCascadeDeletions int
	MaxCascadeWork      int

	// Debug
	Debug      bool
	DebugLocks bool
	// LogLevel sets the minimum log level: "debug", "info", "warn", "error".
	// XOLU_LOG_LEVEL is the primary env var. XOLU_DEBUG=true is a legacy alias
	// that maps to LogLevel "debug". When both are set, XOLU_LOG_LEVEL wins.
	// Defaults to "info" when neither is set.
	LogLevel string

	// NoAscii suppresses the ASCII art box at startup (XOLU_NO_ASCII=true).
	// Useful in container environments or when stdout is a log pipeline.
	NoAscii bool

	// NoStartupText suppresses the startup configuration summary (XOLU_NO_STARTUP_TEXT=true).
	// NoAscii and NoStartupText are independent; either or both may be set.
	NoStartupText bool

	// Authentication
	AuthType  string   // "none", "jwt", "apikey", "bearertoken"
	JWTSecret string   // Secret for JWT validation
	JWTIssuer string   // Expected issuer claim
	APIKeys   []string // Valid API keys (comma-separated in env)
	// InternalToken is the shared secret for the "bearertoken" auth type.
	// The incoming Authorization: Bearer <token> value is compared against
	// this using subtle.ConstantTimeCompare. Typically a 32-byte hex string
	// generated with `openssl rand -hex 32`. Set via XOLU_INTERNAL_TOKEN.
	InternalToken    string
	AuthExcludePaths []string // Paths excluded from auth (e.g., /health)

	// Rate limiting
	RateLimitEnabled bool
	RateLimitRate    int // Requests per window
	RateLimitWindow  int // Window in seconds
	RateLimitByIP    bool

	// TrustedProxies is a comma-separated list of CIDR ranges whose
	// requests are permitted to override the observed peer IP via
	// X-Forwarded-For headers. When empty (default), header-based IP
	// spoofing is refused and the TCP peer is authoritative. See T-38.
	TrustedProxies string `json:"trusted_proxies"`
	RateLimitByKey bool   // Rate limit by API key or JWT subject

	// Metrics
	MetricsEnabled bool
	// MetricsPort, when > 0, starts a dedicated listener that serves only
	// /metrics on the given port. The main API port will no longer expose
	// /metrics, allowing Prometheus scrape traffic to be separated from
	// operational reads and writes. When 0 (the default), /metrics is served
	// on the main port as before. Controlled by XOLU_METRICS_PORT.
	MetricsPort int
	// MetricsHost sets the bind address for the dedicated metrics listener.
	// Only meaningful when MetricsPort > 0. When unset, the value is derived
	// from Host: if Host is a real address (not 0.0.0.0 or ::), it is
	// inherited; otherwise the metrics listener binds to 0.0.0.0. When
	// explicitly set, it always takes precedence. Controlled by XOLU_METRICS_HOST.
	MetricsHost string

	// DynConfigEnabled enables the dynamic configuration system.
	// When false, DynConfigFile and DynConfigAPIEnabled are ignored.
	// Controlled by XOLU_DYNCONFIG_ENABLED.
	DynConfigEnabled bool
	// DynConfigFile is the path to the JSON file that backs the dynamic
	// configuration store. Defaults to {BaseDir}/dynconfig.json.
	// Controlled by XOLU_DYNCONFIG_FILE.
	DynConfigFile string
	// DynConfigReloadSecs is the interval between file reloads in seconds.
	// Must be > 0 when DynConfigEnabled is true. Default: 30.
	// Controlled by XOLU_DYNCONFIG_RELOAD_INTERVAL.
	DynConfigReloadSecs int
	// DynConfigAPIEnabled exposes the admin API endpoints for reading and
	// writing dynamic settings at runtime. When false, settings can only
	// be changed by editing the file directly.
	// Default: false. Controlled by XOLU_DYNCONFIG_API_ENABLED.
	DynConfigAPIEnabled bool

	// CORS
	// CORSOrigins lists allowed origins for cross-origin requests. Empty
	// disables CORS entirely. Use "*" for development only. When combined
	// with cookie-based auth, restrict to specific trusted domains — see
	// the security note on corsMiddleware in server.go.
	CORSOrigins []string

	// Performance tuning — SQLite
	SQLiteMaxOpenConns        int // Max open write connections (0 = backend default)
	SQLiteMaxIdleConns        int // Max idle write connections (0 = backend default)
	SQLiteReadPoolSize        int // Max open read connections (0 = backend default)
	SQLiteContentionThreshold int // Adaptive lock threshold 0-100 (0 = disabled, 95 = default)
	SQLiteBusyTimeout         int // Milliseconds to wait on locked database (0 = use default 5000)
	SQLiteCacheSize           int // Page cache size in KB (0 = use default 2000)

	// SQLitePerFileTenants controls whether each tenant gets its own SQLite
	// database file. When false (default), all tenants share one file and are
	// isolated by the tenant_id column. When true, each tenant gets its own
	// file. Paths are derived from BaseDir by the invariant layout (see
	// pkg/storelayout):
	//
	//   per-file:  tenant 0 -> <BaseDir>/t0000/store/xolu.db
	//              tenant N -> <BaseDir>/tXXXX/store/xolu.db
	//   shared:    all tenants -> <BaseDir>/shared/store/xolu.db
	//
	// Both modes use the same per-tenant table naming (t<XXXX>_* tables); the
	// flag governs only file placement. Choose at deployment time and do not
	// change while data exists — migration requires an explicit export/import.
	// Ignored when StorageType is not "sqlite".
	SQLitePerFileTenants bool

	// Performance tuning — query planner
	// PerformanceProfile selects hardware-specific thresholds for the
	// query planner's push-down decisions. Accepted values:
	//   "auto"      - Run a ~200ms startup micro-benchmark to calibrate (default)
	//   "edge"      - ARM SBCs, gateways (1-2 cores, 1-4 GB RAM)
	//   "vps"       - Small cloud instances (1-2 vCPU, 2-8 GB RAM)
	//   "dedicated" - Bare metal or large instances (4+ cores, 16+ GB)
	PerformanceProfile string

	// Performance tuning — Redis
	RedisPoolSize     int // Redis connection pool size (0 = use default 50)
	RedisMinIdleConns int // Redis minimum idle connections (0 = use default 10)

	// Performance tuning — HTTP server
	HTTPReadTimeout    int // Seconds; max duration for reading request (0 = no timeout)
	HTTPWriteTimeout   int // Seconds; max duration for writing response (0 = no timeout)
	HTTPIdleTimeout    int // Seconds; max duration for keep-alive idle (0 = no timeout)
	HTTPRequestTimeout int // Seconds; per-request middleware timeout (0 = use default 60)

	// Multi-tenancy
	// TenantMode controls tenant isolation behaviour:
	//   "path"   - Tenant routes available with auto-registration, non-tenant routes
	//              use tenant 0 (default)
	//   "strict" - All entity requests require tenant context; non-tenant routes
	//              return 403; tenants must be pre-registered
	TenantMode string

	// TenantAuthMode controls whether an authenticated caller's identity must be
	// authorised for the tenant it requests:
	//   "open"   - Default. Any authenticated caller may act on any tenant.
	//              Correct for single-tenant, trusted-gateway, and edge deployments.
	//   "scoped" - The caller's identity (JWT grant claim, API-key grant, or
	//              bearer=admin) must authorise the requested tenant; otherwise 403.
	//              Requires TenantMode "strict" (the unprefixed tenant-0 routes are
	//              disabled so there is no unauthorised default-tenant path).
	// See docs/proposals/tenant-access-control.md.
	TenantAuthMode string

	// APIKeyGrants maps individual API keys to the tenants they may act on, for
	// TenantAuthMode "scoped". Under "scoped", a key present only in the flat
	// APIKeys list (with no grant here) is rejected. Ignored under "open".
	APIKeyGrants []APIKeyGrant

	// TenantAutoRegister controls whether unknown tenant names are automatically
	// registered on first access. When true, any request to /api/v1/tenant/{name}/...
	// will create the tenant if it doesn't exist. When false (default), unknown
	// tenants return 404. Ignored when TenantMode is "strict".
	TenantAutoRegister bool

	// Timeseries storage (Pebble-backed, requires StorageType = "sqlite")
	TimeseriesEnabled        bool
	TSMemtableSize           int    // bytes, default 67108864 (64 MB)
	TSBlockSize              int    // bytes, default 32768 (32 KB)
	TSCompression            string // "snappy", "zstd", or "none"
	TSL0CompactionThreshold  int    // L0 files before compaction trigger, default 4
	TSMaxOpenFiles           int    // per-tenant Pebble file limit, default 500
	TSDefaultRetentionDays   int    // default 90
	TSCompactionIntervalSecs int    // retention sweep interval in seconds, default 3600
	TSRetentionEnabled       bool   // run background retention goroutine, default false

	// Timeseries query guardrails
	TSQueryTimeoutSecs    int // max execution time per query (0 = default 30s)
	TSMaxQueryEvents      int // max events returned by QueryRange/Latest (0 = default 10000)
	TSMaxScanEvents       int // max events scanned before aborting (0 = default 500000)
	TSMaxRangeDays        int // max From→To window in days (0 = default 366)
	TSMaxBatchSize        int // max events per batch append (0 = default 5000)
	TSMaxResponseBytes    int // max JSON response size in bytes (0 = default 10MB)
	TSMaxAggregateBuckets int // max buckets in a windowed aggregate (0 = default 10000)

	// Timeseries write coalescer tuning.
	// XOLU_TS_COAL_FLUSH_INTERVAL_MS controls how long the coalescer waits
	// before committing accumulated events (default 10ms). Lower values reduce
	// write latency jitter; higher values increase the number of events sharing
	// each fsync at the cost of additional latency. Only relevant when the
	// coalescer is enabled via dynconfig key ts.writecoal.
	TSCoalFlushIntervalMs int // default 10
	// XOLU_TS_COAL_MAX_EVENTS controls the maximum number of events the
	// coalescer accumulates before forcing an early flush (default 2000).
	// Prevents unbounded memory use if the arrival rate is very high.
	TSCoalMaxEvents int // default 2000

	// XOLU_TS_ROLLUP_CASCADE_DELETE controls whether deleting a rollup
	// definition automatically deletes all descendant definitions and stops
	// their workers. When true (default), a single DELETE on a parent removes
	// the entire subtree rooted at that definition. When false, deleting a
	// definition that has descendants returns an error; the caller must delete
	// bottom-up manually.
	TSRollupCascadeDelete bool // default true

	// Blob store — content-addressed object storage on the local filesystem.
	// Blobs never enter SQLite; only their SHA-256 reference is stored in
	// entity fields. Blobs are per-tenant, organised uniformly with the
	// timeseries plane:
	//   {BaseDir}/t{XXXX}/blobs/{xx}/{sha256hex}
	// where t{XXXX} is the tenant directory (tenant 0 included) and {xx} is the
	// first two hex characters of the SHA (git-style prefix). Key aliases live
	// in {BaseDir}/t{XXXX}/blobs/.keys.
	//
	// BlobEnabled must be true for either the JSON blob API (/api/v1/blob/)
	// or the S3-compatible API to function. When false, both are disabled
	// and any attempt to use them returns 501.
	BlobEnabled bool
	// BlobDir is retained for configuration compatibility but is no longer
	// consulted: blobs are placed per tenant under {BaseDir}/t{XXXX}/blobs by
	// the blob manager (see pkg/storelayout.TenantBlobDir). Deprecated.
	BlobDir string

	// CalEnabled controls whether the /api/v2/cal/* endpoints are wired.
	// When true, the server initialises a cal.Manager rooted at BaseDir
	// and registers the four calendar operations (check, openings,
	// propose, confirm) under the v2 tenant scope. When false, the
	// routes are omitted entirely and any attempt to reach them returns
	// the standard 404.
	//
	// Introduced with T-18 in v0.14.7. Default false, matching the v2
	// subsystem posture: opt-in until stable.
	CalEnabled bool

	// BalEnabled controls whether the /api/v2/bal/* endpoints are wired
	// (@B; opt-in until stable, like cal).
	BalEnabled bool
	// BlobMaxSize is the maximum blob size in bytes accepted by PUT/POST.
	// 0 = use default (67108864 = 64 MB). Applies to both the JSON and S3 APIs.
	BlobMaxSize int
	// BlobMaxTotalBytes is the per-tenant total storage cap in bytes.
	// A Put that would push a tenant's total bytes over this limit is rejected
	// with XOLU-BL006. Checked against the sampler cache — a soft cap, not a
	// hard guarantee. 0 means no limit (default).
	BlobMaxTotalBytes int64

	// S3-compatible API — exposes the blob store via an S3-like interface on
	// a dedicated listener so that existing S3 client libraries and tools
	// (rclone, boto3, the AWS CLI, etc.) work without modification.
	//
	// The S3 API is independent of BlobEnabled: setting S3Enabled = true
	// while BlobEnabled = false is a configuration error caught by Validate.
	//
	// Authentication: the S3 API accepts any AWS Signature V4 Authorization
	// header and extracts the access key ID as the tenant identifier. The
	// signature itself is not verified. This is intentional for single-operator
	// deployments; a future release may add full Sig V4 verification.
	// Configure S3 clients with any non-empty secret key value.
	//
	// Bucket semantics: each bucket maps to a tenant. Bucket names must match
	// existing tenant names; auto-registration is not performed on the S3
	// interface regardless of TenantAutoRegister.
	S3Enabled bool
	// S3Host is the bind address for the S3-compatible listener.
	// Default: inherits Host (0.0.0.0).
	// XOLU_S3_ADDR (host:port) and XOLU_S3_HOST / XOLU_S3_PORT are the
	// controlling env vars, following the same precedence as XOLU_METRICS_ADDR.
	S3Host string
	// S3Port is the port for the S3-compatible listener.
	// Default: 9091. Must differ from Port and MetricsPort when S3Enabled is true.
	// 0 means the S3 listener is not started even when S3Enabled is true
	// (useful for testing where the caller manages the listener directly).
	S3Port int
	// S3RequireAuth controls whether requests that arrive without an
	// Authorization header are rejected (true) or silently fall back to
	// treating the bucket name as the tenant (false, default).
	// When false a structured warning is still logged so operators can detect
	// misconfigured clients. Set XOLU_S3_REQUIRE_AUTH=true to enforce.
	S3RequireAuth bool

	// S3KeyGrants maps S3 access keys to their secret and the tenants they may
	// act on, for TenantAuthMode "scoped". Under "scoped" an S3 request must
	// present an access key with a matching grant whose secret authenticates it;
	// the access-key string is no longer trusted as the tenant name. Ignored
	// under "open".
	S3KeyGrants []S3KeyGrant

	// BlobGCEnabled enables the background GC worker.
	// Default: false (GC must be explicitly opted in).
	BlobGCEnabled bool
	// BlobGCIntervalSecs is the time between GC sweeps in seconds.
	// Default: 3600 (1 hour).
	BlobGCIntervalSecs int
	// BlobGCGracePeriodSecs is how long (in seconds) an unreferenced blob must
	// sit in .gc-pending/ before it is hard-deleted.
	// Default: 600 (10 minutes).
	BlobGCGracePeriodSecs int

	// BlobExportSweepEnabled enables the background sweep that deletes
	// expired async-export blobs (T-149, POST .../blob/export). Separate
	// from BlobGCEnabled: export blobs are ordinary blobs from GC's own
	// point of view (referenced by their key alias, so GC alone would
	// never reclaim them) -- this sweep exists specifically to enforce a
	// TTL on top of that, not to replace GC.
	// Default: false (must be explicitly opted in, matching BlobGCEnabled).
	BlobExportSweepEnabled bool
	// BlobExportSweepIntervalSecs is the time between export-sweep passes.
	// Default: 900 (15 minutes).
	BlobExportSweepIntervalSecs int
	// BlobExportTTLSecs is how long (in seconds) a completed export blob
	// is kept before the sweep deletes it -- the team's own framing when
	// this was designed (2026-08-03): "a TTL so that the export expires
	// in the next few hours".
	// Default: 14400 (4 hours).
	BlobExportTTLSecs int

	// BlobUsageSampleIntervalSecs is how often (in seconds) the background
	// usage sampler walks the blob store to update cached totals served by
	// the usage API and telemetry endpoint. 0 disables the sampler.
	// Default: 300 (5 minutes).
	BlobUsageSampleIntervalSecs int
}

func Default

func Default() *Config

Default returns the default configuration

func (*Config) AuthConfig

func (c *Config) AuthConfig() authconfig.Config

AuthConfig extracts the authentication subset of the full server configuration for pkg/middleware.AuthMiddleware (T-19). The server calls this once at startup; the two structures cannot drift because this is the only construction path xolu itself uses.

func (*Config) Tenancy

func (c *Config) Tenancy() TenancyFlags

Tenancy derives the TenancyFlags from the configured mode strings. The implication TenantEnforceGrant ⇒ TenantRequireRoute is applied here, so a caller never has to remember it. (Config validation independently rejects the scoped+path string combination with a helpful error; this derivation is the structural backstop.)

func (*Config) Validate

func (c *Config) Validate() (errs []string, warnings []string)

Validate checks the configuration for invalid or inconsistent values. It returns hard errors (config is broken) and warnings (config is suspicious but functional) as separate slices.

type S3KeyGrant

type S3KeyGrant struct {
	AccessKey string   // the SigV4 access key ID
	Secret    string   // the secret used to authenticate the caller
	Tenants   []string // tenant names this key may act on
	Admin     bool     // true → authorised for any tenant
}

S3KeyGrant binds an S3 access key (and its secret) to the tenants it may act on. Used by the S3 gateway under TenantAuthMode "scoped": the access key identifies the caller, the secret authenticates it, and Tenants/Admin authorise which tenants it may reach. Without a matching grant, a scoped S3 request is denied — the access-key string is no longer trusted as the tenant name. Exactly one of Tenants or Admin should be set.

type TenancyFlags

type TenancyFlags uint8

TenancyFlags is the internal, bit-flag representation of the tenancy operating mode. The user-facing configuration surface remains the two named string fields (TenantMode = "path"|"strict", TenantAuthMode = "open"|"scoped"); this type is what the server consumes, so call sites express intent ("flags.Has(TenantRequireRoute)") instead of re-deriving meaning from string comparisons.

The two concerns are genuinely independent properties, not points on one axis:

TenantRequireRoute  — unprefixed / default-tenant-0 routes are disabled;
                      all entity access must go through /tenant/{id}/.
                      (Set by TenantMode "strict".)
TenantEnforceGrant  — the caller's identity must authorise the requested
                      tenant; otherwise 403. (Set by TenantAuthMode "scoped".)

Representing them as flags makes the design's coherence rule structural rather than a validation afterthought: TenantEnforceGrant *implies* TenantRequireRoute (you cannot enforce per-tenant authorisation while leaving the unauthenticated tenant-0 routes open), so the implication is baked into the derivation below and the incoherent combination is not representable.

const (
	// TenantRequireRoute disables the unprefixed default-tenant routes.
	TenantRequireRoute TenancyFlags = 1 << iota
	// TenantEnforceGrant enforces per-identity tenant authorisation.
	TenantEnforceGrant
)

func (TenancyFlags) Has

func (t TenancyFlags) Has(f TenancyFlags) bool

Has reports whether all bits in f are set.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL