Documentation
¶
Overview ¶
Package config implements YAML configuration loading, environment- variable overlay, CLI flag precedence, strict schema validation, and actionable E_* errors for the GSEER v0.0.1 binary (SPEC-03 §5, §7).
Precedence (highest wins): CLI flags > environment variables > YAML config file > owner-approved defaults (PD-1…PD-9).
Load order (corrected per owner M2 review): YAML → env → flags → defaults (profile-dependent) → hash token → validate. Defaults are applied AFTER env+flags so profile-dependent defaults (database components) use the FINAL resolved profile.
Profiles (SPEC-03 §5) ¶
Two profiles are supported:
- client — CLI-only; queries a configured remote endpoint. No PostgreSQL, no local server.
- self-hosted — runs the local API + dashboard server backed by PostgreSQL. Migrations run on `setup --self-hosted` and `server start` (M3).
Neither profile ships data. v0.0.1 has no public endpoint.
Strict schema (AC-7.x) ¶
Unknown YAML keys are rejected with E_CONFIG_UNKNOWN_KEY. Any `policy.active.*` key is rejected with E_CONFIG_POLICY_ACTIVE_REJECTED and a pointer to v0.0.2 — active collection is not part of v0.0.1 and the config layer enforces this at load time, not just structurally.
`InsecureSkipVerify` and similar unsafe keys are rejected with E_CONFIG_INSECURE for any control-plane endpoint.
Credential handling (honest terminology) ¶
The package distinguishes four credential surfaces:
In-memory token digest — ServerConfig.TokenHash (SHA-256 hex). Computed during Load; the plaintext ServerConfig.Token field is cleared immediately after hashing. The in-memory Config never retains the plaintext token beyond the Load call.
Plaintext token in the YAML config file — the operator's input. The config file is the operator's responsibility; this package does NOT hash the file contents on disk. "Hashed at rest" would be a false claim if the file still contains the plaintext.
Plaintext token in GSEER_TOKEN env var — the operator's input. Same as above: the env var retains the plaintext; only the in-memory Config has the hash.
Database password (DatabaseConfig.Password) — needed for the PostgreSQL connection DSN. NOT hashed; passed to pgx (M3) as part of the connection string. The password is percent-encoded in the DSN via net/url (Finding 4), but the plaintext is required for the connection. This field must NEVER appear in logs or diagnostic output.
M10 API auth requirements (recorded for the future API layer) ¶
The API layer (M10) must:
- Verify Bearer tokens using crypto/subtle.ConstantTimeCompare against TokenHash (never ==, never strings.Equal).
- Require high-entropy tokens (≥ 256 bits / 32 bytes; the .env.example documents `openssl rand -hex 32`).
- Never log or return the token hash in any response or error.
Logging restrictions (M3/M10) ¶
Until a SEC-6 slog redaction handler lands (M10), the following are PENDING and must be enforced by discipline in M3:
- DatabaseConfig.Password must not appear in any log line, error message, or doctor output.
- The effective DSN (which embeds the password) must not be logged directly; M3 must log a redacted form (host:port/db, no userinfo).
- ServerConfig.TokenHash may appear in diagnostics (it's a hash, not plaintext), but the plaintext Token (if transiently present during Load) must never be logged.
The `yaml:"password,omitempty"` struct tag only controls YAML marshaling (omit when empty); it does NOT provide logging redaction.
Index ¶
- Constants
- Variables
- func HashToken(plaintext string) string
- func Load(flags Flags) (*Config, *Error)
- type ClientConfig
- type Config
- type DatabaseConfig
- type DefaultsConfig
- type Error
- type FingerprintsConfig
- type Flags
- type LogConfig
- type PolicyConfig
- type Profile
- type ServerConfig
- type SourceConfig
- type UpdateConfig
Constants ¶
const ( // DefaultListen is the loopback bind for the self-hosted server. // Non-loopback binding requires explicit operator action and auth. DefaultListen = "127.0.0.1:8443" // DefaultLogLevel is the slog level for v0.0.1. DefaultLogLevel = "info" // DefaultLogFormat is the slog format (text for CLI-friendliness). DefaultLogFormat = "text" // DefaultDBPort is the default PostgreSQL port (loopback). DefaultDBPort = 5432 // DefaultDBName is the default database name. DefaultDBName = "gseer" // DefaultDBUser is the default database user. DefaultDBUser = "gseer" // DefaultDBHost is the default database host (loopback). DefaultDBHost = "127.0.0.1" // DefaultMigrationLockTimeoutMs bounds the migration advisory-lock // wait (M3): how long `gseer setup --self-hosted` / `gseer server // start` waits for the golang-migrate pg_advisory_lock before // failing with DB_MIGRATION_LOCK_TIMEOUT. 30s is enough for a // concurrent-start race (the second process waits for the first to // finish) while never leaving the operator facing a frozen CLI. // Range: 1000..300000 ms (validated in validateValues). DefaultMigrationLockTimeoutMs = 30000 // DefaultEvaluationFixtures is the default for policy.evaluation_fixtures. // False on non-dev installs (QS-6): `setup --load-fixtures` is refused // with a confirmation prompt when false. DefaultEvaluationFixtures = false )
const ( // E_CONFIG_PARSE — the YAML file could not be parsed. CodeConfigParse = "E_CONFIG_PARSE" // E_CONFIG_UNKNOWN_KEY — strict schema: an unknown top-level or // nested key appeared in the YAML file. CodeConfigUnknownKey = "E_CONFIG_UNKNOWN_KEY" // E_CONFIG_POLICY_ACTIVE_REJECTED — any `policy.active.*` key. // Active collection is not part of any release until the P2/PAC gates // is normative, not a missing feature. CodeConfigPolicyActiveRejected = "E_CONFIG_POLICY_ACTIVE_REJECTED" // E_CONFIG_INSECURE — an unsafe key was detected (e.g. // InsecureSkipVerify on a control-plane endpoint). Security // close; the rejection is permanent by design (AC-7.x). CodeConfigInsecure = "E_CONFIG_INSECURE" // E_CONFIG_INVALID_VALUE — a known key has an invalid value (e.g. // an unknown log level, a malformed DSN, an invalid listen address). CodeConfigInvalidValue = "E_CONFIG_INVALID_VALUE" // E_CONFIG_PROFILE_CONFLICT — the config mixes keys that don't // belong to the chosen profile (e.g. database.* in a client profile, // or client.endpoint in a self-hosted profile without explanation). CodeConfigProfileConflict = "E_CONFIG_PROFILE_CONFLICT" // E_CONFIG_REQUIRED — a required key for the chosen profile is // missing (e.g. no database DSN in a self-hosted profile). CodeConfigRequired = "E_CONFIG_REQUIRED" // E_CONFIG_FILE_NOT_FOUND — the --config path or default config // file does not exist. CodeConfigFileNotFound = "E_CONFIG_FILE_NOT_FOUND" // E_CLIENT_ONLY — a server command was invoked under a client // profile (SPEC-03 §5, §7). Raised by the CLI layer when the // profile is consulted; the code is defined here so the config // layer can reference it in profile-validation hints. CodeClientOnly = "E_CLIENT_ONLY" )
Stable E_* codes for the config layer (SPEC-03 §7; M2 adds the E_CONFIG_* family). Codes are lowercase-with-underscores; the E_ prefix is reserved for all user-facing error codes across GSEER.
const DefaultAuditSpoolBytes = 10 * 1024 * 1024 // 10 MiB
DefaultAuditSpoolBytes (PD-4) — audit spool size for v0.0.2. NOT used by any v0.0.1 runtime code path.
const DefaultDecompressedCapBytes = 256 * 1024 // 256 KiB
DefaultDecompressedCapBytes (PD-6) — default decompressed response cap for v0.0.2. NOT used by any v0.0.1 runtime code path.
const DefaultDecompressedCapMaxBytes = 1024 * 1024 // 1 MiB
DefaultDecompressedCapMaxBytes (PD-7) — absolute decompressed response cap for v0.0.2. NOT used by any v0.0.1 runtime code path.
const DefaultEmergencyReserveBytes = 64 * 1024 // 64 KiB
DefaultEmergencyReserveBytes (PD-5) — emergency reserve for v0.0.2. NOT used by any v0.0.1 runtime code path.
const DefaultPostgresImage = "postgres:17-alpine"
DefaultPostgresImage — the Compose-provisioned PostgreSQL image (SPEC-02 §1: postgres:17-alpine). Used by the `setup --with-compose` compose-file generation (M2/M11).
const DefaultPostgresMajor = 15
DefaultPostgresMajor (PD-8) — minimum supported PostgreSQL major version. Used by v0.0.1: the doctor check (M3) verifies the server is >= 15.
const DefaultSearchMode = "service"
DefaultSearchMode (PD-1) — the default GQL search mode. Used by v0.0.1: the `gseer search --mode` flag default and the SearchService default (M10).
Variables ¶
var DefaultCollectionPorts = []int{80, 443, 8080, 8443}
DefaultCollectionPorts (PD-2) — initial collection ports for v0.0.2. NOT used by any v0.0.1 runtime code path.
var DefaultProtocols = []string{"https", "tls"}
DefaultProtocols (PD-3) — initial protocols for v0.0.2. NOT used by any v0.0.1 runtime code path.
Functions ¶
func HashToken ¶
HashToken returns the SHA-256 hex digest of the plaintext token. Exported so that the API layer (M10) can hash incoming Bearer tokens for comparison against ServerConfig.TokenHash.
M10 MUST compare using crypto/subtle.ConstantTimeCompare, not ==.
func Load ¶
Load resolves and validates the configuration. Returns a *Config or an *Error (never a generic error — every failure carries a stable E_* code per SPEC-03 §7).
Load order (corrected per owner M2 review Finding 1):
- Resolve config path (--config flag > GSEER_CONFIG env > default).
- Read file; loose-decode into map[string]any.
- scanForPolicyActive — reject any policy.active.* (AC-7.x).
- scanForUnsafeKeys — reject InsecureSkipVerify etc.
- strictDecode into Config (KnownFields(true) — unknown keys → error).
- applyEnv — environment-variable overlay (profile may change).
- applyFlags — CLI flag overrides (profile finalized here).
- applyDefaults — fill in PD-1…PD-9 and v0.0.1 defaults, based on the FINAL resolved profile (so a CLI --profile=client after a YAML self-hosted file does NOT inherit self-hosted DB defaults).
- hashAndClearToken — SHA-256 the token, clear the plaintext.
10. Validate — profile + value validation.
The final precedence: CLI Flags > Environment > YAML > Defaults.
Types ¶
type ClientConfig ¶
type ClientConfig struct {
// Endpoint is the remote GSEER server URL (e.g. https://gseer.example).
// Required for the client profile; ignored for self-hosted.
Endpoint string `yaml:"endpoint"`
}
ClientConfig holds the client-profile settings (SPEC-03 §5).
type Config ¶
type Config struct {
Profile Profile `yaml:"profile"`
Client ClientConfig `yaml:"client"`
Server ServerConfig `yaml:"server"`
Database DatabaseConfig `yaml:"database"`
Policy PolicyConfig `yaml:"policy"`
Log LogConfig `yaml:"log"`
Sources []SourceConfig `yaml:"sources"`
Defaults DefaultsConfig `yaml:"defaults"`
Fingerprints FingerprintsConfig `yaml:"fingerprints"`
Update UpdateConfig `yaml:"update"`
// Source is the file path the config was loaded from, or "env-only"
// when no file was present (defaults + env + flags only). Set by
// Load; not part of the YAML schema.
Source string `yaml:"-"`
}
Config is the validated, in-memory configuration.
type DatabaseConfig ¶
type DatabaseConfig struct {
// DSN is the full connection string. When set, it overrides the
// component fields below. Format:
// postgres://user:password@host:port/dbname?sslmode=disable
DSN string `yaml:"dsn"`
// Component fields (used when DSN is not set).
Host string `yaml:"host"`
Port int `yaml:"port"`
User string `yaml:"user"`
Password string `yaml:"password,omitempty"` // never logged
Name string `yaml:"name"`
// SSLMode controls connection encryption. Loopback-only deployments
// may use "disable"; non-loopback MUST use "require" or "verify-full".
SSLMode string `yaml:"sslmode"`
// MigrationLockTimeoutMs bounds how long the migration runner waits
// for the migration advisory lock before failing with
// DB_MIGRATION_LOCK_TIMEOUT. Implemented as a session-scoped
// statement_timeout inside the migration driver's Lock(): it covers
// ONLY the pg_advisory_lock acquisition and is reset to 0 (verified)
// before any migration SQL runs — slow migrations are never killed
// by the lock-wait limit. Range 1000..300000 ms; default
// DefaultMigrationLockTimeoutMs (applied for explicit-DSN configs
// too — see applyDatabaseDefaults). Used by the storage layer
// (internal/storage/postgres/lock.go + migratedriver.go).
MigrationLockTimeoutMs int `yaml:"migration_lock_timeout_ms"`
}
DatabaseConfig holds the PostgreSQL connection settings (PD-8).
func (DatabaseConfig) EffectiveDSN ¶
func (d DatabaseConfig) EffectiveDSN() string
EffectiveDSN returns the connection string to use, preferring the explicit DSN over the component fields. Used by the storage layer (M3) and the doctor reachability check (M3/M11).
When building from components, the password is percent-encoded via net/url so that special characters (@, :, /, ?, #, spaces, quotes) are handled correctly. Manual string interpolation would produce a malformed DSN for such passwords (owner M2 review Finding 4).
Exported in M3: the storage layer (internal/storage/postgres) and the `gseer db migrate` / `doctor` commands resolve the operator's DSN through this single method so precedence (DSN > components) lives in exactly one place. The raw result embeds the password — callers must pass it only to connection APIs and use postgres.RedactDSN for any display (M2/M3 logging restrictions).
type DefaultsConfig ¶
type DefaultsConfig struct {
// SearchMode (PD-1) — the default GQL search mode. Used by v0.0.1.
SearchMode string `yaml:"search_mode"`
// CollectionPorts (PD-2) — v0.0.2 planning default. NOT used in v0.0.1.
CollectionPorts []int `yaml:"collection_ports"`
// Protocols (PD-3) — v0.0.2 planning default. NOT used in v0.0.1.
Protocols []string `yaml:"protocols"`
// AuditSpoolBytes (PD-4) — v0.0.2 planning default. NOT used in v0.0.1.
AuditSpoolBytes int `yaml:"audit_spool_bytes"`
// EmergencyReserveBytes (PD-5) — v0.0.2 planning default. NOT used in v0.0.1.
EmergencyReserveBytes int `yaml:"emergency_reserve_bytes"`
// DecompressedCapBytes (PD-6) — v0.0.2 planning default. NOT used in v0.0.1.
DecompressedCapBytes int `yaml:"decompressed_cap_bytes"`
// DecompressedCapMaxBytes (PD-7) — v0.0.2 planning default. NOT used in v0.0.1.
DecompressedCapMaxBytes int `yaml:"decompressed_cap_max_bytes"`
// PostgresMajor (PD-8) — minimum supported PostgreSQL version. Used by doctor (M3).
PostgresMajor int `yaml:"postgres_major"`
}
DefaultsConfig records the owner-approved engineering defaults. PD-1 (search mode) is used by v0.0.1 runtime. PD-2…PD-7 are recorded here as v0.0.2 planning defaults; NO v0.0.1 runtime code path reads them. The owner's directive: "PD-2 through PD-7 are approved as engineering defaults only, not authorization to scan."
type Error ¶
type Error struct {
// Code is the stable E_* identifier (e.g. E_CONFIG_UNKNOWN_KEY).
Code string
// Field is the dotted config path of the offending key, when
// applicable (e.g. "policy.active.enabled", "server.listen").
Field string
// Detail is the short "what failed" sentence.
Detail string
// Why is the "likely why" sentence (optional).
Why string
// Next is the "exact next command" or fix instruction.
Next string
}
Error is the structured config-layer error. Every error carries a stable Code (E_* per SPEC-03 §7) and a human-readable message that follows the style guide: what failed → likely why → exact next command.
The Code is stable across releases so that scripts and CI can match on it; the Message may evolve. Errors are never printed with stack traces unless --verbose is set (AC-5.9).
func Validate ¶
Validate is the top-level validation entry point. It runs all five checks in order: profile → values → (policy.active and unsafe keys are checked at load time in loader.go against the raw map, not here, because the strict decode would already have rejected unknown keys).
func ValidateProfile ¶
ValidateProfile checks that the config's fields are consistent with its declared profile. Returns nil if consistent, or an *Error describing the first violation.
type FingerprintsConfig ¶
type FingerprintsConfig struct {
// Disabled turns OFF rule-derived technology enrichment at
// ingestion. Zero value false = enabled (the useful default); the
// inverted name avoids the YAML bool zero-value trap.
Disabled bool `yaml:"disabled"`
// RulesDir optionally loads operator-contributed rules on top of
// the embedded seed set (contribution format: docs/RULES.md). A
// broken operator rule disables the registry — fail-closed, never
// a silent no-op.
RulesDir string `yaml:"rules_dir"`
}
FingerprintsConfig controls the v0.1.0 fingerprint rule engine (internal/fprule). The engine classifies technology evidence that a stored observation ALREADY carries into `fingerprint.technologies` — it never produces observations and never contacts the network.
type Flags ¶
type Flags struct {
// ConfigPath is the --config flag value (path to a YAML file).
ConfigPath string
// Profile is the --profile flag value (client | self-hosted).
Profile string
// Endpoint is the --server flag value (remote endpoint URL for
// the client profile).
Endpoint string
}
Flags holds the CLI flags that participate in config resolution. Only the flags that map to config fields are here; --verbose/--quiet/ --no-color/--output are CLI state, not config, and are owned by the cli package.
type LogConfig ¶
type LogConfig struct {
Level string `yaml:"level"` // debug | info | warn | error
Format string `yaml:"format"` // text | json
}
LogConfig holds slog settings.
type PolicyConfig ¶
type PolicyConfig struct {
// EvaluationFixtures gates `setup --load-fixtures` (QS-6). Default
// false on non-dev installs; when false, the loader refuses with a
// confirmation prompt.
EvaluationFixtures bool `yaml:"evaluation_fixtures"`
}
PolicyConfig holds policy switches. The `active` sub-section is INTENTIONALLY ABSENT — any policy.active.* key is rejected at load time with E_CONFIG_POLICY_ACTIVE_REJECTED (AC-7.x).
type Profile ¶
type Profile string
Profile is the deployment profile (SPEC-03 §5).
const ( // ProfileClient is the CLI-only profile: queries a remote endpoint, // no PostgreSQL, no local server. ProfileClient Profile = "client" // ProfileSelfHosted runs the local API + dashboard backed by // PostgreSQL. Migrations run on setup --self-hosted and server start. ProfileSelfHosted Profile = "self-hosted" )
type ServerConfig ¶
type ServerConfig struct {
// Listen is the bind address for the API + dashboard. Loopback by
// default; non-loopback requires explicit operator action + auth.
Listen string `yaml:"listen"`
// Token is the plaintext API token from config/env. After Load, it
// is hashed to TokenHash and this field is CLEARED so that no
// in-memory representation retains the plaintext beyond Load.
Token string `yaml:"token,omitempty"`
// TokenHash is the SHA-256 hex digest of Token, computed during Load.
// The API verifies incoming Bearer tokens against this hash.
TokenHash string `yaml:"-"`
// TrustedProxies lists the CIDRs (or bare IPs) of reverse proxies in
// front of the server whose X-Forwarded-For header may be honored
// for client-IP resolution (rate limiting, audit). Empty (default)
// = the peer address is used directly and X-Forwarded-For is
// IGNORED — a client can never spoof its identity by setting the
// header. Only name proxies you actually run (e.g. 127.0.0.1 for a
// local nginx). Invalid entries are rejected at config load.
TrustedProxies []string `yaml:"trusted_proxies,omitempty"`
}
ServerConfig holds the self-hosted server settings.
type SourceConfig ¶
type SourceConfig struct {
Name string `yaml:"name"`
Type string `yaml:"type"` // e.g. "ct", "rdap", "dns"
Endpoint string `yaml:"endpoint"` // the source's API URL (passive)
Enabled bool `yaml:"enabled"` // always false in v0.0.1 runtime
}
SourceConfig is a registered data source (config-only in v0.0.1; never collected). The collection framework lands in v0.0.2.
type UpdateConfig ¶
type UpdateConfig struct {
// ManifestURL is the signed update manifest location (https URL or
// local path). The manifest MUST carry an Ed25519 signature that
// verifies against the key embedded in the binary — the URL alone
// proves nothing (docs/UPDATE.md).
ManifestURL string `yaml:"manifest_url"`
}
UpdateConfig points the updater at the signed release channel.