Documentation
¶
Overview ¶
Package config provides configuration management for GNdb.
This package has no I/O dependencies (no file operations, no network calls). Validation functions may write user-facing warnings via gn.Warn().
Configuration Sources ¶
Precedence (highest to lowest): CLI flags > env vars > config.yaml > defaults
Design Principles ¶
- Default config (from New()) is always valid - no validation needed - All mutations go through Option functions - the only way to modify Config - Invalid options are rejected with gn.Warn() - config remains in valid state - ToOptions() converts persistent fields (those in config.yaml) - Environment variables match ToOptions() fields exactly
Persistent vs Runtime Fields ¶
Persistent fields (in ToOptions, config.yaml, and env vars):
- Database: host, port, user, password, database, ssl_mode, batch_size
- Log: level, format, destination
- General: jobs_number
Runtime-only fields (CLI flags only):
- Populate.SourceIDs, ReleaseVersion, ReleaseDate, WithFlatClassification (per-command)
- HomeDir (set once at startup)
Environment Variables ¶
Use GNDB_ prefix with underscores for nesting:
GNDB_DATABASE_HOST=localhost GNDB_DATABASE_PORT=5432 GNDB_LOG_LEVEL=info GNDB_JOBS_NUMBER=8
See .envrc.example for complete list with defaults.
Index ¶
- Variables
- func CacheDir(homeDir string) string
- func ConfigDir(homeDir string) string
- func ConfigFilePath(homeDir string) string
- func CustomSourcesFilePath(homeDir string) string
- func LogDir(homeDir string) string
- func SourcesFilePath(homeDir string) string
- type Config
- type DatabaseConfig
- type ExportConfig
- type LogConfig
- type Option
- func OptDatabaseBatchSize(i int) Option
- func OptDatabaseDatabase(s string) Option
- func OptDatabaseHost(s string) Option
- func OptDatabasePassword(s string) Option
- func OptDatabasePort(i int) Option
- func OptDatabaseSSLMode(s string) Option
- func OptDatabaseUser(s string) Option
- func OptExportOutputDir(s string) Option
- func OptExportParentDir(s string) Option
- func OptExportSourceIDs(ii []int) Option
- func OptExportWithZip(b bool) Option
- func OptHomeDir(s string) Option
- func OptJobsNumber(i int) Option
- func OptLogDestination(s string) Option
- func OptLogFormat(s string) Option
- func OptLogLevel(s string) Option
- func OptPopulateReleaseDate(s string) Option
- func OptPopulateReleaseVersion(s string) Option
- func OptPopulateSourceIDs(ii []int) Option
- func OptPopulateWithFlatClassification(b *bool) Option
- type PopulateConfig
Constants ¶
This section is empty.
Variables ¶
var ( // MinVersionSFGA determines the SFGA version which is still compatible // with GNdb. Versions higher than minimal are all supported. MinVersionSFGA = "v0.3.30" // AppName is used in generating file system paths. AppName = "gndb" )
Functions ¶
func CacheDir ¶
CacheDir returns the directory path for cache files. Returns ~/.cache/gndb by default.
func ConfigDir ¶
ConfigDir returns the directory path for configuration files. Returns ~/.config/gndb by default.
func ConfigFilePath ¶
ConfigFilePath returns the full path to the config.yaml file. Returns ~/.config/gndb/config.yaml by default.
func CustomSourcesFilePath ¶ added in v0.1.2
CustomSourcesFilePath returns the full path to the custom_sources.yaml file. Returns ~/.config/gndb/custom_sources.yaml by default.
func LogDir ¶
LogDir returns the directory path for log files. Returns ~/.local/share/gndb/logs by default.
func SourcesFilePath ¶
SourcesFilePath returns the full path to the sources.yaml file. Returns ~/.config/gndb/sources.yaml by default.
Types ¶
type Config ¶
type Config struct {
// Database contains PostgreSQL connection settings.
Database DatabaseConfig `mapstructure:"database" yaml:"database"`
// Populate contains settings specific to the populate command.
Populate PopulateConfig `mapstructure:"populate" yaml:"populate"`
// Export contains settings specific to the export command.
Export ExportConfig `mapstructure:"export" yaml:"export"`
Log LogConfig `mapstructure:"log" yaml:"log"`
// JobsNumber is the number of concurrent workers for parallel operations.
// Default value is set accoring to the number of available threads.
JobsNumber int `mapstructure:"jobs_number" yaml:"jobs_number"`
// HomeDir determines where config, cache and logs directories reside.
// It must be set by CLI during init, there is no default value for it.
HomeDir string
}
Config represents the complete GNdb configuration.
func New ¶
func New() *Config
New creates a Config with sensible default values. The returned config is always valid and ready to use. Default values can be overridden using Option functions via Update().
func (*Config) ToOptions ¶
ToOptions converts the Config to a slice of Option functions. Only includes persistent fields appropriate for config.yaml. Excludes runtime-only fields (HomeDir, SourceIDs, ReleaseVersion/Date, WithFlatClassification). Used for round-tripping config.yaml ↔ Config conversions.
type DatabaseConfig ¶
type DatabaseConfig struct {
// Host is the PostgreSQL server hostname or IP address.
Host string `mapstructure:"host" yaml:"host"`
// Port is the PostgreSQL server port number.
Port int `mapstructure:"port" yaml:"port"`
// User is the PostgreSQL database username.
User string `mapstructure:"user" yaml:"user"`
// Password is the PostgreSQL database password.
Password string `mapstructure:"password" yaml:"password"`
// Database is the PostgreSQL database name to connect to.
Database string `mapstructure:"database" yaml:"database"`
// SSLMode specifies the SSL connection mode.
// Valid values: "disable", "require", "verify-ca", "verify-full"
SSLMode string `mapstructure:"ssl_mode" yaml:"ssl_mode"`
// BatchSize defines the number of records to process per batch for bulk operations.
// Used in both populate (data import) and optimize (word extraction) phases.
// Larger batches are faster but use more memory. Tune based on available RAM.
// Typical values: 5000-100000 depending on record size and available memory.
BatchSize int `mapstructure:"batch_size" yaml:"batch_size"`
}
DatabaseConfig contains PostgreSQL connection parameters.
type ExportConfig ¶ added in v0.1.3
type ExportConfig struct {
// SourceIDs is the list of data source IDs to export.
// Empty slice means export all sources from the data_sources table.
SourceIDs []int
// OutputDir is the directory where exported files are written.
// Defaults to the current working directory.
OutputDir string
// ParentDir is written to the `parent` field in companion YAML files.
// Defaults to OutputDir. Set to the URL or path from which exported
// files will actually be served, e.g. "http://myserver.org/sfga/".
ParentDir string
// WithZip produces .zip compressed variants alongside .sqlite/.sql.
WithZip bool
}
ExportConfig contains settings specific to the export command. All fields are runtime-only (CLI flags only, not persisted in config.yaml).
type LogConfig ¶
type LogConfig struct {
// Format can be 'json', 'text' or 'tint' (user-facing and colored).
Format string `mapstructure:"format" yaml:"format"`
// Level of logging -- 'error', 'warn', 'info', 'debug'
Level string `mapstructure:"level" yaml:"level"`
// Destination can be a log file (to default place), STDERR or STDOUT
Destination string `mapstructure:"destination" yaml:"destination"`
}
LogConfig provides typical settings for application logs.
type Option ¶
type Option func(*Config)
Option is a function that modifies a Config. Options validate inputs and reject invalid values with warnings.
func OptDatabaseBatchSize ¶
OptDatabaseBatchSize sets the number of records to process per batch. Used for bulk operations in populate and optimize phases.
func OptDatabaseDatabase ¶
OptDatabaseDatabase sets the PostgreSQL database name to connect to.
func OptDatabaseHost ¶
OptDatabaseHost sets the PostgreSQL server hostname or IP address.
func OptDatabasePassword ¶
OptDatabasePassword sets the PostgreSQL database password.
func OptDatabasePort ¶
OptDatabasePort sets the PostgreSQL server port number.
func OptDatabaseSSLMode ¶
OptDatabaseSSLMode sets the SSL connection mode. Valid values: "disable", "require", "verify-ca", "verify-full".
func OptDatabaseUser ¶
OptDatabaseUser sets the PostgreSQL database username.
func OptExportOutputDir ¶ added in v0.1.3
OptExportOutputDir sets the directory where exported files are written. Runtime-only field - not in ToOptions().
func OptExportParentDir ¶ added in v0.1.3
OptExportParentDir sets the value written to the `parent` field in companion YAML files. Defaults to OutputDir if not set. Runtime-only field - not in ToOptions().
func OptExportSourceIDs ¶ added in v0.1.3
OptExportSourceIDs sets the list of data source IDs to export. Empty slice means export all sources from the data_sources table. Runtime-only field - not in ToOptions().
func OptExportWithZip ¶ added in v0.1.3
OptExportWithZip sets whether to produce .zip compressed variants alongside .sqlite/.sql output files. Runtime-only field - not in ToOptions().
func OptHomeDir ¶
OptHomeDir sets the home directory for config, cache, and log locations. Set once at startup from os.UserHomeDir(). Runtime-only field - not in ToOptions().
func OptJobsNumber ¶
OptJobsNumber sets the number of concurrent workers for parallel operations. Default is runtime.NumCPU().
func OptLogDestination ¶
OptLogDestination sets where logs are written. Valid values: "file", "stdin", "stdout".
func OptLogFormat ¶
OptLogFormat sets the log output format. Valid values: "json", "text", "tint".
func OptLogLevel ¶
OptLogLevel sets the logging level. Valid values: "debug", "info", "warn", "error".
func OptPopulateReleaseDate ¶
OptPopulateReleaseDate overrides the release date for a single-source import. Format: YYYY-MM-DD. Only valid when importing one source. Runtime-only field - not in ToOptions().
func OptPopulateReleaseVersion ¶
OptPopulateReleaseVersion overrides the version for a single-source import. Only valid when importing one source. CLI validates this constraint. Runtime-only field - not in ToOptions().
func OptPopulateSourceIDs ¶
OptPopulateSourceIDs sets the list of data source IDs to import. Empty slice means import all sources from sources.yaml. Runtime-only field - not in ToOptions().
func OptPopulateWithFlatClassification ¶
OptPopulateWithFlatClassification sets whether to prefer flat classification over hierarchical parent/child classification from SFGA. Uses pointer to distinguish between unset (nil) and false. Runtime-only field - not in ToOptions().
type PopulateConfig ¶
type PopulateConfig struct {
// SourceIDs is the list of data source IDs to import.
// Empty slice means import all sources from sources.yaml.
// The CLI filters sources and passes only the IDs to process.
// Populate() will load sources.yaml and look up each source by ID.
SourceIDs []int `mapstructure:"source_ids" yaml:"source_ids"`
// ReleaseVersion overrides the version for the data source being imported.
// Only valid when importing a single source (len(SourceIDs) == 1).
// The CLI validates this constraint before calling Populate().
ReleaseVersion string `mapstructure:"release_version" yaml:"release_version"`
// ReleaseDate overrides the release date for the data source being imported.
// Format: YYYY-MM-DD. Only valid when importing a single source (len(SourceIDs) == 1).
// The CLI validates this constraint before calling Populate().
ReleaseDate string `mapstructure:"release_date" yaml:"release_date"`
// WithFlatClassification is true if the 'flat' version of classification
// is preferable instead of parent/child hierarchical classification.
// Note: If flat classification does not exist in SFGA, classification breadcrumbs
// will stay empty even if parent/child hierarchy exists.
// Default: false (hierarchical parent/child classification is preferred)
WithFlatClassification *bool `mapstructure:"with_flat_classification" yaml:"with_flat_classification"`
}
PopulateConfig contains settings specific to the populate command.