config

package
v1.170.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 12, 2026 License: Apache-2.0 Imports: 20 Imported by: 2

README

CLI / Service configuration

Viper provides quite a lot of utilities to retrieve configuration values and integrates well with cobra which is a reference for CLI implementation in go.

Nonetheless viper has some gaps we tried to fill for ease of configuration:

  • configuration value validation at configuration load
  • .env configuration
  • easy mapping/deserialisation between environment variables and complex nested configuration structures.

The idea is to have the ability to have a complex configuration structure for the project so that values can easily be shared throughout it without requiring viper everywhere. We also tried to gather value in a tree structure in order to categorise them into components configuration instead of having all configuration at the same level in a key/value store.

Usage

Configuration structure

The first step is to define a complex configuration structure for your project and add a Validate() method at each level which will be called during load in order to ensure values are as expected. For validation, we use github.com/go-ozzo/ozzo-validation/v4 but other libraries/method could be used. It is also advised to provide a method which returns the structure with defaults.

Please look at the following as an example:

type DummyConfiguration struct {
	Host              string        `mapstructure:"host"`
	Port              int           `mapstructure:"port"`
	DB                string        `mapstructure:"db"`
	User              string        `mapstructure:"user"`
	Password          string        `mapstructure:"password"`
	HealthCheckPeriod time.Duration `mapstructure:"healthcheck_period"`
}

func (cfg *DummyConfiguration) Validate() error {
	// Validate Embedded Structs
	err := ValidateEmbedded(cfg)
	if err != nil {
		return err
	}

	return validation.ValidateStruct(cfg,
		validation.Field(&cfg.Host, validation.Required),
		validation.Field(&cfg.Port, validation.Required),
		validation.Field(&cfg.DB, validation.Required),
		validation.Field(&cfg.User, validation.Required),
		validation.Field(&cfg.Password, validation.Required),
	)
}

func DefaultDummyConfiguration() DummyConfiguration {
	return DummyConfiguration{
		Port:              5432,
		HealthCheckPeriod: time.Second,
	}
}

Note the mapstructure tag. This will be used as part of the environment value name. The structure can be complex and of different levels. The name of the environment value mapped to a configuration field will be the combination of the mapstructure tags separated by underscore _. for example, CLI_HEALTHCHECK_PERIOD environment variable will refer to the HealthCheckPeriod field in the configuration structure above if the prefix used when loading configuration (see below) is CLI.

Configuration load

Load or LoadFromViper will load values from the environment (including from .env files) and assign them to the different configuration fields in the structure passed. Type conversion will be done automatically and if not set and a default value is provided, then the default value will be retained. Configuration values are then validated using the different Validate() methods provided. In order to easily identify environment variables, a prefix can be provided when loading the configuration. This will tell the system to only consider environment variables with this prefix.

Integration with Cobra CLI arguments

cobra and viper are well integrated in the way that you can bind CLI arguments with configuration values. We provide some utilities (BindFlagToEnv) to leverage this binding so that it works with the configuration system described above and limits the number of hardcoded names or global values usually seen in CLI code generated from cobra.

    // Create a viper session instead of using global configuration
	session := viper.New()

	config := &ConfigurationStructConTainingFlag1{}
	defaults := DefaultConfiguration()

	flagSet := pflag.FlagSet{}
	prefix := "ENV_PREFIX"
    // Define CLI flags
	flagSet.String("f", "flag", "a cli flag")
    // Bind flags to environment variables
	err = BindFlagToEnv(session, prefix, "ENV_PREFIX_FLAG1", flagSet.Lookup("flag"))
    ...
    // Load configuration from the environment
    err = LoadFromViper(session, prefix, config, defaults)

Documentation

Overview

Package config provides utilities to load configuration from an environment and perform validation at load time.

  • Copyright (C) 2020-2022 Arm Limited or its affiliates and Contributors. All rights reserved.
  • SPDX-License-Identifier: Apache-2.0

Index

Constants

View Source 
const (
	SensitiveFieldTagMask      = "mask"
	SensitiveFieldTagSensitive = "sensitive"
	SensitiveFieldTagSecret    = "secret"
	SensitiveFieldTagPassword  = "password"
	SensitiveFieldTagCensored  = "censored"
	SensitiveFieldTagRedact    = "redact"
	SensitiveFieldTagPII       = "pii"
	SensitiveFieldTagSens      = "sens"
)
View Source 
const (
	EnvVarSeparator = "_"
	DotEnvFile      = ".env"
)

Variables

View Source 
var CommonFieldTagSecretConverter value.IValueConverter = NewFieldTagSecretConverter(CommonSensitiveFieldTagNames...)

CommonFieldTagSecretConverter masks values whose source configuration field defines any tag listed in CommonSensitiveFieldTagNames.

View Source 
var CommonSensitiveFieldTagNames = []string{
	SensitiveFieldTagMask,
	SensitiveFieldTagSensitive,
	SensitiveFieldTagSecret,
	SensitiveFieldTagPassword,
	SensitiveFieldTagCensored,
	SensitiveFieldTagRedact,
	SensitiveFieldTagPII,
	SensitiveFieldTagSens,
}

CommonSensitiveFieldTagNames lists common struct-tag keys used by Go libraries and applications to identify fields that should not be exposed verbatim in derived configuration views.

These are tag keys checked via `reflect.StructTag.Lookup`, not field names. For example, this list is intended to match fields tagged like `mask:"redact"` or `pii:"true"`, not field names like `Password`, `Secret`, or `Token` on their own.

Recognising these tags helps reduce accidental disclosure of credentials, secrets, PII, and other confidential data in logs, environment-variable dumps, debug output, support bundles, and similar human-readable output. See https://en.wikipedia.org/wiki/Personal_data for general background on PII and related personal-data concepts.

There is no standard Go struct tag for sensitive data. Some libraries use tags such as `sensitive`, `pii`, `sens`, `mask`, or `redact`, while major logging APIs such as `log/slog`, zap, and zerolog commonly address the problem through custom value or object marshalling instead.

This mechanism can support privacy and data-protection controls such as data minimisation and pseudonymisation, but tagging or masking a field does not by itself establish compliance with GDPR or other privacy regulations.

The tag choices are informed by patterns used in similar masking or redaction libraries and articles, including:

View Source 
var SecretConverter value.IValueConverter = value.ValueConverterFunc(func(ctx context.Context, input any) (any, error) {
	if reflection.IsEmpty(input) {
		return "", nil
	}
	converted, err := value.StringConverter.ConvertValue(ctx, input)
	if err != nil {
		return nil, err
	}
	text := fmt.Sprint(converted)
	runes := []rune(text)
	if len(runes) == 0 {
		return "", nil
	}
	first, _ := collection.First(runes)
	last, _ := collection.Last(runes)
	return fmt.Sprintf("%s*****%s", string(first), string(last)), nil
})

SecretConverter converts a value to a masked string.

It first uses value.StringConverter to normalise the input into a string, then preserves the first and last rune with `*****` in between.

Functions

func BindFlagToEnv 

func BindFlagToEnv(viperSession *viper.Viper, envVarPrefix string, envVar string, flag *pflag.Flag) (err error)

BindFlagToEnv binds pflags to environment variable. Envvar is the environment variable string with or without the prefix envVarPrefix

func BindFlagsToEnv added in v1.75.0 

func BindFlagsToEnv(viperSession *viper.Viper, envVarPrefix string, envVar string, flags ...*pflag.Flag) (err error)

BindFlagsToEnv binds a set of pflags to an environment variable. Envvar is the environment variable string with or without the prefix envVarPrefix It is similar to BindFlagToEnv but can be applied to multiple flags. Note: all the flags will have to be of the same type. If more than one flags is changed, the system will pick one at random.

func DetermineConfigurationEnvironmentVariables added in v1.16.0 

func DetermineConfigurationEnvironmentVariables(appName string, configurationToDecode IServiceConfiguration, converters ...valueUtils.IValueConverter) (defaults map[string]any, err error)

DetermineConfigurationEnvironmentVariables returns all the environment variables corresponding to a configuration structure as well as all the default values currently set.

Optional valueUtils.IValueConverter values are applied to each flattened leaf value before the final prefixed map is returned. This is useful when certain source types should be emitted differently in the environment-variable view, for example to unwrap or redact secret-like types, or to apply type-based matching logic before returning the final value.

Value converters may either wrap a type-driven reflection.IValueConverter through NewValueTypeConverter, which follows the same runtime type pattern as mapstructure.DecodeHookFuncType, or map concrete values to boolean match results through NewValueMatchingConverter.

func DetermineConfigurationEnvironmentVariablesWithContext added in v1.170.0 

func DetermineConfigurationEnvironmentVariablesWithContext(ctx context.Context, appName string, configurationToDecode IServiceConfiguration, converters ...valueUtils.IValueConverter) (defaults map[string]any, err error)

DetermineConfigurationEnvironmentVariablesWithContext returns all the environment variables corresponding to a configuration structure as well as all the default values currently set.

func Load 

func Load(envVarPrefix string, configurationToSet IServiceConfiguration, defaultConfiguration IServiceConfiguration) error

Load loads the configuration from the environment (i.e. .env file, environment variables) and puts the entries into the configuration object configurationToSet. If not found in the environment, the values will come from the default values defined in defaultConfiguration. `envVarPrefix` defines a prefix that ENVIRONMENT variables will use. E.g. if your prefix is "spf", the env registry will look for env variables that start with "SPF_". make sure that the tags on the fields of configurationToSet are properly set using only `[_1-9a-zA-Z]` characters.

func LoadFromConfigurationFile added in v1.31.0 

func LoadFromConfigurationFile(viperSession *viper.Viper, configFile string) (err error)

LoadFromConfigurationFile loads the configuration from the environment. If the format is not supported, an error is raised and the same happens if the file cannot be found. Supported formats are the same as what viper(https://github.com/spf13/viper#what-is-viper) supports

func LoadFromEnvironment added in v1.31.0 

func LoadFromEnvironment(viperSession *viper.Viper, envVarPrefix string, configurationToSet IServiceConfiguration, defaultConfiguration IServiceConfiguration, configFile string) error

LoadFromEnvironment is the same as `LoadFromViper` but also gives the ability to load the configuration from a configuration file as long as the format is supported by [Viper](https://github.com/spf13/viper#what-is-viper) Important note: Viper's precedence order is maintained: 1) values set using explicit calls to `Set` 2) flags 3) environment (variables or `.env`) 4) configuration file 5) key/value store 6) default values (set via flag default values, or calls to `SetDefault` or via `defaultConfiguration` argument provided) Nonetheless, when it comes to default values. It differs slightly from Viper as default values from the default Configuration (i.e. `defaultConfiguration` argument provided) will take precedence over defaults set via `SetDefault` or flags unless they are considered empty values according to `reflection.IsEmpty`.

func LoadFromEnvironmentAndSystem added in v1.112.0 

func LoadFromEnvironmentAndSystem(viperSession *viper.Viper, envVarPrefix string, configurationToSet IServiceConfiguration, defaultConfiguration IServiceConfiguration, configFile string, useKeyring bool) error

LoadFromEnvironmentAndSystem is the same as `LoadFromEnvironment` but also gives the ability to load the configuration from system's [keyring service](https://github.com/zalando/go-keyring). Important note: Viper's precedence order is mostly maintained: 1) values defined in keyring (if not empty and keyring is selected - this is the only difference from Viper) 2) values set using explicit calls to `Set` 3) flags 4) environment (variables or `.env`) 5) configuration file 6) key/value store 7) default values (set via flag default values, or calls to `SetDefault` or via `defaultConfiguration` argument provided) Nonetheless, when it comes to default values. It differs slightly from Viper as default values from the default Configuration (i.e. `defaultConfiguration` argument provided) will take precedence over defaults set via `SetDefault` or flags unless they are considered empty values according to `reflection.IsEmpty`.

func LoadFromSystem added in v1.112.0 

func LoadFromSystem(envVarPrefix string, configurationToSet IServiceConfiguration, defaultConfiguration IServiceConfiguration) error

LoadFromSystem is similar to Load but also fetches values from system's [keyring service](https://github.com/zalando/go-keyring).

func LoadFromViper 

func LoadFromViper(viperSession *viper.Viper, envVarPrefix string, configurationToSet IServiceConfiguration, defaultConfiguration IServiceConfiguration) error

LoadFromViper is the same as `Load` but instead of creating a new viper session, reuse the one provided. Important note: Viper's precedence order is maintained: 1) values set using explicit calls to `Set` 2) flags 3) environment (variables or `.env`) 4) key/value store 5) default values (set via flag default values, or calls to `SetDefault` or via `defaultConfiguration` argument provided) Nonetheless, when it comes to default values. It differs slightly from Viper as default values from the default Configuration (i.e. `defaultConfiguration` argument provided) will take precedence over defaults set via `SetDefault` or flags unless they are considered empty values according to `reflection.IsEmpty`.

func LoadFromViperAndSystem added in v1.112.0 

func LoadFromViperAndSystem(viperSession *viper.Viper, envVarPrefix string, configurationToSet IServiceConfiguration, defaultConfiguration IServiceConfiguration) error

LoadFromViperAndSystem is the same as `LoadFromViper` but also fetches values from system's [keyring service](https://github.com/zalando/go-keyring).

func LoadUsing added in v1.145.0 

func LoadUsing(configurationToSet IServiceConfiguration, options ...LoadingOption) (err error)

LoadUsing loads the service configuration using the provided loading options.

Important note: Viper's precedence order is mostly maintained: 1) values defined in keyring (if not empty and keyring is selected - this is the only difference from Viper) 2) values set using explicit calls to `Set` 3) flags 4) environment (variables or `.env`) 5) configuration file 6) key/value store 7) default values (set via flag default values, or calls to `SetDefault` or via `defaultConfiguration` argument provided) Nonetheless, when it comes to default values. It differs slightly from Viper as default values from the default Configuration (i.e. `defaultConfiguration` argument provided) will take precedence over defaults set via `SetDefault` or flags unless they are considered empty values according to `reflection.IsEmpty`.

func NewFieldNameConverter added in v1.170.0 

func NewFieldNameConverter(match collection.MatchingFunction[string], converter value.IValueConverter) value.IValueConverter

NewFieldNameConverter returns a converter that applies converter when the source configuration field name matches.

It uses the Go struct field name recorded in the conversion context. When the field name does not match, or no field metadata is available, the original input value is preserved.

func NewFieldNameSecretConverter added in v1.170.0 

func NewFieldNameSecretConverter(match collection.MatchingFunction[string]) value.IValueConverter

NewFieldNameSecretConverter returns a converter that masks values when the source configuration field name matches a caller-supplied predicate.

This supports name-based detection of sensitive fields using conventions such as `Password`, `Secret`, `APIKey`, `AccessToken`, or `PrivateKey`, even when the source type does not carry explicit sensitivity tags.

The approach mirrors redaction layers often built around structured logging libraries such as `log/slog`, Zap, Logrus, and Zerolog, where field or attribute names are inspected to decide which values should be masked. Those logging libraries do not generally apply name-based redaction by default; the policy is usually implemented in handlers, hooks, formatters, middleware, or application-level wrappers.

Name-based matching is best treated as a defence-in-depth heuristic rather than a primary classification mechanism, and is most effective when used alongside explicit sensitivity annotations such as struct tags.

func NewFieldTagSecretConverter added in v1.170.0 

func NewFieldTagSecretConverter(tagNames ...string) value.IValueConverter

NewFieldTagSecretConverter returns a converter that masks values whose source configuration field defines any of tagNames.

It uses reflection.StructPropertyHasTag against the original configuration field metadata carried through the conversion context, and delegates the masking itself to SecretConverter. This is useful when generating environment-variable views or other derived representations of configuration that should avoid leaking secrets or sensitive personal data.

func NewValueMatchingConverter added in v1.170.0 

func NewValueMatchingConverter[T any](match collection.MatchingFunction[T]) value.IValueConverter

NewValueMatchingConverter applies match to values of type T and replaces the value with the resulting boolean match outcome.

func NewValueTypeConverter added in v1.170.0 

func NewValueTypeConverter(converter reflection.IValueConverter) value.IValueConverter

NewValueTypeConverter wraps a reflection-style converter.

func ValidateEmbedded 

func ValidateEmbedded(cfg Validator) error

ValidateEmbedded uses reflection to find embedded structs and validate them

Types

type IServiceConfiguration 

type IServiceConfiguration interface {
	Validator
}

IServiceConfiguration defines a typical service configuration.

type IValidationError added in v1.88.0 

type IValidationError interface {
	error
	GetMapStructurePath() string
	GetTreePath() string
	GetReason() string
	Unwrap() error
	RecordField(fieldName string, mapStructureFieldName *string, mapStructurePrefix *string)
	RecordPrefix(mapStructurePrefix string)
	fmt.Stringer
	GetTree() []string
	GetMapStructureTree() []string
	GetMapStructurePrefix() *string
}

IValidationError defines a typical structure validation error.

func WrapFieldValidationError added in v1.88.0 

func WrapFieldValidationError(fieldName string, mapStructure, prefix *string, err error) IValidationError

WrapFieldValidationError creates an error resulting from the validation of a field in a structure

func WrapValidationError added in v1.88.0 

func WrapValidationError(prefix *string, err error) IValidationError

WrapValidationError creates an error resulting from the validation of a structure

type LoadingOption added in v1.145.0 

type LoadingOption func(*LoadingOptions) *LoadingOptions

LoadingOption represents a functional option used to modify LoadingOptions during configuration loading.

func DisableKeyring added in v1.145.0 

func DisableKeyring() LoadingOption

DisableKeyring disables the use of the system keyring for loading or storing sensitive configuration values.

func UseKeyring added in v1.145.0 

func UseKeyring() LoadingOption

UseKeyring enables the use of the system keyring for loading or storing sensitive configuration values.

func WithAdditionalDecoderOptions added in v1.145.0 

func WithAdditionalDecoderOptions(hooks ...mapstructure.DecodeHookFunc) LoadingOption

WithAdditionalDecoderOptions appends custom mapstructure decode hooks used when unmarshalling configuration values into Go types.

The provided hooks are added to any decoder options already configured.

func WithConfigFile added in v1.145.0 

func WithConfigFile(path string) LoadingOption

WithConfigFile specifies an explicit configuration file path to load. Passing an empty string is allowed and will be treated as a valid path.

func WithDecoderOptions added in v1.145.0 

func WithDecoderOptions(hooks ...mapstructure.DecodeHookFunc) LoadingOption

WithDecoderOptions sets custom mapstructure decode hooks used when unmarshalling configuration values into Go types.

The provided hooks replace any previously configured decoder options.

func WithDefaultConfiguration added in v1.145.0 

func WithDefaultConfiguration(cfg IServiceConfiguration) LoadingOption

WithDefaultConfiguration sets the baseline configuration that is applied before loading values from other configuration sources.

func WithEnvVarPrefix added in v1.145.0 

func WithEnvVarPrefix(prefix string) LoadingOption

WithEnvVarPrefix sets the prefix applied to environment variables when resolving configuration values.

func WithKeyring added in v1.145.0 

func WithKeyring(enabled bool) LoadingOption

WithKeyring enables or disables the use of the system keyring for loading or storing sensitive configuration values.

func WithViperSession added in v1.145.0 

func WithViperSession(v *viper.Viper) LoadingOption

WithViperSession sets the Viper instance used to load configuration from files, environment variables, and other supported sources.

type LoadingOptions added in v1.145.0 

type LoadingOptions struct {
	// contains filtered or unexported fields
}

LoadingOptions defines the parameters used when loading and decoding service configuration from various sources such as defaults, configuration files, environment variables, and external backends.

It controls how configuration values are discovered, merged, and decoded, including support for custom decode hooks and optional keyring integration.

func DefaultLoadingOptions added in v1.145.0 

func DefaultLoadingOptions() *LoadingOptions

func (*LoadingOptions) HasDecoderOptions added in v1.145.0 

func (o *LoadingOptions) HasDecoderOptions() bool

func (*LoadingOptions) Validate added in v1.145.0 

func (o *LoadingOptions) Validate() (err error)

type Validator 

type Validator interface {
	Validate() error
}

Validator defines an object which can perform some validation on itself.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.