app

Configuration System (pkg/app)

Overview

The shell-operator configuration system provides a three-layer approach for setting every runtime parameter:

CLI flag (explicit)  >  environment variable  >  hardcoded default

The highest-priority source that supplies a value wins. The implementation lives in three files:

How it works

Initialization sequence

main()
  │
  ├─ 1. cfg := app.NewConfig()        // hardcoded defaults
  ├─ 2. app.ParseEnv(cfg)             // env vars overlay defaults
  │
  ├─ 3. app.BindFlags(cfg, root, cmd) // register CLI flags;
  │                                    //   flag default = current cfg value
  │                                    //   (already contains env override)
  │
  └─ 4. cobra.Execute()               // parse CLI args → explicit flags
         │                             //   overwrite cfg fields directly
         └─ applySliceFlags()          // fixup for []string fields

The key insight: BindFlags never reads environment variables itself. It receives a *Config that already has env values merged in and uses those values as flag defaults. When a flag is not passed on the command line, cobra keeps the default — which is the env value (or the hardcoded default if the env var is unset). When a flag is explicitly passed, cobra writes the CLI value into cfg, so CLI always wins.

Step-by-step

1. NewConfig() — creates a Config with all hardcoded defaults (e.g. ListenPort = "9115", ClientQPS = 5).

2. ParseEnv(cfg) — uses caarlos0/env to overlay environment variables on top of the existing cfg values. The library respects envPrefix tags on nested structs (SHELL_OPERATOR_, KUBE_, DEBUG_, etc.) and field-level env tags. A field whose env var is not set keeps its NewConfig default.

3. BindFlags(cfg, rootCmd, startCmd) — registers every flag on the start subcommand with pflag.*Var(...). The default argument for each flag is cfg.<Field> — the value that already includes env overrides. Returns a post-parse fixup function for []string slice flags.

4. cobra.Execute() → parses CLI arguments. Only explicitly passed flags modify cfg. Untouched flags retain the default, which is the env value.

5. applySliceFlags() — special handling for ClientCA (which is []string). If the CLI provided values, they win; otherwise the env value is kept.

Config struct

type Config struct {
    App           AppSettings           `envPrefix:"SHELL_OPERATOR_"`
    Kube          KubeSettings          `envPrefix:"KUBE_"`
    ObjectPatcher ObjectPatcherSettings `envPrefix:"OBJECT_PATCHER_"`
    Admission     AdmissionSettings     `envPrefix:"VALIDATING_WEBHOOK_"`
    Conversion    ConversionSettings    `envPrefix:"CONVERSION_WEBHOOK_"`
    Debug         DebugSettings         `envPrefix:"DEBUG_"`
    Log           LogSettings           `envPrefix:"LOG_"`
}

Each nested struct uses its own envPrefix, and each field inside uses a short env:"..." tag. The final environment variable name is <prefix><field_tag>, for example:

Full reference: flags ↔ env vars ↔ defaults

Application

Kubernetes

Object Patcher

Validating Webhook

Conversion Webhook

Logging

Debug (hidden flags)

Use shell-operator debug-options to see all hidden debug flags.

Priority behavior

Scenario 1: Only env is set

export SHELL_OPERATOR_LISTEN_PORT=8080
shell-operator start
# → cfg.App.ListenPort = "8080"

ParseEnv writes "8080" into cfg. BindFlags uses "8080" as the flag default. No CLI flag is passed, so the default ("8080") is kept.

Scenario 2: Only flag is passed

shell-operator start --listen-port=3000
# → cfg.App.ListenPort = "3000"

NewConfig sets "9115". ParseEnv has nothing to override. BindFlags registers the flag with default "9115". The explicit --listen-port=3000 overwrites cfg.App.ListenPort.

Scenario 3: Both env and flag are set

export SHELL_OPERATOR_LISTEN_PORT=8080
shell-operator start --listen-port=3000
# → cfg.App.ListenPort = "3000"   ← CLI wins

ParseEnv writes "8080". BindFlags registers with default "8080". The explicit flag writes "3000". CLI flag always wins.

Scenario 4: Neither env nor flag

shell-operator start
# → cfg.App.ListenPort = "9115"   ← hardcoded default

Changing priority (env > flag vs. flag > env)

The current implementation enforces flag > env > default. This is the standard behavior expected by most CLI tools (12-factor apps still work because in containers there are usually no CLI overrides, so env is effectively the highest source).

If you need to invert the priority so that env > flag for specific parameters, modify the BindFlags flow:

// After cobra.Execute(), override cfg field from env if the env var is set,
// regardless of whether the flag was passed:
func overrideFromEnv(cfg *Config) {
    if v := os.Getenv("SHELL_OPERATOR_LISTEN_PORT"); v != "" {
        cfg.App.ListenPort = v
    }
}

Call this function after cobra.Execute() for the parameters where env must have higher priority.

Alternatively, you can check whether a flag was explicitly changed by the user:

if !cmd.Flags().Changed("listen-port") {
    // flag was not passed — env value (already in cfg) is in effect
}

pflag.Changed() returns true only if the flag was explicitly set on the command line. This is exactly how the current system works: if Changed is false, the default (env value) remains.

Handling []string (slice) fields

Slice fields like ClientCA require special treatment because pflag.StringArrayVar does not support setting a non-nil default that can be cleanly overridden by CLI. The approach used:

1. Before BindFlags, the env value is captured in a local variable.
2. StringArrayVar binds to a separate CLI-only slice with a nil default.
3. After cobra.Execute(), a fixup function checks: if the CLI slice is non-empty, use it; otherwise fall back to the captured env value.

envClientCA := cfg.Admission.ClientCA   // save env value
var cliClientCA []string
f.StringArrayVar(&cliClientCA, "validating-webhook-client-ca", nil, "...")

// post-parse fixup:
if len(cliClientCA) > 0 {
    cfg.Admission.ClientCA = cliClientCA
} else {
    cfg.Admission.ClientCA = envClientCA
}

Using shell-operator as a Go library

When you embed shell-operator inside another binary (addon-operator, deckhouse, your own controller…), you typically have your own configuration plumbing and don't want shell-operator's env-var parsing or cobra flags to get in the way. The library-facing contract is just two things:

1. Build a *app.Config yourself, however you like — from NewConfig() defaults, ParseEnv(), your own globals, or any combination.
2. Hand that *app.Config to the assembly methods on *ShellOperator.

There are no wrapper constructors like LoadConfig — Config is a plain exported struct, so the consumer is in control. Typical flow:

import (
    shell_operator "github.com/flant/shell-operator/pkg/shell-operator"
    shapp "github.com/flant/shell-operator/pkg/app"
)

// 1. Build the config from whatever sources you have.
cfg := shapp.NewConfig() // start from hardcoded defaults
cfg.App.ListenAddress = myapp.ListenAddress
cfg.App.ListenPort    = myapp.ListenPort
cfg.Kube.Context      = myapp.KubeContext
cfg.Kube.Config       = myapp.KubeConfig
cfg.Kube.ClientQPS    = myapp.KubeClientQPS
cfg.Kube.ClientBurst  = myapp.KubeClientBurst
cfg.ObjectPatcher.KubeClientQPS     = myapp.ObjectPatcherKubeClientQPS
cfg.ObjectPatcher.KubeClientBurst   = myapp.ObjectPatcherKubeClientBurst
cfg.ObjectPatcher.KubeClientTimeout = myapp.ObjectPatcherKubeClientTimeout
cfg.Log.Level = myapp.LogLevel

// 2. Construct the engine and assemble its common parts from the config.
so := shell_operator.NewShellOperator(ctx, ms, hms,
    shell_operator.WithLogger(logger.Named("shell-operator")),
)
err := so.AssembleCommonOperatorFromConfig(cfg, []string{
    "module", "hook", "binding", "queue", "kind",
})

The new AssembleCommonOperatorFromConfig(cfg, labels) method on *ShellOperator is what makes this clean — it derives both KubeClientConfigs (main + object-patcher), the HTTP listen address/port, and the metric prefix from cfg, so the consumer does not have to unpack fields by hand. The older primitive-taking AssembleCommonOperator(listenAddress, listenPort, labels, mainKubeCfg, patcherKubeCfg) is still available for callers that need finer control.

If you also want env-var parsing on top of your own values, call ParseEnv(cfg) between steps 1 and 2 — env values will overlay the fields you already set, mirroring the CLI's defaults → env order. If you want the opposite (your code wins over env), set those fields after ParseEnv.

For the full CLI-style bootstrap (logging, debug server, hooks discovery, admission/conversion webhooks), use shell_operator.Init(ctx, cfg, logger); it now internally delegates to AssembleCommonOperatorFromConfig so the behaviour stays identical.

Forcing cfg into legacy globals

Some debug sub-commands (queue, hook, config) bind their --debug-unix-socket flag to the package-level app.DebugUnixSocket global instead of cfg, because they don't go through the start command path. To guarantee that an outer-program-supplied cfg overrides this global, call app.ApplyConfig(cfg) once after you've built cfg:

cfg := shapp.NewConfig()
cfg.Debug.UnixSocket = "/run/myapp/debug.socket"
shapp.ApplyConfig(cfg) // DebugUnixSocket now equals cfg.Debug.UnixSocket

shell_operator.Init and BindFlags already call ApplyConfig internally, so most callers don't need this. Call it explicitly only when you bypass both — e.g. you build cfg yourself and invoke debug sub-commands directly without Init/BindFlags. The function is idempotent and a nil cfg is a no-op.

Adding a new parameter

1. Add a field to the appropriate settings struct in app_config.go with an env:"TAG_NAME" tag.

2. Set the default in NewConfig() if a non-zero default is needed.

3. Register the flag in the corresponding bind*Flags function in flags.go:

   f.StringVar(&cfg.App.MyParam, "my-param", cfg.App.MyParam, "Description. Can be set with $SHELL_OPERATOR_MY_PARAM.")
   

   The default argument must be cfg.<field> — the value already merged with env.

4. Add a test in app_config_test.go (update the "all flags" and "all envs" tests, and add a priority test if the behavior is non-trivial).

5. For []string fields — follow the ClientCA pattern: capture the env value before binding, use StringArrayVar with nil default, add a fixup in the returned closure.

Advantages

• Single Config struct — all configuration is in one place, easy to pass around, test, and inspect.
• Declarative env mapping — caarlos0/env with struct tags means no hand-written os.Getenv calls for each field; adding a new env var is one struct tag.
• Deterministic priority — the three-layer model (flag > env > default) is enforced structurally, not by convention. BindFlags receives an already-merged config as flag defaults, so the priority emerges from the initialization order.
• No double-read of env — environment is read once in ParseEnv. BindFlags does not read env at all, eliminating a class of bugs where env parsing is duplicated or inconsistent.
• Testable — Config is a plain struct. Tests create a NewConfig(), optionally call ParseEnv, then bindAndParse with arbitrary argv. No global state is mutated (except the legacy DebugUnixSocket global which is kept for backward compatibility with debug sub-commands).
• Cobra-native — the system uses standard cobra/pflag machinery. Flag help strings, type checking, and completion work out of the box.
• Validation — Validate(cfg) checks the final merged config for consistency (required fields, allowed values) in one place.

Disadvantages

• Slice fields need manual fixups — pflag does not cleanly support "use this default unless the flag is explicitly passed" for []string. The capture-and-fixup pattern works but adds boilerplate and cognitive overhead.
• DebugUnixSocket global — debug sub-commands (queue, hook, config) bind their --debug-unix-socket flag to a package-level variable instead of cfg, because they don't go through the start command path. This is a pragmatic compromise but means there are two sources for this one value.
• No built-in config file support — if a .yaml / .toml config file is ever needed, an additional layer would have to be inserted between ParseEnv and BindFlags.
• Env tag duplication — the env variable name appears in both the struct tag (for caarlos0/env) and the flag help string (for the user). They can drift if updated independently.
• caarlos0/env dependency — introduces a third-party library for env parsing. A purely stdlib approach (os.Getenv + manual assignment) would have zero dependencies but more boilerplate.
• Hidden debug flags — debug flags are registered but marked hidden. Users need to know about debug-options subcommand or read the source to discover them.

Testing

Run the configuration tests:

go test ./pkg/app/ -v -run 'TestBindFlags|TestParseEnv|TestCLIFlag|TestEnvOverrides'

The test suite covers:

• All flags correctly populate Config fields.
• All env vars correctly populate Config fields.
• Env vars override hardcoded defaults when no flag is passed.
• Explicit CLI flags override env vars.