cogs

COGS

cogs is a cli tool that allows generation of configuration values through aggregation and resolution of file path (what file holds your config value) and object path (where in the markup data is the config value) pointers.

cogs allows one to deduplicate sources of truth by maintaining a source of reference (the cog file) that points to the location of values (such as port numbers and password strings)

Sources of reference include:

• local files
• remote files (through HTTP GET requests)
• SOPS encrypted files (can also be remote)

installation:

With go:

• clone this repo, cd into it
• go build -o $GOPATH/bin ./cmd/cogs

Without go, PLatform can be Linux/Windows/Darwin:

PL="Darwin" VR="0.8.0" \
  curl -SLk \ 
  "github.com/Bestowinc/cogs/releases/download/v${VR}/cogs_${VR}_${PL}_x86_64.tar.gz" | \
  tar xvz -C /usr/local/bin cogs

help string:

COGS COnfiguration manaGement S

Usage:
  cogs gen <ctx> <cog-file> [options]

Options:
  -h --help        Show this screen.
  --version        Show version.
  --no-enc, -n     Skips fetching encrypted vars.
  --no-decrypt	   Skipts decrypting encrypted vars.
  --envsubst, -e   Perform environmental substitution on the given cog file.
  --keys=<key,>    Include specific keys, comma separated.
  --not=<key,>     Exclude specific keys, comma separated.
  --out=<type>     Configuration output type [default: json].
                   <type>: json, toml, yaml, dotenv, raw.
  
  --export, -x     If --out=dotenv: Prepends "export " to each line.
  --preserve, -p   If --out=dotenv: Preserves variable casing.
  --sep=<sep>      If --out=raw:    Delimits values with a <sep>arator.

cogs gen - outputs a flat and serialized K:V array

annotated spec:

name = "basic example" # every cog manifest should have a name key that corresponds to a string

# key value pairs for a context/ctx are defined under <ctx>.vars
# try running `cogs gen basic basic.cog.toml` to see what output cogs generates
[basic.vars]
var = "var_value"
other_var = "other_var_value"

# if <var>.path is given a string value,
# cogs will look for the key name of <var> in the file that that corresponds to the <var>.path key,
# returning the corresponding value
manifest_var.path = "./test_files/manifest.yaml"
# try removing manifest_var from "./test_files/manifest.yaml" and see what happens

# some variables can set an explicit key name to look for instead of defaulting to look for
# the key name "<var>":
# if <var>.name is defined then cogs will look for a key name that matches <var>.name
look_for_manifest_var.path = "./test_files/manifest.yaml"
look_for_manifest_var.name = "manifest_var"

# dangling variable names should return an error
# try uncommenting the line below and run `cogs gen basic basic.cog.toml`
# empty_var.name = "some_name"

example data:

Example data can be used as a tutorial run cogs gen on the files in the order below, run the command once then read the file to soo how the underlying logic is used:

1. basic example:
   • cogs gen basic basic.cog.toml
2. secret values and paths example:
   • gpg --import ./test_files/sops_functional_tests_key.asc should be run to import the test private key used for encrypted dummy data
   • cogs gen sops secrets.cog.toml
3. read types example:
   • cogs gen kustomize read_types.cog.toml
4. advanced patterns example:
   • cogs gen kustomize read_types.cog.toml
5. envsubst patterns example:
   • NVIM=nvim cogs gen envsubst envsubst.cog.toml --envsubst

envsubst cheatsheet:

Notes and references:

envsubst warning - make sure that any environmental substition declarations allow a file to be parsed as TOML without the usage of the --envsubst flag:

# valid envsubst definitions can be placed anywhere string values are valid
["${ENV}".vars]
thing = "${THING_VAR}"
# the `${ENV}` below creates a TOML read error
[env.vars]${ENV}
thing = "${THING_VAR}"

Further references

• TOML spec
• yq expressions
• envsubst