flate

module
v0.0.0-...-3696fcf Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 26, 2026 License: AGPL-3.0

README

flate

Render and diff Flux GitOps repositories fully offline — one static binary, no cluster, no kubectl, no shellouts.

Tests Build Lint Release License

flate is a Go rewrite of flux-local. Helm, kustomize, go-git, and oras-go are linked as native libraries, so a kind cluster plus a stack of CLIs (helm, kustomize, flux, kubectl) collapse into one binary that runs in CI in seconds, not minutes. Changed-only mode reconciles just the subtree a PR touches, dropping single-file diffs to tens of milliseconds on real home-ops repos.

Install

brew install --cask home-operations/tap/flate
go install github.com/home-operations/flate/cmd/flate@latest
docker pull ghcr.io/home-operations/flate:latest

…or in a GitHub Actions workflow:

- uses: home-operations/flate/action@main

Use

flate get ks       --path ./kubernetes
flate build hr     --path ./kubernetes plex
flate diff ks      --path ./kubernetes --path-orig ../baseline/kubernetes
flate diff images  --path ./kubernetes --path-orig ../baseline/kubernetes -o json
flate test all     --path ./kubernetes

The [name] positional on build/diff/test is matched against the resource's bare name (metadata.name), not namespace/name. Use -n / --namespace to scope.

Every command takes --path <dir> (default .); --path-orig <dir> switches into changed-only mode. flate <verb> --help lists every flag.

Verb Targets Notes
get ks, hr, images, all List or summarize. -o table / yaml / json / name.
build ks, hr, all Render Kustomizations and HelmReleases to YAML or JSON.
diff ks, hr, images Path-keyed diff against --path-orig, rendered via dyff in --output github mode. K8s-aware: list entries match by identifier (container name, env-var name), so a reorder shows as ⇆ order changed instead of a wall of phantom value churn.
test ks, hr, all Pytest-style PASS / FAIL / SKIPPED per resource. Non-zero exit on any failure.

get ks and get hr accept -l/--selector key=value for label filtering. diff accepts --strip-attr <key> (repeatable) to drop annotation/label keys before comparison; the default set covers chart-bump noise (helm.sh/chart, checksum/config, app.kubernetes.io/version, chart). Every subcommand accepts --allow-missing-secrets to soft-skip sources whose auth Secret is missing or PLACEHOLDER-wiped — see Behaviors.

Default output filters. --skip-secrets and --skip-crds both default to truebuild/diff/test strip rendered Secret and CustomResourceDefinition objects from output. Pass --skip-secrets=false / --skip-crds=false to include them; --skip-kinds <kind> (repeatable) drops additional kinds. These are output-stream filters, distinct from --allow-missing-secrets, which gates source fetch on missing auth Secrets.

Changed-only mode

--path-orig flips every command into change-aware reconcile. flate diffs the two paths, walks ownership backwards (longest matching Flux KS spec.path, including spec.components), and reconciles only the touched subtree plus its content dependencies.

In the keep-set: direct file edits, chart sources, KS sourceRef, HR valuesFrom, kustomize components (touching a shared component re-renders every consumer).

Out: dependsOn (reconcile-ordering, not content — skipped resources still get marked Ready so downstream waits unblock) and meta-Kustomizations that don't claim the deeper file.

git worktree add ../baseline main
flate diff ks --path ./kubernetes --path-orig ../baseline/kubernetes

--path can point at a narrow Flux entry like ./kubernetes/flux/cluster; flate iteratively follows each loaded KS's spec.path to discover the rest of the tree.

Source kinds and auth

Kind Status Auth (spec.secretRef)
GitRepository full HTTPS: username + password or bearerToken. SSH: identity (+ optional password, known_hosts).
OCIRepository full .dockerconfigjson. Falls back to --registry-config, then ~/.docker/config.json.
HelmRepository full HTTP basic: username + password. OCI flavor: use a sibling OCIRepository.
HelmChart full Inline (HR.spec.chart) and standalone CRD.
Bucket generic only accesskey + secretkey. aws/gcp/azure fail loud — use static creds.
ExternalArtifact file:// only status.artifact.url must be a local path.

PLACEHOLDER-wiped values (the always-on wipe of cleartext Secret data) are treated as missing — auth fails with a clear "missing username/password" instead of attempting auth with the placeholder string. See Behaviors for --allow-missing-secrets, which soft-skips affected sources end-to-end.

Behaviors

SOPSspec.decryption is not implemented. Encrypted Secret values get wiped to ..PLACEHOLDER_<key>.., same as cleartext values under the always-on wipe. Downstream postBuild.substituteFrom lookups resolve to the placeholder string rather than failing.

spec.suspend — honored on every reconcilable CR. Suspended resources mark Ready / "suspended" and produce no rendered output.

--allow-missing-secrets — off by default. When set, a source whose auth secretRef is missing or PLACEHOLDER-wiped marks Ready / "skipped: …" instead of Failed, and consumers (KS sourceRef, HR chartRef) propagate the skip so flate test reports SKIPPED rather than a cascade of FAILED. Intended for repos that materialize auth on the live cluster via ExternalSecret / SealedSecret. Typos in secretRef.name are rejected at parse time so they don't silently fall into this path. Scope is auth secretRef only — verify, cert, and proxy secretRefs still fail loud, since silently dropping verification or TLS material is a security downgrade.

spec.dependsOn[].readyExpr (CEL) — evaluated against self and dep projections, matching upstream kustomize- and helm-controller binding:

dependsOn:
- name: infra-controllers
  readyExpr: |
    dep.status.conditions.exists(c, c.type == "Healthy" && c.status == "True")

Substitution opt-out — the kustomize.toolkit.fluxcd.io/substitute: disabled label or annotation is honored per-resource, matching kustomize-controller. Used for ConfigMaps embedding bash array expansions envsubst can't parse.

Signature verificationOCIRepository uses cosign keyed mode (spec.verify.secretRef with PEM keys) verified through stdlib crypto, no sigstore dep tree. GitRepository uses PGP via spec.verify.{mode,secretRef}. Cosign keyless and notation are not supported (see Limits).

Library use

pkg/orchestrator is the embed entry point.

import (
    "context"
    "github.com/home-operations/flate/pkg/orchestrator"
)

o, _ := orchestrator.New(orchestrator.Config{Path: "/path/to/cluster"})
res, err := o.Render(context.Background())
// res is non-nil even when err != nil — partial output stays usable.
for id, docs := range res.Manifests {
    // rendered YAML docs for the KS / HR with this id
}
for id, info := range res.Failed {
    // structured failure list keyed by NamedResource
}

Other entry points worth knowing:

  • Orchestrator.WithFetcher(kind, f) — swap any source fetcher (in-memory fakes for tests, custom kinds).
  • Store.OnObject / OnStatus / OnArtifact — typed listeners; payloads are pre-cast.
  • helm.Prepare(hr, lookup, provider) then helmClient.TemplateDocs(...) — render one HelmRelease without the orchestrator. lookup is a manifest.HelmChartLookup (func(ns, name string) *HelmChartSource). kustomize.Prepare(ks, provider) is the symmetric helper for Kustomizations.
  • discovery.Run(ctx, Config{Path, Store, WipeSecrets}) — load phase as a standalone unit.
  • change.Filter.Add(id) — extend the changed-only-mode keep set at runtime when a custom controller emits a child that wasn't visible at filter-build time. Call BEFORE Store.AddObject(child) so the synchronous listener sees the extended set.
  • Store.Mutate[T] — clone-then-AddObject helper encoding the immutability contract. See pkg/manifest/doc.go for the full rule.

Architecture

discovery → Store ⇄ events ⇄ controllers (source · kustomization · helmrelease)

Pipeline: bootstrap-source seed → loader pre-pass excluding configMapGenerator/secretGenerator data files → file walk → spec.path + ResourceSet fixed-point expansion → bootstrap-source aliasing for unresolved GitRepository refs → namespace inheritance → parent index → dependsOn validation + cycle break → change-filter → controllers fire → render → render-time keep-set extension for emitted children → orphan demotion → output.

The Store is the single source of truth. Every stored manifest is immutable; mutation routes through Store.Mutate[T] (clone, mutate, AddObject). Helm chart loads coalesce through a per-path keylock — N parallel reconciles of the same chart issue exactly one parse.

Limits

flate is rendering-only.

  • No SOPS decryption. Values wiped; pre-decrypt if you need them in the diff.
  • No cosign keyless. Keyed verification works end-to-end; keyless logs and renders unverified (no offline trust roots).
  • No notation. Fails loud.
  • No cloud workload identity. spec.serviceAccountName is a no-op; use static creds in a Secret.
  • No healthChecks. flate tracks resource readiness, not status conditions of rendered objects.
  • ResourceSetInputProvider: Static only. Dynamic providers (GitHub, GitLab, OCIArtifactTag, ExternalService) need network access and contribute zero inputs.
  • Diff output isn't a unified-diff patch. flate diff emits dyff path-keyed syntax (@@ <path> @@); GitHub's diff lexer renders it natively but it won't apply with patch / git apply — use the rendered output of flate build if you need a literal patch.

Development

go build ./cmd/flate
go test ./...
go test -race ./...
golangci-lint run ./...

Tool versions pin via mise. Testdata lives in testdata/; test/e2e runs the cobra command tree in-process — no fork/exec, no freshly built binary.

License

AGPL-3.0. flate borrows behavior and test fixtures from flux-local (Apache-2.0).

Directories

Path Synopsis
cmd
flate command
flate — local validator for Flux GitOps repositories.
flate — local validator for Flux GitOps repositories.
internal
cli
Package cli wires flate's command-line interface using cobra.
Package cli wires flate's command-line interface using cobra.
format
Package format provides the table, YAML, JSON, and "name" output modes used across flate's CLI surface.
Package format provides the table, YAML, JSON, and "name" output modes used across flate's CLI surface.
keylock
Package keylock provides per-key mutual-exclusion locks that honor context cancellation.
Package keylock provides per-key mutual-exclusion locks that honor context cancellation.
testrunner
Package testrunner implements `flate test`.
Package testrunner implements `flate test`.
testutil
Package testutil is shared fixture scaffolding for tests across the repo — kept minimal so each test file stays self-describing.
Package testutil is shared fixture scaffolding for tests across the repo — kept minimal so each test file stays self-describing.
pkg
baseline
Package baseline resolves and materializes a git revision into a tempdir so `flate diff` can run without an explicit --path-orig.
Package baseline resolves and materializes a git revision into a tempdir so `flate diff` can run without an explicit --path-orig.
change
Package change computes file-level differences between two filesystem trees and maps them onto the Flux resources they affect.
Package change computes file-level differences between two filesystem trees and maps them onto the Flux resources they affect.
controllers/base
Package base provides the shared lifecycle harness every flate controller wraps around its per-resource reconcile body.
Package base provides the shared lifecycle harness every flate controller wraps around its per-resource reconcile body.
controllers/helmrelease
Package helmrelease implements the HelmReleaseController.
Package helmrelease implements the HelmReleaseController.
controllers/kustomization
Package kustomization reconciles Flux Kustomizations: wait on dependsOn / sourceRef / structural parent, resolve postBuild substitutions, run the kustomize SDK, parse the result back into the Store, and publish a KustomizationArtifact.
Package kustomization reconciles Flux Kustomizations: wait on dependsOn / sourceRef / structural parent, resolve postBuild substitutions, run the kustomize SDK, parse the result back into the Store, and publish a KustomizationArtifact.
controllers/source
Package source reconciles Flux source CRs (GitRepository, OCIRepository, Bucket, ExternalArtifact, HelmRepository) into on-disk artifacts via per-kind Fetcher implementations from pkg/source, then publishes the result to the Store.
Package source reconciles Flux source CRs (GitRepository, OCIRepository, Bucket, ExternalArtifact, HelmRepository) into on-disk artifacts via per-kind Fetcher implementations from pkg/source, then publishes the result to the Store.
depwait
Package depwait blocks a controller until a set of NamedResource dependencies become Ready (or, for kinds without a status pipeline, merely exist) — with per-dep timeouts and fail-fast for missing refs.
Package depwait blocks a controller until a set of NamedResource dependencies become Ready (or, for kinds without a status pipeline, merely exist) — with per-dep timeouts and fail-fast for missing refs.
diff
Package diff produces structured diffs between two sets of rendered Kubernetes manifests.
Package diff produces structured diffs between two sets of rendered Kubernetes manifests.
discovery
Package discovery owns flate's filesystem-to-store hydration phase: walking the user's working tree, expanding spec.path references, aliasing in-cluster-bootstrapped sources, rendering ResourceSets, and computing the structural-parent index.
Package discovery owns flate's filesystem-to-store hydration phase: walking the user's working tree, expanding spec.path references, aliasing in-cluster-bootstrapped sources, rendering ResourceSets, and computing the structural-parent index.
helm
Package helm wraps helm.sh/helm/v4 to render HelmReleases without shelling out to the `helm` binary.
Package helm wraps helm.sh/helm/v4 to render HelmReleases without shelling out to the `helm` binary.
image
Package image discovers container images inside rendered Kubernetes manifests by parsing string values, not by hard-coding a table of kinds and field names.
Package image discovers container images inside rendered Kubernetes manifests by parsing string values, not by hard-coding a table of kinds and field names.
kustomize
Package kustomize wraps sigs.k8s.io/kustomize/api so the rest of flate never invokes the `kustomize` CLI.
Package kustomize wraps sigs.k8s.io/kustomize/api so the rest of flate never invokes the `kustomize` CLI.
loader
Package loader implements Loader — the entry point that scans a local filesystem path, decodes every YAML document found, and adds the recognized Flux objects to a Store.
Package loader implements Loader — the entry point that scans a local filesystem path, decodes every YAML document found, and adds the recognized Flux objects to a Store.
manifest
Package manifest defines the data model for Flux GitOps resources as observed locally in a Git repository.
Package manifest defines the data model for Flux GitOps resources as observed locally in a Git repository.
orchestrator
Package orchestrator wires the controllers together and runs the reconcile loop.
Package orchestrator wires the controllers together and runs the reconcile loop.
resourceset
Package resourceset renders flux-operator ResourceSet CRs offline.
Package resourceset renders flux-operator ResourceSet CRs offline.
selector
Package selector implements metadata filtering used by every flate command.
Package selector implements metadata filtering used by every flate command.
source
Package source implements SourceController — reconciles GitRepository and OCIRepository resources by fetching the underlying artifact into a content-addressed on-disk cache and publishing a *store.GitArtifact or *store.OCIArtifact for downstream controllers.
Package source implements SourceController — reconciles GitRepository and OCIRepository resources by fetching the underlying artifact into a content-addressed on-disk cache and publishing a *store.GitArtifact or *store.OCIArtifact for downstream controllers.
source/atomic
Package atomic carries the small "stage-then-rename" file write helper every flate cache writer used to reinvent (helm.writeAtomic, oci.writeCachedDigest, oci.writeVerifyMarker, blob.Refs.Put).
Package atomic carries the small "stage-then-rename" file write helper every flate cache writer used to reinvent (helm.writeAtomic, oci.writeCachedDigest, oci.writeVerifyMarker, blob.Refs.Put).
source/blob
Package blob is a small content-addressed storage layer for flate's fetched artifacts.
Package blob is a small content-addressed storage layer for flate's fetched artifacts.
source/bucket
Package bucket implements the source.Fetcher for KindBucket (S3-compatible object storage via minio-go).
Package bucket implements the source.Fetcher for KindBucket (S3-compatible object storage via minio-go).
source/cacheroot
Package cacheroot owns the layout of flate's on-disk cache.
Package cacheroot owns the layout of flate's on-disk cache.
source/external
Package external implements the source.Fetcher for KindExternalArtifact (third-party-published artifacts under Flux's source-controller schema).
Package external implements the source.Fetcher for KindExternalArtifact (third-party-published artifacts under Flux's source-controller schema).
source/git
Package git implements the source.Fetcher for KindGitRepository.
Package git implements the source.Fetcher for KindGitRepository.
source/gittree
Package gittree materializes a git commit's tree to disk, writing every blob as a regular file (or real symlink, where applicable), in parallel.
Package gittree materializes a git commit's tree to disk, writing every blob as a regular file (or real symlink, where applicable), in parallel.
source/oci
Package oci implements the source.Fetcher for KindOCIRepository via oras-go.
Package oci implements the source.Fetcher for KindOCIRepository via oras-go.
store
Package store is the central, in-memory state container for the controller pipeline.
Package store is the central, in-memory state container for the controller pipeline.
task
Package task provides a lightweight goroutine lifecycle manager modeled on flux-local's TaskService.
Package task provides a lightweight goroutine lifecycle manager modeled on flux-local's TaskService.
values
Package values implements HelmRelease values resolution and Kustomization postBuild substitution.
Package values implements HelmRelease values resolution and Kustomization postBuild substitution.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL