merger

package
v0.7.9 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 3, 2026 License: Apache-2.0 Imports: 27 Imported by: 0

README

pkg/merger

The merger package resolves and merges Katalog/Komposer YAML files into a single, unified set of CRD definitions. It is the ingestion layer that sits between the raw YAML on disk (or a remote source) and the runtime Katalog struct that drives the operator.

What lives here

File Role
merger.go Merger struct, New, Merge, Enabled, All, Get, ToSpec, ToSecurity, ToNotification, ToProviders, ToUI
file.go loadKatalogFile dispatcher; loadKatalog (Katalog kind); loadKomposer (Komposer kind — imports + inline merge + accumulation)
parse.go parseKatalogDoc — YAML decode + kind validation
file_auth.go loadSourceFileWithAuth — HTTP/S fetch with bearer/basic auth; local file read
helm.go loadHelmSource — Helm chart render → Katalog template extraction
registry.go / registry_v2.go loadRegistrySource — fetch CRDs from the Orkestra registry
helper.go mergeKatalogSecurity, mergeKatalogNotification, checkDuplicate, resolveEnvVar, writeTempFile, gitClone
helm_cache.go Disk cache for git and remote Helm sources (~/.orkestra/helm/) + WarmHelmSource

Merge rules

Katalog  (kind: Katalog)
  Declares CRDs directly in spec.crds.
  Must NOT declare imports — imports are a Komposer concern.

Komposer (kind: Komposer)
  Resolves imports (registry → files → helm) in that order.
  Inline spec.crds are merged last and win on name conflict.
  Top-level fields (security, notification, providers) are
  accumulated across all imports; the Komposer's own block
  wins on conflict.
Deduplication scopes
Scope Mechanism
Within one file's source tree localSeen map[string]string
Across entry-point files seen map[string]string in Merge()
Inline over source valid — triggers mergeCRDEntry
Inline over inline always an error
Top-level field accumulation

When a Komposer references multiple source Katalogs, the merger accumulates their top-level fields so that ork generate rbac and ork generate configmap against a Komposer produce the same output as running against the source Katalogs directly:

Field Accumulation strategy
security mergeKatalogSecurity — non-nil pointer fields in override win
notification mergeKatalogNotification — teams merged by name; override wins per key
providers append all, Komposer's own list replaces if non-empty

Usage

m := merger.New("katalog.yaml", "overrides.yaml")
if err := m.Merge(); err != nil { ... }

kat.Spec         = m.ToSpec()
kat.Security     = m.ToSecurity()
kat.Notification = m.ToNotification()
kat.Providers    = m.ToProviders()

All To* methods panic if called before Merge().

Developer documentation

Full step-by-step documentation is in docs/.

I want to… Go to
Understand the full merge pipeline 01 — Architecture
Understand Katalog vs Komposer rules 02 — Kinds
Add or understand a source type 03 — Imports
Debug duplicate CRD name errors 04 — Deduplication
Understand security/notification/providers inheritance 05 — Top-Level Accumulation
Understand Helm and file source caching 07 — Source Caching

Documentation

Overview

pkg/merger/file.go

pkg/merger/file_auth.go

pkg/merger/helm.go

pkg/merger/helm_cache.go

Disk cache for Helm sources resolved by the merger.

Two namespaces: 

~/.orkestra/helm/git/<sha256>/   — git-sourced charts (repo + ref + chart path)
~/.orkestra/helm/repo/<sha256>/  — remote Helm repository charts (repo + chart + version)

Cache key is the SHA256 of the tuple that uniquely identifies the artifact. Sentinel file: Chart.yaml — if it exists, the cache entry is complete. Callers pass refresh=true to bypass the cache and overwrite the stored copy.

pkg/merger/helper.go

pkg/merger/merger.go

Package merger resolves and merges Katalog and Komposer YAML files into a single unified CRD map that the Katalog runtime consumes. It is the ingestion layer between raw YAML on disk (or remote sources) and the operator's live configuration.

Entry point: New(paths...).Merge() — call once; query with Enabled, All, ToSpec, ToSecurity, ToNotification, and ToProviders.

See README.md for merge rules, source-loading order, and top-level field accumulation semantics.

pkg/merger/parse.go

pkg/merger/pull.go

Exported pull helpers — shared with pkg/motif and any other package that needs to fetch a registry artifact without loading a full Katalog.

Both PullToDir and PullMotifToDir follow a cache-first strategy for OCI artifacts: ~/.orkestra/registry/<host>/<repo>/<version>/ is checked for a sentinel file (katalog.yaml or motif.yaml) before any network call is made. This avoids redundant pulls and removes the need for Docker credential forwarding inside the process — callers should use `ork pull` to populate the cache, then rely on these helpers to read from it.

pkg/merger/registry.go

Index

Constants

This section is empty.

Variables

View Source 
var ParseKatalogDoc = parseKatalogDoc

Export

Functions

func ExportedGitHubRawURL 

func ExportedGitHubRawURL(repoURL, ref, filePath string) string

func ExportedGitLabRawURL 

func ExportedGitLabRawURL(repoURL, ref, filePath string) string

func ExportedIsGitHubURL 

func ExportedIsGitHubURL(u string) bool

func ExportedIsGitLabURL 

func ExportedIsGitLabURL(u string) bool

func ExportedValidatePatternStructure 

func ExportedValidatePatternStructure(dir, url, version string) error

func PullMotifToDir added in v0.3.9 

func PullMotifToDir(url, version string, oci bool, auth *utils.FileAuth) (dir string, cleanup func(), err error)

PullMotifToDir returns the directory for a motif artifact (OCI or Git). For OCI artifacts it checks the local cache (~/.orkestra/registry/) first and returns the cached directory without a network call when available.

For OCI: the entire OCI artifact is pulled — motif.yaml must be at the root. For Git (GitHub/GitLab): only motif.yaml is fetched via raw URL. For generic Git: the repo is cloned and motif.yaml is copied.

Returns (dir, cleanup, err). Always call cleanup() when done.

func PullToDir added in v0.3.9 

func PullToDir(url, version string, oci bool, auth *utils.FileAuth) (dir string, cleanup func(), err error)

PullToDir returns the directory for a registry pattern artifact (OCI or Git). For OCI artifacts it checks the local cache (~/.orkestra/registry/) first and returns the cached directory without a network call when available.

Returns (dir, cleanup, err). Always call cleanup() when done — for cached hits cleanup is a no-op; for fresh pulls it removes the temp dir.

func WarmHelmSource added in v0.7.9 

func WarmHelmSource(src orktypes.HelmSource, refresh bool) error

WarmHelmSource pre-warms the local cache for a HelmSource. Equivalent to what the merger does on first use, but called explicitly by ork pull so subsequent commands are served from cache. When refresh is true the existing cache entry is discarded and re-fetched.

Types

type Merger 

type Merger struct {

	// Refresh bypasses all local caches — git charts, remote Helm repos, and
	// remote file fetches are re-downloaded and the cached copies are overwritten.
	Refresh bool
	// contains filtered or unexported fields
}

Merger loads one or more Katalog files, resolves their imports (files, URLs, Helm charts), merges all CRD entries, and exposes the result through Enabled(), All(), and Get().

Entry point: one or more file paths from the CLI or konstructRuntime. Everything else — source resolution, Helm rendering, deduplication — is internal to the merger.

Merge rules:

  • Sources within a Katalog are loaded before spec.crds
  • Inline spec.crds win on name conflict (local overrides remote)
  • Duplicate names across independent Katalog files are errors
  • disabled CRDs are preserved and filtered by Enabled()

func New 

func New(paths ...string) *Merger

New creates a Merger with the given entry point file paths or URLs. Accepts one or more paths — the same as passing --file multiple times or comma separated.

func (*Merger) APIMetadata 

func (m *Merger) APIMetadata() apiMetadata

APIMetadata returns the merged result as a KatalogMeta with apiversion and kind.

func (*Merger) Add 

func (m *Merger) Add(paths ...string) *Merger

Add appends additional entry point paths after construction. Returns the Merger for chaining.

func (*Merger) All 

func (m *Merger) All() map[string]orktypes.CRDEntry

All returns all CRD entries including disabled ones.

func (*Merger) Count 

func (m *Merger) Count() int

Count returns total CRD count across all imports.

func (*Merger) Enabled 

func (m *Merger) Enabled() map[string]orktypes.CRDEntry

Enabled returns only CRD entries where enabled: true.

func (*Merger) EnabledCount 

func (m *Merger) EnabledCount() int

EnabledCount returns the number of enabled CRDs.

func (*Merger) FirstEntryDir added in v0.4.8 

func (m *Merger) FirstEntryDir() string

FirstEntryDir returns the directory of the first entry point path. Used by the Katalog parser to resolve relative crdFile paths.

func (*Merger) Get 

func (m *Merger) Get(name string) (orktypes.CRDEntry, bool)

Get returns a CRD entry by name. Returns (entry, true) if found.

func (*Merger) GetRegistryURL 

func (m *Merger) GetRegistryURL() string

func (*Merger) Merge 

func (m *Merger) Merge() error

Merge loads all entry points and their declared imports, resolves Helm charts, and produces a single deduplicated CRD map. Safe to call multiple times — re-merges on each call.

func (*Merger) SetRegistryURL 

func (m *Merger) SetRegistryURL(url string)

func (*Merger) ToGateway added in v0.4.9 

func (m *Merger) ToGateway() *orktypes.GatewayConfig

ToGateway returns the gateway deployment config of the merged result. Used by KomposeRuntimeKatalog to populate Katalog.Gateway.

func (*Merger) ToNotification added in v0.2.5 

func (m *Merger) ToNotification() *orktypes.KatalogNotification

ToNotification returns the merged notification configuration of the merged result. When a Komposer references multiple imported Katalogs, teams from all imports are merged — source teams are inherited and the Komposer's own teams win on conflict. Used by KomposeRuntimeKatalog to populate Katalog.Notification.

func (*Merger) ToProfiles added in v0.7.8 

func (m *Merger) ToProfiles() orktypes.ProfileRegistry

ToProfiles returns the merged user-defined profile registry of the merged result. Used by KomposeRuntimeKatalog to populate Katalog.Profiles.

func (*Merger) ToProjectInfo added in v0.3.9 

func (m *Merger) ToProjectInfo() interface{}

ToProjectInfo returns merged project information of the merged result This is used by KomposeRuntimeKatalog to populate Katalog.ProjectInfo.

func (*Merger) ToProviders 

func (m *Merger) ToProviders() []orktypes.KatalogProviderRequirement

ToProviders returns the top-level provider requirements of the merged result. Used by KomposeRuntimeKatalog to populate Katalog.Providers.

func (*Merger) ToSecurity 

func (m *Merger) ToSecurity() orktypes.KatalogSecurity

ToSecurity returns the security config of the merged result as a KatalogSecurity Used by NewKatalog consume the merged result.

func (*Merger) ToSpec 

func (m *Merger) ToSpec() orktypes.KatalogSpec

ToSpec returns the merged result as a KatalogSpec. Used by NewKatalog and generate.Registry to consume the merged result.

func (*Merger) ToUI 

func (m *Merger) ToUI() *orktypes.KatalogForUI

ToUI returns a UI-friendly representation of the merged Katalog. This method extracts only the fields needed for display in the Control Center:

  • API version and kind (always "Katalog" at runtime)
  • Metadata (name, description, version, author, license)
  • All merged CRD definitions

Internal fields (Scheme, GroupVersionKind, etc.) are excluded because they have `yaml:"-" json:"-"` tags and won't be serialized to JSON.

This method is used by the /katalog/raw endpoint to provide a clean, readable view of the Katalog that created the current operator.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.