README
¶
pkg/merger
The merger package resolves and merges Katalog/Komposer YAML files into a single, unified set of CRD definitions. It is the ingestion layer that sits between the raw YAML on disk (or a remote source) and the runtime Katalog struct that drives the operator.
What lives here
| File | Role |
|---|---|
merger.go |
Merger struct, New, Merge, Enabled, All, Get, ToSpec, ToSecurity, ToNotification, ToProviders, ToUI |
file.go |
loadKatalogFile dispatcher; loadKatalog (Katalog kind); loadKomposer (Komposer kind — imports + inline merge + accumulation) |
parse.go |
parseKatalogDoc — YAML decode + kind validation |
file_auth.go |
loadSourceFileWithAuth — HTTP/S fetch with bearer/basic auth; local file read |
helm.go |
loadHelmSource — Helm chart render → Katalog template extraction |
registry.go / registry_v2.go |
loadRegistrySource — fetch CRDs from the Orkestra registry |
helper.go |
mergeKatalogSecurity, mergeKatalogNotification, checkDuplicate, resolveEnvVar, writeTempFile, gitClone |
Merge rules
Katalog (kind: Katalog)
Declares CRDs directly in spec.crds.
Must NOT declare imports — imports are a Komposer concern.
Komposer (kind: Komposer)
Resolves imports (registry → files → helm) in that order.
Inline spec.crds are merged last and win on name conflict.
Top-level fields (security, notification, providers) are
accumulated across all imports; the Komposer's own block
wins on conflict.
Deduplication scopes
| Scope | Mechanism |
|---|---|
| Within one file's source tree | localSeen map[string]string |
| Across entry-point files | seen map[string]string in Merge() |
| Inline over source | valid — triggers mergeCRDEntry |
| Inline over inline | always an error |
Top-level field accumulation
When a Komposer references multiple source Katalogs, the merger accumulates their top-level fields so that ork generate rbac and ork generate configmap against a Komposer produce the same output as running against the source Katalogs directly:
| Field | Accumulation strategy |
|---|---|
security |
mergeKatalogSecurity — non-nil pointer fields in override win |
notification |
mergeKatalogNotification — teams merged by name; override wins per key |
providers |
append all, Komposer's own list replaces if non-empty |
Usage
m := merger.New("katalog.yaml", "overrides.yaml")
if err := m.Merge(); err != nil { ... }
kat.Spec = m.ToSpec()
kat.Security = m.ToSecurity()
kat.Notification = m.ToNotification()
kat.Providers = m.ToProviders()
All To* methods panic if called before Merge().
Developer documentation
Full step-by-step documentation is in docs/.
| I want to… | Go to |
|---|---|
| Understand the full merge pipeline | 01 — Architecture |
| Understand Katalog vs Komposer rules | 02 — Kinds |
| Add or understand a source type | 03 — Imports |
| Debug duplicate CRD name errors | 04 — Deduplication |
| Understand security/notification/providers inheritance | 05 — Top-Level Accumulation |
Documentation
¶
Overview ¶
pkg/merger/file.go
pkg/merger/file_auth.go
pkg/merger/helm.go
pkg/merger/helper.go
pkg/merger/merger.go
Package merger resolves and merges Katalog and Komposer YAML files into a single unified CRD map that the Katalog runtime consumes. It is the ingestion layer between raw YAML on disk (or remote sources) and the operator's live configuration.
Entry point: New(paths...).Merge() — call once; query with Enabled, All, ToSpec, ToSecurity, ToNotification, and ToProviders.
See README.md for merge rules, source-loading order, and top-level field accumulation semantics.
pkg/merger/parse.go
pkg/merger/pull.go
Exported pull helpers — shared with pkg/motif and any other package that needs to fetch a registry artifact without loading a full Katalog.
Both PullToDir and PullMotifToDir follow a cache-first strategy for OCI artifacts: ~/.orkestra/registry/<host>/<repo>/<version>/ is checked for a sentinel file (katalog.yaml or motif.yaml) before any network call is made. This avoids redundant pulls and removes the need for Docker credential forwarding inside the process — callers should use `ork pull` to populate the cache, then rely on these helpers to read from it.
pkg/merger/registry.go
Index ¶
- Variables
- func ExportedGitHubRawURL(repoURL, ref, filePath string) string
- func ExportedGitLabRawURL(repoURL, ref, filePath string) string
- func ExportedIsGitHubURL(u string) bool
- func ExportedIsGitLabURL(u string) bool
- func ExportedValidatePatternStructure(dir, url, version string) error
- func PullMotifToDir(url, version string, oci bool, auth *utils.FileAuth) (dir string, cleanup func(), err error)
- func PullToDir(url, version string, oci bool, auth *utils.FileAuth) (dir string, cleanup func(), err error)
- type Merger
- func (m *Merger) APIMetadata() apiMetadata
- func (m *Merger) Add(paths ...string) *Merger
- func (m *Merger) All() map[string]orktypes.CRDEntry
- func (m *Merger) Count() int
- func (m *Merger) Enabled() map[string]orktypes.CRDEntry
- func (m *Merger) EnabledCount() int
- func (m *Merger) FirstEntryDir() string
- func (m *Merger) Get(name string) (orktypes.CRDEntry, bool)
- func (m *Merger) GetRegistryURL() string
- func (m *Merger) Merge() error
- func (m *Merger) SetRegistryURL(url string)
- func (m *Merger) ToGateway() *orktypes.GatewayConfig
- func (m *Merger) ToNotification() *orktypes.KatalogNotification
- func (m *Merger) ToProjectInfo() interface{}
- func (m *Merger) ToProviders() []orktypes.KatalogProviderRequirement
- func (m *Merger) ToSecurity() orktypes.KatalogSecurity
- func (m *Merger) ToSpec() orktypes.KatalogSpec
- func (m *Merger) ToUI() *orktypes.KatalogForUI
Constants ¶
This section is empty.
Variables ¶
var ParseKatalogDoc = parseKatalogDoc
Export
Functions ¶
func ExportedGitHubRawURL ¶
func ExportedGitLabRawURL ¶
func ExportedIsGitHubURL ¶
func ExportedIsGitLabURL ¶
func PullMotifToDir ¶ added in v0.3.9
func PullMotifToDir(url, version string, oci bool, auth *utils.FileAuth) (dir string, cleanup func(), err error)
PullMotifToDir returns the directory for a motif artifact (OCI or Git). For OCI artifacts it checks the local cache (~/.orkestra/registry/) first and returns the cached directory without a network call when available.
For OCI: the entire OCI artifact is pulled — motif.yaml must be at the root. For Git (GitHub/GitLab): only motif.yaml is fetched via raw URL. For generic Git: the repo is cloned and motif.yaml is copied.
Returns (dir, cleanup, err). Always call cleanup() when done.
func PullToDir ¶ added in v0.3.9
func PullToDir(url, version string, oci bool, auth *utils.FileAuth) (dir string, cleanup func(), err error)
PullToDir returns the directory for a registry pattern artifact (OCI or Git). For OCI artifacts it checks the local cache (~/.orkestra/registry/) first and returns the cached directory without a network call when available.
Returns (dir, cleanup, err). Always call cleanup() when done — for cached hits cleanup is a no-op; for fresh pulls it removes the temp dir.
Types ¶
type Merger ¶
type Merger struct {
// contains filtered or unexported fields
}
Merger loads one or more Katalog files, resolves their imports (files, URLs, Helm charts), merges all CRD entries, and exposes the result through Enabled(), All(), and Get().
Entry point: one or more file paths from the CLI or konstructRuntime. Everything else — source resolution, Helm rendering, deduplication — is internal to the merger.
Merge rules:
- Sources within a Katalog are loaded before spec.crds
- Inline spec.crds win on name conflict (local overrides remote)
- Duplicate names across independent Katalog files are errors
- disabled CRDs are preserved and filtered by Enabled()
func New ¶
New creates a Merger with the given entry point file paths or URLs. Accepts one or more paths — the same as passing --file multiple times or comma separated.
func (*Merger) APIMetadata ¶
func (m *Merger) APIMetadata() apiMetadata
APIMetadata returns the merged result as a KatalogMeta with apiversion and kind.
func (*Merger) Add ¶
Add appends additional entry point paths after construction. Returns the Merger for chaining.
func (*Merger) EnabledCount ¶
EnabledCount returns the number of enabled CRDs.
func (*Merger) FirstEntryDir ¶ added in v0.4.8
FirstEntryDir returns the directory of the first entry point path. Used by the Katalog parser to resolve relative crdFile paths.
func (*Merger) GetRegistryURL ¶
func (*Merger) Merge ¶
Merge loads all entry points and their declared imports, resolves Helm charts, and produces a single deduplicated CRD map. Safe to call multiple times — re-merges on each call.
func (*Merger) SetRegistryURL ¶
func (*Merger) ToGateway ¶ added in v0.4.9
func (m *Merger) ToGateway() *orktypes.GatewayConfig
ToGateway returns the gateway deployment config of the merged result. Used by KomposeRuntimeKatalog to populate Katalog.Gateway.
func (*Merger) ToNotification ¶ added in v0.2.5
func (m *Merger) ToNotification() *orktypes.KatalogNotification
ToNotification returns the merged notification configuration of the merged result. When a Komposer references multiple imported Katalogs, teams from all imports are merged — source teams are inherited and the Komposer's own teams win on conflict. Used by KomposeRuntimeKatalog to populate Katalog.Notification.
func (*Merger) ToProjectInfo ¶ added in v0.3.9
func (m *Merger) ToProjectInfo() interface{}
ToProjectInfo returns merged project information of the merged result This is used by KomposeRuntimeKatalog to populate Katalog.ProjectInfo.
func (*Merger) ToProviders ¶
func (m *Merger) ToProviders() []orktypes.KatalogProviderRequirement
ToProviders returns the top-level provider requirements of the merged result. Used by KomposeRuntimeKatalog to populate Katalog.Providers.
func (*Merger) ToSecurity ¶
func (m *Merger) ToSecurity() orktypes.KatalogSecurity
ToSecurity returns the security config of the merged result as a KatalogSecurity Used by NewKatalog consume the merged result.
func (*Merger) ToSpec ¶
func (m *Merger) ToSpec() orktypes.KatalogSpec
ToSpec returns the merged result as a KatalogSpec. Used by NewKatalog and generate.Registry to consume the merged result.
func (*Merger) ToUI ¶
func (m *Merger) ToUI() *orktypes.KatalogForUI
ToUI returns a UI-friendly representation of the merged Katalog. This method extracts only the fields needed for display in the Control Center:
- API version and kind (always "Katalog" at runtime)
- Metadata (name, description, version, author, license)
- All merged CRD definitions
Internal fields (Scheme, GroupVersionKind, etc.) are excluded because they have `yaml:"-" json:"-"` tags and won't be serialized to JSON.
This method is used by the /katalog/raw endpoint to provide a clean, readable view of the Katalog that created the current operator.
Source Files