merge

package
v1.203.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 2, 2026 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Index

Examples

Constants

View Source
const (
	ListMergeStrategyReplace = "replace"
	ListMergeStrategyAppend  = "append"
	ListMergeStrategyMerge   = "merge"
)

Variables

This section is empty.

Functions

func ApplyDeferredMerges added in v1.201.0

func ApplyDeferredMerges(dctx *DeferredMergeContext, result map[string]interface{}, atmosConfig *schema.AtmosConfiguration, processor YAMLFunctionProcessor) error

ApplyDeferredMerges processes all deferred YAML functions and applies them to the result. This function is called after the initial merge to handle YAML functions that were deferred to avoid type conflicts during merging.

If processor is nil, YAML function strings are kept as-is (for testing or when processing is not needed). If processor is provided, YAML functions are processed to their actual values before merging.

func DeepCopyMap added in v1.195.0

func DeepCopyMap(m map[string]any) (map[string]any, error)

DeepCopyMap performs a deep copy of a map optimized for map[string]any structures. This custom implementation avoids reflection overhead for common cases (maps, slices, primitives) and uses reflection-based normalization for rare complex types (typed slices/maps). Preserves numeric types (unlike JSON which converts all numbers to float64) and is faster than generic reflection-based copying. The data is already in Go map format with custom tags already processed, so we only need structural copying to work around mergo's pointer mutation bug. Uses properly-sized allocations to reduce GC pressure during high-volume operations (118k+ calls per run).

func GetValueAtPath added in v1.201.0

func GetValueAtPath(data map[string]interface{}, path []string) (interface{}, bool)

GetValueAtPath returns the value at a specific path in a nested map structure.

func Merge

func Merge(
	atmosConfig *schema.AtmosConfiguration,
	inputs []map[string]any,
) (map[string]any, error)

Merge takes a list of maps as input, deep-merges the items in the order they are defined in the list, and returns a single map with the merged contents.

func MergeDeferredValues added in v1.201.0

func MergeDeferredValues(values []*DeferredValue, atmosConfig *schema.AtmosConfiguration) (interface{}, error)

MergeDeferredValues merges all values for a single field path.

func MergeWithContext added in v1.191.0

func MergeWithContext(
	atmosConfig *schema.AtmosConfiguration,
	inputs []map[string]any,
	context *MergeContext,
) (map[string]any, error)

MergeWithContext performs a merge operation with file context tracking for better error messages.

func MergeWithOptions added in v1.3.1

func MergeWithOptions(
	atmosConfig *schema.AtmosConfiguration,
	inputs []map[string]any,
	appendSlice bool,
	sliceDeepCopy bool,
) (map[string]any, error)

MergeWithOptions takes a list of maps and options as input, deep-merges the items in the order they are defined in the list, and returns a single map with the merged contents.

func MergeWithOptionsAndContext added in v1.191.0

func MergeWithOptionsAndContext(
	atmosConfig *schema.AtmosConfiguration,
	inputs []map[string]any,
	appendSlice bool,
	sliceDeepCopy bool,
	context *MergeContext,
) (map[string]any, error)

MergeWithOptionsAndContext performs merge with options and context tracking.

func MergeWithProvenance added in v1.194.0

func MergeWithProvenance(
	atmosConfig *schema.AtmosConfiguration,
	inputs []map[string]any,
	ctx *MergeContext,
	positions u.PositionMap,
) (map[string]any, error)

MergeWithProvenance merges multiple maps and records provenance information. It tracks where each value originated (file, line, column) and at what import depth.

Parameters.

  • atmosConfig: Atmos configuration with TrackProvenance flag.
  • inputs: Maps to merge (in order: first is base, last wins).
  • ctx: Merge context with provenance storage.
  • positions: Map of JSONPath -> Position from YAML parsing.

Returns the merged map with provenance recorded in ctx.

func SetValueAtPath added in v1.201.0

func SetValueAtPath(data map[string]interface{}, path []string, value interface{}) error

SetValueAtPath sets a value at a specific path in a nested map structure. Creates intermediate maps as needed. Precondition: data must be a non-nil map (panics if nil).

func WalkAndDeferYAMLFunctions added in v1.201.0

func WalkAndDeferYAMLFunctions(dctx *DeferredMergeContext, data map[string]interface{}, basePath []string) map[string]interface{}

WalkAndDeferYAMLFunctions walks through a map and defers any YAML functions. Returns a new map with YAML functions replaced by nil placeholders.

Types

type DeferredMergeContext added in v1.201.0

type DeferredMergeContext struct {
	// contains filtered or unexported fields
}

DeferredMergeContext tracks all deferred values during the merge process.

func MergeWithDeferred added in v1.201.0

func MergeWithDeferred(
	atmosConfig *schema.AtmosConfiguration,
	inputs []map[string]any,
) (map[string]any, *DeferredMergeContext, error)

MergeWithDeferred performs a merge operation with YAML function deferral. It creates a deferred merge context, walks each input to defer YAML functions, performs the merge, and returns both the result and the deferred context. The caller is responsible for calling ApplyDeferredMerges to process deferred functions.

Integration Example:

// In stack processor (internal/exec/stack_processor_merge.go):
// Replace: finalVars, err := m.Merge(atmosConfig, inputs)
// With:
finalVars, dctx, err := m.MergeWithDeferred(atmosConfig, inputs)
if err != nil {
    return nil, err
}

// After all sections are merged, apply deferred merges with YAML function processor:
processor := &YAMLProcessor{...} // Implement YAMLFunctionProcessor interface
if err := m.ApplyDeferredMerges(dctx, finalVars, atmosConfig, processor); err != nil {
    return nil, err
}

Note: For full integration, YAML function processing during loading must be modified to keep YAML functions as strings rather than processing them immediately. See docs/prd/yaml-function-merge-handling.md for complete implementation plan.

Example

ExampleMergeWithDeferred demonstrates the complete deferred merge workflow. This example shows how YAML functions are detected, deferred during merge, and then applied after merging to avoid type conflicts.

// Create Atmos configuration with replace strategy.
cfg := &schema.AtmosConfiguration{
	Settings: schema.AtmosSettings{
		ListMergeStrategy: "replace",
	},
}

// Simulate configuration from multiple imports.
// In a real scenario, these would come from different YAML files
// in an import chain (e.g., catalog/base.yaml, stacks/dev.yaml, stacks/prod.yaml).
inputs := []map[string]any{
	{
		// Base catalog with YAML function.
		"template_config": "!template '{{ .settings.base }}'",
		"regular_value":   "from-base",
		"nested": map[string]interface{}{
			"yaml_func": "!terraform.output vpc.id",
			"static":    "base-static",
		},
	},
	{
		// Dev environment override.
		"template_config": "!template '{{ .settings.dev }}'",
		"regular_value":   "from-dev",
		"nested": map[string]interface{}{
			"static": "dev-static",
		},
	},
	{
		// Prod environment with concrete value.
		// Without deferred merge, this would cause a type conflict
		// if the YAML function returns a different type.
		"regular_value": "from-prod",
	},
}

// Perform merge with deferred YAML functions.
result, dctx, err := MergeWithDeferred(cfg, inputs)
if err != nil {
	fmt.Printf("Merge error: %v\n", err)
	return
}

// Show what was deferred (sorted for consistent output).
fmt.Println("Deferred YAML functions:")
deferredValues := dctx.GetDeferredValues()
paths := make([]string, 0, len(deferredValues))
for path := range deferredValues {
	paths = append(paths, path)
}
sort.Strings(paths)
for _, path := range paths {
	values := deferredValues[path]
	fmt.Printf("  %s: %d values\n", path, len(values))
	for i, v := range values {
		fmt.Printf("    [%d] precedence=%d value=%v\n", i, v.Precedence, v.Value)
	}
}

// Show merged result (with nil placeholders for YAML functions).
fmt.Println("\nMerged result (before applying deferred):")
fmt.Printf("  template_config: %v\n", result["template_config"])
fmt.Printf("  regular_value: %v\n", result["regular_value"])
if nested, ok := result["nested"].(map[string]interface{}); ok {
	fmt.Printf("  nested.yaml_func: %v\n", nested["yaml_func"])
	fmt.Printf("  nested.static: %v\n", nested["static"])
}

// Apply deferred merges.
// Note: In production, this would process YAML functions by passing a processor.
// For this example, they remain as strings since no processor is provided (nil).
err = ApplyDeferredMerges(dctx, result, cfg, nil)
if err != nil {
	fmt.Printf("Apply error: %v\n", err)
	return
}

fmt.Println("\nFinal result (after applying deferred):")
fmt.Printf("  template_config: %v\n", result["template_config"])
fmt.Printf("  regular_value: %v\n", result["regular_value"])
if nested, ok := result["nested"].(map[string]interface{}); ok {
	fmt.Printf("  nested.yaml_func: %v\n", nested["yaml_func"])
	fmt.Printf("  nested.static: %v\n", nested["static"])
}
Output:

Deferred YAML functions:
  nested.yaml_func: 1 values
    [0] precedence=0 value=!terraform.output vpc.id
  template_config: 2 values
    [0] precedence=0 value=!template '{{ .settings.base }}'
    [1] precedence=1 value=!template '{{ .settings.dev }}'

Merged result (before applying deferred):
  template_config: <nil>
  regular_value: from-prod
  nested.yaml_func: <nil>
  nested.static: dev-static

Final result (after applying deferred):
  template_config: !template '{{ .settings.dev }}'
  regular_value: from-prod
  nested.yaml_func: !terraform.output vpc.id
  nested.static: dev-static
Example (ListMergeStrategies)

ExampleMergeWithDeferred_listMergeStrategies demonstrates how list merge strategies work with deferred YAML functions.

// Test with append strategy.
cfg := &schema.AtmosConfiguration{
	Settings: schema.AtmosSettings{
		ListMergeStrategy: "append",
	},
}

inputs := []map[string]any{
	{
		"items": []interface{}{"item1", "item2"},
	},
	{
		"items": []interface{}{"item3"},
	},
}

result, _, err := MergeWithDeferred(cfg, inputs)
if err != nil {
	fmt.Printf("Error: %v\n", err)
	return
}

fmt.Printf("Append strategy result: %v\n", result["items"])

// Test with merge strategy.
cfg.Settings.ListMergeStrategy = "merge"

inputs = []map[string]any{
	{
		"configs": []interface{}{
			map[string]interface{}{"name": "config1", "enabled": true},
			map[string]interface{}{"name": "config2", "enabled": false},
		},
	},
	{
		"configs": []interface{}{
			map[string]interface{}{"enabled": false, "timeout": 30},
		},
	},
}

result, _, err = MergeWithDeferred(cfg, inputs)
if err != nil {
	fmt.Printf("Error: %v\n", err)
	return
}

fmt.Println("Merge strategy result:")
if configs, ok := result["configs"].([]interface{}); ok {
	for i, config := range configs {
		fmt.Printf("  [%d]: %v\n", i, config)
	}
}
Output:

Append strategy result: [item1 item2 item3]
Merge strategy result:
  [0]: map[enabled:false name:config1 timeout:30]
  [1]: map[enabled:false name:config2]

func NewDeferredMergeContext added in v1.201.0

func NewDeferredMergeContext() *DeferredMergeContext

NewDeferredMergeContext creates a new deferred merge context for tracking deferred values.

func (*DeferredMergeContext) AddDeferred added in v1.201.0

func (dmc *DeferredMergeContext) AddDeferred(path []string, value interface{})

AddDeferred adds a deferred value to the context.

func (*DeferredMergeContext) GetDeferredValues added in v1.201.0

func (dmc *DeferredMergeContext) GetDeferredValues() map[string][]*DeferredValue

GetDeferredValues returns all deferred values. The returned map is the internal storage and can be modified by the caller.

func (*DeferredMergeContext) HasDeferredValues added in v1.201.0

func (dmc *DeferredMergeContext) HasDeferredValues() bool

HasDeferredValues returns true if there are any deferred values.

func (*DeferredMergeContext) IncrementPrecedence added in v1.201.0

func (dmc *DeferredMergeContext) IncrementPrecedence()

IncrementPrecedence increases the precedence counter (call after each import).

type DeferredValue added in v1.201.0

type DeferredValue struct {
	Path       []string    // Field path (e.g., ["components", "terraform", "vpc", "vars", "config"]).
	Value      interface{} // The YAML function string or the final processed value.
	Precedence int         // Merge precedence (higher = later in import chain = higher priority).
	IsFunction bool        // True if Value is still a YAML function string, false if processed.
}

DeferredValue represents a value that contains a YAML function and needs to be processed after the initial merge.

type MergeContext added in v1.191.0

type MergeContext struct {
	// CurrentFile is the file currently being processed.
	CurrentFile string

	// ImportChain tracks the chain of imports leading to the current file.
	// The first element is the root file, the last is the current file.
	ImportChain []string

	// ParentContext is the parent merge context for nested operations.
	ParentContext *MergeContext

	// Provenance stores optional provenance tracking for configuration values.
	// This is nil by default (provenance disabled) for zero performance overhead.
	// Call EnableProvenance() to activate provenance tracking.
	Provenance *ProvenanceStorage

	// Positions stores YAML position information for values in the current file.
	// This maps JSONPath-style paths to line/column positions in the YAML file.
	// Used by MergeWithProvenance to record accurate provenance data.
	Positions u.PositionMap
}

MergeContext tracks file paths and import chains during merge operations to provide better error messages when merge conflicts occur. It also optionally tracks provenance information for configuration values.

MergeContext implements the ProvenanceTracker interface, providing a concrete implementation for stack component provenance tracking.

func NewMergeContext added in v1.191.0

func NewMergeContext() *MergeContext

NewMergeContext creates a new merge context.

func (*MergeContext) Clone added in v1.191.0

func (mc *MergeContext) Clone() *MergeContext

Clone creates a copy of the merge context.

func (*MergeContext) EnableProvenance added in v1.194.0

func (mc *MergeContext) EnableProvenance()

EnableProvenance activates provenance tracking for this context. This must be called before any provenance recording occurs. Once enabled, all merge operations will track the source of each value.

func (*MergeContext) FormatError added in v1.191.0

func (mc *MergeContext) FormatError(err error, additionalInfo ...string) error

FormatError formats an error with merge context information.

func (*MergeContext) GetDepth added in v1.191.0

func (mc *MergeContext) GetDepth() int

GetDepth returns the depth of the import chain.

func (*MergeContext) GetImportChainString added in v1.191.0

func (mc *MergeContext) GetImportChainString() string

GetImportChainString returns a formatted string of the import chain.

func (*MergeContext) GetImportDepth added in v1.194.0

func (c *MergeContext) GetImportDepth() int

GetImportDepth returns the current import depth from the merge context. Returns 0 if context is nil or has no parent chain.

func (*MergeContext) GetProvenance added in v1.194.0

func (mc *MergeContext) GetProvenance(path string) []ProvenanceEntry

GetProvenance returns the provenance chain for a given path. Returns nil if provenance is not enabled or no provenance exists for the path.

func (*MergeContext) GetProvenancePaths added in v1.194.0

func (mc *MergeContext) GetProvenancePaths() []string

GetProvenancePaths returns all paths that have provenance information. Returns nil if provenance is not enabled.

func (*MergeContext) GetProvenanceType added in v1.194.0

func (mc *MergeContext) GetProvenanceType() ProvenanceType

GetProvenanceType returns the appropriate provenance type for the current file. Returns ProvenanceTypeInline if this is the root file (first in import chain), or ProvenanceTypeImport if this is an imported file.

func (*MergeContext) HasFile added in v1.191.0

func (mc *MergeContext) HasFile(filePath string) bool

HasFile checks if a file is already in the import chain (to detect circular imports).

func (*MergeContext) HasProvenance added in v1.194.0

func (mc *MergeContext) HasProvenance(path string) bool

HasProvenance checks if provenance exists for a given path. Returns false if provenance is not enabled.

func (*MergeContext) IsProvenanceEnabled added in v1.194.0

func (mc *MergeContext) IsProvenanceEnabled() bool

IsProvenanceEnabled returns true if provenance tracking is enabled.

func (*MergeContext) RecordProvenance added in v1.194.0

func (mc *MergeContext) RecordProvenance(path string, entry ProvenanceEntry)

RecordProvenance records provenance information for a value at the given path. This is a no-op if provenance is not enabled (Provenance is nil).

func (*MergeContext) WithFile added in v1.191.0

func (mc *MergeContext) WithFile(filePath string) *MergeContext

WithFile creates a new context for processing a specific file.

type ProvenanceEntry added in v1.194.0

type ProvenanceEntry struct {
	// File is the source file path where this value was defined.
	File string

	// Line is the line number (1-indexed) where this value appears.
	Line int

	// Column is the column number (1-indexed) where this value starts.
	Column int

	// Type indicates how this value was introduced (import, inline, override, computed, default).
	Type ProvenanceType

	// ValueHash is a hash of the value for change detection.
	// This allows detecting when a value was overridden with the same value.
	ValueHash string

	// Depth is the inheritance depth (0=parent stack, 1=direct import, 2+=nested imports).
	Depth int
}

ProvenanceEntry represents the provenance information for a single value. It tracks where a value came from in the configuration file hierarchy.

func NewProvenanceEntry added in v1.194.0

func NewProvenanceEntry(params ProvenanceEntryParams) *ProvenanceEntry

NewProvenanceEntry creates a new provenance entry.

func (*ProvenanceEntry) Clone added in v1.194.0

func (p *ProvenanceEntry) Clone() *ProvenanceEntry

Clone creates a deep copy of the provenance entry.

func (*ProvenanceEntry) Equals added in v1.194.0

func (p *ProvenanceEntry) Equals(other *ProvenanceEntry) bool

Equals checks if two provenance entries are equal. Two entries are equal if they have the same file, line, column, type, value hash, and depth.

func (*ProvenanceEntry) IsValid added in v1.194.0

func (p *ProvenanceEntry) IsValid() bool

IsValid checks if the provenance entry has valid data. A valid entry must have a file path and a positive line number.

func (*ProvenanceEntry) String added in v1.194.0

func (p *ProvenanceEntry) String() string

String returns a human-readable representation of the provenance entry.

type ProvenanceEntryParams added in v1.194.0

type ProvenanceEntryParams struct {
	File   string
	Line   int
	Column int
	Type   ProvenanceType
	Value  any
	Depth  int
}

ProvenanceEntryParams contains parameters for creating a provenance entry.

type ProvenanceStorage added in v1.194.0

type ProvenanceStorage struct {
	// contains filtered or unexported fields
}

ProvenanceStorage stores provenance information for configuration values. It maps JSONPath-style paths to a chain of provenance entries showing the inheritance/override history of each value.

func NewProvenanceStorage added in v1.194.0

func NewProvenanceStorage() *ProvenanceStorage

NewProvenanceStorage creates a new provenance storage.

func (*ProvenanceStorage) Clear added in v1.194.0

func (ps *ProvenanceStorage) Clear()

Clear removes all provenance entries.

func (*ProvenanceStorage) Clone added in v1.194.0

func (ps *ProvenanceStorage) Clone() *ProvenanceStorage

Clone creates a deep copy of the provenance storage.

func (*ProvenanceStorage) Get added in v1.194.0

func (ps *ProvenanceStorage) Get(path string) []ProvenanceEntry

Get returns the provenance chain for a given path. Returns nil if no provenance exists for the path. The returned slice is ordered from base → override.

func (*ProvenanceStorage) GetLatest added in v1.194.0

func (ps *ProvenanceStorage) GetLatest(path string) *ProvenanceEntry

GetLatest returns the most recent (final) provenance entry for a path. Returns nil if no provenance exists for the path.

func (*ProvenanceStorage) GetPaths added in v1.194.0

func (ps *ProvenanceStorage) GetPaths() []string

GetPaths returns all paths that have provenance information. The paths are returned in sorted order for consistent iteration.

func (*ProvenanceStorage) Has added in v1.194.0

func (ps *ProvenanceStorage) Has(path string) bool

Has checks if provenance exists for a given path.

func (*ProvenanceStorage) Record added in v1.194.0

func (ps *ProvenanceStorage) Record(path string, entry ProvenanceEntry)

Record adds a provenance entry for a given path. Multiple entries can be recorded for the same path to track inheritance chains.

func (*ProvenanceStorage) Remove added in v1.194.0

func (ps *ProvenanceStorage) Remove(path string)

Remove deletes provenance information for a given path.

func (*ProvenanceStorage) Size added in v1.194.0

func (ps *ProvenanceStorage) Size() int

Size returns the number of paths with provenance information.

type ProvenanceTracker added in v1.194.0

type ProvenanceTracker interface {
	// RecordProvenance records provenance information for a value at the given path.
	// The path should use JSONPath-style syntax (e.g., "vars.tags.environment").
	// Multiple entries for the same path represent the inheritance chain (base → override).
	RecordProvenance(path string, entry ProvenanceEntry)

	// GetProvenance returns the complete provenance chain for a given path.
	// The chain is ordered from base → override, with the last entry being the final value.
	// Returns nil if no provenance exists for the path.
	GetProvenance(path string) []ProvenanceEntry

	// HasProvenance checks if provenance information exists for the given path.
	// Returns false if provenance tracking is disabled or path has no provenance.
	HasProvenance(path string) bool

	// GetProvenancePaths returns all paths that have provenance information.
	// Returns nil if provenance tracking is disabled.
	// The order of paths is not guaranteed.
	GetProvenancePaths() []string

	// IsProvenanceEnabled returns true if provenance tracking is currently active.
	// When disabled, RecordProvenance calls should be no-ops for performance.
	IsProvenanceEnabled() bool

	// EnableProvenance activates provenance tracking for this tracker.
	// This should be called before any provenance recording occurs.
	// Once enabled, all merge/load operations should track provenance.
	EnableProvenance()
}

ProvenanceTracker provides a generic interface for tracking configuration provenance. This interface allows different configuration systems (stacks, atmos.yaml, vendor, workflows) to implement provenance tracking in a consistent way.

Implementations should:

  • Track the source file, line, and column for each configuration value
  • Support hierarchical paths (JSONPath-style) for nested values
  • Record inheritance/override chains when values are merged
  • Be thread-safe if used in concurrent scenarios

Example implementations:

  • MergeContext: Tracks provenance for stack component merging
  • Future: AtmosConfigContext for atmos.yaml imports
  • Future: VendorContext for vendor.yaml provenance
  • Future: WorkflowContext for workflow definitions

type ProvenanceType added in v1.194.0

type ProvenanceType string

ProvenanceType represents the type of provenance entry.

const (
	// ProvenanceTypeImport indicates the value was imported from another file.
	ProvenanceTypeImport ProvenanceType = "import"

	// ProvenanceTypeInline indicates the value was defined inline in the current file.
	ProvenanceTypeInline ProvenanceType = "inline"

	// ProvenanceTypeOverride indicates the value overrides a previous value.
	ProvenanceTypeOverride ProvenanceType = "override"

	// ProvenanceTypeComputed indicates the value was computed (e.g., from a template).
	ProvenanceTypeComputed ProvenanceType = "computed"

	// ProvenanceTypeDefault indicates the value is a default value.
	ProvenanceTypeDefault ProvenanceType = "default"
)

type YAMLFunctionProcessor added in v1.201.0

type YAMLFunctionProcessor interface {
	// ProcessYAMLFunctionString processes a single YAML function string and returns the processed value.
	// The value parameter is the YAML function string (e.g., "!template '{{ .settings.vpc_cidr }}'").
	// Returns the processed value (which may be any type) or an error.
	ProcessYAMLFunctionString(value string) (any, error)
}

YAMLFunctionProcessor is an interface for processing YAML functions. This allows the merge package to process YAML functions without depending on internal/exec.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL