yamlkit

package
v0.1.21 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 28, 2026 License: MIT Imports: 23 Imported by: 26

Documentation

Overview

yamlkit is a package for parsing, traversing, and updating lists of configuration elements, called resources, represented as yaml doc lists.

Index

Constants

View Source 
const (
	PatchFormatJSON = "json"
	PatchFormatYAML = "yaml"
)

Patch format constants used as the "format" field in structural patches.

View Source 
const (
	PlaceHolderBlockApplyString = "confighubplaceholder"
	PlaceHolderBlockApplyInt    = 999999999
)

PlaceHolderBlockApply We will need placeholders for different data types and that fit with different validation rules The string value is all lowercase to comply with DNS label requirements.

View Source 
const EmbeddedAccessorSeparator = "#"
View Source 
const VisitorSetterInvocationFunctionName = "$visitor"

VisitorSetterInvocationFunctionName is a special function name used to indicate that the setter invocation value should be used directly by the visitor, rather than invoking a real function. It is not a valid function name.

Variables

View Source 
var EmbeddedPathNotFound = errors.New("embedded path not found")
View Source 
var NoSubexpressions = errors.New("no capturing subexpressions")
View Source 
var UnsupportedAccessorType = errors.New("accessor type not supported")
View Source 
var UnsupportedValueType = errors.New("only string values supported currently")

Functions

func AddMutations 

func AddMutations(mutations, newMutations api.ResourceMutationList) api.ResourceMutationList

AddMutations merges newMutations into (existing) mutations and returns the result. It's used to accumulate changes over sequential edits, creating a compiled history of all modifications.

Algorithm:

1. Process Each New Mutation

For each mutation in newMutations:

Resource Matching:

  • First tries to match by current ResourceTypeAndName
  • If not found, checks AliasesWithoutScopes to handle renamed resources
  • If still not found, appends as a new resource mutation

Resource-Level Merge Rules: 

| Existing Type    | New Type              | Result              |
|------------------|-----------------------|---------------------|
| Any              | None                  | Keep existing       |
| Any              | Delete or Replace     | Replace with new    |
| None             | Any (non-None)        | Replace with new    |
| Delete           | Any (non-Delete)      | Change to Replace   |
| Add/Update       | Add/Update            | Merge path mutations|

2. Path-Level Merge

When merging Add/Update mutations, the path mutations are combined:

  • Exact path match: Update the value, preserving the original mutation type (except Delete → non-Delete becomes Replace)
  • New path is prefix of existing: Remove existing child paths (the new value supersedes them)
  • New path not found: Add it to the map.

One implication of this approach is that a path and value might appear at the resource level or in the path map at a higher level (path prefix), or even at multiple levels, and can be then overridden by a value of a more specific path. The values need to be patched from least specific to most specific in order to produce the resulting configuration data.

3. Alias Tracking

Merges aliases from both mutations to track all names the resource has had.

Key Behaviors:

  • Accumulative: Designed to be called repeatedly as changes occur
  • Last-write-wins for values: New values replace old values at the same path
  • Type preservation: Original mutation type is preserved (Add stays Add, Update stays Update)
  • Alias awareness: Handles resources that have been renamed between mutations

func ApplyLinePatch added in v0.1.14 

func ApplyLinePatch(target, patch string) (string, bool)

ApplyLinePatch applies a line-level patch (produced by ComputeLinePatch) to a target string. Returns the patched string and true if all hunks applied cleanly. If any hunk fails to apply, returns the partially patched string and false.

func ApplyScalarPatch added in v0.1.14 

func ApplyScalarPatch(target, patch string) (string, bool)

ApplyScalarPatch applies a patch (produced by ComputeScalarPatch) to a target string. It auto-detects the patch format: structural patches (JSON-encoded, start with '{') are applied structurally; line-level text patches (unified diff) are applied with fuzzy context matching.

Returns the patched string and true if the patch applied cleanly. On failure, returns the original target string and false.

func AssociativePathSegment 

func AssociativePathSegment(mergeKey string, mergeKeyValue string, index int) string

AssociativePathSegment builds a path segment encoding both the merge key value and the positional index, using the syntax ?key=value;@index. The merge key value is escaped to handle dots.

func AttributeDetailsEqual 

func AttributeDetailsEqual(details1, details2 *api.AttributeDetails, compareFunctions bool) bool

AttributeDetailsEqual reports whether two sets of attribute details, optionally including getter and setter invocations, match.

func AttributeValueForPath 

func AttributeValueForPath(resourceProvider ResourceProvider, path api.ResolvedPath, resourceInfo *api.ResourceInfo, value any) api.AttributeValue

func ComputeLinePatch added in v0.1.14 

func ComputeLinePatch(previous, modified string) string

ComputeLinePatch computes a line-level diff between two multi-line strings and returns a patch in unified diff text format. The patch can be serialized as a string in MutationInfo.Patch and later applied with ApplyLinePatch.

This uses the Myers diff algorithm via go-diff's DiffLinesToChars to tokenize at line boundaries, producing a minimal edit script that correctly identifies inserted, deleted, and unchanged lines.

func ComputeMutations 

func ComputeMutations(previousParsedData, modifiedParsedData gaby.Container, functionIndex int64, resourceProvider ResourceProvider) (api.ResourceMutationList, error)

ComputeMutations performs a kind of diff between two configuration Units where it determines what modifications were made at the resource/element level and at the path level. They are recorded in a way that can be accumulated and updated over subsequent edits and transformations.

func ComputeMutationsForDocs 

func ComputeMutationsForDocs(rootPath string, previousDoc *gaby.YamlDoc, modifiedDoc *gaby.YamlDoc, functionIndex int64, pathMutationMap api.MutationMap, mergeKeyLookup MergeKeyLookup)

ComputeMutationsForDocs determines the edits that have been performed to transform the previousDoc into modifiedDoc. The resulting mutations are associated with the provided functionIndex. The pathMutationMap is modified in place.

mergeKeyLookup, if non-nil, is called with array paths to determine whether the array is associative (has a merge key). If so, elements are matched by merge key value instead of positional index, and paths use the ?key=value;@index syntax.

func ComputeScalarPatch added in v0.1.14 

func ComputeScalarPatch(previous, modified string) string

ComputeScalarPatch computes a patch for a changed multi-line scalar string value. It tries to parse both values as JSON, then YAML, and computes a structural patch (sub-path mutations) for those formats. Falls back to a line-level text diff.

Structural patches give true three-way merge for embedded JSON/YAML: individual field changes are tracked by path, so independent changes to different fields merge correctly. Line-level patches handle unstructured text (markdown, config files, etc.) with context-based fuzzy matching.

func DeletePaths 

func DeletePaths(
	parsedData gaby.Container,
	resourceTypeToPaths api.ResourceTypeToPathToVisitorInfoType,
	keys []any,
	resourceProvider ResourceProvider,
	whereExpressions []*api.VisitorRelationalExpression,
) error

func DiffPatch 

func DiffPatch(original, modified, targetData []byte, resourceProvider ResourceProvider) ([]byte, bool, error)

DiffPatch compares original and modified YAML content, generates a patch, and applies it to target data

func DiffPatchWithOptions 

func DiffPatchWithOptions(original, modified, targetData []byte, resourceProvider ResourceProvider, omitAdditions bool) ([]byte, bool, error)

DiffPatchWithOptions compares original and modified YAML content, generates a patch, and applies it to target data If omitAdditions is true, mutations of type MutationTypeAdd are filtered out before applying the patch

func EnrichMergeKeysFromDoc added in v0.1.15 

func EnrichMergeKeysFromDoc(doc *gaby.YamlDoc, resourceProvider ResourceProvider, attr *api.AttributeValue)

EnrichMergeKeysFromDoc extracts merge keys from the resolved path of an AttributeValue and adds them as NeededPreferred properties. For each numeric array index in the path, it looks up the merge key via MergeKeyForPath and reads the value from the document. For example, path "spec.template.spec.volumes.1.configMap.name" with merge key "name"="config" at volumes[1] yields NeededPreferred["Name"] = "config".

func EscapeDotsInPathSegment 

func EscapeDotsInPathSegment(segment string) string

EscapeDotsInPathSegment escapes any dots in a path segment for use in whole-path searches because path segments are separated by dots. TODO: Escape more special characters?

func FindMutationIndex 

func FindMutationIndex(mutationSources api.ResourceMutationList, resource api.ResourceInfo, path api.ResolvedPath) (int64, bool)

FindMutationIndex looks up the mutation index for a specific resource and path in a ResourceMutationList. It matches the resource by ResourceTypeAndName, handling aliases and scope changes (same pattern as AddMutations). For the path, it walks up parent paths to find the most specific mutation index, falling back to the resource-level index if no path-level match is found. Returns the mutation index and true if found.

func FindResourceDoc added in v0.1.15 

func FindResourceDoc(
	parsedData gaby.Container,
	resourceProvider ResourceProvider,
	target *api.ResourceInfo,
) (*gaby.YamlDoc, *api.ResourceInfo)

FindResourceDoc finds the document in parsedData that best matches the given target ResourceInfo. It uses the same matching hierarchy as ComputeMutations:

  1. ResourceMergeID match (definite, if both are valid UUIDs)
  2. Exact ResourceName or ResourceNameWithoutScope match
  3. ResourceTypesAreSimilar as a prerequisite for any match

When the target has MatchByIDOnly MutationOption set (e.g., immutable ConfigMaps with hash-suffixed names), only ResourceMergeID matching is used. Returns the matching doc and its ResourceInfo, or (nil, nil) if no match is found.

func FindYAMLPathsByValue 

func FindYAMLPathsByValue(parsedData gaby.Container, resourceProvider ResourceProvider, searchValue any) api.AttributeValueList

FindYAMLPathsByValue searches for all paths that match a specified value in a YAML structure and returns an api.AttributeValueList.

func FunctionInvocationsEqual 

func FunctionInvocationsEqual(fi1, fi2 *api.FunctionInvocation) bool

FunctionInvocationsEqual reports whether two function invocations match.

func GetMutationOptions 

func GetMutationOptions(doc *gaby.YamlDoc, resourceProvider ResourceProvider) []string

GetMutationOptions reads the MutationOptions value from a YAML document using the resource provider's context path.

func GetNeededPaths 

func GetNeededPaths[T api.Scalar](
	parsedData gaby.Container,
	resourceTypeToPaths api.ResourceTypeToPathToVisitorInfoType,
	keys []any,
	resourceProvider ResourceProvider,
	whereExpressions []*api.VisitorRelationalExpression,
) (api.AttributeValueList, error)

GetNeededPaths traverses the specified path patterns of the specified resource types and returns an api.AttributeValueList containing the values and registered information about all of the found attributes matching the path patterns that Need values. Currently "Need" is determined using placeholder values, 999999999 (9 9s) for integers. Use only for ints. Bools have no placeholder value. Use GetNeededStringPaths for strings.

func GetNeededStringPaths 

func GetNeededStringPaths(
	parsedData gaby.Container,
	resourceTypeToPaths api.ResourceTypeToPathToVisitorInfoType,
	keys []any,
	resourceProvider ResourceProvider,
	whereExpressions []*api.VisitorRelationalExpression,
) (api.AttributeValueList, error)

GetNeededStringPaths traverses the specified path patterns of the specified resource types and returns an api.AttributeValueList containing the values and registered information about all of the found string attributes matching the path patterns that Need values. Currently "Need" is determined using placeholder values, "confighubplaceholder" for strings. It can also extract fields embedded in strings using registered embedded accessors.

func GetPathRegistryForAttributeName 

func GetPathRegistryForAttributeName(
	resourceProvider ResourceProvider,
	attributeName api.AttributeName,
) api.ResourceTypeToPathToVisitorInfoType

GetPathRegistryForAttributeName returns the registry for the specified attribute to pass to a visitor function. If the attribute has a non-empty AttributeGroup, the registries for all attributes in the group are combined and returned.

func GetPathVisitorInfo 

func GetPathVisitorInfo(resourceProvider ResourceProvider, resourceType api.ResourceType, path api.UnresolvedPath) *api.PathVisitorInfo

GetPathVisitorInfo returns the path visitor specification for the specified path within the specified resource type to pass to a visitor function. It searches all attribute names in the path registry for the normalized path. GetPathVisitorInfo returns the PathVisitorInfo for the specified path. It searches all attribute names in the path registry, checking the specific resource type across all attributes first, then falling back to ResourceTypeAny. This ensures a specific resource type match always takes priority. The first match provides the base Details (getter/setter invocations). IsNeeded/IsProvided flags and Enricher are collected from all matches. Getter/setter invocations are NOT merged across attribute names because they carry resource-type-specific arguments.

func GetPaths 

func GetPaths[T api.Scalar](
	parsedData gaby.Container,
	resourceTypeToPaths api.ResourceTypeToPathToVisitorInfoType,
	keys []any,
	resourceProvider ResourceProvider,
	whereExpressions []*api.VisitorRelationalExpression,
) (api.AttributeValueList, error)

GetPaths traverses the specified path patterns of the specified resource types and returns an api.AttributeValueList containing the values and registered information about all of the found attributes matching the path patterns. Use only for int and bool attributes. Use GetStringPaths for string attributes.

func GetPathsAnyType 

func GetPathsAnyType(
	parsedData gaby.Container,
	resourceTypeToPaths api.ResourceTypeToPathToVisitorInfoType,
	keys []any,
	resourceProvider ResourceProvider,
	dataType api.DataType,
	neededValuesOnly bool,
	providedValuesOnly bool,
	whereExpressions []*api.VisitorRelationalExpression,
) (api.AttributeValueList, error)

GetPathsAnyType traverses the specified path patterns of the specified resource types and returns an api.AttributeValueList containing the values and registered information about all of the found attributes matching the path patterns.

func GetRegisteredNeededPaths 

func GetRegisteredNeededPaths(resourceProvider ResourceProvider) api.ResourceTypeToPathToVisitorInfoType

GetRegisteredNeededPaths returns a combined registry of all paths marked as IsNeeded across all attributes in the path registry.

func GetRegisteredNeededStringPaths 

func GetRegisteredNeededStringPaths(
	parsedData gaby.Container,
	resourceProvider ResourceProvider,
	whereExpressions []*api.VisitorRelationalExpression,
) (api.AttributeValueList, error)

GetRegisteredNeededStringPaths retrieves Needed values by scanning all attributes for paths marked as IsNeeded.

func GetRegisteredProvidedPaths 

func GetRegisteredProvidedPaths(resourceProvider ResourceProvider) api.ResourceTypeToPathToVisitorInfoType

GetRegisteredProvidedPaths returns a combined registry of all paths marked as IsProvided across all attributes in the path registry.

func GetRegisteredProvidedStringPaths 

func GetRegisteredProvidedStringPaths(
	parsedData gaby.Container,
	resourceProvider ResourceProvider,
	whereExpressions []*api.VisitorRelationalExpression,
) (api.AttributeValueList, error)

GetRegisteredProvidedStringPaths retrieves Provided values by scanning all attributes for paths marked as IsProvided. Resources with IgnoreProvided annotation are skipped.

func GetResourceCategoryTypeName 

func GetResourceCategoryTypeName(doc *gaby.YamlDoc, resourceProvider ResourceProvider) (api.ResourceCategory, api.ResourceType, api.ResourceName, error)

func GetResourceInfo 

func GetResourceInfo(doc *gaby.YamlDoc, resourceProvider ResourceProvider) (*api.ResourceInfo, error)

func GetStringPaths 

func GetStringPaths(
	parsedData gaby.Container,
	resourceTypeToPaths api.ResourceTypeToPathToVisitorInfoType,
	keys []any,
	resourceProvider ResourceProvider,
	whereExpressions []*api.VisitorRelationalExpression,
) (api.AttributeValueList, error)

GetStringPaths traverses the specified path patterns of the specified resource types and returns an api.AttributeValueList containing the values and registered information about all of the found string attributes matching the path patterns. It can also extract fields embedded in strings using registered embedded accessors.

func GetToolchainPath 

func GetToolchainPath(rp ResourceProvider) string

func GetVisitorMapForPath 

func GetVisitorMapForPath(resourceProvider ResourceProvider, rt api.ResourceType, path api.UnresolvedPath) api.ResourceTypeToPathToVisitorInfoType

GetVisitorMapForPath is used to get visitor info for a resolved path.

func GetVisitorOptions added in v0.1.15 

func GetVisitorOptions(doc *gaby.YamlDoc, resourceProvider ResourceProvider) []string

GetVisitorOptions reads the VisitorOptions value from a YAML document using the resource provider's context path.

func IsMultiLineString added in v0.1.14 

func IsMultiLineString(s string) bool

IsMultiLineString returns true if the string contains embedded newlines (not just a trailing newline), indicating it is a multi-line string value that may benefit from line-level diffing. YAML scalar serialization via gaby appends a trailing newline, so a single-line value like "alice" becomes "alice\n" — this function correctly returns false for such values.

func IsNumber 

func IsNumber(s string) bool

IsNumber reports whether a strings characters are all within ['0'-'9'].

func IsNumeric 

func IsNumeric(c rune) bool

IsNumeric reports whether a character is within ['0'-'9'].

func IsPatchableString added in v0.1.14 

func IsPatchableString(s string) bool

IsPatchableString returns true if the string may benefit from structured or line-level patching rather than wholesale replacement. This includes:

  • Multi-line strings (for line-level text diff)
  • JSON objects or arrays (for structural JSON diff)
  • Strings with YAML structure (for structural YAML diff)

func JoinPathSegments 

func JoinPathSegments(segments []string) string

JoinPathSegments escapes any dots in path segments and joins them for use in whole-path searches.

func LowerFirst 

func LowerFirst(s string) string

LowerFirst lowercases the first character, which is useful for converting PascalCase to camelCase

func MatchesWhereResourceExpressions 

func MatchesWhereResourceExpressions(doc *gaby.YamlDoc, resourceInfo *api.ResourceInfo, expressions []*api.VisitorRelationalExpression) (bool, error)

MatchesWhereResourceExpressions evaluates each expression against a resource. For paths with the "ConfigHub." prefix, values are resolved from ResourceInfo metadata. For other paths, values are resolved from the resource's YAML document using YamlSafePathGetValueAnyType. Returns false if any expression doesn't match. Keep consistent with ValidWhereResourcePaths.

func Normalize 

func Normalize(parsedData gaby.Container, resourceProvider ResourceProvider) error

Normalize iterates over all resources in the parsed configuration data and assigns a ResourceMergeID to any resource that doesn't already have one. ResourceMergeIDs are random UUIDs stored as context annotations on resources. If a resource has a legacy ResourceID but no ResourceMergeID, the legacy ResourceID is migrated: its value is written to the new ResourceMergeID path and the old ResourceID path is deleted.

func PatchMutations 

func PatchMutations(parsedData gaby.Container, mutationsPredicates, mutationsPatch api.ResourceMutationList, resourceProvider ResourceProvider) (gaby.Container, error)

PatchMutations applies a set of mutations to configuration data, effectively "replaying" recorded changes onto a YAML document. It's the inverse of ComputeMutations - where ComputeMutations determines what changed, PatchMutations applies those changes.

mutationsPatch is sometimes generated from other configuration units, such as in the canonical case of upgrade from upstream. Or may be generated from past revisions or even live state. So it may not match the provided configuration data in some ways, such as resource names and whole resources. By default all resources and paths are patchable. Predicates are used to preserve existing changes. mutationsPredicates is expected to have been generated from the mutations corresponding to the configuration data being patched. So it is expected to match the contents of parsedData. It is acceptable for mutationsPredicates to be nil.

Algorithm:

1. Process Each Document

For each document in parsedData:

Resource Matching:

  • Tries to find matching patch by current resource name
  • Falls back to originalName annotation (for cloned resources)
  • Falls back to AliasesWithoutScopes from predicates

Predicate Filtering (Resource Level):

  • If predicate exists and Predicate == false, skip the entire resource

Resource-Level Mutations: 

| MutationType     | Action                                           |
|------------------|--------------------------------------------------|
| Add / Replace    | Replace entire document with the mutation's Value|
| Delete           | Set document to nil (filtered on serialization)  |
| None             | Skip (no changes)                                |
| Update           | Process path-level mutations                     |

2. Path-Level Mutations

For Update mutations, process each path in PathMutationMap:

  • Sort paths: Process parent paths before children (lexicographic sort)

  • Predicate filtering: Check path and all parent paths for Predicate == false

  • Apply by type:

    | MutationType | Action | |------------------|-------------------------------------------| | Add / Replace | Set value at path (overwrites) | | Update | Merge value at path (preserves comments) | | Delete | Remove the path from document |

Key Behaviors:

  • Alias awareness: Matches resources even when names differ between patch and target
  • Predicate filtering: Allows selective application of changes
  • Comment preservation: Update mutations try to preserve YAML comments
  • Parent-first ordering: Ensures parent paths are applied before children
  • Graceful handling: Logs errors but continues processing other mutations

func PathIsResolved 

func PathIsResolved(path string, includeAt bool) bool

func RegisterPathsByAttributeName 

func RegisterPathsByAttributeName(
	resourceProvider ResourceProvider,
	attributeName api.AttributeName,
	resourceType api.ResourceType,
	pathInfos api.PathToVisitorInfoType,
	details *AttributeRegistrationDetails,
	isNeeded bool,
	isProvided bool,
)

RegisterPathsByAttributeName registers the specified path visitor specifications under the designated attribute name and resource type, and adds the provided getter and setter invocations, merging with existing registrations at the same paths, if any. If requested, the registered paths will be normalized so that associative lookups and array indices will be converted to wildcards, which is desired when matching all paths to the attribute. AttributeNameResourceName is used for references to resource names. Other attribute names are used for specific setters and/or getters, especially for attributes that appear in multiple resource types and/or locations. Provided values are special in that they represent sources of values for attributes of the specified attribute name, though they are logically distinct kinds of attributes.

func Reset 

func Reset(parsedData gaby.Container, mutationsPredicates api.ResourceMutationList, resourceProvider ResourceProvider) error

func ResolveAssociativeSegments 

func ResolveAssociativeSegments(doc *gaby.YamlDoc, path string) string

ResolveAssociativeSegments resolves ?key=value;@index segments in a path to numeric indices by looking up elements in the document. It tries key=value match first, then falls back to the positional index. Returns the resolved path with numeric indices only.

func ResourceAndCategoryTypeMaps 

func ResourceAndCategoryTypeMaps(parsedData gaby.Container, resourceProvider ResourceProvider) (
	resourceMap ResourceNameToCategoryTypesMap,
	categoryTypeMap ResourceCategoryTypeToNamesMap,
	err error,
)

ResourceAndCategoryTypeMaps returns maps of all resources in the provided list of parsed YAML documents, from from names to categories+types and categories+types to names.

func ResourceTypesForAttribute 

func ResourceTypesForAttribute(attributeName api.AttributeName, resourceProvider ResourceProvider) []api.ResourceType

ResourceTypesForAttribute returns a list of resource types associated with the specified attribute.

func ResourceTypesForPathMap 

func ResourceTypesForPathMap(pathMap map[api.ResourceType][]string) []api.ResourceType

ResourceTypesForPathMap returns a list of resource types from a path map.

func StripAssociativeSegments 

func StripAssociativeSegments(path string) string

StripAssociativeSegments converts ?key=value;@index segments to just the numeric index. For ?key=@index (direct index), extracts just the index. Non-associative segments are passed through as-is.

func StripComments 

func StripComments(yamlData []byte) ([]byte, error)

StripComments removes all comments from YAML data while preserving the structure and values. This is useful when comparing YAML documents where comments should be ignored.

func SubtractMutations 

func SubtractMutations(mutations, subtractMutations api.ResourceMutationList) api.ResourceMutationList

SubtractMutations removes mutations that overlap with subtractMutations from mutations. It's used in three-way merging to ensure that changes made in a target unit take precedence over changes from a source unit.

Use Case:

When merging source unit changes into a target unit:

  • Source: base → sourceEnd (upstream changes)
  • Target: base → target (local customizations)
  • Result: SubtractMutations(sourceMutations, targetMutations) gives changes that won't overwrite local customizations

Both operands are expected to be mutation diffs produced by ComputeMutations. ComputeMutations generates Add, Delete, Update, and None mutations (not Replace). None means the resource was present but not changed. AddMutations is what converts Delete followed by Add to Replace. Replace is handled here just in case, but not expected. Update at the resource level has an empty Value - all values are in the PathMutationMap.

Algorithm:

1. Resource Matching

For each mutation, find corresponding subtraction mutation by:

  • Current ResourceTypeAndName
  • AliasesWithoutScopes from either mutation (handles renamed resources)

2. Resource-Level Subtraction 

| Subtract Type | Mutation Type | Result                                     |
|---------------|---------------|--------------------------------------------|
| Delete        | Any           | Remove entirely (target deleted)           |
| Replace       | Any           | Remove entirely (target redefined)         |
| None          | Any           | Keep mutation (target didn't change it)    |
| Any           | None          | Keep mutation (source didn't change it)    |
| Update/Add    | Delete        | Remove (don't delete what target modified) |
| Update/Add    | Update/Add    | Process path-level subtraction             |

3. Path-Level Subtraction

For each path in the mutation's PathMutationMap:

  • Case 1 - Exact match: Path exists in subtractMutations → remove it
  • Case 2 - Subtract path is prefix: e.g., subtract has spec.containers.0, mutation has spec.containers.0.image → remove it (parent was changed)
  • Case 3 - Mutation path is prefix: e.g., mutation has spec.containers.0 (whole block), subtract has spec.containers.0.image → if the mutation is a Delete, skip it entirely (can't partially un-delete). Otherwise, keep the mutation and add the subtractMutation paths. PatchMutations processes paths from least specific to most specific, so the subtractMutation's more specific paths will override the mutation's value.

Key Behaviors:

  • Target precedence: Changes in subtractMutations take priority
  • Alias awareness: Matches resources across renames
  • Partial expansion: Only expands paths as needed, keeping unaffected branches whole
  • Type conversion: If all paths are subtracted from an Update, it becomes None

func TransformConfig 

func TransformConfig(
	originalData []byte,
	resourceProvider ResourceProvider,
	transform func(parsedData gaby.Container) ([]byte, error),
) ([]byte, bool, error)

TransformConfig applies a mutation function to configuration data, preserving YAML comments by diffing the changes and patching them onto the original data. This is a general-purpose mechanism that can be reused for any config transformation that operates on comment-stripped data (Starlark, CEL, etc.).

The transform function receives comment-stripped parsed YAML and returns modified YAML bytes. If the transform function returns nil bytes, the original data is returned unchanged.

func UpdatePathsFunction 

func UpdatePathsFunction[T api.Scalar](
	parsedData gaby.Container,
	resourceTypeToPaths api.ResourceTypeToPathToVisitorInfoType,
	keys []any,
	resourceProvider ResourceProvider,
	updater func(T, VisitorContext) T,
	upsert bool,
	whereExpressions []*api.VisitorRelationalExpression,
) error

UpdatePathsFunction traverses the specified path patterns of the specified resource types. The updater function simply needs to return the new attribute value, which must be of the type of the generic type parameter.

func UpdatePathsFunctionDoc 

func UpdatePathsFunctionDoc(
	parsedData gaby.Container,
	resourceTypeToPaths api.ResourceTypeToPathToVisitorInfoType,
	keys []any,
	resourceProvider ResourceProvider,
	updater func(*gaby.YamlDoc, VisitorContext) *gaby.YamlDoc,
	upsert bool,
	whereExpressions []*api.VisitorRelationalExpression,
) error

UpdatePathsFunctionDoc traverses the specified path patterns of the specified resource types. The updater function simply needs to return the new attribute value, which must be a YamlDoc.

func UpdatePathsSetterArgument 

func UpdatePathsSetterArgument(
	parsedData gaby.Container,
	resourceTypeToPaths api.ResourceTypeToPathToVisitorInfoType,
	keys []any,
	resourceProvider ResourceProvider,
	upsert bool,
	whereExpressions []*api.VisitorRelationalExpression,
) error

UpdatePathsSetterArgument traverses the specified path patterns of the specified resource types. For each path, if the visitor context has Details with a SetterInvocation using VisitorSetterInvocationFunctionName, the value at the path is set to the first argument's Value. Supports string, int, and bool values. Skips paths where the current value is already set to a non-placeholder value. Otherwise, the path is not updated.

func UpdatePathsValue 

func UpdatePathsValue[T api.Scalar](
	parsedData gaby.Container,
	resourceTypeToPaths api.ResourceTypeToPathToVisitorInfoType,
	keys []any,
	resourceProvider ResourceProvider,
	newValue T,
	upsert bool,
	whereExpressions []*api.VisitorRelationalExpression,
) error

UpdatePathsValue traverses the specified path patterns of the specified resource types and updates the attributes with the provided value.

func UpdateStringPaths 

func UpdateStringPaths(
	parsedData gaby.Container,
	resourceTypeToPaths api.ResourceTypeToPathToVisitorInfoType,
	keys []any,
	resourceProvider ResourceProvider,
	newValue string,
	upsert bool,
	whereExpressions []*api.VisitorRelationalExpression,
) error

UpdateStringPaths traverses the specified path patterns of the specified resource types and updates the attributes with the provided value. It can also inject fields embedded in strings using registered embedded accessors.

func UpdateStringPathsFunction 

func UpdateStringPathsFunction(
	parsedData gaby.Container,
	resourceTypeToPaths api.ResourceTypeToPathToVisitorInfoType,
	keys []any,
	resourceProvider ResourceProvider,
	updater func(string) string,
	upsert bool,
	whereExpressions []*api.VisitorRelationalExpression,
) error

UpdateStringPathsFunction traverses the specified path patterns of the specified resource types. The updater function simply needs to return the new attribute value. It can also inject fields embedded in strings using registered embedded accessors.

func VetPathsSetterArgument 

func VetPathsSetterArgument(
	parsedData gaby.Container,
	resourceTypeToPaths api.ResourceTypeToPathToVisitorInfoType,
	keys []any,
	resourceProvider ResourceProvider,
	whereExpressions []*api.VisitorRelationalExpression,
) (api.ValidationResult, error)

VetPathsSetterArgument traverses the specified path patterns and validates that current values match the expected default values from $visitor setter invocations. Returns a ValidationResult with Passed=false and FailedAttributes listing any mismatched paths.

func VisitPaths 

func VisitPaths[T api.Scalar](
	parsedData gaby.Container,
	resourceTypeToPaths api.ResourceTypeToPathToVisitorInfoType,
	keys []any,
	output any,
	resourceProvider ResourceProvider,
	visitor VisitorFunc[T],
	upsert bool,
	whereExpressions []*api.VisitorRelationalExpression,
) (any, error)

VisitPaths is a simple wrapper of the base visitor function. It traverses the specified path patterns of the specified resource types within the parsed configuration YAML document list.

func VisitPathsAnyType 

func VisitPathsAnyType(
	parsedData gaby.Container,
	resourceTypeToPaths api.ResourceTypeToPathToVisitorInfoType,
	keys []any,
	output any,
	resourceProvider ResourceProvider,
	visitor VisitorFuncAnyType,
	upsert bool,
	whereExpressions []*api.VisitorRelationalExpression,
) (any, error)

VisitPathsAnyType is a simple wrapper of the base visitor function. It traverses the specified path patterns of the specified resource types within the parsed configuration YAML document list.

func VisitPathsDoc 

func VisitPathsDoc(
	parsedData gaby.Container,
	resourceTypeToPaths api.ResourceTypeToPathToVisitorInfoType,
	keys []any,
	output any,
	resourceProvider ResourceProvider,
	visitor VisitorFuncDoc,
	upsert bool,
	whereExpressions []*api.VisitorRelationalExpression,
) (any, error)

VisitPathsDoc is the base visitor function. It traverses the specified path patterns of the specified resource types within the parsed configuration YAML document list.

func VisitResources 

func VisitResources(parsedData gaby.Container, output any, resourceProvider ResourceProvider, visitor ResourceVisitorFunc) (any, error)

VisitResources iterates over all of the resources/elements in a configuration unit and passes metadata about the resource as well as the document itself to a visitor function.

func VisitResourcesFiltered 

func VisitResourcesFiltered(parsedData gaby.Container, output any, resourceProvider ResourceProvider, whereExpressions []*api.VisitorRelationalExpression, visitor ResourceVisitorFunc) (any, error)

VisitResourcesFiltered iterates over resources, skipping those that don't match the whereExpressions. When whereExpressions is nil or empty, it behaves identically to VisitResources.

func VisitorInfoEqual 

func VisitorInfoEqual(pathVisitorInfo1, pathVisitorInfo2 *api.PathVisitorInfo, compareFunctions bool) bool

VisitorInfoEqual reports whether two path visitor specifications, optionally including getter and setter invocations, match.

func YamlSafePathGetDoc 

func YamlSafePathGetDoc(
	doc *gaby.YamlDoc,
	resolvedPath api.ResolvedPath,
	notFoundOk bool,
) (*gaby.YamlDoc, bool, error)

YamlSafePathGetDoc returns a document node at a fully resolved path and whether it was found. An error indicates a parsing error. An error is also returned if the path is expected to exist.

func YamlSafePathGetValue 

func YamlSafePathGetValue[T api.Scalar](
	doc *gaby.YamlDoc,
	resolvedPath api.ResolvedPath,
	notFoundOk bool,
) (T, bool, error)

YamlSafePathGetValue returns a value at a fully resolved path and whether it was found. An error indicates a parsing error or that the value was not of the expected type.

func YamlSafePathGetValueAnyType 

func YamlSafePathGetValueAnyType(
	doc *gaby.YamlDoc,
	resolvedPath api.ResolvedPath,
	notFoundOk bool,
) (any, bool, error)

YamlSafePathGetValueAnyType returns a value at a fully resolved path and whether it was found. An error indicates a parsing error.

Types

type AttributeEnricher added in v0.1.15 

type AttributeEnricher func(doc *gaby.YamlDoc, attrInfo *api.AttributeValue, isProvided bool) error

AttributeEnricher is a function that enriches an AttributeValue with properties after it is extracted by a visitor. It receives the resource doc for context and a flag indicating whether the value is a provided value. It populates ProvidedProperties, NeededRequired, and/or NeededPreferred on the attribute's Details.

type AttributeRegistrationDetails added in v0.1.15 

type AttributeRegistrationDetails struct {
	GetterInvocation *api.FunctionInvocation
	SetterInvocation *api.FunctionInvocation
	api.AttributeNeedsProvidesDetails
	Enricher AttributeEnricher
}

AttributeRegistrationDetails specifies getter/setter invocations and an optional Enricher function for use when registering paths via RegisterPathsByAttributeName.

type EmbeddedAccessor 

type EmbeddedAccessor interface {
	// ExistsP reports whether the specified attribute or subpart exists within
	// the string at the specified YAML document node.
	ExistsP(scalarYamlDoc *gaby.YamlDoc, path string) bool

	// SetP sets the specified attribute or subpart within the string at the
	// specified YAML document node.
	SetP(scalarYamlDoc *gaby.YamlDoc, value any, path string) error

	// Data returns the value of the specified attribute or subpart embedded
	// within the string at the specified YAML document node.
	Data(scalarYamlDoc *gaby.YamlDoc, path string) (any, error)

	// Replace replaces the value of the specified attribute or subpart within
	// the provided string.
	Replace(currentFieldValue string, value any, path string) (string, error)

	// Extract returns the value of the specified attribute or subpart within the
	// provided string.
	Extract(currentFieldValue, path string) (any, error)
}

EmbeddedAccessor is used to access attributes embedded in data formats encoded within string values within a YAML document. For instance, YAML might be encoded within a YAML value. Or it could be as simple as a structured string with distinct sections and separators, such as a container image or URL.

func GetEmbeddedAccessor 

func GetEmbeddedAccessor(embeddedAccessorType api.EmbeddedAccessorType, config string) (EmbeddedAccessor, error)

type JSONAccessor added in v0.1.14 

type JSONAccessor struct{}

JSONAccessor is an EmbeddedAccessor that accesses fields within a JSON string value embedded in a YAML scalar. The path uses dot-separated segments to navigate the parsed JSON structure.

For example, given a YAML field containing the JSON string '{"a":{"b":"hello"}}':

  • Extract(jsonStr, "a.b") returns "hello"
  • Replace(jsonStr, "world", "a.b") returns '{"a":{"b":"world"}}'

The config string is not used (pass "" when creating).

func (*JSONAccessor) Data added in v0.1.14 

func (ja *JSONAccessor) Data(scalarYamlDoc *gaby.YamlDoc, path string) (any, error)

func (*JSONAccessor) ExistsP added in v0.1.14 

func (ja *JSONAccessor) ExistsP(scalarYamlDoc *gaby.YamlDoc, path string) bool

func (*JSONAccessor) Extract added in v0.1.14 

func (ja *JSONAccessor) Extract(currentFieldValue, path string) (any, error)

func (*JSONAccessor) Replace added in v0.1.14 

func (ja *JSONAccessor) Replace(currentFieldValue string, value any, path string) (string, error)

func (*JSONAccessor) SetP added in v0.1.14 

func (ja *JSONAccessor) SetP(scalarYamlDoc *gaby.YamlDoc, value any, path string) error

type LineAccessor added in v0.1.14 

type LineAccessor struct{}

LineAccessor is an EmbeddedAccessor that accesses individual lines of a multi-line string value by line number. The path is a 1-based line number (as a string).

For example, given a multi-line string "line one\nline two\nline three\n":

  • ExistsP(doc, "2") returns true
  • Data(doc, "2") returns "line two"
  • SetP(doc, "new line two", "2") replaces line 2
  • Extract("line one\nline two\n", "1") returns "line one"

The config string is not used (pass "" when creating).

func (*LineAccessor) Data added in v0.1.14 

func (la *LineAccessor) Data(scalarYamlDoc *gaby.YamlDoc, path string) (any, error)

func (*LineAccessor) ExistsP added in v0.1.14 

func (la *LineAccessor) ExistsP(scalarYamlDoc *gaby.YamlDoc, path string) bool

func (*LineAccessor) Extract added in v0.1.14 

func (la *LineAccessor) Extract(currentFieldValue, path string) (any, error)

func (*LineAccessor) Replace added in v0.1.14 

func (la *LineAccessor) Replace(currentFieldValue string, value any, path string) (string, error)

func (*LineAccessor) SetP added in v0.1.14 

func (la *LineAccessor) SetP(scalarYamlDoc *gaby.YamlDoc, value any, path string) error

type MergeKeyEntry added in v0.1.15 

type MergeKeyEntry struct {
	Key   string // merge key field name (e.g., "name")
	Value string // merge key value (e.g., "config")
}

MergeKeyEntry represents a merge key/value pair extracted from an associative path segment.

func ExtractMergeKeysFromPath added in v0.1.15 

func ExtractMergeKeysFromPath(path string) []MergeKeyEntry

ExtractMergeKeysFromPath extracts merge key/value pairs from associative path segments. Path segments of the form ?key=value;@index yield {Key: key, Value: value}.

type MergeKeyLookup 

type MergeKeyLookup func(path string) (string, bool)

MergeKeyLookup is a function that returns the merge key field name for a given array path, if one exists. It is used by ComputeMutationsForDocs to match array elements by merge key value instead of positional index.

type RegexpAccessor 

type RegexpAccessor struct {
	RegexpString string
	Regexp       *regexp.Regexp
	SubexpNames  []string
}

RegexpAccessor is an EmbeddedAccessor that uses regular expressions to extract and insert subparts of a structured string value.

func (*RegexpAccessor) Data 

func (ra *RegexpAccessor) Data(scalarYamlDoc *gaby.YamlDoc, path string) (any, error)

func (*RegexpAccessor) ExistsP 

func (ra *RegexpAccessor) ExistsP(scalarYamlDoc *gaby.YamlDoc, path string) bool

func (*RegexpAccessor) Extract 

func (ra *RegexpAccessor) Extract(currentFieldValue, path string) (any, error)

func (*RegexpAccessor) Replace 

func (ra *RegexpAccessor) Replace(currentFieldValue string, value any, path string) (string, error)

func (*RegexpAccessor) SetP 

func (ra *RegexpAccessor) SetP(scalarYamlDoc *gaby.YamlDoc, value any, path string) error

type ResolvedPathInfo 

type ResolvedPathInfo struct {
	Path          api.ResolvedPath
	PathArguments []api.FunctionArgument
}

ResolvedPathInfo contains a fully resolved path and any named path parameters specified in the unresolved path expression (using ?, *?, or *@).

func ResolveAssociativePaths 

func ResolveAssociativePaths(
	doc *gaby.YamlDoc,
	unresolvedPath api.UnresolvedPath,
	resolvedPath api.ResolvedPath,
	upsert bool,
	accessor EmbeddedAccessor,
) ([]ResolvedPathInfo, error)

ResolveAssociativePaths resolves an associative path with associative lookups (?) and wildcards (*, *?, *@) into specific resolved paths and discovered path parameters. See the documentation for api.UnresolvedPath for more details. If accessor is non-nil, it is used as a fallback for associative lookups on scalar array elements where the element has no map fields (e.g., pflags like "--key=value").

type ResourceCategoryTypeToNamesMap 

type ResourceCategoryTypeToNamesMap map[api.ResourceCategoryType][]api.ResourceName

type ResourceInfoToDocMap 

type ResourceInfoToDocMap map[api.ResourceInfo]int

func ResourceToDocMap 

func ResourceToDocMap(parsedData gaby.Container, resourceProvider ResourceProvider) (resourceMap ResourceInfoToDocMap, err error)

ResourceToDocMap returns a map of all resources in the provided list of parsed YAML documents to their document index.

type ResourceNameToCategoryTypesMap 

type ResourceNameToCategoryTypesMap map[api.ResourceName][]api.ResourceCategoryType

type ResourceProvider 

type ResourceProvider interface {
	DefaultResourceCategory() api.ResourceCategory
	ResourceCategoryGetter(doc *gaby.YamlDoc) (api.ResourceCategory, error)
	ResourceTypeGetter(doc *gaby.YamlDoc) (api.ResourceType, error)
	ResourceNameGetter(doc *gaby.YamlDoc) (api.ResourceName, error)
	// ResourceMergeIDGetter returns the resource merge ID, checking the new ResourceMergeID
	// path first and falling back to the legacy ResourceID path for backward compatibility.
	ResourceMergeIDGetter(doc *gaby.YamlDoc) (string, error)
	// Deprecated: Use ResourceMergeIDGetter instead.
	ResourceIDGetter(doc *gaby.YamlDoc) (string, error)
	// ResourceNameStableCoreGetter returns the stable core of the resource name, with
	// generated prefixes and suffixes stripped. Returns empty string if not present.
	ResourceNameStableCoreGetter(doc *gaby.YamlDoc) (api.ResourceName, error)
	RemoveScopeFromResourceName(resourceName api.ResourceName) api.ResourceName
	ScopelessResourceNamePath() api.ResolvedPath
	SetResourceName(doc *gaby.YamlDoc, name string) error
	// SetResourceMergeID sets the resource merge ID at the new ResourceMergeID path.
	SetResourceMergeID(doc *gaby.YamlDoc, id string) error
	// Deprecated: Use SetResourceMergeID instead.
	SetResourceID(doc *gaby.YamlDoc, id string) error
	// DeleteResourceMergeID deletes the resource merge ID from both the new ResourceMergeID
	// path and the legacy ResourceID path.
	DeleteResourceMergeID(doc *gaby.YamlDoc) error
	// Deprecated: Use DeleteResourceMergeID instead.
	DeleteResourceID(doc *gaby.YamlDoc) error
	ResourceTypesAreSimilar(resourceTypeA, resourceTypeB api.ResourceType) bool
	TypeDescription() string
	NormalizeName(name string) string
	NameSeparator() string
	ContextPath(contextField string) string
	GetPathRegistry() api.AttributeNameToResourceTypeToPathToVisitorInfoType
	GetAttributeRegistry() api.AttributeNameToAttributeDescriptor
	GetRegistry() *ResourceProviderRegistry
	// MergeKeyForPath returns the merge key field name for the given resource type
	// and array path, if one exists. The path should use dot-separated segments
	// where array indices may be numeric or wildcards. The implementation normalizes
	// numeric indices to wildcards for lookup. Returns ("", false) if no merge key
	// is defined for the path.
	MergeKeyForPath(resourceType api.ResourceType, path string) (string, bool)
	// IsMapKeyPath returns true if the given path is a freeform map (e.g., labels,
	// annotations) whose children are dynamic keys rather than schema fields.
	// During path normalization, child segments of map paths are converted to wildcards.
	IsMapKeyPath(resourceType api.ResourceType, path string) bool
	GetToolchainType() workerapi.ToolchainType
}

The ResourceProvider interface is used to perform toolchain-specific operations.

type ResourceProviderRegistry added in v0.1.15 

type ResourceProviderRegistry struct {
	PathRegistry      api.AttributeNameToResourceTypeToPathToVisitorInfoType
	AttributeRegistry api.AttributeNameToAttributeDescriptor
}

ResourceProviderRegistry holds the path and attribute registries common to all ResourceProvider implementations.

func NewResourceProviderRegistry added in v0.1.15 

func NewResourceProviderRegistry() ResourceProviderRegistry

NewResourceProviderRegistry creates a new ResourceProviderRegistry with initialized maps.

func (*ResourceProviderRegistry) GetAttributeRegistry added in v0.1.15 

func (r *ResourceProviderRegistry) GetAttributeRegistry() api.AttributeNameToAttributeDescriptor

func (*ResourceProviderRegistry) GetPathRegistry added in v0.1.15 

func (r *ResourceProviderRegistry) GetPathRegistry() api.AttributeNameToResourceTypeToPathToVisitorInfoType

func (*ResourceProviderRegistry) GetRegistry added in v0.1.15 

func (r *ResourceProviderRegistry) GetRegistry() *ResourceProviderRegistry

type ResourceTypeToPathPrefixSetType 

type ResourceTypeToPathPrefixSetType map[api.ResourceType]map[string]struct{}

type ResourceVisitorFunc 

type ResourceVisitorFunc func(doc *gaby.YamlDoc, output any, index int, resourceInfo *api.ResourceInfo) (any, []error)

ResourceVisitorFunc defines the signature of functions invoked by the resource visitor function.

type VisitorContext 

type VisitorContext struct {
	api.AttributeInfo // includes Path and Info
	Arguments         []api.FunctionArgument
	EmbeddedPath      string
	Accessor          EmbeddedAccessor
	PathVisitorInfo   *api.PathVisitorInfo
}

VisitorContext contains information passed to visitor functions for each path traversed.

type VisitorFunc 

type VisitorFunc[T api.Scalar] func(doc *gaby.YamlDoc, output any, context VisitorContext, currentValue T) (any, error)

VisitorFunc defines the signature of functions invoked by the visitor functions.

type VisitorFuncAnyType 

type VisitorFuncAnyType func(doc *gaby.YamlDoc, output any, context VisitorContext, currentValue any) (any, error)

VisitorFuncAnyType defines the signature of functions invoked by the visitor functions.

type VisitorFuncDoc 

type VisitorFuncDoc func(doc *gaby.YamlDoc, output any, context VisitorContext, currentDoc *gaby.YamlDoc) (any, error)

VisitorFuncDoc defines the signature of functions invoked by the visitor function.

type YAMLAccessor added in v0.1.14 

type YAMLAccessor struct{}

YAMLAccessor is an EmbeddedAccessor that accesses fields within a YAML string value embedded in a YAML scalar. The path uses dot-separated segments to navigate the parsed YAML structure.

For example, given a YAML field containing the string "a:\n b: hello\n":

  • Extract(yamlStr, "a.b") returns "hello"
  • Replace(yamlStr, "world", "a.b") returns "a:\n b: world\n"

The config string is not used (pass "" when creating).

func (*YAMLAccessor) Data added in v0.1.14 

func (ya *YAMLAccessor) Data(scalarYamlDoc *gaby.YamlDoc, path string) (any, error)

func (*YAMLAccessor) ExistsP added in v0.1.14 

func (ya *YAMLAccessor) ExistsP(scalarYamlDoc *gaby.YamlDoc, path string) bool

func (*YAMLAccessor) Extract added in v0.1.14 

func (ya *YAMLAccessor) Extract(currentFieldValue, path string) (any, error)

func (*YAMLAccessor) Replace added in v0.1.14 

func (ya *YAMLAccessor) Replace(currentFieldValue string, value any, path string) (string, error)

func (*YAMLAccessor) SetP added in v0.1.14 

func (ya *YAMLAccessor) SetP(scalarYamlDoc *gaby.YamlDoc, value any, path string) error

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.