parse

package
v1.22.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 5, 2025 License: Apache-2.0 Imports: 56 Imported by: 4

Documentation

Index

Constants

View Source
const (
	// RenderingInProgress means that the configs are still being rendered by Config Sync.
	RenderingInProgress string = "Rendering is still in progress"

	// RenderingSucceeded means that the configs have been rendered successfully.
	RenderingSucceeded string = "Rendering succeeded"

	// RenderingFailed means that the configs have failed to be rendered.
	RenderingFailed string = "Rendering failed"

	// RenderingSkipped means that the configs don't need to be rendered.
	RenderingSkipped string = "Rendering skipped"

	// RenderingRequired means that the configs require rendering but the
	// hydration-controller is not currently running.
	RenderingRequired string = "Rendering required but is currently disabled"
	// RenderingNotRequired means that the configs do not require rendering but the
	// hydration-controller is currently running.
	RenderingNotRequired string = "Rendering not required but is currently enabled"
)

Variables

This section is empty.

Functions

func BadScopeErr

func BadScopeErr(resource client.Object, want declared.Scope) status.ResourceError

BadScopeErr reports that the passed resource declares a Namespace for a different Namespace repository.

func OptionsForScope

func OptionsForScope(options validate.Options, scope declared.Scope) validate.Options

OptionsForScope returns new Options that have been updated for the given Scope.

Types

type EventHandler added in v1.20.0

type EventHandler struct {
	Context           context.Context
	Reconciler        Reconciler
	NSControllerState *namespacecontroller.State
}

EventHandler is a events.Subscriber implementation that handles events and triggers the RunFunc when appropriate.

func NewEventHandler added in v1.20.0

func NewEventHandler(ctx context.Context, r Reconciler, nsControllerState *namespacecontroller.State) *EventHandler

NewEventHandler builds an EventHandler

func (*EventHandler) Handle added in v1.20.0

func (s *EventHandler) Handle(event events.Event) events.Result

Handle an Event and return the Result. - FullSyncEventType - Reset the cache and sync from scratch. - SyncEventType - Sync from the cache, priming the cache from disk, if necessary. - StatusUpdateEventType - Update the RSync status with status from the Remediator & NSController. - NamespaceSyncEventType - Sync from the cache, if the NSController requested one. - RetrySyncEventType - Sync from the cache, if one of the following cases is detected:

  • Remediator or Reconciler reported a management conflict
  • Reconciler requested a retry due to error
  • Remediator requested a watch update

type FileSource

type FileSource struct {
	// SourceDir is the path to the symbolic link of the source repository.
	SourceDir cmpath.Absolute
	// HydratedRoot is the path to the root of the hydrated directory.
	HydratedRoot string
	// RepoRoot is the absolute path to the parent directory of SourceRoot and HydratedRoot.
	RepoRoot cmpath.Absolute
	// HydratedLink is the relative path to the symbolic link of the hydrated configs.
	HydratedLink string
	// SyncDir is the path to the directory of policies within the source repository.
	SyncDir cmpath.Relative
	// SourceType is the type of the source repository, must be git or oci.
	SourceType configsync.SourceType
	// SourceRepo is the source repo to sync.
	SourceRepo string
	// SourceBranch is the branch of the source repo to sync.
	SourceBranch string
	// SourceRev is the revision of the source repo to sync.
	SourceRev            string
	ReconcilerSignalsDir cmpath.Absolute
}

FileSource includes all settings to configure where a Parser reads files from.

type Files added in v1.17.0

type Files struct {
	// TODO: unwrap composition
	FileSource
}

Files lists files in a repository and ensures the source repository hasn't been modified from HEAD.

type GitSourceSpec added in v1.19.0

type GitSourceSpec struct {
	Repo     string
	Revision string
	Branch   string
	Dir      string
}

GitSourceSpec is a SourceSpec for the Git SourceType

func (GitSourceSpec) Equals added in v1.19.0

func (g GitSourceSpec) Equals(other SourceSpec) bool

Equals returns true if the specified SourceSpec equals this GitSourceSpec, including type and all field values.

type HelmSourceSpec added in v1.19.0

type HelmSourceSpec struct {
	Repo    string
	Version string
	Chart   string
}

HelmSourceSpec is a SourceSpec for the Helm SourceType

func (HelmSourceSpec) Equals added in v1.19.0

func (h HelmSourceSpec) Equals(other SourceSpec) bool

Equals returns true if the specified SourceSpec equals this HelmSourceSpec, including type and all field values.

type OCISourceSpec added in v1.19.0

type OCISourceSpec struct {
	Image string
	Dir   string
}

OCISourceSpec is a SourceSpec for the OCI SourceType

func (OCISourceSpec) Equals added in v1.19.0

func (o OCISourceSpec) Equals(other SourceSpec) bool

Equals returns true if the specified SourceSpec equals this OCISourceSpec, including type and all field values.

type Options added in v1.17.0

type Options struct {
	// Files lists Files in the source of truth.
	// TODO: compose Files without extending.
	Files

	// Clock is used for time tracking, namely to simplify testing by allowing
	// a fake clock, instead of a RealClock.
	Clock clock.Clock

	// ConfigParser defines the minimum interface required for Reconciler to use a
	// ConfigParser to read configs from a filesystem.
	ConfigParser filesystem.ConfigParser

	// ClusterName is the name of the cluster we're syncing configuration to.
	ClusterName string

	// Client knows how to read objects from a Kubernetes cluster and update
	// status.
	Client client.Client

	// ReconcilerName is the name of the reconciler resources, such as service
	// account, service, deployment and etc.
	ReconcilerName string

	// SyncName is the name of the RootSync or RepoSync object.
	SyncName string

	// Scope defines the scope of the reconciler, either root or namespaced.
	Scope declared.Scope

	// DiscoveryClient is how the Parser learns what types are currently
	// available on the cluster.
	DiscoveryClient discovery.ServerResourcer

	// Converter uses the DiscoveryClient to encode the declared fields of
	// objects in Git.
	Converter *declared.ValueConverter

	// WebhookEnabled indicates whether the Webhook is currently enabled
	WebhookEnabled bool

	// DeclaredResources is the set of valid source objects, managed by the
	// Updater and shared with the Parser & Remediator.
	// This is used by the Parser to validate that CRDs can only be removed from
	// the source when all of its CRs are removed as well.
	DeclaredResources *declared.Resources
}

Options holds configuration and dependencies required by all parsers.

type Parser

type Parser interface {
	// ParseSource parses the source manifest files and returns the objects.
	ParseSource(ctx context.Context, state *sourceState) ([]ast.FileObject, status.MultiError)
}

Parser represents a parser that can be pointed at and continuously parse a source.

type ReconcileResult added in v1.21.0

type ReconcileResult struct {
	SourceChanged bool
	Success       bool
}

ReconcileResult encapsulates the result of a reconciler.Reconcile. This simply allows explicitly naming return values in a way that makes the implementation easier to read.

type Reconciler added in v1.20.0

type Reconciler interface {
	// Options returns the ReconcilerOptions used by this reconciler.
	Options() *ReconcilerOptions

	// ReconcilerState returns the current state of the parser/reconciler.
	ReconcilerState() *ReconcilerState

	// Reconcile the cluster with the source config.
	//
	// Reconcile has multiple phases:
	// - Fetch - Checks the shared filesystem for new source commits fetched by
	//     one of the *-sync sidecars.
	// - Render - Checks the shared filesystem for source rendered by the
	//     hydration-controller sidecar using helm or kustomize, if required.
	// - Read - Reads the fetch and render status from the shared filesystem and
	//     lists the source config files.
	// - Parse - Parses resource objects from the source config files, validates
	//    them, and adds custom metadata.
	// - Update (aka Sync) - Updates the cluster and remediator to reflect the
	//     latest resource object manifests in the source.
	Reconcile(ctx context.Context, trigger string) ReconcileResult

	// UpdateSyncStatus updates the RSync status to reflect asynchronous status
	// changes made by the remediator between Reconcile calls.
	// Returns an error if the status update failed or was cancelled.
	UpdateSyncStatus(context.Context) error
}

Reconciler reconciles the cluster with config from the source of truth. TODO: Move to reconciler package; requires unwinding dependency cycles

func NewRepoSyncReconciler added in v1.21.0

func NewRepoSyncReconciler(recOpts *ReconcilerOptions, parseOpts *Options) Reconciler

NewRepoSyncReconciler creates a new reconciler for reconciling namespace-scoped resources configured with a RepoSync.

func NewRootSyncReconciler added in v1.21.0

func NewRootSyncReconciler(recOpts *ReconcilerOptions, parseOpts *RootOptions) Reconciler

NewRootSyncReconciler creates a new reconciler for reconciling cluster-scoped and namespace-scoped resources configured with a RootSync.

type ReconcilerOptions added in v1.20.0

type ReconcilerOptions struct {
	// Extend parser options to ensure they're using the same dependencies.
	*Options

	// Updater syncs the source from the parsed cache to the cluster.
	*Updater

	// FullSyncPeriod is the max time between full reconciliations.
	// When elapsed, the next reconcile attempt will reset the source cache,
	// triggering full reconciliation.
	FullSyncPeriod time.Duration

	// StatusUpdatePeriod is how long the Parser waits between updates of the
	// sync status, to account for management conflict errors from the Remediator.
	StatusUpdatePeriod time.Duration

	// RenderingEnabled indicates whether the hydration-controller is currently
	// running for this reconciler.
	RenderingEnabled bool
}

ReconcilerOptions holds configuration for the reconciler.

type ReconcilerState added in v1.20.0

type ReconcilerState struct {
	// contains filtered or unexported fields
}

ReconcilerState is the current state of the Reconciler, including progress indicators and in-memory cache for each of the reconcile stages: - Fetch - Render/Hydrate - Read - Parse/Validate - Update

ReconcilerState also includes a cache of the RSync spec and status (ReconcilerStatus).

TODO: break up cacheForCommit into phase-based caches TODO: move sourceState into ReconcilerState so the RSync spec and status are next to each other

func (*ReconcilerState) IsFullSyncRequired added in v1.21.0

func (s *ReconcilerState) IsFullSyncRequired(now metav1.Time, fullSyncPeriod time.Duration) bool

IsFullSyncRequired returns true if the specified period has elapsed since the last time SetLastFullSyncTime was called.

func (*ReconcilerState) RecordFailure added in v1.21.0

func (s *ReconcilerState) RecordFailure(c clock.Clock, errs status.MultiError)

RecordFailure is called when a sync attempt errors. It invalidates the checkpoint, requests a retry, and logs the errors. Does not reset the cacheForCommit.

func (*ReconcilerState) RecordFullSyncStart added in v1.21.0

func (s *ReconcilerState) RecordFullSyncStart(now metav1.Time)

RecordFullSyncStart is called when a full sync attempt starts. It resets the cacheForCommit, which tells the parser to re-parse from source, but keeps the last known read status, to skip re-read on next sync attempt. Retry request will not be reset, to avoid resetting the backoff retries.

func (*ReconcilerState) RecordReadFailure added in v1.21.0

func (s *ReconcilerState) RecordReadFailure()

RecordReadFailure is called when read errors after a source change is detected. It resets the cacheForCommit, forces re-reading and re-parsing.

func (*ReconcilerState) RecordReadSuccess added in v1.21.0

func (s *ReconcilerState) RecordReadSuccess(source *sourceState)

RecordReadSuccess is called when read succeeds after a source change is detected. It resets the cacheForCommit, which skips re-reading, but forces re-parsing.

func (*ReconcilerState) RecordRenderInProgress added in v1.21.0

func (s *ReconcilerState) RecordRenderInProgress()

RecordRenderInProgress is called when waiting for rendering status. It resets the cacheForCommit, which tells the next reconcile attempt to re-parse from source.

func (*ReconcilerState) RecordSyncSuccess added in v1.21.0

func (s *ReconcilerState) RecordSyncSuccess(c clock.Clock)

RecordSyncSuccess is called after a successful sync. It records the last known source path, as well as timestamps for last updated and last transitioned.

func (*ReconcilerState) SyncErrors added in v1.20.0

func (s *ReconcilerState) SyncErrors() status.MultiError

SyncErrors returns all the sync errors, including remediator errors, validation errors, applier errors, and watch update errors.

type ReconcilerStatus added in v1.19.0

type ReconcilerStatus struct {
	// SourceStatus tracks info from the `Status.Source` field of a RepoSync/RootSync.
	SourceStatus *SourceStatus

	// RenderingStatus tracks info from the `Status.Rendering` field of a RepoSync/RootSync.
	RenderingStatus *RenderingStatus

	// SyncStatus tracks info from the `Status.Sync` field of a RepoSync/RootSync.
	SyncStatus *SyncStatus
}

ReconcilerStatus represents the status of the reconciler.

func (*ReconcilerStatus) DeepCopy added in v1.19.0

func (s *ReconcilerStatus) DeepCopy() *ReconcilerStatus

DeepCopy returns a deep copy of the receiver. Warning: Go errors are not copy-able. So this isn't a true deep-copy.

type RenderingStatus added in v1.19.0

type RenderingStatus struct {
	// Spec represents the source specification that this status corresponds to.
	// The spec is stored in the status so we can distinguish if the status
	// reflects the latest spec or not.
	Spec       SourceSpec
	Commit     string
	Message    string
	Errs       status.MultiError
	LastUpdate metav1.Time
	// RequiresRendering indicates whether the sync source has dry configs
	// only used internally (not surfaced on RSync status)
	RequiresRendering bool
}

RenderingStatus represents the status of the rendering stage of the pipeline.

func (*RenderingStatus) DeepCopy added in v1.19.0

func (rs *RenderingStatus) DeepCopy() *RenderingStatus

DeepCopy returns a deep copy of the receiver. Warning: Go errors are not copy-able. So this isn't a true deep-copy.

func (*RenderingStatus) Equals added in v1.19.0

func (rs *RenderingStatus) Equals(other *RenderingStatus) bool

Equals returns true if the specified RenderingStatus equals this RenderingStatus, excluding the LastUpdate timestamp.

type RootOptions added in v1.17.0

type RootOptions struct {
	// Extend parse.Options
	*Options

	// SourceFormat defines the structure of the Root repository. Only the Root
	// repository may be SourceFormatHierarchy; all others are implicitly
	// SourceFormatUnstructured.
	SourceFormat configsync.SourceFormat

	// NamespaceStrategy indicates the NamespaceStrategy to be used by this
	// reconciler.
	NamespaceStrategy configsync.NamespaceStrategy

	// DynamicNSSelectorEnabled represents whether the NamespaceSelector's dynamic
	// mode is enabled. If it is enabled, NamespaceSelector will also select
	// resources matching the on-cluster Namespaces.
	// Only Root reconciler may have dynamic NamespaceSelector enabled because
	// RepoSync can't manage NamespaceSelectors.
	DynamicNSSelectorEnabled bool

	// NSControllerState stores whether the Namespace Controller schedules a sync
	// event for the reconciler thread, along with the cached NamespaceSelector
	// and selected namespaces.
	// Only Root reconciler may have Namespace Controller state because
	// RepoSync can't manage NamespaceSelectors.
	NSControllerState *namespacecontroller.State
}

RootOptions includes options specific to RootSync objects.

type SourceSpec added in v1.19.0

type SourceSpec interface {
	// Equals returns true if the specified SourceSpec equals this
	// SourceSpec, including type and all field values.
	Equals(SourceSpec) bool
}

SourceSpec is a representation of the source specification that is cached and stored in the RSync status for each stage in the pipeline.

For the purposes of deciding when to skip updates, the SourceSpec is comparable for equality. If not equal, an update is necessary.

func SourceSpecFromFileSource added in v1.19.0

func SourceSpecFromFileSource(source FileSource, sourceType configsync.SourceType, commit string) SourceSpec

SourceSpecFromFileSource builds a SourceSpec from the FileSource. The type of SourceSpec depends on the SourceType. Commit is only necessary for Helm sources, because the chart Version is parsed from the "commit" string (`chart:version`).

type SourceStatus added in v1.19.0

type SourceStatus struct {
	// Spec represents the source specification that this status corresponds to.
	// The spec is stored in the status so we can distinguish if the status
	// reflects the latest spec or not.
	Spec       SourceSpec
	Commit     string
	Errs       status.MultiError
	LastUpdate metav1.Time
}

SourceStatus represents the status of the source stage of the pipeline.

func (*SourceStatus) DeepCopy added in v1.19.0

func (gs *SourceStatus) DeepCopy() *SourceStatus

DeepCopy returns a deep copy of the receiver. Warning: Go errors are not copy-able. So this isn't a true deep-copy.

func (*SourceStatus) Equals added in v1.19.0

func (gs *SourceStatus) Equals(other *SourceStatus) bool

Equals returns true if the specified SourceStatus equals this SourceStatus, excluding the LastUpdate timestamp.

type SyncErrorCache added in v1.19.0

type SyncErrorCache struct {
	// contains filtered or unexported fields
}

SyncErrorCache is a collection of sync errors, locked for thread-safe use.

func NewSyncErrorCache added in v1.19.0

func NewSyncErrorCache(conflictHandler conflict.Handler, fightHandler fight.Handler) *SyncErrorCache

NewSyncErrorCache constructs a new SyncErrorCache with shared handlers

func (*SyncErrorCache) AddApplyError added in v1.19.0

func (s *SyncErrorCache) AddApplyError(err status.Error)

AddApplyError adds an apply error to the list of cached apply errors.

func (*SyncErrorCache) ConflictHandler added in v1.19.0

func (s *SyncErrorCache) ConflictHandler() conflict.Handler

ConflictHandler returns the thread-safe handler of resource & management fights

func (*SyncErrorCache) Errors added in v1.19.0

func (s *SyncErrorCache) Errors() status.MultiError

Errors returns the latest known set of errors from the updater and remediator.

func (*SyncErrorCache) FightHandler added in v1.19.0

func (s *SyncErrorCache) FightHandler() fight.Handler

FightHandler returns the thread-safe handler of controller fights

func (*SyncErrorCache) ResetApplyErrors added in v1.19.0

func (s *SyncErrorCache) ResetApplyErrors()

ResetApplyErrors deletes all cached apply errors.

func (*SyncErrorCache) SetValidationErrs added in v1.19.0

func (s *SyncErrorCache) SetValidationErrs(errs status.MultiError)

SetValidationErrs replaces the cached validation errors

func (*SyncErrorCache) SetWatchErrs added in v1.19.0

func (s *SyncErrorCache) SetWatchErrs(errs status.MultiError)

SetWatchErrs replaces the cached watch errors. These come from updating the watches, not watch event errors.

type SyncStatus added in v1.19.0

type SyncStatus struct {
	// Spec represents the source specification that this status corresponds to.
	// The spec is stored in the status so we can distinguish if the status
	// reflects the latest spec or not.
	Spec       SourceSpec
	Syncing    bool
	Commit     string
	Errs       status.MultiError
	LastUpdate metav1.Time
}

SyncStatus represents the status of the sync stage of the pipeline.

func (*SyncStatus) DeepCopy added in v1.19.0

func (ss *SyncStatus) DeepCopy() *SyncStatus

DeepCopy returns a deep copy of the receiver. Warning: Go errors are not copy-able. So this isn't a true deep-copy.

func (*SyncStatus) Equals added in v1.19.0

func (ss *SyncStatus) Equals(other *SyncStatus) bool

Equals returns true if the specified SyncStatus equals this SyncStatus, excluding the LastUpdate timestamp.

type SyncStatusClient added in v1.20.0

type SyncStatusClient interface {
	// GetReconcilerStatus reads the status of the reconciler from the RSync status.
	GetReconcilerStatus(ctx context.Context) (*ReconcilerStatus, status.Error)
	// SetSourceStatus sets the source status and syncing condition on the RSync.
	SetSourceStatus(ctx context.Context, newStatus *SourceStatus) status.Error
	// SetRenderingStatus sets the rendering status and syncing condition on the RSync.
	SetRenderingStatus(ctx context.Context, oldStatus, newStatus *RenderingStatus) status.Error
	// SetSyncStatus sets the sync status and syncing condition on the RSync.
	SetSyncStatus(ctx context.Context, newStatus *SyncStatus) status.Error
	// SetRequiresRenderingAnnotation sets the requires-rendering annotation on the RSync.
	SetRequiresRenderingAnnotation(ctx context.Context, renderingRequired bool) status.Error
	// SetImageToSyncAnnotation sets the source annotations on the RSync.
	SetImageToSyncAnnotation(ctx context.Context, commit string) status.Error
}

SyncStatusClient provides methods to read and write RSync object status.

type Updater added in v1.17.0

type Updater struct {
	// Scope defines the scope of the reconciler, either root or namespaced.
	Scope declared.Scope
	// Resources is a set of resources declared in the source of truth.
	Resources *declared.Resources
	// Remediator is the interface Remediator implements that accepts a new set of
	// declared configuration.
	Remediator remediator.Interface
	// Applier is a bulk client for applying a set of desired resource objects and
	// tracking them in a ResourceGroup inventory.
	Applier applier.Applier
	// SyncErrorCache caches the sync errors from the various reconciler
	// sub-components running in parallel. This allows batching updates and
	// pushing them asynchronously.
	SyncErrorCache *SyncErrorCache
	// contains filtered or unexported fields
}

Updater mutates the most-recently-seen versions of objects stored in memory.

func (*Updater) HasManagementConflict added in v1.19.0

func (u *Updater) HasManagementConflict() bool

HasManagementConflict returns true when conflict errors have been encountered by the Applier or Remediator for at least one currently managed object.

func (*Updater) ManagementConflicts added in v1.19.0

func (u *Updater) ManagementConflicts() []status.ManagementConflictError

ManagementConflicts returns a list of conflict errors encountered by the Applier or Remediator.

func (*Updater) Remediating added in v1.19.0

func (u *Updater) Remediating() bool

Remediating returns true if the Remediator is remediating.

func (*Updater) Update added in v1.17.0

func (u *Updater) Update(ctx context.Context, cache *cacheForCommit) status.MultiError

Update does the following: 1. Pauses the remediator 2. Validates and sterilizes the objects 3. Updates the declared resource objects in memory 4. Applies the objects 5. Updates the remediator watches 6. Restarts the remediator

Any errors returned will be prepended with any known conflict errors from the remediator. This is required to preserve errors that have been reported by another reconciler.

Directories

Path Synopsis
Package events contains a set of Events sent by Publishers.
Package events contains a set of Events sent by Publishers.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL