migration

package
v0.8.7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 2, 2026 License: Apache-2.0 Imports: 16 Imported by: 0

Documentation

Index

Constants

View Source
const (
	StateUninitialized = "uninitialized"
	StateInitialized   = "initialized"
	StateLagsOk        = "lags_ok"
	StateFenced        = "fenced"
	StatePromoted      = "promoted"
	StateSwitched      = "switched"
)

FSM State constants

View Source
const (
	EventInitialize  = "initialize"
	EventWaitForLags = "wait_for_lags"
	EventFence       = "fence"
	EventPromote     = "promote"
	EventSwitch      = "switch"
)

FSM Event constants

Variables

This section is empty.

Functions

func BuildClusterLinkConfig added in v0.8.1

func BuildClusterLinkConfig(config *MigrationConfig, apiKey, apiSecret string) clusterlink.Config

BuildClusterLinkConfig assembles a clusterlink.Config from a migration config plus runtime API credentials. Centralized here so the bookend callers in cmd/migration/execute don't duplicate the field layout.

func DisableOffsetSync added in v0.8.1

func DisableOffsetSync(
	ctx context.Context,
	cl clusterlink.Service,
	clCfg clusterlink.Config,
	config *MigrationConfig,
	persist func() error,
) error

DisableOffsetSync runs the pre-execute disable bookend for the --pause-consumer-offset-sync flow.

No-op cases (return nil without contacting the cluster link):

  • intent flag not set
  • migration is already at StateSwitched (re-run after success)
  • PauseConsumerOffsetSyncFlipped is already true (resume after partial failure)

Active path: re-queries the cluster link for drift, calls AlterConfigs to set consumer.offset.sync.enable=false, sets PauseConsumerOffsetSyncFlipped, and persists the marker before returning.

A non-nil error stops execution before the FSM runs (R12).

func RestoreOffsetSync added in v0.8.1

func RestoreOffsetSync(
	_ context.Context,
	cl clusterlink.Service,
	clCfg clusterlink.Config,
	config *MigrationConfig,
	persist func() error,
)

RestoreOffsetSync runs the post-execute restore bookend. Soft-failure semantics: ListConfigs or AlterConfigs failure prints a remediation message to stderr but does NOT propagate an error, because the switchover itself succeeded (R13). The PauseConsumerOffsetSyncFlipped marker stays true on failure so the state file records that a restore is still owed.

Diff-based restore: when MigrationConfig.ClusterLinkConfigs holds an init snapshot, restore queries the cluster link for its current consumer.offset.* state and re-applies snapshot values for keys that the disable bookend cleared (current is missing, empty, or "false"). Keys whose current value looks like a deliberate post-disable operator change (non-empty, non-"false", differs from snapshot) are left alone. When the snapshot is empty (defensive fallback for unforeseen state-file lifecycles), restore performs a single SET consumer.offset.sync.enable=true.

The ctx argument is accepted for symmetry with DisableOffsetSync but the network calls use a fresh background ctx with a per-call timeout. The restore must run even when the parent ctx is already cancelled (e.g. a signal arrived between orchestrator.Execute returning and the bookend firing) — that is the case the soft-fail semantic exists for. Each PUT gets its own timeout budget so a slow endpoint on one key cannot starve the rest of the restore.

func WarnIfPausedOnExecuteFailure added in v0.8.1

func WarnIfPausedOnExecuteFailure(config *MigrationConfig, execErr error)

WarnIfPausedOnExecuteFailure prints a stderr remediation message when orchestrator.Execute returns an error while the cluster link is still in the disabled state (PauseConsumerOffsetSyncFlipped=true). The restore bookend only runs after a successful execute, so without this warning the operator has no signal that the cluster link is paused.

Soft-fail: never returns an error — this is best-effort messaging on top of the underlying execute error.

Types

type ExecutionParams

type ExecutionParams struct {
	LagThreshold     int64
	ClusterApiKey    string
	ClusterApiSecret string
}

ExecutionParams holds runtime parameters needed during migration execution

type MigrationConfig added in v0.8.4

type MigrationConfig struct {
	MigrationId  string `json:"migration_id"`
	CurrentState string `json:"current_state"`

	// Gateway configuration
	KubeConfigPath string `json:"kube_config_path"`

	// Source cluster configuration
	SourceBootstrap string `json:"source_bootstrap"`

	// Destination cluster configuration
	ClusterBootstrap    string   `json:"cluster_bootstrap"`
	ClusterId           string   `json:"cluster_id"`
	ClusterRestEndpoint string   `json:"cluster_rest_endpoint"`
	ClusterLinkName     string   `json:"cluster_link_name"`
	Topics              []string `json:"topics"`

	// Migration runtime data (populated during initialization)
	ClusterLinkTopics  []string          `json:"cluster_link_topics"`
	ClusterLinkConfigs map[string]string `json:"cluster_link_configs"`

	// Operator intent: pause cluster-link consumer offset sync for the duration of execute.
	// PauseConsumerOffsetSync records the operator's choice at init time.
	// PauseConsumerOffsetSyncFlipped is set when kcp has executed the disable AlterConfigs and
	// not yet restored — supports drift detection, idempotent resume, and remediation messaging.
	PauseConsumerOffsetSync        bool `json:"pause_consumer_offset_sync"`
	PauseConsumerOffsetSyncFlipped bool `json:"pause_consumer_offset_sync_flipped"`

	// Gateway CR configuration
	InitialCrName    string `json:"initial_cr_name"`
	K8sNamespace     string `json:"k8s_namespace"`
	InitialCrYAML    []byte `json:"initial_cr_yaml"`
	FencedCrYAML     []byte `json:"fenced_cr_yaml"`
	SwitchoverCrYAML []byte `json:"switchover_cr_yaml"`
}

MigrationConfig holds all domain configuration for a migration This is pure data with no behavior - just fields that get serialized

type MigrationOrchestrator

type MigrationOrchestrator struct {
	// contains filtered or unexported fields
}

MigrationOrchestrator manages the FSM lifecycle and coordinates workflow execution

func NewMigrationOrchestrator

func NewMigrationOrchestrator(
	config *MigrationConfig,
	workflow *MigrationWorkflow,
	migrationState *MigrationState,
	stateFilePath string,
) *MigrationOrchestrator

NewMigrationOrchestrator creates a new migration orchestrator with injected dependencies

func (*MigrationOrchestrator) Execute

func (o *MigrationOrchestrator) Execute(ctx context.Context, lagThreshold int64, clusterApiKey, clusterApiSecret string) error

Execute runs the full migration workflow from the current state

func (*MigrationOrchestrator) Initialize

func (o *MigrationOrchestrator) Initialize(ctx context.Context, clusterApiKey, clusterApiSecret string) error

Initialize triggers the initialization event

type MigrationState added in v0.8.4

type MigrationState struct {
	Migrations   []MigrationConfig  `json:"migrations"`
	KcpBuildInfo types.KcpBuildInfo `json:"kcp_build_info"`
	Timestamp    time.Time          `json:"timestamp"`
}

MigrationState represents the migration state file structure This is a dedicated state file for migration commands (init, execute, list)

func NewMigrationState added in v0.8.4

func NewMigrationState() *MigrationState

NewMigrationState creates a new empty MigrationState with metadata

func NewMigrationStateFromFile added in v0.8.4

func NewMigrationStateFromFile(filePath string) (*MigrationState, error)

NewMigrationStateFromFile loads a MigrationState from a JSON file

func (*MigrationState) GetMigrationById added in v0.8.4

func (ms *MigrationState) GetMigrationById(migrationId string) (*MigrationConfig, error)

GetMigrationById retrieves a migration by its ID

func (*MigrationState) UpsertMigration added in v0.8.4

func (ms *MigrationState) UpsertMigration(config MigrationConfig)

UpsertMigration adds a new migration or updates an existing one by ID

func (*MigrationState) WriteToFile added in v0.8.4

func (ms *MigrationState) WriteToFile(filePath string) error

WriteToFile saves the MigrationState to a JSON file using atomic write

type MigrationWorkflow

type MigrationWorkflow struct {
	// contains filtered or unexported fields
}

func NewMigrationWorkflow

func NewMigrationWorkflow(
	gatewayService gateway.Service,
	clusterLinkService clusterlink.Service,
) *MigrationWorkflow

func NewMigrationWorkflowWithOffsets

func NewMigrationWorkflowWithOffsets(
	gatewayService gateway.Service,
	clusterLinkService clusterlink.Service,
	sourceOffset offset.Provider,
	destinationOffset offset.Provider,
) *MigrationWorkflow

func (*MigrationWorkflow) CheckLags

func (s *MigrationWorkflow) CheckLags(
	ctx context.Context,
	config *MigrationConfig,
	lagThreshold int64,
	clusterApiKey, clusterApiSecret string,
) error

CheckLags polls source and destination offsets until lag is below threshold

func (*MigrationWorkflow) FenceGateway

func (s *MigrationWorkflow) FenceGateway(ctx context.Context, config *MigrationConfig) error

FenceGateway applies the fenced gateway CR YAML to block traffic and waits for the Confluent operator to report the gateway as Ready at the new spec generation. The wait runs without a deadline by default — the operator drives convergence and the user can Ctrl-C if a rollout wedges. An optional per-workflow rolloutTimeout caps the wait when set (via SetRolloutTimeout).

func (*MigrationWorkflow) Initialize

func (s *MigrationWorkflow) Initialize(
	ctx context.Context,
	config *MigrationConfig,
	clusterApiKey, clusterApiSecret string,
) error

func (*MigrationWorkflow) PromoteTopics

func (s *MigrationWorkflow) PromoteTopics(ctx context.Context, config *MigrationConfig, clusterApiKey, clusterApiSecret string) error

PromoteTopics polls offsets and promotes mirror topics that reach zero lag

func (*MigrationWorkflow) SetPromoteBatchSize added in v0.8.7

func (s *MigrationWorkflow) SetPromoteBatchSize(n int)

SetPromoteBatchSize caps how many mirror topics are promoted per batch during PromoteTopics. A value of 0 (the default) means unlimited — all zero-lag topics are promoted at once. When set (>0), each batch is promoted and fully confirmed STOPPED before the next batch is submitted.

func (*MigrationWorkflow) SetRolloutTimeout added in v0.8.1

func (s *MigrationWorkflow) SetRolloutTimeout(d time.Duration)

SetRolloutTimeout sets the deadline applied to gateway-readiness waits. A value of 0 means no deadline.

func (*MigrationWorkflow) SwitchGateway

func (s *MigrationWorkflow) SwitchGateway(ctx context.Context, config *MigrationConfig) error

SwitchGateway applies the switchover gateway CR YAML to point to Confluent Cloud and waits for the operator to report the gateway as Ready. The wait uses the same no-deadline-by-default behavior as FenceGateway.

type WorkflowStep

type WorkflowStep struct {
	Event       string
	Description string
	FromState   string
	ToState     string
	UserMessage string // Emoji-prefixed message shown to the user
}

WorkflowStep defines a single step in the migration workflow

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL