Documentation
¶
Index ¶
- Constants
- func BuildClusterLinkConfig(config *MigrationConfig, apiKey, apiSecret string) clusterlink.Config
- func DisableOffsetSync(ctx context.Context, cl clusterlink.Service, clCfg clusterlink.Config, ...) error
- func RestoreOffsetSync(_ context.Context, cl clusterlink.Service, clCfg clusterlink.Config, ...)
- func WarnIfPausedOnExecuteFailure(config *MigrationConfig, execErr error)
- type ExecutionParams
- type MigrationConfig
- type MigrationOrchestrator
- type MigrationState
- type MigrationWorkflow
- func (s *MigrationWorkflow) CheckLags(ctx context.Context, config *MigrationConfig, lagThreshold int64, ...) error
- func (s *MigrationWorkflow) FenceGateway(ctx context.Context, config *MigrationConfig) error
- func (s *MigrationWorkflow) Initialize(ctx context.Context, config *MigrationConfig, ...) error
- func (s *MigrationWorkflow) PromoteTopics(ctx context.Context, config *MigrationConfig, ...) error
- func (s *MigrationWorkflow) SetPromoteBatchSize(n int)
- func (s *MigrationWorkflow) SetRolloutTimeout(d time.Duration)
- func (s *MigrationWorkflow) SwitchGateway(ctx context.Context, config *MigrationConfig) error
- type WorkflowStep
Constants ¶
const ( StateUninitialized = "uninitialized" StateInitialized = "initialized" StateLagsOk = "lags_ok" StateFenced = "fenced" StatePromoted = "promoted" StateSwitched = "switched" )
FSM State constants
const ( EventInitialize = "initialize" EventWaitForLags = "wait_for_lags" EventFence = "fence" EventPromote = "promote" EventSwitch = "switch" )
FSM Event constants
Variables ¶
This section is empty.
Functions ¶
func BuildClusterLinkConfig ¶ added in v0.8.1
func BuildClusterLinkConfig(config *MigrationConfig, apiKey, apiSecret string) clusterlink.Config
BuildClusterLinkConfig assembles a clusterlink.Config from a migration config plus runtime API credentials. Centralized here so the bookend callers in cmd/migration/execute don't duplicate the field layout.
func DisableOffsetSync ¶ added in v0.8.1
func DisableOffsetSync( ctx context.Context, cl clusterlink.Service, clCfg clusterlink.Config, config *MigrationConfig, persist func() error, ) error
DisableOffsetSync runs the pre-execute disable bookend for the --pause-consumer-offset-sync flow.
No-op cases (return nil without contacting the cluster link):
- intent flag not set
- migration is already at StateSwitched (re-run after success)
- PauseConsumerOffsetSyncFlipped is already true (resume after partial failure)
Active path: re-queries the cluster link for drift, calls AlterConfigs to set consumer.offset.sync.enable=false, sets PauseConsumerOffsetSyncFlipped, and persists the marker before returning.
A non-nil error stops execution before the FSM runs (R12).
func RestoreOffsetSync ¶ added in v0.8.1
func RestoreOffsetSync( _ context.Context, cl clusterlink.Service, clCfg clusterlink.Config, config *MigrationConfig, persist func() error, )
RestoreOffsetSync runs the post-execute restore bookend. Soft-failure semantics: ListConfigs or AlterConfigs failure prints a remediation message to stderr but does NOT propagate an error, because the switchover itself succeeded (R13). The PauseConsumerOffsetSyncFlipped marker stays true on failure so the state file records that a restore is still owed.
Diff-based restore: when MigrationConfig.ClusterLinkConfigs holds an init snapshot, restore queries the cluster link for its current consumer.offset.* state and re-applies snapshot values for keys that the disable bookend cleared (current is missing, empty, or "false"). Keys whose current value looks like a deliberate post-disable operator change (non-empty, non-"false", differs from snapshot) are left alone. When the snapshot is empty (defensive fallback for unforeseen state-file lifecycles), restore performs a single SET consumer.offset.sync.enable=true.
The ctx argument is accepted for symmetry with DisableOffsetSync but the network calls use a fresh background ctx with a per-call timeout. The restore must run even when the parent ctx is already cancelled (e.g. a signal arrived between orchestrator.Execute returning and the bookend firing) — that is the case the soft-fail semantic exists for. Each PUT gets its own timeout budget so a slow endpoint on one key cannot starve the rest of the restore.
func WarnIfPausedOnExecuteFailure ¶ added in v0.8.1
func WarnIfPausedOnExecuteFailure(config *MigrationConfig, execErr error)
WarnIfPausedOnExecuteFailure prints a stderr remediation message when orchestrator.Execute returns an error while the cluster link is still in the disabled state (PauseConsumerOffsetSyncFlipped=true). The restore bookend only runs after a successful execute, so without this warning the operator has no signal that the cluster link is paused.
Soft-fail: never returns an error — this is best-effort messaging on top of the underlying execute error.
Types ¶
type ExecutionParams ¶
ExecutionParams holds runtime parameters needed during migration execution
type MigrationConfig ¶ added in v0.8.4
type MigrationConfig struct {
MigrationId string `json:"migration_id"`
CurrentState string `json:"current_state"`
// Gateway configuration
KubeConfigPath string `json:"kube_config_path"`
// Source cluster configuration
SourceBootstrap string `json:"source_bootstrap"`
// Destination cluster configuration
ClusterBootstrap string `json:"cluster_bootstrap"`
ClusterId string `json:"cluster_id"`
ClusterRestEndpoint string `json:"cluster_rest_endpoint"`
ClusterLinkName string `json:"cluster_link_name"`
Topics []string `json:"topics"`
// Migration runtime data (populated during initialization)
ClusterLinkTopics []string `json:"cluster_link_topics"`
ClusterLinkConfigs map[string]string `json:"cluster_link_configs"`
// Operator intent: pause cluster-link consumer offset sync for the duration of execute.
// PauseConsumerOffsetSync records the operator's choice at init time.
// PauseConsumerOffsetSyncFlipped is set when kcp has executed the disable AlterConfigs and
// not yet restored — supports drift detection, idempotent resume, and remediation messaging.
PauseConsumerOffsetSync bool `json:"pause_consumer_offset_sync"`
PauseConsumerOffsetSyncFlipped bool `json:"pause_consumer_offset_sync_flipped"`
// Gateway CR configuration
InitialCrName string `json:"initial_cr_name"`
K8sNamespace string `json:"k8s_namespace"`
InitialCrYAML []byte `json:"initial_cr_yaml"`
FencedCrYAML []byte `json:"fenced_cr_yaml"`
SwitchoverCrYAML []byte `json:"switchover_cr_yaml"`
}
MigrationConfig holds all domain configuration for a migration This is pure data with no behavior - just fields that get serialized
type MigrationOrchestrator ¶
type MigrationOrchestrator struct {
// contains filtered or unexported fields
}
MigrationOrchestrator manages the FSM lifecycle and coordinates workflow execution
func NewMigrationOrchestrator ¶
func NewMigrationOrchestrator( config *MigrationConfig, workflow *MigrationWorkflow, migrationState *MigrationState, stateFilePath string, ) *MigrationOrchestrator
NewMigrationOrchestrator creates a new migration orchestrator with injected dependencies
func (*MigrationOrchestrator) Execute ¶
func (o *MigrationOrchestrator) Execute(ctx context.Context, lagThreshold int64, clusterApiKey, clusterApiSecret string) error
Execute runs the full migration workflow from the current state
func (*MigrationOrchestrator) Initialize ¶
func (o *MigrationOrchestrator) Initialize(ctx context.Context, clusterApiKey, clusterApiSecret string) error
Initialize triggers the initialization event
type MigrationState ¶ added in v0.8.4
type MigrationState struct {
Migrations []MigrationConfig `json:"migrations"`
KcpBuildInfo types.KcpBuildInfo `json:"kcp_build_info"`
Timestamp time.Time `json:"timestamp"`
}
MigrationState represents the migration state file structure This is a dedicated state file for migration commands (init, execute, list)
func NewMigrationState ¶ added in v0.8.4
func NewMigrationState() *MigrationState
NewMigrationState creates a new empty MigrationState with metadata
func NewMigrationStateFromFile ¶ added in v0.8.4
func NewMigrationStateFromFile(filePath string) (*MigrationState, error)
NewMigrationStateFromFile loads a MigrationState from a JSON file
func (*MigrationState) GetMigrationById ¶ added in v0.8.4
func (ms *MigrationState) GetMigrationById(migrationId string) (*MigrationConfig, error)
GetMigrationById retrieves a migration by its ID
func (*MigrationState) UpsertMigration ¶ added in v0.8.4
func (ms *MigrationState) UpsertMigration(config MigrationConfig)
UpsertMigration adds a new migration or updates an existing one by ID
func (*MigrationState) WriteToFile ¶ added in v0.8.4
func (ms *MigrationState) WriteToFile(filePath string) error
WriteToFile saves the MigrationState to a JSON file using atomic write
type MigrationWorkflow ¶
type MigrationWorkflow struct {
// contains filtered or unexported fields
}
func NewMigrationWorkflow ¶
func NewMigrationWorkflow( gatewayService gateway.Service, clusterLinkService clusterlink.Service, ) *MigrationWorkflow
func NewMigrationWorkflowWithOffsets ¶
func NewMigrationWorkflowWithOffsets( gatewayService gateway.Service, clusterLinkService clusterlink.Service, sourceOffset offset.Provider, destinationOffset offset.Provider, ) *MigrationWorkflow
func (*MigrationWorkflow) CheckLags ¶
func (s *MigrationWorkflow) CheckLags( ctx context.Context, config *MigrationConfig, lagThreshold int64, clusterApiKey, clusterApiSecret string, ) error
CheckLags polls source and destination offsets until lag is below threshold
func (*MigrationWorkflow) FenceGateway ¶
func (s *MigrationWorkflow) FenceGateway(ctx context.Context, config *MigrationConfig) error
FenceGateway applies the fenced gateway CR YAML to block traffic and waits for the Confluent operator to report the gateway as Ready at the new spec generation. The wait runs without a deadline by default — the operator drives convergence and the user can Ctrl-C if a rollout wedges. An optional per-workflow rolloutTimeout caps the wait when set (via SetRolloutTimeout).
func (*MigrationWorkflow) Initialize ¶
func (s *MigrationWorkflow) Initialize( ctx context.Context, config *MigrationConfig, clusterApiKey, clusterApiSecret string, ) error
func (*MigrationWorkflow) PromoteTopics ¶
func (s *MigrationWorkflow) PromoteTopics(ctx context.Context, config *MigrationConfig, clusterApiKey, clusterApiSecret string) error
PromoteTopics polls offsets and promotes mirror topics that reach zero lag
func (*MigrationWorkflow) SetPromoteBatchSize ¶ added in v0.8.7
func (s *MigrationWorkflow) SetPromoteBatchSize(n int)
SetPromoteBatchSize caps how many mirror topics are promoted per batch during PromoteTopics. A value of 0 (the default) means unlimited — all zero-lag topics are promoted at once. When set (>0), each batch is promoted and fully confirmed STOPPED before the next batch is submitted.
func (*MigrationWorkflow) SetRolloutTimeout ¶ added in v0.8.1
func (s *MigrationWorkflow) SetRolloutTimeout(d time.Duration)
SetRolloutTimeout sets the deadline applied to gateway-readiness waits. A value of 0 means no deadline.
func (*MigrationWorkflow) SwitchGateway ¶
func (s *MigrationWorkflow) SwitchGateway(ctx context.Context, config *MigrationConfig) error
SwitchGateway applies the switchover gateway CR YAML to point to Confluent Cloud and waits for the operator to report the gateway as Ready. The wait uses the same no-deadline-by-default behavior as FenceGateway.