Documentation
¶
Overview ¶
Package controlmanager watches for workloads that disappeared from under Miabi: an app whose active release container is gone from its node, a service app whose swarm service is gone from its cluster, and a volume whose data is gone from its node. Its scope is restoring what Miabi already decided, where Miabi decided it; it never places or moves a workload.
Lost data is never repaired unattended. A volume that is gone is reported with the backup to restore it from, and every app mounting it is blocked: recreating the volume would hand the app an empty one, and starting a database on that initializes a new, empty cluster over the data it should have kept. So far nothing is acted on at all: findings become timeline events, alerts, metrics and an admin report.
Index ¶
- type AppStore
- type Auditor
- type BlockedApp
- type ClusterManager
- type ConfigStore
- type DatabaseBackupStore
- type DatabaseStore
- type DeploymentStore
- type Finding
- type GatewayEnsurer
- type Mode
- type NodeDocker
- type NodePlacement
- type Recorder
- type Redeployer
- type ReleaseStore
- type Restore
- type ServerStore
- type Service
- func (s *Service) BlockedRedeploy(appID uint) (string, bool)
- func (s *Service) Mode() Mode
- func (s *Service) SetBackups(volumes VolumeBackupStore, databases DatabaseBackupStore)
- func (s *Service) SetEnforcement(r Redeployer, placement NodePlacement, configs ConfigStore, auditor Auditor)
- func (s *Service) SetGateways(servers ServerStore, ensurer GatewayEnsurer)
- func (s *Service) Status() Status
- func (s *Service) Sweep(ctx context.Context) error
- func (s *Service) Tick() error
- type Settings
- type Skip
- type Status
- type VolumeBackupStore
- type VolumeStore
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type AppStore ¶
type AppStore interface {
ListReconcilable() ([]models.Application, error)
}
AppStore lists the apps expected to be running. Satisfied by *repositories.ApplicationRepository.
type Auditor ¶
Auditor records what the control manager did, so an app that restarted itself at 03:00 is explainable without reading logs. Satisfied by *audit.Logger.
type BlockedApp ¶
type BlockedApp struct {
AppID uint `json:"app_id"`
Name string `json:"name"`
Reason string `json:"reason"`
}
BlockedApp is an app whose data is gone. Starting it again would give it an empty volume and call that recovery, so it needs a person to restore the volume first — Miabi cannot put the data back.
type ClusterManager ¶
type ClusterManager interface {
Manager(ctx context.Context, clusterID uint) (docker.Client, error)
}
ClusterManager reaches a cluster's swarm manager. Satisfied by *cluster.Service.
type ConfigStore ¶
ConfigStore resolves a config an app mounts. Satisfied by *repositories.ConfigRepository.
type DatabaseBackupStore ¶
type DatabaseBackupStore interface {
ListByInstance(instanceID uint) ([]models.DatabaseBackupSet, error)
}
DatabaseBackupStore lists an instance's recovery points, which is what a database is restored from. Satisfied by *repositories.DatabaseBackupSetRepository. Optional.
type DatabaseStore ¶
type DatabaseStore interface {
ListAllInstances() ([]models.DatabaseInstance, error)
SetVolumeEngineCreatedAt(id uint, at string) error
}
DatabaseStore does the same for a database instance's data volume. Satisfied by *repositories.DatabaseRepository.
type DeploymentStore ¶
type DeploymentStore interface {
InProgressAppIDs() ([]uint, error)
FindByID(id uint) (*models.Deployment, error)
}
DeploymentStore reports the apps with a deploy under way, and how a redeploy the control manager started is getting on. Satisfied by *repositories.DeploymentRepository.
type Finding ¶
type Finding struct {
drift.Item
WorkspaceID uint `json:"workspace_id"`
ServerID uint `json:"server_id"`
ClusterID uint `json:"cluster_id"`
FirstSeenAt time.Time `json:"first_seen_at"`
LastSeenAt time.Time `json:"last_seen_at"`
// Confirmed is set once enough sweeps agree. Only confirmed findings are recorded as events and counted
// in metrics.
Confirmed bool `json:"confirmed"`
// Attempts, NextAttemptAt and BreakerOpen report what enforcement has tried: how many redeploys, when the
// next one is due under backoff, and whether it has given up and left this to a person.
Attempts int `json:"attempts,omitempty"`
NextAttemptAt *time.Time `json:"next_attempt_at,omitempty"`
BreakerOpen bool `json:"breaker_open,omitempty"`
// Restore is set on lost data: the backup to bring it back from.
Restore *Restore `json:"restore,omitempty"`
// Blocked marks a workload nothing may start again, because the volume holding its data is gone.
Blocked bool `json:"blocked,omitempty"`
// BlockedReason says which volume, for a report that stands on its own.
BlockedReason string `json:"blocked_reason,omitempty"`
// contains filtered or unexported fields
}
Finding is a workload a sweep found missing or replaced. Findings live only in the leading process's memory, so a new leader starts over and confirms again before it reports anything.
type GatewayEnsurer ¶
GatewayEnsurer redeploys a node's own gateway, idempotently. Satisfied by the wiring in internal/routes, which mints the node's gateway token and calls edgegateway.Ensure. It is never used for a gateway Miabi did not deploy: an imported one, including the platform's own on the manager, belongs to whoever installed it.
type NodeDocker ¶
type NodeDocker interface {
For(serverID uint) (docker.Client, error)
ConnectedSince(serverID uint) (time.Time, bool)
}
NodeDocker resolves node engines. Satisfied by *nodes.Clients.
type NodePlacement ¶
NodePlacement reports whether a node may take a workload. Satisfied by *node.Service: a cordoned node is being drained, and recreating containers on it would work against the operator who cordoned it.
type Recorder ¶
type Recorder interface {
Emit(workspaceID, appID uint, t models.AppEventType, sev models.AppEventSeverity, message string, meta map[string]string, actorID *uint)
EmitDatabase(workspaceID, databaseID uint, name string, t models.AppEventType, sev models.AppEventSeverity, message string, meta map[string]string, actorID *uint)
}
Recorder writes timeline events. Satisfied by *events.Service.
type Redeployer ¶
type Redeployer interface {
ReconcileRedeploy(app *models.Application, reason string) (*models.Deployment, error)
}
Redeployer redeploys an app where it already is: the deploy is routed by the app's own ServerID and ClusterID, so nothing is ever placed or moved. Implemented by *application.Service.
type ReleaseStore ¶
ReleaseStore resolves an app's active release. Satisfied by *repositories.ReleaseRepository.
type Restore ¶
type Restore struct {
// From names where the data comes back from: a volume archive, or a database recovery point.
From string `json:"from"`
// Available is false when there is no completed backup to restore, which is the worst case and the
// one worth saying out loud.
Available bool `json:"available"`
BackupID uint `json:"backup_id,omitempty"`
Ref string `json:"ref,omitempty"`
CreatedAt time.Time `json:"created_at,omitzero"`
SizeBytes int64 `json:"size_bytes,omitempty"`
}
Restore is the backup a report of lost data points at. Miabi never restores one by itself: which recovery point to accept, and the data loss between it and now, are a person's call.
type ServerStore ¶
ServerStore lists the nodes, so the gateway that fronts each one can be checked too. Satisfied by *node.Service.
type Service ¶
type Service struct {
// contains filtered or unexported fields
}
Service sweeps the platform for missing workloads.
func New ¶
func New(nodes NodeDocker, clusters ClusterManager, apps AppStore, releases ReleaseStore, deploys DeploymentStore, volumes VolumeStore, databases DatabaseStore, events Recorder, settings Settings) *Service
New builds the control manager. It does nothing until Tick is scheduled.
func (*Service) BlockedRedeploy ¶
BlockedRedeploy reports whether an app must not be started again, and why. Whatever comes to act on findings must ask first: Docker creates a named volume silently when a container starts without one, so redeploying an app whose volume is gone turns data loss into an empty volume nobody notices.
func (*Service) Mode ¶
Mode returns the configured mode. Off and enforce are both deliberate choices, so they are matched exactly; anything else observes, because an unrecognized value must neither switch detection off nor start redeploying things on its own.
func (*Service) SetBackups ¶
func (s *Service) SetBackups(volumes VolumeBackupStore, databases DatabaseBackupStore)
SetBackups wires the backup histories a report of lost data points at. Nil-safe: without them a finding still reports the loss, with no backup to name.
func (*Service) SetEnforcement ¶
func (s *Service) SetEnforcement(r Redeployer, placement NodePlacement, configs ConfigStore, auditor Auditor)
SetEnforcement wires what acting on a finding needs. Without a Redeployer nothing is ever redeployed, whatever the mode says.
func (*Service) SetGateways ¶
func (s *Service) SetGateways(servers ServerStore, ensurer GatewayEnsurer)
SetGateways wires node-gateway watching: the nodes to check, and how to put a Miabi-deployed gateway back. Nil-safe — without them gateways are simply not watched.
func (*Service) Status ¶
Status reports the last sweep. A standby control plane never sweeps, so it has none to report.
type Skip ¶
type Skip struct {
Scope string `json:"scope"` // node | cluster
ID uint `json:"id"`
Reason string `json:"reason"`
}
Skip is a node or cluster a sweep could not observe. Its items are unknown, never missing: an offline node is not an empty one.
type Status ¶
type Status struct {
Mode Mode `json:"mode"`
LastSweepAt *time.Time `json:"last_sweep_at"`
SweepMillis int64 `json:"sweep_ms"`
Findings []Finding `json:"findings"`
Blocked []BlockedApp `json:"blocked"`
Skipped []Skip `json:"skipped"`
}
Status is the control manager's last sweep.
type VolumeBackupStore ¶
type VolumeBackupStore interface {
ListByVolume(volumeID uint) ([]models.VolumeBackup, error)
}
VolumeBackupStore lists a volume's archives, so a report of lost data can name the one to restore. Satisfied by *repositories.VolumeBackupRepository. Optional: without it a finding simply suggests none.