processmanager

package
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 30, 2026 License: EPL-2.0 Imports: 37 Imported by: 0

Documentation

Index

Constants

View Source
const (
	// DrainQuiesceComplete means grace/force finished and verify passed.
	DrainQuiesceComplete = "complete"
	// DrainQuiesceTimedOut means the runtime was unreachable or drain could not finish.
	DrainQuiesceTimedOut = "timedOut"
	// DrainQuiesceVerifyFailed means leftovers remained after SIGKILL.
	DrainQuiesceVerifyFailed = "verifyFailed"
)
View Source
const (
	// IntervalInMinutes is the sliding window used to detect restart loops.
	IntervalInMinutes = 10
	// AbnormalNumberOfRestarts is how many recorded crash/recreate events
	// inside the window mark a microservice stuck.
	AbnormalNumberOfRestarts = 10
)
View Source
const (
	ContainerManagerModuleName = "Container Manager"
)
View Source
const ExecStartGateTimeout = 15 // seconds — use with time.Second in callers
View Source
const LabeledWorkloadStopTimeoutSec int64 = 30

LabeledWorkloadStopTimeoutSec is the CRI StopContainer SIGTERM budget for labeled workloads (longer than the runtime 10s default so flocked VOLUME mounts can exit cleanly). Stops stay concurrent and must fit the 120s data-plane stop budget.

View Source
const (
	ProcessManagerModuleName = "Process Manager"
)

Variables

View Source
var (
	ErrExecSessionNotFound      = errors.New("exec session not found")
	ErrExecSessionOwnerMismatch = errors.New("exec session owner mismatch")
	ErrExecSessionDuplicate     = errors.New("exec session already registered")
	ErrExecStartTimeout         = errors.New("exec start timeout")
)
View Source
var ErrControlPlaneNotFound = errors.New("control plane deployment not found")

Functions

func AdvanceRunningErrorClear added in v1.1.0

func AdvanceRunningErrorClear()

AdvanceRunningErrorClear clears the current error text once a workload has stayed running for the grace period. It uses the status already stored and does not inspect the runtime. The usage sampler can call this on its own loop.

func BeginDataPlaneDrainHold added in v1.1.0

func BeginDataPlaneDrainHold() error

BeginDataPlaneDrainHold tells the control plane to pause reconcile until the data-plane socket is back. The file is the signal; the API socket is not required.

func BeginQuiesceForDataPlaneDrain added in v1.0.1

func BeginQuiesceForDataPlaneDrain()

BeginQuiesceForDataPlaneDrain pauses reconcile for coordinated data-plane MS drain. Reconcile resumes via TryResumeReconcileAfterDataPlaneEngineReady once the engine is ready.

func ConfigureControlPlaneRestartForTest

func ConfigureControlPlaneRestartForTest(eng engine.ContainerEngine, engineName string, recreateFn func(*models.ControlPlaneDeployment, bool, int64) error)

ConfigureControlPlaneRestartForTest wires engine metadata and optional recreate hook for facade restart tests.

func ConfigureEngineForTest

func ConfigureEngineForTest(eng engine.ContainerEngine)

ConfigureEngineForTest wires a test engine into the process manager singleton.

func DataPlaneDrainHoldActive added in v1.1.0

func DataPlaneDrainHoldActive() bool

DataPlaneDrainHoldActive reports whether a data-plane stop is in progress.

func DrainLabeledWorkloads added in v1.1.0

func DrainLabeledWorkloads(ctx context.Context, runtime LabeledWorkloadRuntime, timeout time.Duration, stopTimeoutSec int64) error

DrainLabeledWorkloads issues concurrent CRI StopContainer calls for labeled running workloads and returns when the set is empty or the budget expires.

func DrainRuntimeForDataPlaneStop added in v1.0.1

func DrainRuntimeForDataPlaneStop(timeout time.Duration) error

DrainRuntimeForDataPlaneStop drains labeled ioFog microservice containers before the embedded data-plane containerd stops. Control-plane attach-only stop must not call this path (see supervisor shouldDrainRuntimeOnControlStop).

Reconcile is quiesced for the drain and stays quiesced until the runtime engine socket is healthy again (see TryResumeReconcileAfterDataPlaneEngineReady).

func EndDataPlaneDrainHold added in v1.1.0

func EndDataPlaneDrainHold() error

EndDataPlaneDrainHold clears the reconcile pause. Call it when drain did not stop containerd, or after the data plane is ready again.

func ErrIfDataPlaneDrainHold added in v1.1.0

func ErrIfDataPlaneDrainHold() error

ErrIfDataPlaneDrainHold returns ErrReconcilePaused while a data-plane stop is in progress. Create must return that error instead of calling the runtime.

func IsControllerWorkload

func IsControllerWorkload(labels map[string]string, containerID, image string, dep *models.ControlPlaneDeployment) bool

IsControllerWorkload reports whether labels, container ID, and image match the singleton control plane deployment record (full controller identity).

func IsEdgeletSelfContainer

func IsEdgeletSelfContainer(labels map[string]string, containerImage string) bool

IsEdgeletSelfContainer reports whether the container is the edgelet daemon running inside its own image (EDGELET_DAEMON=container). Set EDGELET_IMAGE on the host container to the image ref used at deploy time so the watchdog can skip self-removal.

func IsLabeledWorkloadDrainTimeout added in v1.1.0

func IsLabeledWorkloadDrainTimeout(err error) bool

IsLabeledWorkloadDrainTimeout reports whether drain stopped because the budget expired.

func IsQuiesced

func IsQuiesced() bool

IsQuiesced reports whether reconcile is paused (e.g. pending engine restart).

func IsQuiescedForDataPlaneDrain added in v1.0.1

func IsQuiescedForDataPlaneDrain() bool

IsQuiescedForDataPlaneDrain reports whether reconcile is held for a data-plane stop until the runtime engine socket becomes healthy again.

func ListVolumePathHolders added in v1.1.0

func ListVolumePathHolders(paths []string) ([]int, error)

ListVolumePathHolders returns host PIDs that have an open file on or under the given directories. It does not delete volume files.

func ListVolumeTreeHolders added in v1.1.0

func ListVolumeTreeHolders(diskDirectory string) ([]int, error)

ListVolumeTreeHolders returns host PIDs that still have an open file under volumes/data or volumes/shared. It does not delete volume files.

func LocalWorkloadsOutOfScope added in v1.1.0

func LocalWorkloadsOutOfScope(watchdogEnabled bool) bool

LocalWorkloadsOutOfScope reports whether local microservices and local models are excluded because watchdog is enabled.

func MarkWorkloadsUsingCatalogItem added in v1.1.0

func MarkWorkloadsUsingCatalogItem(name, source string)

MarkWorkloadsUsingCatalogItem marks workloads whose catalog lists name. A local item wakes local workloads. A fleet item wakes controller workloads.

func ObserveDataPlaneDrainHold added in v1.1.0

func ObserveDataPlaneDrainHold()

ObserveDataPlaneDrainHold pauses reconcile while the hold file exists and resumes only after that file is gone and the containerd socket is present.

func PathHoldsVolumeTree added in v1.1.0

func PathHoldsVolumeTree(target string, trees []string) bool

PathHoldsVolumeTree reports whether target is the tree root or a path under it.

func PersistentVolumeTrees added in v1.1.0

func PersistentVolumeTrees(diskDirectory string) []string

PersistentVolumeTrees returns the host trees that exclusive-lock VOLUME workloads open: volumes/data (private) and volumes/shared.

func RegisterExecSessionForTest

func RegisterExecSessionForTest(rec *ExecSessionRecord) error

RegisterExecSessionForTest inserts a registry row for handler/integration tests.

func ResetExecRegistryForTest

func ResetExecRegistryForTest()

ResetExecRegistryForTest clears the exec session registry on the singleton.

func ResetExecStartGateTimeoutForTest

func ResetExecStartGateTimeoutForTest()

ResetExecStartGateTimeoutForTest restores the production sync start gate duration.

func ResetProcessManagerEngineForTest

func ResetProcessManagerEngineForTest()

ResetProcessManagerEngineForTest clears test engine wiring on the singleton so later tests that expect an uninitialized process manager are not polluted.

func RuntimeEngineAvailable added in v1.0.1

func RuntimeEngineAvailable() bool

RuntimeEngineAvailable reports whether the process manager has a wired container engine.

func SetDataPlaneDrainHoldPath added in v1.1.0

func SetDataPlaneDrainHoldPath(path string)

SetDataPlaneDrainHoldPath overrides the hold file (tests only).

func SetExecStartGateTimeoutForTest

func SetExecStartGateTimeoutForTest(d time.Duration)

SetExecStartGateTimeoutForTest shortens the sync start gate for unit tests.

func SetInstanceForTest added in v1.0.1

func SetInstanceForTest(pm *ProcessManager) func()

SetInstanceForTest replaces the ProcessManager singleton for tests and returns a restore func.

func SetQuiesced

func SetQuiesced(v bool)

SetQuiesced blocks or resumes reconcile scheduling.

func TryResumeReconcileAfterDataPlaneEngineReady added in v1.0.1

func TryResumeReconcileAfterDataPlaneEngineReady()

TryResumeReconcileAfterDataPlaneEngineReady clears data-plane drain quiesce after the runtime engine is healthy again (runtime split attach path). A drain-active file keeps the pause until the data plane clears it.

Types

type ContainerManager

type ContainerManager struct {
	// contains filtered or unexported fields
}

ContainerManager manages container operations via a ContainerEngine.

func NewContainerManager

func NewContainerManager(eng engine.ContainerEngine, microserviceManager MicroserviceManagerInterface, engineName string) *ContainerManager

NewContainerManager creates a new ContainerManager

func (*ContainerManager) AddContainer

func (cm *ContainerManager) AddContainer(ctx context.Context, ms *models.Microservice) error

AddContainer creates and starts a container for a microservice. It holds IsUpdating=true for the duration so the reconciliation loop treats this microservice as "in-flight" and does not enqueue a second ADD task.

func (*ContainerManager) GetContainerForMicroservice

func (cm *ContainerManager) GetContainerForMicroservice(microserviceUUID string) (*engine.Container, error)

GetContainerForMicroservice returns the container for a microservice, using DB-first lookup when available (iofog engine) with label-based fallback.

func (*ContainerManager) KillContainerByMicroserviceUUID

func (cm *ContainerManager) KillContainerByMicroserviceUUID(ctx context.Context, microserviceUUID string) error

KillContainerByMicroserviceUUID forcefully stops a container by microservice UUID.

func (*ContainerManager) RecreateContainer

func (cm *ContainerManager) RecreateContainer(ctx context.Context, ms *models.Microservice, opts RecreateOptions) (string, error)

RecreateContainer removes an existing workload container when present, then creates and starts a new one from the microservice spec.

func (*ContainerManager) RemoveContainerByID

func (cm *ContainerManager) RemoveContainerByID(ctx context.Context, containerID string, withCleanup bool, removeImage bool) error

RemoveContainerByID removes a container by concrete engine-assigned container ID. This is primarily used by watchdog unknown-container cleanup where there is no guaranteed iofog microservice UUID mapping.

func (*ContainerManager) RemoveContainerByMicroserviceUUID

func (cm *ContainerManager) RemoveContainerByMicroserviceUUID(ctx context.Context, microserviceUUID string, withCleanup bool, removeImage bool) error

RemoveContainerByMicroserviceUUID removes a container by microservice UUID. withCleanup is passed to engine.RemoveContainer. Persistent VOLUME data is bind-mounted and is not deleted when the container is removed. removeImage controls whether the container image is also removed after container deletion — set true for normal lifecycle deletions, false for the deprovision path.

func (*ContainerManager) RemoveContainerRuntimeForEngineSwitch

func (cm *ContainerManager) RemoveContainerRuntimeForEngineSwitch(ctx context.Context, containerID string) error

RemoveContainerRuntimeForEngineSwitch removes a workload container without deleting deploy spec rows.

func (*ContainerManager) StartContainerByMicroserviceUUID

func (cm *ContainerManager) StartContainerByMicroserviceUUID(ctx context.Context, microserviceUUID string) error

StartContainerByMicroserviceUUID starts a container by microservice UUID.

func (*ContainerManager) StopContainerByMicroserviceUUID

func (cm *ContainerManager) StopContainerByMicroserviceUUID(ctx context.Context, microserviceUUID string) error

StopContainerByMicroserviceUUID stops a container by microservice UUID

func (*ContainerManager) UpdateContainer

func (cm *ContainerManager) UpdateContainer(ctx context.Context, ms *models.Microservice, withCleanup bool) error

UpdateContainer updates a container for a microservice

type ContainerTask

type ContainerTask struct {
	Action           TaskAction
	MicroserviceUUID string
	OperationID      string // correlation id for structured logging (set on first execute)
	Retries          int
	Command          []string
	ExecID           string
}

ContainerTask represents a task to be executed on a container

func NewContainerTask

func NewContainerTask(action TaskAction, microserviceUUID string) *ContainerTask

NewContainerTask creates a new ContainerTask

func NewContainerTaskWithExec

func NewContainerTaskWithExec(action TaskAction, microserviceUUID string, command []string) *ContainerTask

NewContainerTaskWithExec creates a new ContainerTask for exec operations

func (*ContainerTask) IncrementRetries

func (t *ContainerTask) IncrementRetries()

IncrementRetries increments the retry count

type DrainQuiesceResult added in v1.1.0

type DrainQuiesceResult struct {
	Status string
}

DrainQuiesceResult is the programmatic outcome of grace → force → verify.

func QuiesceLabeledWorkloads added in v1.1.0

func QuiesceLabeledWorkloads(ctx context.Context, runtime LabeledWorkloadRuntime, timeout time.Duration, deps QuiesceDeps) DrainQuiesceResult

QuiesceLabeledWorkloads runs CRI SIGTERM, deletes labeled containers and pod sandboxes, then SIGKILLs volume-tree holders that survived that delete. An exited container or a remaining sandbox is not clear: force-kill and shim reap must not run while containerd still owns the task.

func (DrainQuiesceResult) Verified added in v1.1.0

func (r DrainQuiesceResult) Verified() bool

Verified reports whether drain+verify completed and shim reap may proceed.

type ExecOwner

type ExecOwner string

ExecOwner identifies who owns an interactive exec session.

const (
	ExecOwnerController ExecOwner = "controller"
	ExecOwnerLocal      ExecOwner = "local"
)

type ExecSessionCallbackInterface

type ExecSessionCallbackInterface interface {
	GetStdinReader() io.Reader
	GetStdoutWriter() io.Writer
	GetStderrWriter() io.Writer
	OnComplete()
	OnError(err error)
	IsRunning() bool
}

ExecSessionCallbackInterface defines the interface for exec session callbacks

type ExecSessionRecord

type ExecSessionRecord struct {
	SessionID     string
	MSUUID        string
	ContainerID   string
	RuntimeExecID string
	Owner         ExecOwner
	Started       bool
}

ExecSessionRecord tracks one interactive exec session in the ProcessManager registry.

type ExecSessionRegistry

type ExecSessionRegistry struct {
	// contains filtered or unexported fields
}

ExecSessionRegistry tracks interactive exec sessions keyed by wire sessionId.

func NewExecSessionRegistry

func NewExecSessionRegistry() *ExecSessionRegistry

NewExecSessionRegistry returns an empty exec session registry.

func (*ExecSessionRegistry) Get

func (r *ExecSessionRegistry) Get(sessionID string) (*ExecSessionRecord, bool)

Get returns a copy of the session record.

func (*ExecSessionRegistry) ListControllerSessionsForMS

func (r *ExecSessionRegistry) ListControllerSessionsForMS(msUUID string) []ExecSessionRecord

ListControllerSessionsForMS returns controller-owned sessions for one microservice.

func (*ExecSessionRegistry) ListInteractiveForMS

func (r *ExecSessionRegistry) ListInteractiveForMS(msUUID string) []ExecSessionRecord

ListInteractiveForMS returns all controller and local interactive sessions for one microservice.

func (*ExecSessionRegistry) MarkStarted

func (r *ExecSessionRegistry) MarkStarted(sessionID string)

MarkStarted marks a session as successfully started.

func (*ExecSessionRegistry) Register

func (r *ExecSessionRegistry) Register(rec *ExecSessionRecord) error

Register inserts a pending session record. sessionId must be unique.

func (*ExecSessionRegistry) Release

func (r *ExecSessionRegistry) Release(sessionID string, owner ExecOwner) (*ExecSessionRecord, error)

Release removes a session when owner matches.

func (*ExecSessionRegistry) Remove

func (r *ExecSessionRegistry) Remove(sessionID string)

Remove deletes a session regardless of owner (internal cleanup).

func (*ExecSessionRegistry) RuntimeExecIDsForContainer

func (r *ExecSessionRegistry) RuntimeExecIDsForContainer(containerID string) map[string]struct{}

RuntimeExecIDsForContainer returns registered runtime exec ids for a container (for orphan sweep).

func (*ExecSessionRegistry) SetRuntimeExecID

func (r *ExecSessionRegistry) SetRuntimeExecID(sessionID, engineExecID string)

SetRuntimeExecID updates the engine-native exec handle after CreateExecSession. Edgelet engines use the pre-create hint; docker/podman replace it with the daemon-assigned id.

type LabeledWorkloadRuntime added in v1.1.0

type LabeledWorkloadRuntime interface {
	ListLabeledRunning(ctx context.Context) ([]string, error)
	StopContainer(ctx context.Context, id string, timeoutSec int64) error
}

LabeledWorkloadRuntime lists and stops labeled Edgelet workloads while the container runtime is still up. Control-plane API access is not required.

type LocalDeployProgressCallback

type LocalDeployProgressCallback func(stage string, message string)

LocalDeployProgressCallback reports local deployment runtime stage transitions.

type MicroserviceManagerInterface

type MicroserviceManagerInterface interface {
	GetLatestMicroservices() []*models.Microservice
	GetCurrentMicroservices() []*models.Microservice
	FindLatestMicroserviceByUUID(uuid string) *models.Microservice
	GetRegistry(id int) *models.Registry
	SetCurrentMicroservices(microservices []*models.Microservice)
}

MicroserviceManagerInterface defines the interface for microservice management This will be implemented by FieldAgent or a dedicated MicroserviceManager

type ProcessManager

type ProcessManager struct {
	// contains filtered or unexported fields
}

ProcessManager manages container lifecycle via a ContainerEngine.

func GetInstance

func GetInstance() *ProcessManager

GetInstance returns the singleton ProcessManager instance

func NewProcessManagerForHandlerTest added in v1.0.1

func NewProcessManagerForHandlerTest(eng engine.ContainerEngine) *ProcessManager

NewProcessManagerForHandlerTest returns a minimal process manager wired for EdgeletAPI handler tests.

func (*ProcessManager) CleanupForEngineSwitch

func (pm *ProcessManager) CleanupForEngineSwitch(ctx context.Context) error

CleanupForEngineSwitch stops/removes all managed workload containers and clears ephemeral DB state.

func (*ProcessManager) ConfiguredMicroserviceImages added in v1.1.0

func (pm *ProcessManager) ConfiguredMicroserviceImages() []string

ConfiguredMicroserviceImages returns image names that scheduled prune must keep: controller-managed desired microservices plus local-deployed workloads that are not gone. Stopped or restarting local deploys stay in the keep-set so their images are not deleted before reconcile recreates the container.

func (*ProcessManager) CreateControllerExecSession

func (pm *ProcessManager) CreateControllerExecSession(sessionID, msUUID string, command []string, callback ExecSessionCallbackInterface) error

CreateControllerExecSession registers a controller exec session, starts the shell, and blocks until running or timeout.

func (*ProcessManager) CreateExecSession

func (pm *ProcessManager) CreateExecSession(microserviceUUID string, command []string, callback ExecSessionCallbackInterface) (string, error)

CreateExecSession creates and starts an exec session for a microservice. Legacy field-agent path — returns the runtime exec id (not wire sessionId). Prefer CreateControllerExecSession / CreateLocalExecSession for Plan 23 multi-session exec.

func (*ProcessManager) CreateLocalExecSession

func (pm *ProcessManager) CreateLocalExecSession(localSessionID, msUUID string, command []string, callback ExecSessionCallbackInterface) error

CreateLocalExecSession registers a local EdgeletAPI exec session with the local runtime id pattern. Interactive shells wait for the process to be running before POST returns; one-shot commands (ms exec <id> -- cmd) start I/O asynchronously local exec path.

func (*ProcessManager) DeleteControlPlane

func (pm *ProcessManager) DeleteControlPlane() error

DeleteControlPlane stops the controller container, removes volumes, and deletes the singleton row.

func (*ProcessManager) DrainRuntimeForDataPlaneShutdown added in v1.0.1

func (pm *ProcessManager) DrainRuntimeForDataPlaneShutdown(timeout time.Duration) error

DrainRuntimeForDataPlaneShutdown tears down labeled workload containers from the runtime before the embedded data-plane stops. Deploy specs in SQLite are preserved; ephemeral runtime refs are cleared so reconcile recreates sandboxes after the socket returns (BR-24-I1, BR-24-I3).

func (*ProcessManager) DrainRuntimeForShutdown

func (pm *ProcessManager) DrainRuntimeForShutdown(timeout time.Duration) error

DrainRuntimeForShutdown best-effort drains runtime tasks before manager teardown. It is used during daemon shutdown so containerd can stop without lingering shims.

func (*ProcessManager) GetContainerByID

func (pm *ProcessManager) GetContainerByID(containerID string) (*engine.Container, error)

GetContainerByID resolves one container by concrete container id.

func (*ProcessManager) GetContainerByIDPrefix

func (pm *ProcessManager) GetContainerByIDPrefix(prefix string) (*engine.Container, []string, error)

GetContainerByIDPrefix resolves a container by ID prefix. Returns the matched container and a list of matching IDs for ambiguity reporting.

func (*ProcessManager) GetContainerForMicroservice

func (pm *ProcessManager) GetContainerForMicroservice(microserviceUUID string) (*engine.Container, error)

GetContainerForMicroservice resolves runtime container by microservice UUID.

func (*ProcessManager) GetContainerSandboxID added in v1.1.0

func (pm *ProcessManager) GetContainerSandboxID(containerID string) (string, error)

GetContainerSandboxID returns the pause / sandbox id for a workload container.

func (*ProcessManager) GetExecSessionExitCode

func (pm *ProcessManager) GetExecSessionExitCode(execID string) (int, error)

GetExecSessionExitCode returns the exit status for completed exec sessions.

func (*ProcessManager) GetExecSessionStatus

func (pm *ProcessManager) GetExecSessionStatus(execID string) (any, error)

GetExecSessionStatus reports whether the exec process identified by execID is running.

func (*ProcessManager) GetLatestMicroservices

func (pm *ProcessManager) GetLatestMicroservices() []*models.Microservice

GetLatestMicroservices returns the latest microservices from the microservice manager

func (*ProcessManager) GetMicroserviceUUIDForContainer

func (pm *ProcessManager) GetMicroserviceUUIDForContainer(container engine.Container) string

GetMicroserviceUUIDForContainer derives a microservice selector from a container.

func (*ProcessManager) GetModuleIndex

func (pm *ProcessManager) GetModuleIndex() int

GetModuleIndex returns the module index

func (*ProcessManager) GetName

func (pm *ProcessManager) GetName() string

GetName returns the module name

func (*ProcessManager) GetSession

func (pm *ProcessManager) GetSession(sessionID string) (*ExecSessionRecord, bool)

GetSession returns the registry record for attach/status lookups.

func (*ProcessManager) InspectContainerRaw

func (pm *ProcessManager) InspectContainerRaw(containerID string) (map[string]any, error)

InspectContainerRaw returns full engine-native inspect payload for a container.

func (*ProcessManager) KillMicroservice

func (pm *ProcessManager) KillMicroservice(microserviceUUID string) error

KillMicroservice sends a forceful kill signal to a runtime microservice container.

func (*ProcessManager) LaunchLocalMicroservice

func (pm *ProcessManager) LaunchLocalMicroservice(ms *models.Microservice, registry *models.Registry, hostIP string) (string, error)

LaunchLocalMicroservice creates and starts a locally deployed microservice.

func (*ProcessManager) LaunchLocalMicroserviceWithProgress

func (pm *ProcessManager) LaunchLocalMicroserviceWithProgress(ms *models.Microservice, registry *models.Registry, hostIP string, progress LocalDeployProgressCallback) (string, error)

LaunchLocalMicroserviceWithProgress creates and starts a locally deployed microservice while reporting stage transitions.

func (*ProcessManager) ListAllContainers added in v1.1.0

func (pm *ProcessManager) ListAllContainers() ([]engine.Container, error)

ListAllContainers returns every listed container from the active engine.

func (*ProcessManager) ListControllerSessionsForMS

func (pm *ProcessManager) ListControllerSessionsForMS(msUUID string) []ExecSessionRecord

ListControllerSessionsForMS lists controller-owned sessions for status reporting (Plan 23-3).

func (*ProcessManager) ListImages

func (pm *ProcessManager) ListImages() ([]engine.ImageInfo, error)

ListImages returns local runtime images from the active engine.

func (*ProcessManager) LoadImageFromPath

func (pm *ProcessManager) LoadImageFromPath(path string) ([]engine.LoadedImage, error)

LoadImageFromPath imports an image archive from daemon-local path.

func (*ProcessManager) MarkReconcile added in v1.1.0

func (pm *ProcessManager) MarkReconcile(uuid string)

MarkReconcile records a workload for the next pass and wakes the monitor. Marking the same UUID again is safe.

func (*ProcessManager) NoteRuntimeEventStreamHealthy added in v1.1.0

func (pm *ProcessManager) NoteRuntimeEventStreamHealthy()

NoteRuntimeEventStreamHealthy clears the down flag after a successful subscribe.

func (*ProcessManager) NoteRuntimeEventStreamSettled added in v1.1.0

func (pm *ProcessManager) NoteRuntimeEventStreamSettled()

NoteRuntimeEventStreamSettled allows the next outage to log again.

func (*ProcessManager) NoteRuntimeEventStreamUnavailable added in v1.1.0

func (pm *ProcessManager) NoteRuntimeEventStreamUnavailable() bool

NoteRuntimeEventStreamUnavailable marks the stream down. It reports whether this is the start of the outage so the caller logs once, not on every tick.

func (*ProcessManager) PruneContainers

func (pm *ProcessManager) PruneContainers() (*engine.ContainerPruneReport, error)

PruneContainers prunes stopped/orphaned containers.

func (*ProcessManager) PruneDanglingImages

func (pm *ProcessManager) PruneDanglingImages() (*engine.ImagePruneReport, error)

PruneDanglingImages prunes dangling images.

func (*ProcessManager) PruneVolumes

func (pm *ProcessManager) PruneVolumes() (*engine.VolumePruneReport, error)

PruneVolumes prunes unused/orphaned volume artifacts.

func (*ProcessManager) PullImage

func (pm *ProcessManager) PullImage(imageRef string, registry *models.Registry, platform string) error

PullImage pulls an image using optional registry credentials and platform selector.

func (*ProcessManager) PullImageWithProgress

func (pm *ProcessManager) PullImageWithProgress(imageRef string, registry *models.Registry, platform string, onProgress func(float32)) error

PullImageWithProgress pulls an image and reports progress percent when available.

func (*ProcessManager) ReconcilePending added in v1.1.0

func (pm *ProcessManager) ReconcilePending(uuid string) bool

ReconcilePending reports whether a workload is waiting for the next pass.

func (*ProcessManager) ReconcileRuntimeEvent added in v1.1.0

func (pm *ProcessManager) ReconcileRuntimeEvent(uuid, kind string)

ReconcileRuntimeEvent reconciles one workload from a container lifecycle event. Exit, OOM, and delete use the existing reconcile path. Start refreshes status and IP and does not compare the container spec. A restart wait that is already scheduled keeps its parked start; this wake does not enqueue that start.

func (*ProcessManager) ReleaseExecSession

func (pm *ProcessManager) ReleaseExecSession(sessionID string, owner ExecOwner) error

ReleaseExecSession stops the runtime exec and removes the registry entry when owner matches.

func (*ProcessManager) RemoveContainerByContainerID

func (pm *ProcessManager) RemoveContainerByContainerID(containerID string) error

RemoveContainerByContainerID removes a runtime container by concrete container id.

func (*ProcessManager) RemoveImage

func (pm *ProcessManager) RemoveImage(selector string) error

RemoveImage removes an image by ID or name reference.

func (*ProcessManager) RemoveMicroservice

func (pm *ProcessManager) RemoveMicroservice(microserviceUUID string) error

RemoveMicroservice removes a runtime microservice container.

func (*ProcessManager) ResizeExecSession

func (pm *ProcessManager) ResizeExecSession(execID string, cols, rows uint32) error

ResizeExecSession resizes a running tty exec session.

func (*ProcessManager) RestartControlPlaneDeployment

func (pm *ProcessManager) RestartControlPlaneDeployment(item *models.ControlPlaneDeployment, pullImage bool) error

RestartControlPlaneDeployment bounces the singleton control plane container without deleting the system_control_plane row or re-registering with Pot.

func (*ProcessManager) RestartMicroservice

func (pm *ProcessManager) RestartMicroservice(microserviceUUID string) error

RestartMicroservice performs stop then start for a runtime microservice container.

func (*ProcessManager) RuntimeEventStreamDegraded added in v1.1.0

func (pm *ProcessManager) RuntimeEventStreamDegraded() bool

RuntimeEventStreamDegraded reports whether container events are currently down.

func (*ProcessManager) SetControlPlanePullOnRecreate

func (pm *ProcessManager) SetControlPlanePullOnRecreate(pull bool)

SetControlPlanePullOnRecreate requests an image pull on the next control plane generation recreate.

func (*ProcessManager) SetEngine

func (pm *ProcessManager) SetEngine(eng engine.ContainerEngine, engineName string)

SetEngine swaps the active container engine without restarting the process manager.

func (*ProcessManager) SetWatchdogLocalKnowledgeCallback added in v1.1.0

func (pm *ProcessManager) SetWatchdogLocalKnowledgeCallback(fn func())

SetWatchdogLocalKnowledgeCallback runs when watchdog is on so local Knowledge rows and trees are removed (fleet-desired Knowledge is kept).

func (*ProcessManager) SetWatchdogLocalModelsCallback added in v1.1.0

func (pm *ProcessManager) SetWatchdogLocalModelsCallback(fn func())

SetWatchdogLocalModelsCallback runs when watchdog is on so local Model rows and trees are removed (fleet-desired models are kept).

func (*ProcessManager) Start

func (pm *ProcessManager) Start(eng engine.ContainerEngine, microserviceManager MicroserviceManagerInterface) error

Start starts the ProcessManager with a given container engine.

func (*ProcessManager) StartMicroservice

func (pm *ProcessManager) StartMicroservice(microserviceUUID string) error

StartMicroservice starts a runtime microservice container.

func (*ProcessManager) Stop

func (pm *ProcessManager) Stop() error

Stop stops the ProcessManager

func (*ProcessManager) StopAllInteractiveForMicroservice

func (pm *ProcessManager) StopAllInteractiveForMicroservice(msUUID string)

StopAllInteractiveForMicroservice stops all controller and local interactive exec sessions for one MS. Healthcheck exec paths are excluded — they never enter the registry.

func (*ProcessManager) StopExecSession

func (pm *ProcessManager) StopExecSession(_ string, execID string) error

StopExecSession kills and deregisters the exec process in the engine. Called when the controller closes the WebSocket so the exec ID can be reused.

func (*ProcessManager) StopMicroservice

func (pm *ProcessManager) StopMicroservice(microserviceUUID string) error

StopMicroservice stops a runtime microservice container.

func (*ProcessManager) StopRunningMicroservices

func (pm *ProcessManager) StopRunningMicroservices(iofogUUID string, withCleanup bool) error

StopRunningMicroservices stops all running microservices matching the iofogUuid. When withCleanup=true (deprovision flow), containers are also removed along with their volumes.

func (*ProcessManager) StopRunningMicroservicesWithScope

func (pm *ProcessManager) StopRunningMicroservicesWithScope(iofogUUID string, withCleanup bool, includeLocal bool) error

StopRunningMicroservicesWithScope stops running microservices matching iofogUuid. includeLocal=false preserves local-scope workloads during deprovision cleanup.

func (*ProcessManager) StreamMicroserviceLogs

func (pm *ProcessManager) StreamMicroserviceLogs(microserviceUUID string, cfg *engine.TailConfig, handler engine.LogTailHandler) error

StreamMicroserviceLogs streams logs for one runtime microservice using the provided tail handler.

func (*ProcessManager) SyncApplyControlPlaneDeployment

func (pm *ProcessManager) SyncApplyControlPlaneDeployment(item *models.ControlPlaneDeployment, progress LocalDeployProgressCallback) error

SyncApplyControlPlaneDeployment persists and synchronously launches or recreates the controller container.

func (*ProcessManager) TailMicroserviceLogs

func (pm *ProcessManager) TailMicroserviceLogs(microserviceUUID string, cfg *engine.TailConfig) ([]map[string]any, error)

TailMicroserviceLogs returns bounded logs for one runtime microservice.

func (*ProcessManager) Update

func (pm *ProcessManager) Update()

Update notifies the ProcessManager of a full reload. Startup and reconnect use this so every workload is reconciled once.

type QuiesceDeps added in v1.1.0

type QuiesceDeps struct {
	DiskDirectory  string
	StopTimeoutSec int64
	ListPIDs       func(ctx context.Context) ([]int, error)
	ListTasks      func(ctx context.Context) ([]string, error)
	VolumeHolders  func(diskDirectory string) ([]int, error)
	// Release deletes labeled containers and pod sandboxes after SIGTERM.
	// Force-kill runs only when Release leaves no CRI object behind.
	Release func(ctx context.Context) error
	Kill    func(pid int) error
	SelfPID int
	// ProtectPID is true for processes that must survive force-kill: the
	// data-plane parent, the live containerd child, and shims still attached
	// to that child's socket.
	ProtectPID    func(pid int) bool
	AfterKillWait time.Duration
	// RemainingStop is the CRI stop budget still left on the quiesce deadline.
	// Release reads it so a retry does not start a fresh stop timeout.
	RemainingStop *atomic.Int64
	// Poll is the delay between CRI delete retries. Zero uses the drain poll interval.
	Poll time.Duration
	// Sleep and Now override the clock in tests.
	Sleep func(time.Duration)
	Now   func() time.Time
}

QuiesceDeps customizes leftover PID discovery and signals (tests inject fakes).

type RecreateOptions

type RecreateOptions struct {
	PullImage   bool
	WithCleanup bool
	RemoveImage bool
}

RecreateOptions controls remove+create lifecycle recreation.

type RestartStuckChecker

type RestartStuckChecker struct {
	// contains filtered or unexported fields
}

RestartStuckChecker tracks real crash/recreate events and consecutive-failure backoff.

func GetRestartStuckChecker

func GetRestartStuckChecker() *RestartStuckChecker

GetRestartStuckChecker returns the singleton RestartStuckChecker instance

func (*RestartStuckChecker) ArmFailure added in v1.1.0

func (r *RestartStuckChecker) ArmFailure(microserviceUUID string) time.Duration

ArmFailure increments consecutive failures and returns the delay until the next attempt.

func (*RestartStuckChecker) Clear

func (r *RestartStuckChecker) Clear(microserviceUUID string)

Clear removes all tracking data for a microservice (useful for cleanup)

func (*RestartStuckChecker) ConsecutiveFailures added in v1.1.0

func (r *RestartStuckChecker) ConsecutiveFailures(microserviceUUID string) int

ConsecutiveFailures is the in-memory consecutive failure count n.

func (*RestartStuckChecker) ConsumeCatalogReadySkip added in v1.1.0

func (r *RestartStuckChecker) ConsumeCatalogReadySkip(microserviceUUID string) bool

ConsumeCatalogReadySkip returns true once when catalog transitions from waiting to Ready.

func (*RestartStuckChecker) IsStuck

func (r *RestartStuckChecker) IsStuck(microserviceUUID string) bool

IsStuck reports whether the microservice has enough recorded crash/recreate events in the sliding window. It does not record a new event.

func (*RestartStuckChecker) IsStuckInContainerCreation

func (r *RestartStuckChecker) IsStuckInContainerCreation(microserviceUUID string) bool

IsStuckInContainerCreation reports whether start/create attempts in the sliding window exceeded the threshold. It does not record a new attempt.

func (*RestartStuckChecker) MarkCatalogWaiting added in v1.1.0

func (r *RestartStuckChecker) MarkCatalogWaiting(microserviceUUID string)

MarkCatalogWaiting notes that start is blocked on catalog becoming Ready.

func (*RestartStuckChecker) ObserveRunning added in v1.1.0

func (r *RestartStuckChecker) ObserveRunning(microserviceUUID string)

ObserveRunning clears the in-episode crash flag so the next crash is a new event.

func (*RestartStuckChecker) ParkTask added in v1.1.0

func (r *RestartStuckChecker) ParkTask(task *ContainerTask)

ParkTask holds a task retry until DelayUntil elapses.

func (*RestartStuckChecker) Ready added in v1.1.0

func (r *RestartStuckChecker) Ready(microserviceUUID string, skip recreateSkip) bool

Ready reports whether a crash-driven recreate may be enqueued now.

func (*RestartStuckChecker) RecordFailedStart added in v1.1.0

func (r *RestartStuckChecker) RecordFailedStart(microserviceUUID string)

RecordFailedStart records a failed start or container-create attempt.

func (*RestartStuckChecker) RecordIfNew added in v1.1.0

func (r *RestartStuckChecker) RecordIfNew(microserviceUUID string) bool

RecordIfNew records a crash/recreate event once per episode. A later ObserveRunning (or rebuild / stable-running reset) starts a new episode.

func (*RestartStuckChecker) RemainingDelay added in v1.1.0

func (r *RestartStuckChecker) RemainingDelay(microserviceUUID string) time.Duration

RemainingDelay is how long crash-driven recreate should still wait.

func (*RestartStuckChecker) ResetAfterRebuild added in v1.1.0

func (r *RestartStuckChecker) ResetAfterRebuild(microserviceUUID string)

ResetAfterRebuild clears stuck timestamps and backoff so the operator retry can proceed.

func (*RestartStuckChecker) ResetBackoff added in v1.1.0

func (r *RestartStuckChecker) ResetBackoff(microserviceUUID string)

ResetBackoff clears consecutive-failure delay after stable running or rebuild.

func (*RestartStuckChecker) TakeDueTasks added in v1.1.0

func (r *RestartStuckChecker) TakeDueTasks() []*ContainerTask

TakeDueTasks returns parked retries whose wait has elapsed.

type TaskAction

type TaskAction string

TaskAction represents the type of container task

const (
	TaskActionAdd               TaskAction = "ADD"
	TaskActionUpdate            TaskAction = "UPDATE"
	TaskActionRemove            TaskAction = "REMOVE"
	TaskActionRemoveWithCleanup TaskAction = "REMOVE_WITH_CLEAN_UP"
	TaskActionStop              TaskAction = "STOP"
	TaskActionCreateExec        TaskAction = "CREATE_EXEC"
	TaskActionGetExecStatus     TaskAction = "GET_EXEC_STATUS"
	TaskActionKillExec          TaskAction = "KILL_EXEC"
)

type TaskQueue

type TaskQueue struct {
	// contains filtered or unexported fields
}

TaskQueue manages container tasks in a thread-safe queue

func NewTaskQueue

func NewTaskQueue(bufferSize int) *TaskQueue

NewTaskQueue creates a new TaskQueue with the specified buffer size

func (*TaskQueue) Add

func (q *TaskQueue) Add(task *ContainerTask)

Add adds a task to the queue. Silently drops the task if the queue is full (capacity = 100); this should never happen under normal operating conditions.

func (*TaskQueue) Close

func (q *TaskQueue) Close()

Close closes the task queue channel.

func (*TaskQueue) Get

func (q *TaskQueue) Get(ctx context.Context) (*ContainerTask, bool)

Get blocks until a task is available or the context is canceled. Returns (task, true) on success and (nil, false) when the context is done.

func (*TaskQueue) Size

func (q *TaskQueue) Size() int

Size returns the current number of tasks in the queue.

func (*TaskQueue) TryGet

func (q *TaskQueue) TryGet() (*ContainerTask, bool)

TryGet attempts to retrieve a task from the queue without blocking.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL