Documentation
¶
Overview ¶
Package command provides L4 (DeviceLatent) command queue primitives for unicast device commands with durable ack semantics.
Lifecycle ¶
Commands move through a state machine driven by distinct Queue methods, each triggering exactly one transition:
Enqueue → Pending
Dequeue → Pending → Sent (claim + lease, single atomic step)
Report → Sent → Delivered (optional: device acknowledged receipt)
Ack → any non-terminal → {Succeeded / Failed / Expired / Canceled}
Cancel → any non-terminal → Canceled
Pending: enqueued, awaiting send to device transport. Sent: transmitted to device transport; delivery attempt counter incremented. Delivered: device reported receipt and began execution (optional state). Succeeded: device confirmed successful execution. Failed: permanent failure (device error or retries exhausted). Expired: deadline elapsed before completion. Canceled: explicitly canceled by operator/system.
Devices MAY skip Report and go directly from Sent to Succeeded via Ack(Success); in that case DeliveredAt remains nil, marking the absence of the intermediate event.
Three-tier timeouts ¶
Each entry carries Timeouts:
- ScheduleToSend: max duration from creation (Pending) to Sent.
- SendToComplete: max duration from Sent to a terminal state.
- OverallDeadline: absolute max duration from creation to any terminal state.
The Sweeper worker evaluates these deadlines periodically (via ActiveScanner.ScanActive) and calls Queue.Ack(..., AckTimeout, now) to transition overdue entries to StatusExpired.
Facade ¶
Queue is the service-facing interface. All state transitions are owned by Queue methods; services MUST NOT mutate entry.Status directly.
ActiveScanner is the role-based scan port used by Sweeper and by operational views that list non-terminal entries. It is intentionally distinct from Queue.Dequeue (the claim-with-lease primary consumer path).
QueueRegistrar is an optional Cell-side interface; the runtime consumer lives in runtime/command (queue discovery + dispatch registry).
Testing ¶
The commandtest sub-package provides [commandtest.InMemQueue], an in-memory implementation of Queue + ActiveScanner + Writer suitable for unit tests and examples. Not suitable for production: single-process only.
References ¶
ref: kubernetes/kubernetes client-go util/workqueue/queue.go@master — Add/Get/Done primitive shape influences Enqueue/Dequeue/Ack; workqueue intentionally omits reason semantics, GoCell adds AckReason (mapped in SDK/runtime-equivalent layers, not app code).
ref: nats-io/nats.go jetstream/message.go@main — InProgress maps to Report (Sent→Delivered, distinct network event); Ack/Nak/Term are single-step terminal dispositions; ConsumerInfo (ops view) is separate from Fetch (primary consume). GoCell follows the same split: ActiveScanner vs Dequeue.
ref: temporalio/sdk-go internal/internal_task_handlers.go@master — ScheduledTime, StartedTime, CompletedTime are distinct events on the history; GoCell SentAt/DeliveredAt/CompletedAt mirror this per-event recording (not batched at Ack time).
ref: temporal activity timeouts — ScheduleToStart / StartToClose / Heartbeat / ScheduleToClose four-tier model; GoCell simplifies to three tiers (ScheduleToSend / SendToComplete / OverallDeadline).
Index ¶
- Constants
- func AdvanceCommand(entry *Entry, to Status, now time.Time) error
- func ResetForRetry(entry *Entry) error
- func Transition(from, to Status) error
- type AckReason
- type ActiveScanner
- type AuthzFunc
- type EnqueueOptions
- type Entry
- type ExpiryTransition
- type Queue
- type QueueRegistrar
- type ScanFilter
- type Status
- type Sweeper
- type SweeperOption
- type TimeoutPhase
- type Timeouts
- type Writer
Constants ¶
const DefaultLeaseDuration = 5 * time.Minute
DefaultLeaseDuration is the suggested default lease when callers don't specify one in Queue.Dequeue. Intentionally mirrors kernel/idempotency.DefaultLeaseTTL (5 minutes) so L4 command leases and L2 consumer leases default to the same timeout. Update both if this default changes.
Variables ¶
This section is empty.
Functions ¶
func AdvanceCommand ¶
AdvanceCommand validates a transition and applies the timestamp side effects that the kernel owns. This is a kernel-internal helper used by Queue implementations; service/application code MUST use Queue methods directly (Enqueue/Dequeue/Report/Ack/ExtendLease/Cancel), which internally delegate to AdvanceCommand.
Side effects by target status:
- Sent: sets SentAt, increments Attempt
- Delivered: sets DeliveredAt
- Succeeded/Failed/Expired/Canceled: sets CompletedAt
- Pending: ResetForRetry path; use ResetForRetry directly, not AdvanceCommand
Returns an error if the transition is invalid per statusTransitions, or if a required prerequisite timestamp is missing (e.g., Sent→Delivered without SentAt).
func ResetForRetry ¶
ResetForRetry resets a command back to Pending for retry. This is the only sanctioned way to retry a command — adapters MUST NOT directly mutate Status, SentAt, or other fields to simulate retry.
Allowed source states:
- StatusSent: transport delivery failed, retry from scratch
- StatusFailed: explicit operator/system retry of a failed command
Disallowed source states:
- StatusPending: already pending (caller bug, no-op not allowed)
- StatusDelivered: device ACK'd receipt, resending would duplicate execution
- StatusSucceeded/StatusExpired/StatusCanceled: semantically final
Side effects:
- Status → StatusPending
- SentAt, DeliveredAt, CompletedAt → nil
- Attempt is preserved (tracks total attempts across retries)
- All other fields (ID, DeviceID, Payload, Timeouts, Metadata, CreatedAt) preserved
func Transition ¶
Transition validates a state transition from → to and returns an error if the transition is not allowed. This is a pure validation function; it does NOT mutate any state.
Types ¶
type AckReason ¶
type AckReason uint8
AckReason classifies how a command ended. Every reason maps to a terminal status.
ref: JetStream Ack/Nak/Term disposition — only terminal outcomes.
func (AckReason) TargetStatus ¶
TargetStatus returns the terminal Status produced by this AckReason. Panics on invalid reason — callers must guard with Valid() at boundaries.
type ActiveScanner ¶
type ActiveScanner interface {
// ScanActive returns non-terminal entries matching filter, ordered by
// CreatedAt ascending (FIFO). Implementations MAY impose an adapter-level
// result cap but MUST document it.
ScanActive(ctx context.Context, filter ScanFilter) ([]Entry, error)
// GetCommand returns a single command by ID. Returns ErrCommandNotFound when not found.
GetCommand(ctx context.Context, id string) (*Entry, error)
}
ActiveScanner queries all non-terminal command entries matching a filter. This is the role-based scan port used by Sweeper (timeout detection) and operational views (list pending/in-flight commands). It is deliberately distinct from Queue.Dequeue (the claim-with-lease primary consumer path): scanners do not advance state or issue leases, and scanners are allowed to see Sent/Delivered entries that Dequeue must hide.
ref: Temporal HistoryService active-timer scan + JetStream ConsumerInfo — separate from the primary Fetch/dispatch path; read-only lens over non-terminal entries.
type AuthzFunc ¶
AuthzFunc is a caller-supplied permission hook invoked at the Enqueue boundary. Pass nil to skip (demo/test mode). See docs for T3 DEVICE-ENQUEUE-RBAC.
type EnqueueOptions ¶
type EnqueueOptions struct {
// IdempotencyKey enforces STATE-AWARE ACTIVE uniqueness: at most one command
// with this key may be NON-terminal (Pending/Sent/Delivered) at a time. A
// re-enqueue under the same key is coalesced (no-op, nil error) while a holder
// is non-terminal, and is ADMITTED once the prior holder reaches a terminal
// status (Succeeded/Failed/Expired/Canceled) — so a later retry of the same
// logical command enqueues a fresh entry. Empty = no uniqueness.
//
// This is the River UniqueOpts.ByState / Temporal single-open-execution model:
// the queue (the owner of active-command state) is the correctness authority
// for "at most one in-flight per identity"; producers MUST NOT reconstruct it
// from side state. Uniqueness is keyed on Status, so terminal transitions
// (Ack/Cancel/Sweeper-AckTimeout) release the key automatically — there is no
// separate release call to forget. PG enforces it with a partial unique index
// `WHERE … AND status IN (Pending,Sent,Delivered)`; the in-mem store derives it
// from Status. Both are pinned by the commandtest conformance suite
// (Enqueue/ActiveKeyBlocksAcrossNonTerminal + Enqueue/KeyReleasedOn*).
//
// CAUTION: a command that never reaches terminal would hold its key forever and
// block all retries — so a producer requesting active-uniqueness MUST also give
// the command a terminal-guaranteeing deadline (see runtime/command
// WithActiveUniqueness, which couples the two so the unsafe combination is
// inexpressible).
IdempotencyKey string
// Authz is invoked before any write; return non-nil to reject. Use nil to skip.
Authz AuthzFunc
}
EnqueueOptions configures a single Enqueue call. Lease duration is not set at enqueue time — it is determined by the leaseDuration parameter of Queue.Dequeue. DefaultLeaseDuration is the recommended default for Dequeue callers.
type Entry ¶
type Entry struct {
ID string
DeviceID string
CommandType string // e.g., "reboot", "cert-renew", "config-push"
Payload []byte
Status Status
Metadata map[string]string // extensible key-value pairs
Timeouts Timeouts
// Attempt tracks the current delivery attempt (0 = first attempt).
// Design note: there is no explicit "Retrying" status. Retry is modeled
// as the same Pending→Sent arc with an incremented Attempt counter. This
// avoids a combinatorial explosion of states (Retrying×{Sent,Delivered})
// and keeps the transition table compact. Adapters inspect Attempt to
// decide whether to retry or transition to Failed.
//
// Use ResetForRetry to move a command back to Pending for retry —
// do NOT directly mutate Status/SentAt fields.
Attempt int
CreatedAt time.Time
SentAt *time.Time // set when Status transitions to Sent
DeliveredAt *time.Time // set when device ACKs receipt
CompletedAt *time.Time // set when terminal state reached
}
Entry represents a single L4 command enqueued for device execution.
The lifecycle: Pending → Sent → Delivered → Succeeded/Failed/Expired/Canceled. Queue implementations persist and advance the status through Queue methods.
func NewEntry ¶
func NewEntry(id, deviceID, commandType string, payload []byte, timeouts Timeouts, now time.Time) Entry
NewEntry creates an Entry in Pending status with the given parameters. This is the only sanctioned way to create a command entry; it enforces:
- Status = StatusPending (callers cannot create non-Pending commands)
- SentAt / DeliveredAt / CompletedAt = nil (no phase timestamps at creation)
- Attempt = 0
- CreatedAt = now (caller-provided, not wall-clock)
The now parameter makes the constructor pure and deterministically testable.
ref: Temporal StartWorkflowExecution — callers submit intent + timeouts, status/timestamps are owned by the server. Time is an explicit event parameter, never read from wall clock inside the state machine.
func (*Entry) DeadlineFor ¶
func (e *Entry) DeadlineFor(phase TimeoutPhase) time.Time
DeadlineFor returns the absolute deadline time for the given timeout phase. Returns zero Time if no timeout is configured for that phase, or if the prerequisite timestamp is not yet set (e.g., PhaseSendToComplete before Sent).
This is a pure calculation — adapter code compares the result to time.Now().
func (*Entry) LogValue ¶
LogValue implements slog.LogValuer so that Entry can be logged without leaking arbitrary Payload bytes. Payload is replaced by a byte-count summary "<REDACTED bytes=N>" to prevent accidental sensitive data exposure in logs while still giving operators the payload size for debugging.
func (*Entry) ValidateNew ¶
ValidateNew checks that required fields are present, status is valid, timeouts are non-negative, and creation-time invariants hold. This is a creation-time validator — it enforces that the entry is in its initial state (Pending, no timestamps, Attempt=0). It is NOT suitable for validating an entry that has been advanced through the lifecycle.
Creation-time invariants (enforced by NewEntry):
- Status must be StatusPending
- SentAt, DeliveredAt, CompletedAt must be nil
- Attempt must be 0
These constraints ensure that callers cannot bypass the state machine by constructing an Entry with an arbitrary status or timestamps.
type ExpiryTransition ¶
type ExpiryTransition struct {
CommandID string
From Status
To Status // typically StatusExpired
Reason string // e.g. "phase_overall_deadline"
}
ExpiryTransition describes a single status change recommended by SweepOnce.
func SweepOnce ¶
func SweepOnce(entries []Entry, now time.Time) []ExpiryTransition
SweepOnce is a pure function: given a snapshot of non-terminal entries and the current time, returns the list of transitions adapters should apply. Callers pass entries in ANY non-terminal status (Pending / Sent / Delivered). Terminal entries are silently ignored.
Priority: PhaseOverall > PhaseSendToComplete > PhaseScheduleToSend (first match wins; returned Reason identifies which phase triggered).
type Queue ¶
type Queue interface {
Enqueue(ctx context.Context, entry Entry, opts EnqueueOptions) error
Dequeue(ctx context.Context, targetID string, n int, leaseDuration time.Duration) ([]Entry, error)
Report(ctx context.Context, commandID string, now time.Time) error
Ack(ctx context.Context, commandID string, reason AckReason, now time.Time) error
ExtendLease(ctx context.Context, commandID string, extension time.Duration, now time.Time) error
Cancel(ctx context.Context, commandID string, now time.Time) error
}
Queue is the kernel-level L4 command queue facade. Implementations live in adapters/postgres or in-memory (commandtest package).
Event-driven state machine: each state transition is triggered by a distinct Queue method call, not chained inside Ack:
Enqueue → StatusPending (created) Dequeue → StatusPending → StatusSent (claim + lease, single atomic step) Report → StatusSent → StatusDelivered (device acknowledged receipt) Ack → any non-terminal → terminal (single atomic step) ExtendLease → renew existing lease Cancel → any non-terminal → StatusCanceled (operator action)
Ack is single-step atomic: it does NOT chain Pending→Sent→Delivered→terminal. Callers skipping Report (e.g., acking directly from StatusSent) produce a StatusSent → StatusSucceeded transition with DeliveredAt left nil, indicating the device never reported intermediate delivery.
ref: Temporal RecordActivityTaskHeartbeat + RespondActivityTaskCompleted — distinct RPCs for in-progress signal vs terminal; state transitions are recorded at the event, not batched at ack time. ref: JetStream InProgress vs Ack/Nak/Term — distinct client methods for in-flight continuation vs disposition.
type QueueRegistrar ¶
type QueueRegistrar interface {
RegisterCommandQueue(q Queue)
}
QueueRegistrar is an optional interface a Cell may implement to receive its command.Queue dependency from the runtime; it is the optional injection-direction interface for command queue handles; the cell-side equivalent is collapsed into cell.Registrar. Runtimes SHOULD probe this via type assertion during Cell.Init.
The concrete Queue instance is owned by the composition root (or a runtime/command discovery phase), not the Cell.
The runtime consumer lives in runtime/command.DiscoverQueueRegistrars; see that package for wiring examples. The Sweeper is driven as a reconcile.Reconciler via a kernel/reconcile.Loop.
type ScanFilter ¶
type ScanFilter struct {
// DeviceID, if non-empty, restricts results to one device. Empty = all devices.
DeviceID string
// Statuses, if non-empty, restricts results to the listed statuses.
// Empty slice = all non-terminal statuses (Pending / Sent / Delivered).
// Terminal statuses in Statuses are silently ignored — scanners return
// only non-terminal entries.
Statuses []Status
}
ScanFilter narrows an ActiveScanner.ScanActive call. Empty fields mean "no filter" (all non-terminal entries across all devices).
type Status ¶
type Status uint8
Status represents the lifecycle state of an L4 command entry. L4 commands target devices with long-latency round-trips (IoT command ack, certificate renewal, etc.).
ref: ThingsBoard RPC status model (QUEUED→SENT→DELIVERED→SUCCESSFUL); ref: Temporal Nexus operations (SCHEDULED→STARTED→terminal, three-tier timeouts).
const ( // StatusPending: enqueued, awaiting send to device transport. // // IMPORTANT: iota+1 ensures the zero value (0) is NOT a valid status. // A forgotten/uninitialised Entry.Status will not silently appear valid. StatusPending Status = iota + 1 // = 1 StatusSent // transmitted to device transport StatusDelivered // device ACK'd receipt (not execution) StatusSucceeded // device confirmed successful execution StatusFailed // permanent failure (device error or retries exhausted) StatusExpired // deadline elapsed before completion StatusCanceled // explicitly canceled by operator/system )
func (Status) CanTransitionTo ¶
CanTransitionTo reports whether s can transition to target.
func (Status) IsTerminal ¶
IsTerminal reports whether s is a terminal (final) state. Terminal states: Succeeded, Failed, Expired, Canceled.
func (Status) ValidTransitions ¶
ValidTransitions returns the set of states reachable from s. Returns nil for terminal states.
type Sweeper ¶
type Sweeper struct {
// contains filtered or unexported fields
}
Sweeper is a kernel-level command expiry actor. It implements reconcile.Reconciler: each Reconcile call scans non-terminal commands (via ActiveScanner) and terminates expired entries (via Queue.Ack(AckTimeout)). It is the reconcile runtime's first real consumer — a reconcile.Loop drives it on a TickerTrigger cadence (the per-tick control shell that used to live in runtime/command has been deleted).
Time source: the Sweeper holds the cell's business clock (clk). Reconcile — whose signature carries no "now" — sources the sweep time from clk.Now(), following the reconcile convention that a Reconciler owns its own domain clock. Control-plane scheduling (the tick cadence, startup probe, requeue timers) is NOT the Sweeper's concern: it is owned by reconcile.Loop's sealed real-only clock and the injected TickerTrigger clock. The lower-level SweepTick(ctx, now) primitive still takes an explicit "now", keeping business-plane sweep logic deterministically testable without clock injection.
All non-built fields are unexported — callers cannot populate them via `&command.Sweeper{scanner: ...}` literals. The remaining attack surface is the bare zero-value literal `&command.Sweeper{}`, which produces an instance with nil scanner / queue / clk and would panic on the first sweep. The unexported `built` sentinel + head fail-closed (SweepTick and Reconcile) turns that panic into a clean error: only NewSweeper sets `built=true`, so any literal-zero Sweeper short-circuits before dereferencing scanner / clk.
Filter narrows the scan; zero value (default) means "all devices, all non-terminal statuses". Adapters decide whether ScanFilter is honored efficiently (e.g., indexed by device_id) or scanned in memory.
ref: Temporal HistoryService timer scan loop — role-based periodic scan over active timers; disposition (expire vs retry) is a separate decision. ref: kubernetes-sigs/controller-runtime pkg/reconcile/reconcile.go (Reconciler). ref: kernel/outbox.ConsumerBase.built — same sentinel pattern.
func NewSweeper ¶
func NewSweeper(scanner ActiveScanner, queue Queue, clk clock.Clock, opts ...SweeperOption) (*Sweeper, error)
NewSweeper constructs a Sweeper. scanner and queue are required interface dependencies; nil triggers fail-fast per validation.IsNilInterface (typed-nil safety, ≈ OUTBOX-SERVICE-01 pattern). clk is the mandatory business clock — Reconcile sources the sweep "now" from clk.Now(); a nil clock is a programmer error and panics (clock.MustHaveClock) per CLOCK-POSITIONAL-INJECTION-01.
Example:
sweeper, err := command.NewSweeper(scanner, queue, clk,
command.WithSweeperFilter(command.ScanFilter{DeviceID: "dev-1"}),
)
if err != nil {
return fmt.Errorf("sweeper: %w", err)
}
func (*Sweeper) Reconcile ¶
Reconcile implements reconcile.Reconciler. The Sweeper is a bulk scan-all actor: req.EntityID is ignored. A TickerTrigger emits Request{} (empty EntityID) as the "resync-all" pulse, and the Sweeper has no per-entity path — every call performs one full SweepTick at clk.Now().
Error semantics: any sweep error (scan failure or aggregated Ack failures) is returned as-is. The reconcile.Loop classifies a bare error as transient and backs off + retries on the next tick; nothing the Sweeper produces is a reconcile.PermanentError (a transient DB/broker fault is always retryable). On success Reconcile returns the zero Result{} (RequeueAfter == 0), so the Loop re-observes at its configured tick interval — the TickerTrigger cadence is the real driver, making the sweep level-triggered.
The head guard mirrors SweepTick's: a nil receiver or zero-value &command.Sweeper{} (built==false) fails closed with an error BEFORE dereferencing s.clk, instead of panicking. A Sweeper wired through reconcile.New(...).Build() can never be in that state (Build rejects a typed-nil reconciler), so this is defense-in-depth.
func (*Sweeper) SweepTick ¶
SweepTick executes a single sweep: read non-terminal entries via ScanActive, compute expirations via SweepOnce, terminate each expired entry via Queue.Ack(AckTimeout).
Error semantics:
- ScanActive error: immediately short-circuits the tick and returns the scan error. No Ack calls are made (partial-tick risk is avoided).
- Ack errors (scan succeeded): all Ack errors from the full transition set are aggregated via errors.Join and returned. A single Ack failure does not abort remaining Ack calls.
The caller (control plane) is responsible for logging and metrics.
Two head guards close the literal-construction attack surface:
- `if s == nil` — `var s *Sweeper; s.SweepTick(ctx, now)` would otherwise panic on the s.built read below; the guard returns a typed error.
- `if !s.built` — closes the zero-value `&command.Sweeper{}` literal bypass (only NewSweeper sets built=true).
func (*Sweeper) Validate ¶
Validate reports whether the Sweeper is ready to run, with NO side effects (no scan, no Ack). It is the readiness gate reconcile.Loop invokes before Start (the reconcilerReadinessChecker seam) so a misconstructed sweeper fails startup (bootstrap rolls back) instead of starting and erroring on every tick (review P2-1).
It catches exactly the cases SweepTick's head guards catch — nil receiver and the zero-value &command.Sweeper{} literal (built==false) — plus a defensive nil scanner/queue check (NewSweeper already guarantees these when built, but Validate is the single readiness contract so it states the full invariant).
type SweeperOption ¶
type SweeperOption func(*Sweeper)
SweeperOption configures optional Sweeper fields. Pass into NewSweeper as variadic args. The required positional dependencies (scanner, queue) are NOT exposed as options so the type system enforces their presence.
func WithSweeperFilter ¶
func WithSweeperFilter(f ScanFilter) SweeperOption
WithSweeperFilter narrows the scan to a specific device or status set. Default: zero-value ScanFilter (all devices, all non-terminal statuses).
type TimeoutPhase ¶
type TimeoutPhase uint8
TimeoutPhase identifies which phase of the command lifecycle a deadline applies to.
const ( // PhaseScheduleToSend: max duration from creation (Pending) to Sent. PhaseScheduleToSend TimeoutPhase = iota + 1 // PhaseSendToComplete: max duration from Sent to a terminal state. PhaseSendToComplete // PhaseOverall: absolute max duration from creation to terminal state. PhaseOverall )
type Timeouts ¶
type Timeouts struct {
// ScheduleToSend: max wait from creation to device transport delivery.
ScheduleToSend time.Duration
// SendToComplete: max wait from Sent to terminal state (Succeeded/Failed).
SendToComplete time.Duration
// OverallDeadline: absolute max from creation to any terminal state.
OverallDeadline time.Duration
}
Timeouts configures per-phase deadlines for an L4 command. Zero duration means no timeout for that phase.
Adapter responsibility: the kernel defines timeout configuration and pure deadline calculation (via Entry.DeadlineFor). The adapter MUST run a periodic sweeper (e.g., every 30s–60s) that queries non-terminal commands, computes DeadlineFor for each active phase, and calls Queue.Ack(..., AckTimeout, now) when now exceeds the deadline.
ref: Temporal Nexus operations — ScheduleToCloseTimeout, ScheduleToStartTimeout, StartToCloseTimeout. GoCell simplifies to three tiers.
type Writer ¶
type Writer interface {
// WriteCommand persists a command entry atomically with business state.
// Consistency: L4 (DeviceLatent).
WriteCommand(ctx context.Context, entry Entry) error
}
Writer persists L4 command entries within a transaction. Primarily used by adapter internals and test fixtures to seed entries bypassing Enqueue validation; service/application code MUST use Queue.Enqueue instead.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package commandtest provides shared conformance assertions for command.Queue + command.ActiveScanner implementations.
|
Package commandtest provides shared conformance assertions for command.Queue + command.ActiveScanner implementations. |