README
¶
Stovepipe internal message-queue contract
Wire payloads for the queues internal to the Stovepipe pipeline. It is internal — used only within the Stovepipe domain — so it lives under stovepipe/core rather than api/ (Bazel visibility keeps it domain-scoped).
Payloads are defined in proto3 (proto/, generated into protopb/) and serialized as protobuf JSON (protojson), so the MySQL-backed queue keeps storing self-describing JSON. The contract package adds only generic glue — Marshal/Unmarshal and the TopicKeys reflection lookup — and owns the TopicKey constants for the stages it carries. Each payload declares the topic key(s) that carry it via the topic_keys proto option (defined in api/base/messagequeue); a contract test round-trips every payload and asserts each topic key is bound to exactly one message.
Stages
- process (
TopicKeyProcess,ProcessRequest) — ingest publishes the minted request id here once it accepts a new head; the process controller reloads theRequestfrom storage and decides the build strategy. Only the id travels: producer and consumer share the store, so messages stay small and redelivery is idempotent. - build (
TopicKeyBuild,BuildRequest) — process publishes the request id here once its build scope (BuildStrategy/BaseURI) is decided; the build controller reloads theRequest, triggers the build, and persists the resultingBuild. Partitioned by request id. - buildsignal (
TopicKeyBuildSignal,BuildSignal) — build publishes the build id here after triggering; buildsignal holds the delivery between polls until the build reaches a terminal status. Partitioned by build id, so each build's poll loop is an independent partition. - buildanalysis (
TopicKeyBuildAnalysis,BuildAnalysis) — identifies a persisted build for issue detection, partitioned by build id. Build publishes a start notice and buildsignal publishes a completion notice with a distinct message id. See build analysis for polling and persistence behavior. - record (
TopicKeyRecord,Record) — buildsignal publishes the request id after persisting the terminal outcome; record reloads the request and records its validation fact. Partitioned by request id.
Issue tracking handoff
IssueTrack binds to TopicKeyIssueTrack and identifies a persisted build and its queue. Notifications are partitioned by queue so both running and final issue-state updates share one serialized path, independently of build-partitioned analysis polling. The message carries identifiers rather than a failure collection; per-build observations are read through BuildIssueStore.List.
Publish only after detection and observation persistence succeed. With finalize unset, the notification permits opening newly detected issues, not resolution by absence. With finalize set, it requests final reconciliation after a successful terminal analysis. Publish that notification even when no failures were detected; terminal Build status alone does not replace the handoff. Publication retries must also follow duplicate observation writes so a crash between persistence and publication does not strand work.
Per-build observations accumulate historical evidence; finalization does not turn them into a replacement snapshot or override the detector's absence semantics. The reconciler must establish the authoritative final failure set and applicable validation scope before resolving issues, reject older requests, and ignore running notifications after finalization. Notification identity must distinguish new observations from finalization while remaining stable on publication retries. Business behavior follows the payload, not a parsed message ID.
Only the handoff contract is defined. No publisher, consumer, topic registration, reconciliation progress, or writes to open issues and lifecycle history are enabled.
Documentation
¶
Overview ¶
Package messagequeue holds Stovepipe's internal message-queue contract: the wire payloads for the pipeline queues Stovepipe owns, defined by the proto files in proto/ and generated into protopb/. The proto is the language-neutral authority; the generated Go types in protopb are the binding for Go callers.
It is internal — used only within the Stovepipe domain — so it lives under stovepipe/core rather than api/. The message types are generated into protopb; this package adds only generic protojson glue (Marshal/Unmarshal) and the topic-key reflection lookup (TopicKeys), so there is no per-message serialization code. Payloads are serialized as protobuf JSON, not binary, so the MySQL-backed queue keeps storing self-describing JSON. The topic key that carries each payload is declared on the message itself via the topic_keys proto option (see api/base/messagequeue).
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Marshal ¶
Marshal serializes any contract message to protojson bytes for the queue payload, keeping the proto field names (snake_case) on the wire.
Types ¶
type BuildAnalysis ¶
type BuildAnalysis = protopb.BuildAnalysis
BuildAnalysis identifies the build whose issue observations are polled.
type BuildRequest ¶
type BuildRequest = protopb.BuildRequest
BuildRequest is the payload process/analyze publishes to the build stage: the request id to build.
type BuildSignal ¶
type BuildSignal = protopb.BuildSignal
BuildSignal is the payload build publishes to the buildsignal stage, and buildsignal re-publishes to itself while polling: the build id to poll.
type FixRequest ¶
type FixRequest = protopb.FixRequest
FixRequest identifies the build whose issues should be considered for repair.
type IssueTrack ¶
type IssueTrack = protopb.IssueTrack
IssueTrack identifies persisted build observations for queue-scoped reconciliation.
type ProcessRequest ¶
type ProcessRequest = protopb.ProcessRequest
ProcessRequest is the payload ingest publishes to the process stage: the minted request id to validate.
type Record ¶
Record is the payload buildsignal publishes to the record stage once a build reaches a terminal status: the build id to record.
type TopicKey ¶
TopicKey is the typed identifier used to look up a queue backend, topic name, and subscription config in a consumer.TopicRegistry. The constants below are the logical topic keys for Stovepipe's internal pipeline stages; they are the same strings each message lists in its topic_keys option.
const ( // TopicKeyProcess carries newly accepted requests from ingest to the process // stage. ingest publishes a ProcessRequest (the request id) here; the process // controller consumes it, reloads the Request, and decides the build strategy. TopicKeyProcess TopicKey = "process" // TopicKeyBuild carries requests whose build scope has been decided from // process/analyze to the build stage. The producer publishes a BuildRequest // (the request id) here; the build controller consumes it, reloads the // Request, and triggers the build. Partitioned by request id. TopicKeyBuild TopicKey = "build" // TopicKeyBuildSignal carries builds to poll from build to the buildsignal // stage, and from buildsignal back to itself between polls. Producers // publish a BuildSignal (the build id) here; the buildsignal controller // consumes it, polls the build runner, and records terminal status. // Partitioned by build id, so each build's poll loop is an independent // partition. TopicKeyBuildSignal TopicKey = "buildsignal" // TopicKeyBuildAnalysis carries build IDs for issue detection, partitioned by build ID. TopicKeyBuildAnalysis TopicKey = "buildanalysis" // TopicKeyIssueTrack carries persisted observation notifications, partitioned by queue. TopicKeyIssueTrack TopicKey = "issuetrack" // TopicKeyFix carries build IDs for repair submission, partitioned by build ID. TopicKeyFix TopicKey = "fix" // TopicKeyRecord carries a request whose build reached a terminal status from // buildsignal to the record stage. The buildsignal controller publishes a // Record (the request id) here once, and only once, a build reaches a // terminal status; non-terminal polls never publish here. Partitioned by // request id. TopicKeyRecord TopicKey = "record" )
Source Files
Directories