README
¶
Stovepipe internal message-queue contract
Wire payloads for the queues internal to the Stovepipe pipeline. It is internal — used only within the Stovepipe domain — so it lives under stovepipe/core rather than api/ (Bazel visibility keeps it domain-scoped).
Payloads are defined in proto3 (proto/, generated into protopb/) and serialized as protobuf JSON (protojson), so the MySQL-backed queue keeps storing self-describing JSON. The contract package adds only generic glue — Marshal/Unmarshal and the TopicKeys reflection lookup — and owns the TopicKey constants for the stages it carries. Each payload declares the topic key(s) that carry it via the topic_keys proto option (defined in api/base/messagequeue); a contract test round-trips every payload and asserts each topic key is bound to exactly one message.
Stages
- process (
TopicKeyProcess,ProcessRequest) — ingest publishes the minted request id here once it accepts a new head; the process controller reloads theRequestfrom storage and decides the build strategy. Only the id travels: producer and consumer share the store, so messages stay small and redelivery is idempotent. - build (
TopicKeyBuild,BuildRequest) — process/analyze publishes the request id here once its build scope (BuildStrategy/BaseURI) is decided; the build controller reloads theRequest, triggers the build, and persists the resultingBuild. Partitioned by request id. - buildsignal (
TopicKeyBuildSignal,BuildSignal) — build publishes the build id here after triggering; buildsignal re-publishes to itself between polls until the build reaches a terminal status. Partitioned by build id, so each build's poll loop is an independent partition. See doc/rfc/stovepipe/steps/build.md and buildsignal.md.
See doc/rfc/messagequeue-contract.md for the contract conventions and api/runway/messagequeue for the external reference example.
Documentation
¶
Overview ¶
Package messagequeue holds Stovepipe's internal message-queue contract: the wire payloads for the pipeline queues Stovepipe owns, defined by the proto files in proto/ and generated into protopb/. The proto is the language-neutral authority; the generated Go types in protopb are the binding for Go callers.
It is internal — used only within the Stovepipe domain — so it lives under stovepipe/core rather than api/. The message types are generated into protopb; this package adds only generic protojson glue (Marshal/Unmarshal) and the topic-key reflection lookup (TopicKeys), so there is no per-message serialization code. Payloads are serialized as protobuf JSON, not binary, so the MySQL-backed queue keeps storing self-describing JSON. The topic key that carries each payload is declared on the message itself via the topic_keys proto option (see api/base/messagequeue).
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Marshal ¶
Marshal serializes any contract message to protojson bytes for the queue payload, keeping the proto field names (snake_case) on the wire.
Types ¶
type BuildRequest ¶
type BuildRequest = protopb.BuildRequest
BuildRequest is the payload process/analyze publishes to the build stage: the request id to build.
type BuildSignal ¶
type BuildSignal = protopb.BuildSignal
BuildSignal is the payload build publishes to the buildsignal stage, and buildsignal re-publishes to itself while polling: the build id to poll.
type ProcessRequest ¶
type ProcessRequest = protopb.ProcessRequest
ProcessRequest is the payload ingest publishes to the process stage: the minted request id to validate.
type Record ¶
Record is the payload buildsignal publishes to the record stage once a build reaches a terminal status: the build id to record.
type TopicKey ¶
TopicKey is the typed identifier used to look up a queue backend, topic name, and subscription config in a consumer.TopicRegistry. The constants below are the logical topic keys for Stovepipe's internal pipeline stages; they are the same strings each message lists in its topic_keys option.
const ( // TopicKeyProcess carries newly accepted requests from ingest to the process // stage. ingest publishes a ProcessRequest (the request id) here; the process // controller consumes it, reloads the Request, and decides the build strategy. TopicKeyProcess TopicKey = "process" // TopicKeyBuild carries requests whose build scope has been decided from // process/analyze to the build stage. The producer publishes a BuildRequest // (the request id) here; the build controller consumes it, reloads the // Request, and triggers the build. Partitioned by request id. TopicKeyBuild TopicKey = "build" // TopicKeyBuildSignal carries builds to poll from build to the buildsignal // stage, and from buildsignal back to itself between polls. Producers // publish a BuildSignal (the build id) here; the buildsignal controller // consumes it, polls the build runner, and records terminal status. // Partitioned by build id, so each build's poll loop is an independent // partition. TopicKeyBuildSignal TopicKey = "buildsignal" // TopicKeyRecord carries a request whose build reached a terminal status from // buildsignal to the record stage. The buildsignal controller publishes a // Record (the request id) here once, and only once, a build reaches a // terminal status; non-terminal polls never publish here. Partitioned by // request id. TopicKeyRecord TopicKey = "record" )
Source Files
Directories