README
¶
SubmitQueue
SubmitQueue service layout:
gateway/— Gateway service: entry point forPing,Land,Cancel, request-summary, request-history, and queue-listing RPCs. It also consumes request-log events and maintains the public request projections.orchestrator/— Orchestrator service: coordinates validation, dependency analysis, speculation, builds, landing, cancellation, conclusion, hooks, and DLQ reconciliation.extension/— SubmitQueue-specific extension contracts and implementations, including storage, queue configuration, change providers, validation, conflict analysis, speculation, and build runners.entity/— SubmitQueue-specific domain entities.core/— Infrastructure shared across SubmitQueue's own services, including request and batch lifecycle helpers, change-set resolution, and internal topic keys.
Cross-domain building blocks live outside this directory: shared entities in platform/base/, shared extensions in platform/extension/, and cross-domain infrastructure such as the consumer framework in platform/.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package client is the SubmitQueue client: dialling a gateway, the calls a caller makes against it, and the terminal view of what a queue is doing.
|
Package client is the SubmitQueue client: dialling a gateway, the calls a caller makes against it, and the terminal view of what a queue is doing. |
|
Package core groups infrastructure shared across SubmitQueue's own services (gateway and orchestrator) — the SubmitQueue-scoped analogue of the repo-level core/.
|
Package core groups infrastructure shared across SubmitQueue's own services (gateway and orchestrator) — the SubmitQueue-scoped analogue of the repo-level core/. |
|
changeset
Package changeset resolves batch identity into the changes a batch contains.
|
Package changeset resolves batch identity into the changes a batch contains. |
|
changeset/fake
Package fake provides an in-memory changeset.Resolver for tests and examples.
|
Package fake provides an in-memory changeset.Resolver for tests and examples. |
|
changeset/mock
Package mock is a generated GoMock package.
|
Package mock is a generated GoMock package. |
|
messagequeue
Package messagequeue holds SubmitQueue's internal message-queue contract: the wire payloads for the pipeline queues SubmitQueue owns, defined by the proto files in proto/ and generated into protopb/.
|
Package messagequeue holds SubmitQueue's internal message-queue contract: the wire payloads for the pipeline queues SubmitQueue owns, defined by the proto files in proto/ and generated into protopb/. |
|
topickey
Package topickey re-exports SubmitQueue pipeline stage identifiers from submitqueue/core/messagequeue, the package that owns the topic-key constants and the payloads bound to them.
|
Package topickey re-exports SubmitQueue pipeline stage identifiers from submitqueue/core/messagequeue, the package that owns the topic-key constants and the payloads bound to them. |
|
extension
|
|
|
buildrunner/buildkite
Package buildkite implements buildrunner.BuildRunner backed by the Buildkite CI platform.
|
Package buildkite implements buildrunner.BuildRunner backed by the Buildkite CI platform. |
|
buildrunner/fake
Package fake provides a buildrunner.BuildRunner whose outcome is driven by the triggered changes.
|
Package fake provides a buildrunner.BuildRunner whose outcome is driven by the triggered changes. |
|
buildrunner/githubactions
Package githubactions implements buildrunner.BuildRunner backed by GitHub Actions workflow_dispatch.
|
Package githubactions implements buildrunner.BuildRunner backed by GitHub Actions workflow_dispatch. |
|
buildrunner/mock
Package mock is a generated GoMock package.
|
Package mock is a generated GoMock package. |
|
changeprovider/fake
Package fake provides a changeprovider.ChangeProvider whose outcome is driven by the input change.
|
Package fake provides a changeprovider.ChangeProvider whose outcome is driven by the input change. |
|
changeprovider/git
Package git provides a changeprovider.ChangeProvider that reads change metadata out of a git repository, for a remote that offers no API to ask.
|
Package git provides a changeprovider.ChangeProvider that reads change metadata out of a git repository, for a remote that offers no API to ask. |
|
changeprovider/mock
Package mock is a generated GoMock package.
|
Package mock is a generated GoMock package. |
|
changeprovider/routing
Package routing provides a ChangeProvider that dispatches URIs to downstream change providers based on the URI's change type.
|
Package routing provides a ChangeProvider that dispatches URIs to downstream change providers based on the URI's change type. |
|
conflict/all
Package all provides a conflict.Analyzer that pessimistically reports a conflict against every in-flight batch.
|
Package all provides a conflict.Analyzer that pessimistically reports a conflict against every in-flight batch. |
|
conflict/fake
Package fake provides a conflict.Analyzer that decorates an existing analyzer: it delegates to the wrapped implementation for the happy path, but injects an error when a caller-supplied predicate matches.
|
Package fake provides a conflict.Analyzer that decorates an existing analyzer: it delegates to the wrapped implementation for the happy path, but injects an error when a caller-supplied predicate matches. |
|
conflict/mock
Package mock is a generated GoMock package.
|
Package mock is a generated GoMock package. |
|
conflict/none
Package none provides a conflict.Analyzer that never reports a conflict.
|
Package none provides a conflict.Analyzer that never reports a conflict. |
|
conflict/pathoverlap
Package pathoverlap provides a conflict.Analyzer that reports a conflict between two batches when the paths they change share a key.
|
Package pathoverlap provides a conflict.Analyzer that reports a conflict between two batches when the paths they change share a key. |
|
queueconfig/mock
Package mock is a generated GoMock package.
|
Package mock is a generated GoMock package. |
|
queueconfig/yaml
Package yaml provides a YAML-file-backed implementation of queueconfig.Store.
|
Package yaml provides a YAML-file-backed implementation of queueconfig.Store. |
|
speculation/allocator
Package allocator defines the budget-spending composition point used by the standard Speculator.
|
Package allocator defines the budget-spending composition point used by the standard Speculator. |
|
speculation/allocator/mock
Package mock is a generated GoMock package.
|
Package mock is a generated GoMock package. |
|
speculation/allocator/sticky
Package sticky provides an allocator.Allocator that fills only free budget slots and leaves every in-flight build running — it never preempts.
|
Package sticky provides an allocator.Allocator that fills only free budget slots and leaves every in-flight build running — it never preempts. |
|
speculation/generator
Package generator defines the candidate-stream composition point used by the standard Speculator.
|
Package generator defines the candidate-stream composition point used by the standard Speculator. |
|
speculation/generator/bestfirst
Package bestfirst provides a probability-ordered speculation path generator.
|
Package bestfirst provides a probability-ordered speculation path generator. |
|
speculation/generator/mock
Package mock is a generated GoMock package.
|
Package mock is a generated GoMock package. |
|
speculation/scorer/evidence
Package evidence revises a base Scorer's price with factors for observed batch progress.
|
Package evidence revises a base Scorer's price with factors for observed batch progress. |
|
speculation/scorer/fake
Package fake provides a scorer.Scorer that decorates an existing scorer: it delegates to the wrapped implementation for the happy path, but injects an error when a change URI carries a failure marker of the form "sq-fake=<token>":
|
Package fake provides a scorer.Scorer that decorates an existing scorer: it delegates to the wrapped implementation for the happy path, but injects an error when a change URI carries a failure marker of the form "sq-fake=<token>": |
|
speculation/scorer/mock
Package mock is a generated GoMock package.
|
Package mock is a generated GoMock package. |
|
speculation/speculator/mock
Package mock is a generated GoMock package.
|
Package mock is a generated GoMock package. |
|
speculation/speculator/standard
Package standard provides the standard speculator.Speculator: it composes a Generator and an Allocator so a queue's candidate scoring and its budget rationing policy can vary independently without changing the Speculator.
|
Package standard provides the standard speculator.Speculator: it composes a Generator and an Allocator so a queue's candidate scoring and its budget rationing policy can vary independently without changing the Speculator. |
|
storage
Package storage holds SubmitQueue's persistence contracts: the store interfaces both services are written against, the error vocabulary their implementations return, and the queue config their factories resolve against.
|
Package storage holds SubmitQueue's persistence contracts: the store interfaces both services are written against, the error vocabulary their implementations return, and the queue config their factories resolve against. |
|
storage/mock
Package mock is a generated GoMock package.
|
Package mock is a generated GoMock package. |
|
validator/fake
Package fake provides a validator.Validator that always passes.
|
Package fake provides a validator.Validator that always passes. |
|
validator/mock
Package mock is a generated GoMock package.
|
Package mock is a generated GoMock package. |
|
gateway
|
|
|
extension/storage
Package storage resolves the gateway's queue-scoped stores: the append-only request log and the three read models behind request-summary retrieval and List.
|
Package storage resolves the gateway's queue-scoped stores: the append-only request log and the three read models behind request-summary retrieval and List. |
|
extension/storage/mock
Package mock is a generated GoMock package.
|
Package mock is a generated GoMock package. |
|
Package orchestrator declares the SubmitQueue orchestrator's pipeline topology, extension seams, and controller set.
|
Package orchestrator declares the SubmitQueue orchestrator's pipeline topology, extension seams, and controller set. |
|
controller/build
Package build starts the builds a head batch's speculation paths have been funded for.
|
Package build starts the builds a head batch's speculation paths have been funded for. |
|
controller/buildsignal
Package buildsignal implements the build poll loop.
|
Package buildsignal implements the build poll loop. |
|
controller/cancel
Package cancel implements the orchestrator-side cancel controller.
|
Package cancel implements the orchestrator-side cancel controller. |
|
controller/dependencyanalysis
Package dependencyanalysis decides which batch carries a request, resolves what that batch must serialize behind, and promotes it from Creating to Created.
|
Package dependencyanalysis decides which batch carries a request, resolves what that batch must serialize behind, and promotes it from Creating to Created. |
|
controller/dlq
Package dlq contains controllers that consume messages from per-topic dead-letter queues and reconcile the affected request and batch entities into a terminal failed state.
|
Package dlq contains controllers that consume messages from per-topic dead-letter queues and reconcile the affected request and batch entities into a terminal failed state. |
|
controller/land
Package land implements the trigger stage for the asynchronous land.
|
Package land implements the trigger stage for the asynchronous land. |
|
controller/landconflictsignal
Package landconflictsignal consumes merge-conflict results from Runway's signal queue, correlates them to the land request by the echoed id, and either advances the request to the batch stage (landable) or fails it (conflicted).
|
Package landconflictsignal consumes merge-conflict results from Runway's signal queue, correlates them to the land request by the echoed id, and either advances the request to the batch stage (landable) or fails it (conflicted). |
|
controller/landsignal
Package landsignal consumes merge results from Runway's merge-signal queue, correlates them to the batch by the echoed id, and transitions the batch to a terminal state — Succeeded when Runway's merge completed the land, Failed when it could not — then fans the batch out to conclude (so member requests pick up the outcome) and speculate (so dependents can re-plan).
|
Package landsignal consumes merge results from Runway's merge-signal queue, correlates them to the batch by the echoed id, and transitions the batch to a terminal state — Succeeded when Runway's merge completed the land, Failed when it could not — then fans the batch out to conclude (so member requests pick up the outcome) and speculate (so dependents can re-plan). |
|
controller/speculate
Package speculate plans a queue's speculative builds and finalizes each batch's outcome from their results.
|
Package speculate plans a queue's speculative builds and finalizes each batch's outcome from their results. |
|
core/batch
Package batch provides the shared primitives for moving a batch through its lifecycle states while keeping the queue's per-state membership records (entity.QueueBatchState) in step.
|
Package batch provides the shared primitives for moving a batch through its lifecycle states while keeping the queue's per-state membership records (entity.QueueBatchState) in step. |
|
extension/storage
Package storage resolves the orchestrator's queue-scoped stores: the pipeline working state behind validation, batching, speculation, building and landing.
|
Package storage resolves the orchestrator's queue-scoped stores: the pipeline working state behind validation, batching, speculation, building and landing. |
|
extension/storage/mock
Package mock is a generated GoMock package.
|
Package mock is a generated GoMock package. |
Click to show internal directories.
Click to hide internal directories.