Documentation
¶
Overview ¶
Package poll implements the GitLab cron-polling event dispatch loop. It discovers events from the GitLab API, converts them to NormalizedEvents, routes them through the dispatch core, and dispatches agent stages via API-triggered pipelines.
Index ¶
- Constants
- func EnsureDispatchSecret(ctx context.Context, client forge.Client, owner, repo string) (secret string, created bool, err error)
- func ReadWebhookPayloadFile(path string) ([]byte, error)
- func SeedGitLabPollStateBranches(ctx context.Context, client forge.Client, owner, repo, dispatchSecret string) (bool, error)
- type Dispatch
- type EventNote
- type GitLabClient
- type Issue
- type LabelAuthor
- type LabelState
- type MergeRequest
- type Note
- type Options
- type PendingLabel
- type Poller
- type ProjectEvent
- type ResourceLabelEvent
- type RoutableEvent
- type UserRef
- type WebhookEvent
Constants ¶
const ( // PollStateBranchSlash is written by the slash poll (*/5). PollStateBranchSlash = "fullsend-poll-state-slash" // PollStateBranchEvents is written by the event poll (2,17,32,47). PollStateBranchEvents = "fullsend-poll-state-events" // PollStateFileName is the single HMAC-signed document on each branch. PollStateFileName = "state.json" )
Poller state lives on two dedicated, unprotected branches so the bot PAT can run at Developer access (30) instead of Maintainer (40). Each poll mode owns its own branch so concurrent slash+events runs cannot clobber each other, and force-re-root pruning cannot drop a sibling file. See #7343 and ADR 0067.
Variables ¶
This section is empty.
Functions ¶
func EnsureDispatchSecret ¶ added in v0.44.0
func EnsureDispatchSecret(ctx context.Context, client forge.Client, owner, repo string) (secret string, created bool, err error)
EnsureDispatchSecret returns the project's existing FULLSEND_DISPATCH_SECRET CI/CD variable value, or generates and stores a new one (masked, protected, like the bot PAT) if none is set. created is true when a new secret was written. Without this, poll-state HMAC signing requires a manual operator step and the poller fails closed.
func ReadWebhookPayloadFile ¶ added in v0.46.0
ReadWebhookPayloadFile reads the native webhook body from path — the value of GitLab's file-type TRIGGER_PAYLOAD variable. It refuses a symlink or any non-regular file (so a crafted path cannot redirect the read out of the runner-provided location) and bounds the read to the size BuildWebhookEvents accepts.
func SeedGitLabPollStateBranches ¶ added in v0.44.0
func SeedGitLabPollStateBranches(ctx context.Context, client forge.Client, owner, repo, dispatchSecret string) (bool, error)
SeedGitLabPollStateBranches creates both poll-state branches with an HMAC-signed state.json, using the operator's Maintainer-capable client so legacy CI/CD variables can still be read. *Fast fields are written to the slash branch; *Full fields plus LabelState go to the events branch.
A missing branch is seeded from any present legacy variables, or with an empty signed baseline when none are present. An already-present state.json is left untouched so live poller writes are not clobbered. dispatchSecret must be non-empty: unsigned documents would be discarded as tampered by the runtime poller.
Returns true if at least one branch was written.
Types ¶
type Dispatch ¶
type Dispatch struct {
Stage string `json:"stage"`
EventType string `json:"event_type"`
EventPayloadB64 string `json:"event_payload_b64"`
ResourceKey string `json:"resource_key"`
MRAuthorID int `json:"mr_author_id,omitempty"`
ActorID int `json:"actor_id,omitempty"`
IsFork bool `json:"is_fork"`
IID int `json:"iid,omitempty"`
}
Dispatch represents a single API-triggered pipeline dispatch record.
type EventNote ¶
type EventNote struct {
ID int `json:"id"`
NoteableType string `json:"noteable_type"`
NoteableIID int `json:"noteable_iid"`
Body string `json:"body"`
}
EventNote is the embedded note object in a project event.
type GitLabClient ¶
type GitLabClient interface {
ListIssuesUpdatedSince(ctx context.Context, owner, repo string, since time.Time) ([]Issue, error)
ListMergeRequestsUpdatedSince(ctx context.Context, owner, repo string, since time.Time) ([]MergeRequest, error)
// ListProjectEvents returns events matching targetType (use lowercase
// "note" for the request param). The after parameter is date-only
// (ISO 8601 date, exclusive): implementations must widen to at least
// since.AddDate(0,0,-1) and apply client-side timestamp filtering.
ListProjectEvents(ctx context.Context, owner, repo string, targetType string, after time.Time) ([]ProjectEvent, error)
// ListIssueNotes MUST return notes in ascending created_at order.
ListIssueNotes(ctx context.Context, owner, repo string, issueIID int) ([]Note, error)
ListMergeRequestNotes(ctx context.Context, owner, repo string, mrIID int) ([]Note, error)
// ListResourceLabelEvents MUST return events in ascending ID order
// (the poller iterates in reverse to find the most recent "add").
ListResourceLabelEvents(ctx context.Context, owner, repo string, issueIID int) ([]ResourceLabelEvent, error)
// GetFileContentAtRef retrieves a file at a specific ref (branch,
// tag, or SHA). Returns forge.ErrNotFound if the file or ref does
// not exist. Used to load the HMAC-signed poll-state document.
GetFileContentAtRef(ctx context.Context, owner, repo, path, ref string) ([]byte, error)
// GetBranchRef returns the HEAD commit SHA for the named branch.
// Returns forge.ErrNotFound if the branch does not exist. Used to
// pin the CAS parent SHA at poll-state load time.
GetBranchRef(ctx context.Context, owner, repo, branch string) (string, error)
// CommitFileToBranch commits a single file to branch without force.
// expectedSHA is the branch tip observed at load time and is sent as
// start_sha so a concurrent writer surfaces forge.ErrNonFastForward.
// An empty expectedSHA (no branch observed at load time) still commits
// with start_sha pinned to the repository root, but leaves force unset
// so a concurrent first writer racing branch creation surfaces the same
// forge.ErrNonFastForward instead of being silently overwritten.
// The commit message is suffixed with [skip ci] when not already present.
CommitFileToBranch(ctx context.Context, owner, repo, branch, path, message string, content []byte, expectedSHA string) error
// ForceCommitFileToBranch force-updates branch to a single-file
// commit re-rooted on a fixed base SHA. The branch is created if
// it does not exist. History is pruned to base + 1 commit.
// Used for install-time seeding; runtime persist uses CommitFileToBranch.
ForceCommitFileToBranch(ctx context.Context, owner, repo, branch, path, message string, content []byte) error
// DeleteRef deletes a git ref (e.g., "heads/fullsend-poll-state-slash").
// Returns forge.ErrNotFound if the ref does not exist. Used to
// discard a tampered poll-state branch.
DeleteRef(ctx context.Context, owner, repo, refPath string) error
GetAuthenticatedUser(ctx context.Context) (string, error)
GetAuthenticatedUserID(ctx context.Context) (int, error)
// CreateNoteAwardEmoji adds an emoji reaction. noteableType must be
// "Issue" or "MergeRequest" to select the correct API endpoint.
CreateNoteAwardEmoji(ctx context.Context, owner, repo string, noteableType string, noteableIID, noteID int, emoji string) error
GetIssue(ctx context.Context, owner, repo string, issueIID int) (*Issue, error)
GetMergeRequest(ctx context.Context, owner, repo string, mrIID int) (*MergeRequest, error)
// GetMemberAccessLevel returns the access level for a project member.
// Implementations MUST use the /members/all/:user_id endpoint to
// include inherited (group-level) membership, not /members/:user_id
// which only returns direct members.
GetMemberAccessLevel(ctx context.Context, owner, repo string, userID int) (int, error)
GetProjectPath(ctx context.Context, projectID int) (string, error)
// CreatePipeline creates a new pipeline on the given ref with the
// given variables. Returns the pipeline ID and web URL.
CreatePipeline(ctx context.Context, owner, repo, ref string, variables map[string]string) (int64, string, error)
// CreatePipelineWithInputs creates a new pipeline on the given ref
// using typed GitLab CI/CD pipeline inputs (spec:inputs) instead of
// user-defined pipeline variables. Unlike CreatePipeline, this
// remains usable when a project's
// ci_pipeline_variables_minimum_override_role is
// forge.PipelineVarOverrideNoOneAllowed, because that setting does
// not govern pipeline inputs (#7850). dispatch() uses this for a
// target repository whose committed wrapper declares the typed
// contract (see usesTypedDispatch) and falls back to CreatePipeline
// otherwise, since a poller binary upgrade is not synchronized with
// that repository's own scaffold migration.
CreatePipelineWithInputs(ctx context.Context, owner, repo, ref string, inputs map[string]forge.PipelineInputValue) (int64, string, error)
// CreateChildPipelineWithInputs starts a downstream pipeline on ref
// through the pipeline trigger API, authenticated by the running
// job's CI_JOB_TOKEN rather than the client's PAT (#8438). GitLab
// records it with source "pipeline" and attributes it to the user who
// owns the calling job, so it runs on a protected branch whenever that
// user may, independently of the Poller credential's protected-ref
// access. Implementations MUST NOT retry: an ambiguous outcome could
// otherwise start a duplicate agent pipeline. Returns the pipeline ID
// and web URL.
CreateChildPipelineWithInputs(ctx context.Context, owner, repo, ref, jobToken string, inputs map[string]forge.PipelineInputValue) (int64, string, error)
}
GitLabClient defines the GitLab API surface the poller requires. This interface is separate from forge.Client because the poller needs GitLab-specific methods (label events, project events) that are not part of the forge-neutral abstraction. Phase 1 wiring will provide a concrete type that satisfies this interface, backed by the forge.Client credential and HTTP plumbing.
All List* methods MUST exhaust pagination (per_page=100, follow x-next-page) and return the complete result set. Returning only page 1 (GitLab default: 20 items) causes silent event loss.
type Issue ¶
type Issue struct {
IID int `json:"iid"`
Title string `json:"title"`
State string `json:"state"`
Labels []string `json:"labels"`
Author UserRef `json:"author"`
ClosedBy UserRef `json:"closed_by"`
ClosedAt time.Time `json:"closed_at"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
Issue represents a GitLab issue as returned by the API. Author is a nested object in the GitLab v4 response.
type LabelAuthor ¶
LabelAuthor identifies who applied a label.
type LabelState ¶
LabelState tracks previously-seen labels per issue IID.
type MergeRequest ¶
type MergeRequest struct {
IID int `json:"iid"`
Title string `json:"title"`
State string `json:"state"`
Labels []string `json:"labels"`
SourceProjectID int `json:"source_project_id"`
TargetProjectID int `json:"target_project_id"`
SourceBranch string `json:"source_branch"`
TargetBranch string `json:"target_branch"`
Author UserRef `json:"author"`
MergeUser UserRef `json:"merge_user"`
MergedBy UserRef `json:"merged_by"`
ClosedBy UserRef `json:"closed_by"`
MergedAt time.Time `json:"merged_at"`
ClosedAt time.Time `json:"closed_at"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
MergeRequest represents a GitLab merge request as returned by the API. Fields are derived from nested API objects (author, merge_user); GitLab does not expose flat author_id/merged_by_id fields on MRs.
type Note ¶
type Note struct {
ID int `json:"id"`
Body string `json:"body"`
Author UserRef `json:"author"`
CreatedAt time.Time `json:"created_at"`
}
Note represents a GitLab note (comment) on an issue or MR.
type Options ¶
type Options struct {
BotUserID int
GitLabURL string
PipelineRef string // git ref for API-triggered pipelines (required; resolved at CLI wiring time)
PollJobURL string // back-link to the poller CI job (optional; from CI_JOB_URL)
DispatchSecret string // HMAC shared secret for signing dispatch variables (optional; from FULLSEND_DISPATCH_SECRET)
Mode string // "slash" (slash commands only) or "events" (full discovery, filters /fs-* notes); empty uses events discovery path but includes all notes for backward compatibility
// ChildDispatchJobToken, when set, selects the child-pipeline dispatch
// route (#8438): agent pipelines are started with this CI_JOB_TOKEN
// through the pipeline trigger API (source "pipeline", attributed to
// the owner of the running job) instead of with the Poller credential
// through the pipelines API (source "api"). It requires the typed
// pipeline-input contract. Empty keeps the Poller API route.
ChildDispatchJobToken string
}
Options configures the poller.
type PendingLabel ¶ added in v0.46.0
type PendingLabel struct {
IID int `json:"iid"`
Label string `json:"label"`
EventID int `json:"event_id,omitempty"`
At int64 `json:"at"`
Labels []string `json:"labels,omitempty"`
ActorID int `json:"actor_id,omitempty"`
ActorLogin string `json:"actor_login,omitempty"`
ActorBot bool `json:"actor_bot,omitempty"`
// OccurredAt is the addition's own occurrence time in Unix milliseconds,
// kept apart from At (the issue snapshot time) so a pending-only retry
// keeps the dispatch key's retention anchored to the occurrence. Zero
// for handoffs written before the field existed.
OccurredAt int64 `json:"occurred_at,omitempty"`
// SnapshotAt is RoutableEvent.SnapshotAt in Unix milliseconds: the issue
// snapshot time the legacy label key derives from when At is the label
// event's own time. Zero when At is the snapshot time.
SnapshotAt int64 `json:"snapshot_at,omitempty"`
// Unresolved marks issue-level reconciliation work rather than a known
// occurrence: a writer could not resolve which occurrence of Label on
// IID is current (a label-event lookup failed), so it hands the poller
// the obligation to resolve and dispatch it. The marker is independent
// of LabelState and of the updated_at watermark, so a competing poll
// advancing the watermark cannot lose it. EventID, At and the actor
// fields are unset; the poller binds them when it resolves the marker.
Unresolved bool `json:"unresolved,omitempty"`
}
PendingLabel is a label addition whose webhook dispatch failed, kept in poll state until the poller dispatches it (or its retry budget is spent). It carries what the poller needs to rebuild the event without rediscovering it from the issue list: At is the occurrence time in Unix milliseconds. ActorID, ActorLogin and ActorBot are the actor the webhook builder bound to this exact addition; they are carried in the signed handoff so a retry routes the occurrence under its own actor (checking that actor's current access), never under whoever applied the label last.
type Poller ¶
type Poller struct {
// contains filtered or unexported fields
}
Poller discovers GitLab events and dispatches agent stages.
func New ¶
func New(client GitLabClient, router dispatch.EventRouter, projectPath string, opts Options) *Poller
New creates a Poller for the given project.
func (*Poller) BuildWebhookEvents ¶ added in v0.46.0
BuildWebhookEvents converts a native GitLab webhook body (the contents of the file TRIGGER_PAYLOAD points to) into validated webhook events routed through the Poller's router. It returns an error — and no events — when any part of the payload cannot be validated against the re-fetch (fail closed). Bot-authored events are filtered with the same rules the poller applies.
func (*Poller) Run ¶
Run executes a single poll cycle: read watermark, discover events, filter, deduplicate, convert to NormalizedEvent, route, dispatch, and advance the watermark.
Conversion, routing, and dispatch failures are returned after poll state is persisted so the poll job fails instead of reporting a healthy cycle. Failed events remain retryable until maxEventRetries. The cycle that exhausts the budget also fails (the event is dropped); later rediscovery of an already-dropped event is a skip, not a new failure, so the 30s watermark overlap cannot pin the poll job red.
Poll mode is determined by Options.Mode:
- "slash": fast poll — only /fs-* slash commands via the Events API
- "events": full discovery — labels, merges, MR opens, closed-unmerged MRs, non-slash notes (filters out /fs-* notes)
- "": backward compatibility — uses events discovery path but does not filter /fs-* notes
func (*Poller) RunWebhook ¶ added in v0.46.0
RunWebhook builds events from a native GitLab webhook body and dispatches their routed stages, deduplicated against the poller's per-mode state. It fails closed — dispatching nothing — when the payload cannot be validated.
type ProjectEvent ¶
type ProjectEvent struct {
ID int `json:"id"`
Author UserRef `json:"author"`
Note EventNote `json:"note"`
CreatedAt time.Time `json:"created_at"`
}
ProjectEvent represents an event from the GitLab Events API.
type ResourceLabelEvent ¶
type ResourceLabelEvent struct {
ID int `json:"id"`
Action string `json:"action"`
Label struct {
Name string `json:"name"`
} `json:"label"`
User UserRef `json:"user"`
CreatedAt time.Time `json:"created_at"`
}
ResourceLabelEvent represents a label change event from the GitLab API.
type RoutableEvent ¶
type RoutableEvent struct {
Type string
Action string // MR lifecycle action for Type=="mr_event": "opened", "closed", or empty (merged)
IID int
UpdatedAt time.Time
Labels []string // full label set at event time
ChangedLabel string // the label that was added/removed (label events only)
// LabelEventID is the GitLab resource label event ID of the label
// addition (label events only; 0 when unknown). Both the webhook builder
// and poll discovery take it from the same resource label events API, so
// it is the stable occurrence identity a remove-then-re-add cannot share.
LabelEventID int
// OccurredAt is when the selected label addition happened, from its
// resource label event (poll discovery only; zero when unknown).
// UpdatedAt stays the issue snapshot time that drives the watermark and
// the legacy label keys; OccurredAt only extends dispatch-key retention
// (see dispatchedAtUnix) so a fresh addition seen through an older
// snapshot is not pruned as old.
OccurredAt time.Time
// SnapshotAt is the issue's updated_at as read when this label addition was
// observed, when that differs from UpdatedAt (webhook events carry the
// label event's own time in UpdatedAt). Pollers from before label event IDs
// key an addition on the issue's updated_at, so the legacy compatibility
// key derives from SnapshotAt; zero means UpdatedAt is the snapshot time.
SnapshotAt time.Time
NoteBody string
NoteID int
NoteAuthorID int
NoteAuthorLogin string
IsBot bool
MRSource int
MRTarget int
SourceBranch string
TargetBranch string
MRAuthorID int
MRAuthorLogin string
MergedByLogin string
}
RoutableEvent is an intermediate representation of a detected change, produced by event discovery and consumed by the poll loop for NormalizedEvent conversion and dispatch.
func (RoutableEvent) FallbackLabelKey ¶ added in v0.46.0
func (e RoutableEvent) FallbackLabelKey() string
FallbackLabelKey returns the millisecond-timestamp key this label event carries when its resource label event ID is unknown, or "" for a non-label event or one that already uses that key. A retry count recorded while the ID lookup failed lives under this key, so it must still be found once a later cycle resolves the ID.
func (RoutableEvent) Key ¶
func (e RoutableEvent) Key() string
Key returns a deduplication key for the event. Note events use their globally unique NoteID. Label events are keyed on the resource label event ID when known (a remove-then-re-add is a new event, hence a new key); otherwise on a millisecond timestamp so a re-add within one second is still distinct. MR open events include Action so they do not collide with a merge of the same IID in the same second. MR merge events use type+IID+timestamp (Action empty) for stable dispatched-key persistence.
Keys persisted by earlier versions carry Unix-second label timestamps; see LegacyLabelKey and labelKeyAtOrAfter for how those stay recognised.
func (RoutableEvent) LegacyLabelKey ¶ added in v0.46.0
func (e RoutableEvent) LegacyLabelKey() string
LegacyLabelKey returns the Unix-second label key earlier versions persisted for this event, or "" for a non-label event.
type UserRef ¶
UserRef is a minimal user reference from the GitLab API. The Bot field is not present in all API responses (notably the Notes API author object omits it). Use isBotEvent() for reliable bot detection, which combines the API field, botUserID, and username-pattern heuristics.
type WebhookEvent ¶ added in v0.46.0
type WebhookEvent struct {
Event RoutableEvent
Normalized dispatch.NormalizedEvent
ActorID int
Stages []string
Deferred error
}
WebhookEvent is one validated webhook transition, ready for the shared poll dispatch path. Event is what dispatch() consumes (it carries the same Key() the poller would compute for the same change), Normalized is the routing input produced by toNormalizedEvent, ActorID is the actor bound to the transition, and Stages is the router's decision (nil when the Poller has no router). Deferred is set only for a validated label addition whose normalization or routing failed: the label provenance checks passed, so the occurrence is not dropped but handed to the poller for retry (see dispatchWebhookEvents), and Normalized, ActorID and Stages are unset.