poll

package
v0.46.0-rc.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 9, 2026 License: Apache-2.0 Imports: 21 Imported by: 0

Documentation

Overview

Package poll implements the GitLab cron-polling event dispatch loop. It discovers events from the GitLab API, converts them to NormalizedEvents, routes them through the dispatch core, and dispatches agent stages via API-triggered pipelines.

Index

Constants

View Source
const (
	// PollStateBranchSlash is written by the slash poll (*/5).
	PollStateBranchSlash = "fullsend-poll-state-slash"
	// PollStateBranchEvents is written by the event poll (2,17,32,47).
	PollStateBranchEvents = "fullsend-poll-state-events"
	// PollStateFileName is the single HMAC-signed document on each branch.
	PollStateFileName = "state.json"
)

Poller state lives on two dedicated, unprotected branches so the bot PAT can run at Developer access (30) instead of Maintainer (40). Each poll mode owns its own branch so concurrent slash+events runs cannot clobber each other, and force-re-root pruning cannot drop a sibling file. See #7343 and ADR 0067.

Variables

This section is empty.

Functions

func EnsureDispatchSecret added in v0.44.0

func EnsureDispatchSecret(ctx context.Context, client forge.Client, owner, repo string) (secret string, created bool, err error)

EnsureDispatchSecret returns the project's existing FULLSEND_DISPATCH_SECRET CI/CD variable value, or generates and stores a new one (masked, protected, like the bot PAT) if none is set. created is true when a new secret was written. Without this, poll-state HMAC signing requires a manual operator step and the poller fails closed.

func ReadWebhookPayloadFile added in v0.46.0

func ReadWebhookPayloadFile(path string) ([]byte, error)

ReadWebhookPayloadFile reads the native webhook body from path — the value of GitLab's file-type TRIGGER_PAYLOAD variable. It refuses a symlink or any non-regular file (so a crafted path cannot redirect the read out of the runner-provided location) and bounds the read to the size BuildWebhookEvents accepts.

func SeedGitLabPollStateBranches added in v0.44.0

func SeedGitLabPollStateBranches(ctx context.Context, client forge.Client, owner, repo, dispatchSecret string) (bool, error)

SeedGitLabPollStateBranches creates both poll-state branches with an HMAC-signed state.json, using the operator's Maintainer-capable client so legacy CI/CD variables can still be read. *Fast fields are written to the slash branch; *Full fields plus LabelState go to the events branch.

A missing branch is seeded from any present legacy variables, or with an empty signed baseline when none are present. An already-present state.json is left untouched so live poller writes are not clobbered. dispatchSecret must be non-empty: unsigned documents would be discarded as tampered by the runtime poller.

Returns true if at least one branch was written.

Types

type Dispatch

type Dispatch struct {
	Stage           string `json:"stage"`
	EventType       string `json:"event_type"`
	EventPayloadB64 string `json:"event_payload_b64"`
	ResourceKey     string `json:"resource_key"`
	MRAuthorID      int    `json:"mr_author_id,omitempty"`
	ActorID         int    `json:"actor_id,omitempty"`
	IsFork          bool   `json:"is_fork"`
	IID             int    `json:"iid,omitempty"`
}

Dispatch represents a single API-triggered pipeline dispatch record.

type EventNote

type EventNote struct {
	ID           int    `json:"id"`
	NoteableType string `json:"noteable_type"`
	NoteableIID  int    `json:"noteable_iid"`
	Body         string `json:"body"`
}

EventNote is the embedded note object in a project event.

type GitLabClient

type GitLabClient interface {
	ListIssuesUpdatedSince(ctx context.Context, owner, repo string, since time.Time) ([]Issue, error)
	ListMergeRequestsUpdatedSince(ctx context.Context, owner, repo string, since time.Time) ([]MergeRequest, error)
	// ListProjectEvents returns events matching targetType (use lowercase
	// "note" for the request param). The after parameter is date-only
	// (ISO 8601 date, exclusive): implementations must widen to at least
	// since.AddDate(0,0,-1) and apply client-side timestamp filtering.
	ListProjectEvents(ctx context.Context, owner, repo string, targetType string, after time.Time) ([]ProjectEvent, error)
	// ListIssueNotes MUST return notes in ascending created_at order.
	ListIssueNotes(ctx context.Context, owner, repo string, issueIID int) ([]Note, error)
	ListMergeRequestNotes(ctx context.Context, owner, repo string, mrIID int) ([]Note, error)
	// ListResourceLabelEvents MUST return events in ascending ID order
	// (the poller iterates in reverse to find the most recent "add").
	ListResourceLabelEvents(ctx context.Context, owner, repo string, issueIID int) ([]ResourceLabelEvent, error)
	// GetFileContentAtRef retrieves a file at a specific ref (branch,
	// tag, or SHA). Returns forge.ErrNotFound if the file or ref does
	// not exist. Used to load the HMAC-signed poll-state document.
	GetFileContentAtRef(ctx context.Context, owner, repo, path, ref string) ([]byte, error)
	// GetBranchRef returns the HEAD commit SHA for the named branch.
	// Returns forge.ErrNotFound if the branch does not exist. Used to
	// pin the CAS parent SHA at poll-state load time.
	GetBranchRef(ctx context.Context, owner, repo, branch string) (string, error)
	// CommitFileToBranch commits a single file to branch without force.
	// expectedSHA is the branch tip observed at load time and is sent as
	// start_sha so a concurrent writer surfaces forge.ErrNonFastForward.
	// An empty expectedSHA (no branch observed at load time) still commits
	// with start_sha pinned to the repository root, but leaves force unset
	// so a concurrent first writer racing branch creation surfaces the same
	// forge.ErrNonFastForward instead of being silently overwritten.
	// The commit message is suffixed with [skip ci] when not already present.
	CommitFileToBranch(ctx context.Context, owner, repo, branch, path, message string, content []byte, expectedSHA string) error
	// ForceCommitFileToBranch force-updates branch to a single-file
	// commit re-rooted on a fixed base SHA. The branch is created if
	// it does not exist. History is pruned to base + 1 commit.
	// Used for install-time seeding; runtime persist uses CommitFileToBranch.
	ForceCommitFileToBranch(ctx context.Context, owner, repo, branch, path, message string, content []byte) error
	// DeleteRef deletes a git ref (e.g., "heads/fullsend-poll-state-slash").
	// Returns forge.ErrNotFound if the ref does not exist. Used to
	// discard a tampered poll-state branch.
	DeleteRef(ctx context.Context, owner, repo, refPath string) error
	GetAuthenticatedUser(ctx context.Context) (string, error)
	GetAuthenticatedUserID(ctx context.Context) (int, error)
	// CreateNoteAwardEmoji adds an emoji reaction. noteableType must be
	// "Issue" or "MergeRequest" to select the correct API endpoint.
	CreateNoteAwardEmoji(ctx context.Context, owner, repo string, noteableType string, noteableIID, noteID int, emoji string) error
	GetIssue(ctx context.Context, owner, repo string, issueIID int) (*Issue, error)
	GetMergeRequest(ctx context.Context, owner, repo string, mrIID int) (*MergeRequest, error)
	// GetMemberAccessLevel returns the access level for a project member.
	// Implementations MUST use the /members/all/:user_id endpoint to
	// include inherited (group-level) membership, not /members/:user_id
	// which only returns direct members.
	GetMemberAccessLevel(ctx context.Context, owner, repo string, userID int) (int, error)
	GetProjectPath(ctx context.Context, projectID int) (string, error)
	// CreatePipeline creates a new pipeline on the given ref with the
	// given variables. Returns the pipeline ID and web URL.
	CreatePipeline(ctx context.Context, owner, repo, ref string, variables map[string]string) (int64, string, error)
	// CreatePipelineWithInputs creates a new pipeline on the given ref
	// using typed GitLab CI/CD pipeline inputs (spec:inputs) instead of
	// user-defined pipeline variables. Unlike CreatePipeline, this
	// remains usable when a project's
	// ci_pipeline_variables_minimum_override_role is
	// forge.PipelineVarOverrideNoOneAllowed, because that setting does
	// not govern pipeline inputs (#7850). dispatch() uses this for a
	// target repository whose committed wrapper declares the typed
	// contract (see usesTypedDispatch) and falls back to CreatePipeline
	// otherwise, since a poller binary upgrade is not synchronized with
	// that repository's own scaffold migration.
	CreatePipelineWithInputs(ctx context.Context, owner, repo, ref string, inputs map[string]forge.PipelineInputValue) (int64, string, error)
}

GitLabClient defines the GitLab API surface the poller requires. This interface is separate from forge.Client because the poller needs GitLab-specific methods (label events, project events) that are not part of the forge-neutral abstraction. Phase 1 wiring will provide a concrete type that satisfies this interface, backed by the forge.Client credential and HTTP plumbing.

All List* methods MUST exhaust pagination (per_page=100, follow x-next-page) and return the complete result set. Returning only page 1 (GitLab default: 20 items) causes silent event loss.

type Issue

type Issue struct {
	IID       int       `json:"iid"`
	Title     string    `json:"title"`
	State     string    `json:"state"`
	Labels    []string  `json:"labels"`
	Author    UserRef   `json:"author"`
	ClosedBy  UserRef   `json:"closed_by"`
	ClosedAt  time.Time `json:"closed_at"`
	CreatedAt time.Time `json:"created_at"`
	UpdatedAt time.Time `json:"updated_at"`
}

Issue represents a GitLab issue as returned by the API. Author is a nested object in the GitLab v4 response.

type LabelAuthor

type LabelAuthor struct {
	ID       int
	Username string
	IsBot    bool
}

LabelAuthor identifies who applied a label.

type LabelState

type LabelState map[int][]string

LabelState tracks previously-seen labels per issue IID.

type MergeRequest

type MergeRequest struct {
	IID             int       `json:"iid"`
	Title           string    `json:"title"`
	State           string    `json:"state"`
	Labels          []string  `json:"labels"`
	SourceProjectID int       `json:"source_project_id"`
	TargetProjectID int       `json:"target_project_id"`
	SourceBranch    string    `json:"source_branch"`
	TargetBranch    string    `json:"target_branch"`
	Author          UserRef   `json:"author"`
	MergeUser       UserRef   `json:"merge_user"`
	MergedBy        UserRef   `json:"merged_by"`
	ClosedBy        UserRef   `json:"closed_by"`
	MergedAt        time.Time `json:"merged_at"`
	ClosedAt        time.Time `json:"closed_at"`
	CreatedAt       time.Time `json:"created_at"`
	UpdatedAt       time.Time `json:"updated_at"`
}

MergeRequest represents a GitLab merge request as returned by the API. Fields are derived from nested API objects (author, merge_user); GitLab does not expose flat author_id/merged_by_id fields on MRs.

type Note

type Note struct {
	ID        int       `json:"id"`
	Body      string    `json:"body"`
	Author    UserRef   `json:"author"`
	CreatedAt time.Time `json:"created_at"`
}

Note represents a GitLab note (comment) on an issue or MR.

type Options

type Options struct {
	BotUserID      int
	GitLabURL      string
	PipelineRef    string // git ref for API-triggered pipelines (required; resolved at CLI wiring time)
	PollJobURL     string // back-link to the poller CI job (optional; from CI_JOB_URL)
	DispatchSecret string // HMAC shared secret for signing dispatch variables (optional; from FULLSEND_DISPATCH_SECRET)
	Mode           string // "slash" (slash commands only) or "events" (full discovery, filters /fs-* notes); empty uses events discovery path but includes all notes for backward compatibility
}

Options configures the poller.

type PendingLabel added in v0.46.0

type PendingLabel struct {
	IID        int      `json:"iid"`
	Label      string   `json:"label"`
	EventID    int      `json:"event_id,omitempty"`
	At         int64    `json:"at"`
	Labels     []string `json:"labels,omitempty"`
	ActorID    int      `json:"actor_id,omitempty"`
	ActorLogin string   `json:"actor_login,omitempty"`
	ActorBot   bool     `json:"actor_bot,omitempty"`
	// OccurredAt is the addition's own occurrence time in Unix milliseconds,
	// kept apart from At (the issue snapshot time) so a pending-only retry
	// keeps the dispatch key's retention anchored to the occurrence. Zero
	// for handoffs written before the field existed.
	OccurredAt int64 `json:"occurred_at,omitempty"`
	// SnapshotAt is RoutableEvent.SnapshotAt in Unix milliseconds: the issue
	// snapshot time the legacy label key derives from when At is the label
	// event's own time. Zero when At is the snapshot time.
	SnapshotAt int64 `json:"snapshot_at,omitempty"`
	// Unresolved marks issue-level reconciliation work rather than a known
	// occurrence: a writer could not resolve which occurrence of Label on
	// IID is current (a label-event lookup failed), so it hands the poller
	// the obligation to resolve and dispatch it. The marker is independent
	// of LabelState and of the updated_at watermark, so a competing poll
	// advancing the watermark cannot lose it. EventID, At and the actor
	// fields are unset; the poller binds them when it resolves the marker.
	Unresolved bool `json:"unresolved,omitempty"`
}

PendingLabel is a label addition whose webhook dispatch failed, kept in poll state until the poller dispatches it (or its retry budget is spent). It carries what the poller needs to rebuild the event without rediscovering it from the issue list: At is the occurrence time in Unix milliseconds. ActorID, ActorLogin and ActorBot are the actor the webhook builder bound to this exact addition; they are carried in the signed handoff so a retry routes the occurrence under its own actor (checking that actor's current access), never under whoever applied the label last.

type Poller

type Poller struct {
	// contains filtered or unexported fields
}

Poller discovers GitLab events and dispatches agent stages.

func New

func New(client GitLabClient, router dispatch.EventRouter, projectPath string, opts Options) *Poller

New creates a Poller for the given project.

func (*Poller) BuildWebhookEvents added in v0.46.0

func (p *Poller) BuildWebhookEvents(ctx context.Context, raw []byte) ([]WebhookEvent, error)

BuildWebhookEvents converts a native GitLab webhook body (the contents of the file TRIGGER_PAYLOAD points to) into validated webhook events routed through the Poller's router. It returns an error — and no events — when any part of the payload cannot be validated against the re-fetch (fail closed). Bot-authored events are filtered with the same rules the poller applies.

func (*Poller) Run

func (p *Poller) Run(ctx context.Context) error

Run executes a single poll cycle: read watermark, discover events, filter, deduplicate, convert to NormalizedEvent, route, dispatch, and advance the watermark.

Conversion, routing, and dispatch failures are returned after poll state is persisted so the poll job fails instead of reporting a healthy cycle. Failed events remain retryable until maxEventRetries. The cycle that exhausts the budget also fails (the event is dropped); later rediscovery of an already-dropped event is a skip, not a new failure, so the 30s watermark overlap cannot pin the poll job red.

Poll mode is determined by Options.Mode:

  • "slash": fast poll — only /fs-* slash commands via the Events API
  • "events": full discovery — labels, merges, MR opens, closed-unmerged MRs, non-slash notes (filters out /fs-* notes)
  • "": backward compatibility — uses events discovery path but does not filter /fs-* notes

func (*Poller) RunWebhook added in v0.46.0

func (p *Poller) RunWebhook(ctx context.Context, raw []byte) error

RunWebhook builds events from a native GitLab webhook body and dispatches their routed stages, deduplicated against the poller's per-mode state. It fails closed — dispatching nothing — when the payload cannot be validated.

type ProjectEvent

type ProjectEvent struct {
	ID        int       `json:"id"`
	Author    UserRef   `json:"author"`
	Note      EventNote `json:"note"`
	CreatedAt time.Time `json:"created_at"`
}

ProjectEvent represents an event from the GitLab Events API.

type ResourceLabelEvent

type ResourceLabelEvent struct {
	ID     int    `json:"id"`
	Action string `json:"action"`
	Label  struct {
		Name string `json:"name"`
	} `json:"label"`
	User      UserRef   `json:"user"`
	CreatedAt time.Time `json:"created_at"`
}

ResourceLabelEvent represents a label change event from the GitLab API.

type RoutableEvent

type RoutableEvent struct {
	Type         string
	Action       string // MR lifecycle action for Type=="mr_event": "opened", "closed", or empty (merged)
	IID          int
	UpdatedAt    time.Time
	Labels       []string // full label set at event time
	ChangedLabel string   // the label that was added/removed (label events only)
	// LabelEventID is the GitLab resource label event ID of the label
	// addition (label events only; 0 when unknown). Both the webhook builder
	// and poll discovery take it from the same resource label events API, so
	// it is the stable occurrence identity a remove-then-re-add cannot share.
	LabelEventID int
	// OccurredAt is when the selected label addition happened, from its
	// resource label event (poll discovery only; zero when unknown).
	// UpdatedAt stays the issue snapshot time that drives the watermark and
	// the legacy label keys; OccurredAt only extends dispatch-key retention
	// (see dispatchedAtUnix) so a fresh addition seen through an older
	// snapshot is not pruned as old.
	OccurredAt time.Time
	// SnapshotAt is the issue's updated_at as read when this label addition was
	// observed, when that differs from UpdatedAt (webhook events carry the
	// label event's own time in UpdatedAt). Pollers from before label event IDs
	// key an addition on the issue's updated_at, so the legacy compatibility
	// key derives from SnapshotAt; zero means UpdatedAt is the snapshot time.
	SnapshotAt      time.Time
	NoteBody        string
	NoteID          int
	NoteAuthorID    int
	NoteAuthorLogin string
	IsBot           bool
	MRSource        int
	MRTarget        int
	SourceBranch    string
	TargetBranch    string
	MRAuthorID      int
	MRAuthorLogin   string
	MergedByLogin   string
}

RoutableEvent is an intermediate representation of a detected change, produced by event discovery and consumed by the poll loop for NormalizedEvent conversion and dispatch.

func (RoutableEvent) FallbackLabelKey added in v0.46.0

func (e RoutableEvent) FallbackLabelKey() string

FallbackLabelKey returns the millisecond-timestamp key this label event carries when its resource label event ID is unknown, or "" for a non-label event or one that already uses that key. A retry count recorded while the ID lookup failed lives under this key, so it must still be found once a later cycle resolves the ID.

func (RoutableEvent) Key

func (e RoutableEvent) Key() string

Key returns a deduplication key for the event. Note events use their globally unique NoteID. Label events are keyed on the resource label event ID when known (a remove-then-re-add is a new event, hence a new key); otherwise on a millisecond timestamp so a re-add within one second is still distinct. MR open events include Action so they do not collide with a merge of the same IID in the same second. MR merge events use type+IID+timestamp (Action empty) for stable dispatched-key persistence.

Keys persisted by earlier versions carry Unix-second label timestamps; see LegacyLabelKey and labelKeyAtOrAfter for how those stay recognised.

func (RoutableEvent) LegacyLabelKey added in v0.46.0

func (e RoutableEvent) LegacyLabelKey() string

LegacyLabelKey returns the Unix-second label key earlier versions persisted for this event, or "" for a non-label event.

type UserRef

type UserRef struct {
	ID       int    `json:"id"`
	Username string `json:"username"`
	Bot      bool   `json:"bot"`
}

UserRef is a minimal user reference from the GitLab API. The Bot field is not present in all API responses (notably the Notes API author object omits it). Use isBotEvent() for reliable bot detection, which combines the API field, botUserID, and username-pattern heuristics.

type WebhookEvent added in v0.46.0

type WebhookEvent struct {
	Event      RoutableEvent
	Normalized dispatch.NormalizedEvent
	ActorID    int
	Stages     []string
	Deferred   error
}

WebhookEvent is one validated webhook transition, ready for the shared poll dispatch path. Event is what dispatch() consumes (it carries the same Key() the poller would compute for the same change), Normalized is the routing input produced by toNormalizedEvent, ActorID is the actor bound to the transition, and Stages is the router's decision (nil when the Poller has no router). Deferred is set only for a validated label addition whose normalization or routing failed: the label provenance checks passed, so the occurrence is not dropped but handed to the poller for retry (see dispatchWebhookEvents), and Normalized, ActorID and Stages are unset.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL