ingest

package
v0.26.15 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 13, 2026 License: Apache-2.0 Imports: 19 Imported by: 0

Documentation

Index

Constants

View Source
const (
	EventPullRequest       = "PullRequestEvent"
	EventPullRequestReview = "PullRequestReviewEvent"
	EventIssueComment      = "IssueCommentEvent"
	EventIssues            = "IssuesEvent"
)

GitHub event type constants.

Variables

View Source
var ErrArchiveNotFound = errors.New("archive not found")

ErrArchiveNotFound is returned when a GH Archive hourly file is not yet published.

Functions

func Backfill

func Backfill(ctx context.Context, store *postgres.Store, days int) error

Backfill processes historical GH Archive hours in reverse-chronological order (most recent first) so that on initial deployment, archive data is populated going back the specified number of days.

func Run

func Run(ctx context.Context, store *postgres.Store) error

Run processes one or more hourly GH Archive dumps.

Types

type Aggregator

type Aggregator struct {
	// contains filtered or unexported fields
}

Aggregator collects events into per-contributor hourly summaries and per-PR observations for the merge graph.

func NewAggregator

func NewAggregator(hour time.Time) *Aggregator

NewAggregator creates an aggregator for the given hour (truncated to the hour boundary).

func (*Aggregator) Add

func (a *Aggregator) Add(ev Event)

Add processes a single event. Bot actors are skipped for the per-contributor summaries (we don't credit bots with activity) but PullRequestEvent observations are recorded in the merge graph regardless of actor — the merge action's actor is typically a CI bot, and that observation IS the merge graph's reason to exist.

func (*Aggregator) Count

func (a *Aggregator) Count() int

Count returns the number of unique contributors aggregated.

func (*Aggregator) Hour

func (a *Aggregator) Hour() time.Time

Hour returns the hour this aggregator is collecting for.

func (*Aggregator) PREvents

func (a *Aggregator) PREvents() []PREvent

PREvents returns all collected PR-level observations for this hour. Each (repo, pr_number, action) tuple is unique within an hour; the storage layer COALESCEs across hours into a single PR row whose author / opened_at / merged_at / closed_at columns fill in as the corresponding events arrive.

func (*Aggregator) Results

func (a *Aggregator) Results() []Summary

Results returns all summaries.

type ArchiveReader

type ArchiveReader struct {
	// contains filtered or unexported fields
}

func NewArchiveReader

func NewArchiveReader(baseURL string) *ArchiveReader

func (*ArchiveReader) Stream

func (r *ArchiveReader) Stream(ctx context.Context, hour time.Time, fn func(Event)) error

Stream downloads the archive for the given hour and calls fn for each relevant event. Transient transport failures and 5xx responses are retried with exponential backoff; 404 short-circuits as ErrArchiveNotFound (the file is not yet published, later hours won't be either).

func (*ArchiveReader) URL

func (r *ArchiveReader) URL(t time.Time) string

type Event

type Event struct {
	Type      string
	Action    string
	Actor     string
	Repo      string
	CreatedAt time.Time
	// PRNumber is populated for PullRequestEvent only; zero for other
	// event types. Used by the merge-graph aggregator to correlate
	// opened/merged/closed events on the same PR (the actor differs:
	// opens are attributed to the author, merges are usually attributed
	// to a CI bot in modern OSS workflows).
	PRNumber int
}

type PREvent

type PREvent struct {
	Repo    string
	Number  int
	Action  string    // opened, merged, closed
	Author  string    // populated for opened action; empty otherwise
	OccurAt time.Time // event timestamp
}

PREvent is one observation of a PullRequestEvent, persisted to the merge-graph table so opened/merged/closed actions on the same PR can be joined later. Without this, "PRs merged authored by X" can't be computed: GH Archive attributes the merged action to whoever clicked merge (usually a CI bot), not the PR author.

Per-event records are keyed by (repo, pr_number) at storage time; the storage layer COALESCEs fields so each action stamps only its own column without clobbering earlier observations of the same PR.

type Summary

type Summary struct {
	Username      string
	PRsOpened     int
	PRsMerged     int
	PRsClosed     int
	ReviewsGiven  int
	IssueComments int
	IssuesOpened  int
	IssuesClosed  int
	Repos         map[string]bool
}

Summary is the per-contributor hourly aggregation result.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL