Documentation
¶
Overview ¶
Package github fetches data from GitHub's GraphQL API.
Index ¶
- Constants
- type Client
- func (c *Client) EnableAutoMerge(ctx context.Context, pullRequestID string) error
- func (c *Client) FillForbiddenStars(ctx context.Context, stars map[string]int, reasons map[string]string)
- func (c *Client) GetStars(ctx context.Context, repos []Repo) (map[string]int, map[string]string, error)
- func (c *Client) RepoIDs(ctx context.Context, repos []Repo) (map[string]int64, error)
- func (c *Client) Repository(owner, repo string) *Repository
- func (c *Client) Tags(ctx context.Context, repos []Repo) (*Sweep, error)
- func (c *Client) Versions(ctx context.Context, repos []Repo) (*Sweep, error)
- type Repo
- type Repository
- func (r *Repository) Blobs(ctx context.Context, logger *slog.Logger, expressions []string) (map[string]string, error)
- func (r *Repository) FilesTwoDeep(ctx context.Context, expressions []string) (map[string][]string, error)
- func (r *Repository) OIDs(ctx context.Context, expressions []string) (map[string]string, error)
- func (r *Repository) OpenPullRequestHeads(ctx context.Context) ([]string, error)
- func (r *Repository) Subtrees(ctx context.Context, expression string) ([]string, error)
- type Sweep
Constants ¶
const BatchSize = 50
BatchSize is the number of repositories queried in one GraphQL request. GitHub charges GraphQL by node count rather than by request, so batching turns a sweep over ~2,300 packages from that many REST calls into ~46 requests.
100 aliases per query makes GitHub answer some of them with RESOURCE_LIMITS_EXCEEDED, which silently drops those repositories from the result. 75 and below were clean when measured, and 50 leaves room for the limit to move.
const SweepDepth = 10
SweepDepth is how many versions of each package a sweep looks at.
A sweep decides whether a package has anything new, and the newest releases are where anything new appears once the package's history is in the registry. Asking for more costs nothing in rate limit — GitHub charges a GraphQL query by what it returns, and a batch of 50 stays at one point either way — but it makes each answer bigger for something a sweep never reads.
What a sweep can't see is a release published with an old date, which lands further down the list. Those are for the reconciliation that walks the whole history to find.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client queries GitHub's GraphQL API.
func (*Client) EnableAutoMerge ¶
EnableAutoMerge turns on auto-merge for a pull request.
This is a GraphQL mutation because the REST API has no equivalent. Auto-merge is what makes CI the gate: the pull request merges itself once the checks pass, and stays open when they don't.
func (*Client) FillForbiddenStars ¶
func (c *Client) FillForbiddenStars(ctx context.Context, stars map[string]int, reasons map[string]string)
FillForbiddenStars looks up, without a token, the repositories an organization's IP allow list refused.
Anonymous requests are not covered by an allow list, so this recovers what the authenticated query couldn't read. It is deliberately narrow: anonymous requests are limited to 60 an hour, and only the refused repositories are retried. A repository that genuinely doesn't exist is left alone, since asking again would only spend that budget.
func (*Client) GetStars ¶
func (c *Client) GetStars(ctx context.Context, repos []Repo) (map[string]int, map[string]string, error)
GetStars returns the stargazer count of each repository, and why any of them couldn't be read.
A repository that can't be read is left out of the result rather than failing the whole batch, because one missing package must not stop the rest from being initialized. The reason is returned so the caller can say more than that it failed: a deleted repository and an organization refusing the request look the same from here otherwise.
A batch that fails outright is treated the same way. Returning nothing would throw away every count already read, and building the state from scratch asks for more than two thousand of them in 47 batches: one failing request would leave every package looking equally unused and the run processing them in name order.
func (*Client) RepoIDs ¶ added in v0.12.0
RepoIDs returns each repository's numeric id, fifty repositories to a request.
func (*Client) Repository ¶ added in v0.11.0
func (c *Client) Repository(owner, repo string) *Repository
Repository returns a reader of the repository.
func (*Client) Tags ¶
Tags returns the newest tags of each repository, newest first.
Some packages are versioned by their tags rather than their releases — flutter tags every build and publishes the SDK elsewhere — and aqua-registry says so with version_source.
func (*Client) Versions ¶
Versions returns the newest versions of each repository, newest first, and why any of them couldn't be read.
One request covers fifty repositories, and GitHub charges it as one point against a budget of its own: a sweep over the whole registry costs about forty-seven points out of five thousand an hour, and none of the REST budget that generating spends.
Conditional requests would have been the other way to do this, and can't be. The REST listing carries each asset's download count, which changes whenever anybody downloads one, so its ETag changes with it: the packages a sweep most wants to skip are the ones whose ETag never holds. GraphQL doesn't support ETags at all, and doesn't need them, because it returns only the fields that were asked for.
type Repository ¶ added in v0.11.0
type Repository struct {
// contains filtered or unexported fields
}
Repository reads the files and trees of one repository.
func (*Repository) Blobs ¶ added in v0.11.0
func (r *Repository) Blobs(ctx context.Context, logger *slog.Logger, expressions []string) (map[string]string, error)
Blobs returns the text of the file each expression names, such as "main:pkgs/69/1790772769/registry.yaml", keyed by the expression.
An expression naming nothing is left out rather than failing the rest: that is what the head branch of a pull request closed since it was listed looks like, and the files the other expressions name are no less there for it.
A file GitHub wouldn't send whole is left out too, and said so. It would parse as something the file doesn't say.
func (*Repository) FilesTwoDeep ¶ added in v0.12.0
func (r *Repository) FilesTwoDeep(ctx context.Context, expressions []string) (map[string][]string, error)
FilesTwoDeep returns, for each expression naming a tree, the files two levels under it, such as "v1.0.0/registry-1.json" under "main:pkgs/67/1790772767/versions". An expression naming nothing has no entry.
func (*Repository) OIDs ¶ added in v0.12.0
OIDs returns the sha of the object each expression names, keyed by the expression.
func (*Repository) OpenPullRequestHeads ¶ added in v0.11.0
func (r *Repository) OpenPullRequestHeads(ctx context.Context) ([]string, error)
OpenPullRequestHeads returns the head branch of every open pull request.
func (*Repository) Subtrees ¶ added in v0.11.0
Subtrees returns the directories two levels under the tree an expression names, such as "69/1790772769" under "main:pkgs".
One request for every package the registry holds: the two levels are names and nothing else, a few dozen bytes a package. Reading the tree recursively instead would list every version of every package, and GitHub stops a recursive listing at 100,000 entries.
A tree the expression doesn't name holds nothing, which is what a registry no package has been moved into yet looks like.
type Sweep ¶ added in v0.0.20
type Sweep struct {
// Versions is the newest versions of each repository, newest first.
Versions map[string][]string
// Names is what each repository is called now. It differs from the key when the
// repository has been renamed or transferred since the registry recorded it:
// GitHub answers a query made with the old name and says which name it answered
// for, so a sweep finds this out without asking anything extra.
Names map[string]string
// IDs is each repository's numeric id on GitHub. A name can be taken over by another
// repository and an id can't, so it is what says the repository is still the one a
// package was taken over from.
IDs map[string]int64
// Reasons says why a repository couldn't be read.
Reasons map[string]string
}
Sweep is what one look at the registry's repositories found, keyed throughout by the name that was asked for.
Source Files