connector

package
v0.4.10 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 16, 2026 License: Apache-2.0 Imports: 3 Imported by: 0

Documentation

Overview

Package connector defines the Connector and Sink interfaces and the Provenance struct returned by each extraction.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Connector

type Connector interface {
	Name() string
	Ping(ctx context.Context) error
	Extract(ctx context.Context, repo Repo, window Window, sink Sink) Provenance
}

type EndpointStatus

type EndpointStatus struct {
	Accessible bool   `json:"accessible"`
	Reason     string `json:"reason,omitempty"`
}

type Prefetcher

type Prefetcher interface {
	Prefetch(ctx context.Context, slug string, window Window) error
}

Prefetcher is the optional interface a Connector may implement to allow its slow per-repo work to start during the run.go clone phase, in parallel with the actual clone. Prefetch must be safe to call concurrently against distinct slugs and must stash its result somewhere Extract can find it later (typically a per-slug cache held on the connector itself). Extract remains the canonical entry point for row emission; Prefetch is purely a wall-clock hint. See ADR 022 for the connector-contract baseline and the #71 ADR for this extension.

type Provenance

type Provenance struct {
	Connector          string                    `json:"connector"`
	Repo               string                    `json:"repo"`
	WindowCovered      Window                    `json:"window_covered"`
	RowsReturned       map[string]int            `json:"rows_returned"`
	PaginationComplete bool                      `json:"pagination_complete"`
	RateLimitTruncated bool                      `json:"rate_limit_truncated"`
	Errors             map[string]string         `json:"errors"`
	Endpoints          map[string]EndpointStatus `json:"endpoints,omitempty"`
	// Flags carries boolean per-extraction signals the manifest aggregates
	// across repos. Currently used for "mailmap_applied". A flag absent from
	// the map reads as false; the aggregator ANDs across all provenances of
	// the same connector to derive the manifest-wide value.
	Flags map[string]bool `json:"flags,omitempty"`

	// GraphQLPointsUsed is the total number of GraphQL rate-limit points
	// consumed across all requests in this extraction. Zero when no GraphQL
	// calls were made (e.g. REST-only connectors).
	GraphQLPointsUsed int `json:"graphql_points_used,omitempty"`
	// GraphQLPointsRemaining is the remaining GitHub GraphQL rate-limit budget
	// as of the last observed response. Zero when no GraphQL calls were made.
	GraphQLPointsRemaining int `json:"graphql_points_remaining,omitempty"`

	// ConfigDepth records operator-declared extraction-depth overrides that
	// narrow what data was captured. Absent keys mean the connector ran at
	// full depth. The analyser reads this to interpret reduced row counts as
	// "out of scope" rather than "no signal". Currently used by the github
	// connector for "pr_window" and "pr_history_sample".
	ConfigDepth map[string]string `json:"config_depth,omitempty"`

	// Sampling holds per-bucket statistics when sparse-historical PR sampling
	// is active (pr_inflection + pr_history_sample configured). Nil when the
	// connector ran at full PR fidelity. The analyser uses the per-bucket
	// target/actual/total counts to compute confidence intervals on metrics
	// derived from pre-bracket sparse data.
	Sampling *SamplingProvenance `json:"sampling,omitempty"`
}

func NewProvenance

func NewProvenance(name, repo string, w Window) Provenance

func (*Provenance) Merge

func (p *Provenance) Merge(other Provenance)

Merge folds other into p in place. Used when a single Extract pass runs multiple goroutines that write disjoint provenance fragments (see `github.Connector.Extract`'s clone-bound vs API-bound split, #71).

Policy:

  • RowsReturned counters are summed.
  • Errors are first-wins per context: p's existing entry sticks, other's fills only previously-empty contexts. Callers organise goroutines so contexts don't collide; on collision the deterministic policy is to keep p's.
  • PaginationComplete is ANDed.
  • RateLimitTruncated is ORed.
  • Endpoints and Flags: other's entry fills if p has none for the key; existing entries on p are preserved.

Window, Connector, Repo are not merged — those are set at NewProvenance time and other's values must match. The caller is expected to pass a fragment built from the same (connector, repo, window).

type Repo

type Repo struct {
	Slug          string
	DefaultBranch string
	HeadSHA       string
	Team          string
	Clone         string
}

type SampleBucket added in v0.4.9

type SampleBucket struct {
	Month     string `json:"month"`               // "2022-01" or "2022-01-W1" for weekly sub-buckets
	Target    int    `json:"target"`              // requested N per bucket
	Actual    int    `json:"actual"`              // PRs emitted
	Total     int    `json:"total"`               // totalCount from GraphQL search
	Truncated bool   `json:"truncated,omitempty"` // true when Total > 1000 (search cap)
}

SampleBucket records the extraction result for one month (or week) bucket in the pre-bracket sparse slice.

type SamplingProvenance added in v0.4.9

type SamplingProvenance struct {
	InflectionDate string         `json:"inflection_date"` // "2023-06-01"
	BracketWindow  string         `json:"bracket_window"`  // "12m"
	BracketStart   string         `json:"bracket_start"`   // "2022-06-01"
	BracketEnd     string         `json:"bracket_end"`     // "2026-06-15"
	Strategy       string         `json:"strategy"`        // "search_default_relevance" | "random"
	Buckets        []SampleBucket `json:"buckets"`
}

SamplingProvenance records the sparse-historical PR sampling configuration and per-bucket extraction results. Present only when pr_inflection is set.

type Sink

type Sink interface {
	InsertRepo(model.Repo) error
	InsertTeamRepo(team, repo string) error
	InsertRepoLanguage(model.RepoLanguage) error
	InsertBranch(model.Branch) error
	InsertBranchProtection(model.BranchProtection) error
	InsertCodeowner(model.Codeowner) error

	InsertCommit(model.Commit) error
	InsertCommitFile(model.CommitFile) error
	InsertCommitCoauthor(model.CommitCoauthor) error

	InsertPR(model.PR) error
	InsertPRCommit(model.PRCommit) error
	InsertReview(model.Review) error
	InsertPRComment(model.PRComment) error
	InsertPRReviewRequest(model.PRReviewRequest) error
	InsertPRLabel(model.PRLabel) error

	InsertBuild(model.Build) error
	InsertBuildJob(model.BuildJob) error
	InsertDeploy(model.Deploy) error
	InsertRelease(model.Release) error

	InsertIncident(model.Incident) error
	InsertDefect(model.Defect) error

	InsertFileMetric(model.FileMetric) error
	InsertHarnessArtifact(model.HarnessArtifact) error
	InsertFileComplexityHistory(model.FileComplexityHistory) error
	InsertRepoFile(model.RepoFile) error
}

Sink is the typed insertion surface every connector writes against. Methods are typed per canonical table so the compiler enforces that connectors cannot invent tables outside the schema.

type Window

type Window struct {
	Start time.Time `json:"start"`
	End   time.Time `json:"end"`
}

func (Window) Contains

func (w Window) Contains(t time.Time) bool

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL