Documentation
¶
Overview ¶
Package pg provides PostgreSQL implementations for commitgraph.
Package pg provides PostgreSQL operations for commitgraph repos.
Package pg provides PostgreSQL operations for commitgraph user_aliases.
Package pg provides PostgreSQL implementations for commitgraph.
Example (Invariant4WithRealDatabase) ¶
Example_invariant4WithRealDatabase shows how to run these invariants against a real database (for CI setup).
// In CI, create a fixture database:
//
// 1. Run initial schema migration
// 2. Insert valid test data
// 3. Insert deliberate violations (see TestInvariant4_FixtureViolations)
// 4. Run all three invariant queries
// 5. Verify each returns exactly the expected violation rows
// 6. Clean up fixture database
// Example code structure:
/*
db, err := sql.Open("postgres", "fixture-db-connection-string")
if err != nil {
log.Fatal(err)
}
defer db.Close()
// Run query (a)
rowsA, err := db.Query(`
SELECT rut.repo_id, rut.user_id, rut.tool
FROM repo_user_daily_tool rut
LEFT JOIN users u ON rut.user_id = u.user_id
WHERE u.user_id IS NULL
`)
if err != nil {
log.Fatal(err)
}
defer rowsA.Close()
var orphanCount int
for rowsA.Next() {
orphanCount++
}
// Verify we caught exactly 1 violation (our fixture)
if orphanCount != 1 {
log.Fatalf("Expected 1 orphan user_id violation, got %d", orphanCount)
}
// Similar checks for queries (b) and (c)...
*/
// This function is illustrative documentation only (the body above is a
// comment, not executable code) and intentionally has no "// Output:"
// comment — a prior version claimed a fake "Output:" that nothing in
// this function actually produced, which made `go test` fail.
Index ¶
- Constants
- func BatchUpsertUsers(ctx context.Context, db *sql.Tx, logins []string) (map[string]int64, error)
- func UpsertRepo(ctx context.Context, db *sql.Tx, provider, repoFullName string) (int64, error)
- type AliasIngester
- type AliasRow
- type DBExecutor
- type ExclusionInfo
- type ExclusionOp
- type ExclusionRequest
- type Executor
- type IdentityIngester
- type RepoExcluder
- type Result
- type Row
- type Rows
- type SQLExecutor
- func (e *SQLExecutor) ExecContext(ctx context.Context, query string, args ...interface{}) (Result, error)
- func (e *SQLExecutor) QueryContext(ctx context.Context, query string, args ...interface{}) (Rows, error)
- func (e *SQLExecutor) QueryRowContext(ctx context.Context, query string, args ...interface{}) Row
Examples ¶
Constants ¶
const BatchUsersUpsertQuery = `` /* 135-byte string literal not displayed */
BatchUsersUpsertQuery is a SQL query that batch upserts multiple user logins and returns the complete login -> user_id mapping.
This query:
- Accepts an array of logins via unnest($1::text[])
- Inserts new users with those logins (profile_url and avatar_url are NULL initially)
- On conflict (login already exists), performs a no-op update (preserves the existing row's data, only login is reassigned to itself)
- Returns all logins with their user_ids, both newly created and pre-existing
The ON CONFLICT clause deliberately uses "DO UPDATE SET login = excluded.login" rather than "DO NOTHING": Postgres does not produce RETURNING rows for conflicts resolved by DO NOTHING, only for rows actually written by INSERT or UPDATE. Without the no-op update, a conflicting (pre-existing) login would be silently dropped from the result set, and the caller would need a second SELECT round trip to fill in the gaps. The no-op update keeps the whole batch upsert to exactly one round trip.
The query is idempotent: re-running with the same login set returns consistent results.
Usage:
rows, err := db.Query(ctx, BatchUsersUpsertQuery, pq.Array([]string{"alice", "bob"}))
for rows.Next() {
var login string
var userID int64
rows.Scan(&login, &userID)
// login -> userID mapping
}
Example SQL test:
SELECT * FROM users; -- Initial state: empty SELECT * FROM unnest(ARRAY['alice', 'bob', 'charlie']) AS login; -- Input: three logins -- Execute the query: INSERT INTO users (login) SELECT unnest(ARRAY['alice', 'bob', 'charlie']::text[]) ON CONFLICT (login) DO UPDATE SET login = excluded.login RETURNING login, user_id; -- Returns: alice -> 1, bob -> 2, charlie -> 3 -- Re-run with same + new logins: INSERT INTO users (login) SELECT unnest(ARRAY['alice', 'bob', 'diana']::text[]) ON CONFLICT (login) DO UPDATE SET login = excluded.login RETURNING login, user_id; -- Returns: alice -> 1, bob -> 2, diana -> 4 -- (alice and bob reuse existing IDs, diana gets new ID)
const RepoUpsertQuery = `` /* 176-byte string literal not displayed */
RepoUpsertQuery is a single parameterized upsert that allocates or reuses the surrogate repo_id for a (provider, repo_full_name) pair. Per docs/plan/plan.md "Postgres schema": allocation is one upsert-returning-id per repo, not per commit — a single round trip per scan job, negligible against the clone itself.
ON CONFLICT ... DO UPDATE (not DO NOTHING) is required for two reasons:
- Postgres does not emit a RETURNING row for a conflict resolved by DO NOTHING, only for rows actually written by INSERT or UPDATE — DO NOTHING would return no rows at all on the (common) second-and-later call for an existing repo.
- repo_full_name must be refreshed in place on a GitHub rename (plan.md edge case 4: "Repo renamed on GitHub — the surrogate repo_id survives it; the repos.repo_full_name row is updated"). DO NOTHING would leave the stale name in place.
const UsersSelectByLoginsQuery = `
SELECT login, user_id
FROM users
WHERE login = ANY($1::text[])
`
UsersSelectByLoginsQuery is a fallback query that retrieves user_ids for existing logins. This is useful when you need to get the mapping without attempting insertion.
Use this when: - You know all logins already exist and want to avoid the overhead of INSERT attempts - You need to distinguish between "not found" and "newly inserted" cases
Returns only logins that already exist in the users table.
Variables ¶
This section is empty.
Functions ¶
func BatchUpsertUsers ¶
BatchUpsertUsers executes BatchUsersUpsertQuery in exactly one SQL round trip and returns the complete login -> user_id map for every login in the input slice, whether it was just inserted or already existed.
db is typically a *sql.Tx so the upsert participates in the caller's transaction alongside the rest of a scan-job write; a *sql.DB also satisfies the same QueryContext method and works for standalone callers.
An empty logins slice returns an empty (non-nil) map without touching the database.
func UpsertRepo ¶
UpsertRepo allocates or reuses the surrogate repo_id for (provider, repoFullName) in exactly one round trip, using RepoUpsertQuery. It is called once per repo scan job, before any rollup rows are written for that repo.
The first call for a new (provider, repoFullName) pair inserts a row and returns a fresh repo_id. Every subsequent call for the same pair returns the same repo_id (idempotent), even across a GitHub rename that changes repoFullName for an already-known provider — the ON CONFLICT target is the UNIQUE (provider, repo_full_name) constraint from the current schema, so a genuine rename (which changes repo_full_name) is only picked up here once the caller passes the new name; see plan.md's identity-resolution notes for how renames are detected upstream of this call.
Types ¶
type AliasIngester ¶
type AliasIngester struct {
// contains filtered or unexported fields
}
AliasIngester handles user_alias upsert operations.
func NewAliasIngester ¶
func NewAliasIngester(db DBExecutor) *AliasIngester
NewAliasIngester creates a new alias ingester. Accepts *sql.DB or *sql.Conn (both implement the DBExecutor interface).
func (*AliasIngester) DeleteAdminAliases ¶
func (a *AliasIngester) DeleteAdminAliases(ctx context.Context, sourceLogins []string) (int64, error)
DeleteAdminAliases removes specific admin aliases by source_login. Returns the number of rows deleted.
func (*AliasIngester) GetAdminAliases ¶
GetAdminAliases retrieves all current admin aliases from the database. Returns a map of source_login -> target_login for reason='admin' only.
func (*AliasIngester) UpsertAliases ¶
func (a *AliasIngester) UpsertAliases(ctx context.Context, rows []AliasRow) error
UpsertAliases performs a bulk upsert of alias rows.
Uses ON CONFLICT (source_login) DO UPDATE to handle re-runs safely: - Updates target_login, reason, and created_at if source_login exists - Inserts new row if source_login doesn't exist
This is idempotent - re-running after ConfigMap changes updates existing rows rather than erroring or duplicating.
type AliasRow ¶
type AliasRow struct {
SourceLogin string // The login to alias from (PRIMARY KEY)
TargetLogin string // The canonical login to alias to
Reason string // 'admin' or 'name-match'
CreatedAt time.Time // When this alias was created
}
AliasRow represents a single user_aliases row.
type DBExecutor ¶
type DBExecutor interface {
ExecContext(ctx context.Context, query string, args ...interface{}) (sql.Result, error)
QueryRowContext(ctx context.Context, query string, args ...interface{}) *sql.Row
QueryContext(ctx context.Context, query string, args ...interface{}) (*sql.Rows, error)
}
DBExecutor is the database operations interface. This matches database/sql's DB and Conn interfaces.
type ExclusionInfo ¶
type ExclusionInfo struct {
Provider string `json:"provider"`
RepoFullName string `json:"repo_full_name"`
ExcludedAt time.Time `json:"excluded_at"`
ExcludedReason string `json:"excluded_reason"`
}
ExclusionInfo holds information about an excluded repo.
type ExclusionOp ¶
type ExclusionOp string
ExclusionOp represents the type of exclusion operation.
const ( // OpExclude applies an exclusion OpExclude ExclusionOp = "exclude" // OpClear removes an exclusion OpClear ExclusionOp = "clear" )
type ExclusionRequest ¶
type ExclusionRequest struct {
Provider string // e.g., "github"
RepoFullName string // e.g., "owner/name"
ExcludedAt *time.Time // NULL for clear operations
ExcludedReason string // Human-readable reason (required for exclude, empty for clear)
Operator string // Who is performing this action
}
ExclusionRequest represents a request to apply or clear an exclusion.
type Executor ¶
type Executor interface {
ExecContext(ctx context.Context, query string, args ...interface{}) (Result, error)
QueryRowContext(ctx context.Context, query string, args ...interface{}) Row
QueryContext(ctx context.Context, query string, args ...interface{}) (Rows, error)
}
Executor is the database operations interface. This is a subset of database/sql's DB and Conn interfaces, allowing for both transactional and non-transactional use.
type IdentityIngester ¶
type IdentityIngester struct {
// contains filtered or unexported fields
}
IdentityIngester implements identity.DB for PostgreSQL.
func NewIdentityIngester ¶
func NewIdentityIngester(db Executor) *IdentityIngester
NewIdentityIngester creates a new PostgreSQL identity ingester.
func (*IdentityIngester) IngestEmailResolution ¶
func (i *IdentityIngester) IngestEmailResolution(ctx context.Context, rows []identity.ResolutionRow) (*identity.IngestResult, error)
IngestEmailResolution performs a bulk upsert of email resolution rows using the ON CONFLICT rule from plan.md:
ON CONFLICT (email) DO UPDATE
SET login = excluded.login, source = excluded.source,
resolved_at = excluded.resolved_at
WHERE excluded.source = 'manual'
OR (email_resolution.source <> 'manual'
AND excluded.resolved_at > email_resolution.resolved_at)
This implements the conflict resolution rule:
- Manual source always wins (overwrites any existing row)
- Non-manual sources win only if existing row is also non-manual AND the new resolved_at is newer
- Otherwise the existing row is preserved
The implementation uses a single bulk INSERT with ON CONFLICT DO UPDATE and a WHERE clause on the UPDATE to implement the selective conflict rule. This is efficient for large batches (349K+ rows from claude-leaderboard seed).
Returns identity.IngestResult with counts of ingested and skipped rows, including a breakdown of skip reasons.
type RepoExcluder ¶
type RepoExcluder struct {
// contains filtered or unexported fields
}
RepoExcluder handles repo-level exclusion operations.
func NewRepoExcluder ¶
func NewRepoExcluder(db Executor) *RepoExcluder
NewRepoExcluder creates a new repo excluder.
func (*RepoExcluder) ApplyExclusion ¶
func (r *RepoExcluder) ApplyExclusion(ctx context.Context, req ExclusionRequest) (int64, error)
ApplyExclusion applies or clears a repo exclusion.
For exclude operations: sets excluded_at = now() with the provided reason. For clear operations: sets excluded_at = NULL and excluded_reason = NULL.
This returns the number of rows affected (should be 1 if repo exists, 0 otherwise).
func (*RepoExcluder) GetExclusion ¶
func (r *RepoExcluder) GetExclusion(ctx context.Context, provider, repoFullName string) (*time.Time, string, error)
GetExclusion retrieves the current exclusion status for a repo. Returns (excluded_at, excluded_reason, nil) or (nil, "", error).
func (*RepoExcluder) ListExclusions ¶
func (r *RepoExcluder) ListExclusions(ctx context.Context) ([]ExclusionInfo, error)
ListExclusions retrieves all currently excluded repos. Returns a slice of (provider, repo_full_name, excluded_at, excluded_reason).
type Row ¶
type Row interface {
Scan(dest ...interface{}) error
}
Row is the interface returned by QueryRowContext (subset of sql.Row).
type SQLExecutor ¶
type SQLExecutor struct {
// contains filtered or unexported fields
}
SQLExecutor wraps *sql.DB to implement the Executor interface.
func NewSQLExecutor ¶
func NewSQLExecutor(db *sql.DB) *SQLExecutor
NewSQLExecutor creates a new SQLExecutor from *sql.DB.
func (*SQLExecutor) ExecContext ¶
func (*SQLExecutor) QueryContext ¶
func (*SQLExecutor) QueryRowContext ¶
func (e *SQLExecutor) QueryRowContext(ctx context.Context, query string, args ...interface{}) Row