README
¶
Rollup Package
The rollup package provides rollup computation for commitgraph v2.
Purpose
This package implements the core rollup computation that aggregates AI-tool-tagged commits by (user, repo, tool, day) while applying date quarantine filtering to exclude out-of-range commits from the rollup table.
Date Quarantine
The quarantine filter excludes commits with committed_at outside the range [2005-01-01, today+1] (UTC) from the rollup computation:
- Lower bound: 2005-01-01 00:00:00 UTC (inclusive)
- Upper bound: Today + 1 day 23:59:59.999999999 UTC (inclusive)
This prevents malformed or maliciously-dated commits (e.g., the 2170 incident) from corrupting the rollup while preserving the raw data in the Parquet artifact.
Usage
Creating Quarantine Bounds
import "github.com/jedarden/commitgraph/pkg/rollup"
// Create bounds for the current date
bounds := rollup.NewQuarantineBounds(time.Now().UTC())
Filtering Commits
// Check if a commit date is within bounds
if bounds.IsIncluded(commit.CommittedAt) {
// Include in rollup
}
Computing Rollups
commits := []rollup.Commit{
{
SHA: "abc123",
AuthorEmail: "user@example.com",
CommittedAt: time.Date(2024, 6, 15, 0, 0, 0, 0, time.UTC),
Tools: []string{"claude"},
},
}
// Compute rollup with date filtering
rollupRows := rollup.ComputeRollup(commits, repoID, bounds)
// rollupRows contains only commits within the date bounds
// The original commits slice is preserved for Parquet writing
Architecture
The rollup computation is designed to preserve the raw/filtered split:
- Rollup computation (step 4): Excludes out-of-range commits via
ComputeRollup() - Parquet artifact (step 5b): Writes all commits with unclamped
committed_atvalues
This separation ensures:
- Ranking queries never see out-of-range dates
- Raw data is preserved for re-detection and analysis
- The 2170 incident cannot recur
Testing
The package includes comprehensive tests covering:
- Boundary conditions (2004-12-31, 2005-01-01, today+1, today+2)
- The 2170 incident scenario
- Aggregation by day and tool
- Parquet data preservation
Run tests with:
go test ./pkg/rollup/... -v
Acceptance Criteria
All acceptance criteria from bead cg-19os are met:
- ✅ 2004-12-31 excluded from rollup
- ✅ 2005-01-01 included in rollup
- ✅ today+1 included, today+2 excluded
- ✅ Parquet artifact preserves unclamped
committed_at - ✅ Synthetic 2170 fixture produces zero rollup rows
Integration
This package is used by:
migration/migrate_corpus.py(via Go bindings or CLI)- Future clone-worker implementation (step 4 rollup computation)
References
- Plan:
docs/plan/plan.md- Architecture, quarantine section - Schema:
migrations/001_initial_schema.sql-repo_user_daily_tooltable - Bead: cg-19os - Original task specification
Documentation
¶
Overview ¶
Package rollup provides rollup computation for commitgraph.
The rollup aggregates AI-tool-tagged commits by (user, repo, tool, day) while applying date quarantine filtering to exclude out-of-range commits from the rollup table while preserving them in the raw Parquet artifact.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Commit ¶
type Commit struct {
SHA string // Commit SHA
AuthorEmail string // Author email (for identity resolution)
AuthorName string // Author name
CommittedAt time.Time // Commit date (UTC)
Message string // Commit message
Tools []string // Detected AI tools (empty if no AI tool detected)
}
Commit represents a single commit for rollup computation.
type QuarantineBounds ¶
type QuarantineBounds struct {
// MinDate is the lower bound (inclusive): 2005-01-01 UTC
MinDate time.Time
// MaxDate is the upper bound (inclusive): today+1 UTC
MaxDate time.Time
}
QuarantineBounds defines the valid date range for rollup computation. Commits with committed_at outside this range are excluded from the rollup but preserved in the raw Parquet artifact.
func NewQuarantineBounds ¶
func NewQuarantineBounds(today time.Time) QuarantineBounds
NewQuarantineBounds creates bounds for the current date. MinDate is fixed at 2005-01-01 UTC. MaxDate is the current UTC date + 1 day.
func (QuarantineBounds) IsIncluded ¶
func (qb QuarantineBounds) IsIncluded(committedAt time.Time) bool
IsIncluded returns true if the given committed_at falls within the quarantine bounds [MinDate, MaxDate] (inclusive on both ends). MinDate includes the entire day starting at 2005-01-01 00:00:00 UTC. MaxDate includes the entire day ending at today+1 23:59:59.999999999 UTC.
type RollupRow ¶
type RollupRow struct {
UserEmail string // Author email (resolved to user_id later)
RepoID int64 // Repository ID
Tool string // AI tool name
Day time.Time // Day (UTC, midnight)
Count int // Number of commits
}
RollupRow represents a single rollup aggregation row.
func ComputeRollup ¶
func ComputeRollup(commits []Commit, repoID int64, bounds QuarantineBounds) []RollupRow
ComputeRollup computes (user, repo, tool, day, count) aggregations from the given commits, applying date quarantine filtering.
Commits with committed_at outside the quarantine bounds are excluded from the rollup entirely. The caller is responsible for preserving the raw commit data (including unclamped committed_at) in the Parquet artifact.
Parameters:
- commits: All commits for a repo (may include out-of-range dates)
- repoID: Repository ID for rollup rows
- bounds: Quarantine date bounds
Returns:
- Rollup rows for commits within the date bounds
Source Files