Documentation
¶
Index ¶
- Variables
- func CopyToTempTable[T any](ctx context.Context, conn PgxSession, tempTable string, rows []T, ...) (cleanup func(), err error)
- func IsSuiteImportable(name string) bool
- func ParseGormLogLevel(logLevel string) (gormlogger.LogLevel, error)
- func RefreshByPhase(matviews []PostgresView, refreshFn func([]PostgresView))
- type DB
- func (d *DB) CleanupPartitions(dryRun bool) (detached, dropped int, err error)
- func (d *DB) CopyFrom(ctx context.Context, table string, columns []string, rowSrc pgx.CopyFromSource) (int64, error)
- func (d *DB) DetachOldPartitions(retentionDays int, dryRun bool) (int, error)
- func (d *DB) DropDetachedPartitions(detachedDays int, dryRun bool) (int, error)
- func (d *DB) EnsurePartitions(releases []string, startDate, endDate time.Time, dryRun bool) (int, error)
- func (d *DB) PartitionedTables() []string
- func (d *DB) UpdateSchema(reportEnd *time.Time) error
- type Option
- type PgxSession
- type PostgresFunction
- type PostgresView
- type SchemaHashType
- type TempColumn
Constants ¶
This section is empty.
Variables ¶
var PostgresFunctions = []PostgresFunction{
{
Name: "job_results",
Definition: jobResultFunction,
},
{
Name: "test_results",
Definition: testResultFunction,
},
}
var PostgresMatViews = []PostgresView{}
TODO: for historical sippy we need to specify the pinnedDate and not use NOW
var PostgresViews = []PostgresView{}
PostgresViews are regular, non-materialized views:
Functions ¶
func CopyToTempTable ¶
func CopyToTempTable[T any]( ctx context.Context, conn PgxSession, tempTable string, rows []T, cols []TempColumn[T], ) (cleanup func(), err error)
func IsSuiteImportable ¶
IsSuiteImportable checks if a suite name should be imported based on the explicit testSuites list or dynamic patterns.
func ParseGormLogLevel ¶
func ParseGormLogLevel(logLevel string) (gormlogger.LogLevel, error)
func RefreshByPhase ¶
func RefreshByPhase(matviews []PostgresView, refreshFn func([]PostgresView))
RefreshByPhase groups matviews by RefreshPhase and calls refreshFn for each phase in order. All matviews in a phase are passed to refreshFn together (the caller is responsible for concurrent execution within a phase).
Types ¶
type DB ¶
type DB struct {
DB *gorm.DB
// BatchSize is used for how many insertions we should do at once. Postgres supports
// a maximum of 2^16 records per insert.
BatchSize int
// GoparPartitions provides partition creation/management operations
GoparPartitions *partitioning.DB_PARTITIONS
}
func (*DB) CleanupPartitions ¶
CleanupPartitions performs the full partition lifecycle cleanup: 1. Detaches partitions older than 100 days 2. Drops detached partitions older than 110 days
This provides a 10-day safety window between detachment and permanent deletion.
Parameters:
- dryRun: If true, only preview what would be done
Returns the number of partitions detached and dropped.
func (*DB) CopyFrom ¶
func (d *DB) CopyFrom(ctx context.Context, table string, columns []string, rowSrc pgx.CopyFromSource) (int64, error)
CopyFrom uses the PostgreSQL COPY protocol to perform bulk data insertion, streaming all rows in a single protocol-level operation. Callers provide a pgx.CopyFromSource (e.g. pgx.CopyFromRows or pgx.CopyFromSlice) so they can choose whether to pre-allocate rows or generate them lazily.
Columns not listed receive their DEFAULT (e.g. serial id, deleted_at NULL).
func (*DB) DetachOldPartitions ¶
DetachOldPartitions detaches partitions older than the specified retention period. This is a safer alternative to immediate deletion - detached partitions can be archived or reviewed before permanent deletion.
Parameters:
- retentionDays: Age threshold in days (e.g., 100 means detach partitions older than 100 days)
- dryRun: If true, only preview what would be detached
Returns the total number of partitions detached across all tables.
func (*DB) DropDetachedPartitions ¶
DropDetachedPartitions drops partitions that have been detached for longer than the specified period. This permanently deletes the data.
Parameters:
- detachedDays: Minimum age in days since detachment (e.g., 110 means drop partitions detached more than 110 days ago)
- dryRun: If true, only preview what would be dropped
Returns the total number of partitions dropped across all tables.
func (*DB) EnsurePartitions ¶
func (d *DB) EnsurePartitions(releases []string, startDate, endDate time.Time, dryRun bool) (int, error)
EnsurePartitions creates missing partitions for all managed partitioned tables. It uses LIST→RANGE nested partitioning where:
- Level 1: LIST partition by release (e.g., "4.17", "4.18")
- Level 2: RANGE sub-partition by timestamp (daily granularity)
Parameters:
- releases: List of releases to create partitions for (e.g., ["4.17", "4.18", "4.19"])
- startDate: Start date for partition creation
- endDate: End date for partition creation
- dryRun: If true, only preview what would be created
Returns the total number of partitions created across all tables.
func (*DB) PartitionedTables ¶
PartitionedTables returns the list of tables that are partitioned and managed by gopar partition lifecycle management.
type PgxSession ¶
type PgxSession interface {
Exec(ctx context.Context, sql string, arguments ...any) (pgconn.CommandTag, error)
Query(ctx context.Context, sql string, args ...any) (pgx.Rows, error)
CopyFrom(ctx context.Context, tableName pgx.Identifier, columnNames []string, rowSrc pgx.CopyFromSource) (int64, error)
}
PgxSession is satisfied by both *pgx.Conn and pgx.Tx.
type PostgresFunction ¶
type PostgresView ¶
type PostgresView struct {
// Name is the name of the materialized view in postgres.
Name string
// Definition is the material view definition.
Definition string
// ReplaceStrings is a map of strings we want to replace in the create view statement, allowing for re-use.
ReplaceStrings map[string]string
// IndexColumns are the columns to create a unique index for. Will be named idx_[Name] and automatically
// replaced if changes are made to these values. IndexColumns are required as we need them defined to be able to
// refresh materialized views concurrently. (avoiding locking reads for several minutes while we update)
IndexColumns []string
// AdditionalIndexes are non-unique indexes to create on the materialized view for query performance.
// Each entry is a raw column expression (e.g. "release, timestamp DESC") and will be named
// idx_[Name]_[sequence].
AdditionalIndexes []string
// RefreshPhase controls the order in which matviews are refreshed. All matviews
// in phase 0 refresh first (concurrently), then all in phase 1, and so on.
// Use this when a matview reads from another matview and needs it to be up-to-date.
// The default zero value means phase 0.
RefreshPhase int
}
type SchemaHashType ¶
type SchemaHashType string
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
These types are used to decode information from ci-search, but we don't want to expose these for anyone else.
|
These types are used to decode information from ci-search, but we don't want to expose these for anyone else. |