backfill

package
v0.16.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 18, 2025 License: Apache-2.0 Imports: 12 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	DefaultBatchSize int           = 1000
	DefaultDelay     time.Duration = 0
)
View Source 
const CNeedsBackfillColumn = "_pgroll_needs_backfill"

CNeedsBackfillColumn is the name of the internal column created by pgroll to mark rows that must be backfilled

Variables

This section is empty.

Functions

func TriggerFunctionName added in v0.14.0 

func TriggerFunctionName(tableName, columnName string) string

TriggerFunctionName returns the name of the trigger function for a given table and column.

func TriggerName added in v0.14.0 

func TriggerName(tableName, columnName string) string

TriggerName returns the name of the trigger for a given table and column.

Types

type Backfill 

type Backfill struct {
	*Config
	// contains filtered or unexported fields
}

func New 

func New(conn db.DB, c *Config) *Backfill

New creates a new backfill operation with the given options. The backfill is not started until `Start` is invoked.

func (*Backfill) CreateTriggers added in v0.14.0 

func (bf *Backfill) CreateTriggers(ctx context.Context, j *Job) error

CreateTriggers creates the triggers for the tables before starting the backfill.

func (*Backfill) Start 

func (bf *Backfill) Start(ctx context.Context, table *schema.Table) error

Start updates all rows in the given table, in batches, using the following algorithm: 1. Get the primary key column for the table. 2. Get the first batch of rows from the table, ordered by the primary key. 3. Update each row in the batch, setting the value of the primary key column to itself. 4. Repeat steps 2 and 3 until no more rows are returned.

type CallbackFn 

type CallbackFn func(done int64, total int64)

type Config added in v0.10.0 

type Config struct {
	// contains filtered or unexported fields
}

func NewConfig added in v0.10.0 

func NewConfig(opts ...OptionFn) *Config

func (*Config) AddCallback added in v0.10.0 

func (c *Config) AddCallback(fn CallbackFn)

AddCallback adds a callback to the backfill operation. Callbacks are invoked after each batch is processed.

type Job added in v0.14.0 

type Job struct {
	Tables []*schema.Table
	// contains filtered or unexported fields
}

Job is a collection of all tables that need to be backfilled and their associated triggers.

func NewJob added in v0.14.0 

func NewJob(schemaName, latestSchema string) *Job

func (*Job) AddTask added in v0.14.0 

func (j *Job) AddTask(t *Task)

type OperationTrigger added in v0.14.0 

type OperationTrigger struct {
	Name           string
	Direction      TriggerDirection
	Columns        map[string]*schema.Column
	TableName      string
	PhysicalColumn string
	SQL            string
}

type OptionFn 

type OptionFn func(*Config)

func WithBatchDelay 

func WithBatchDelay(delay time.Duration) OptionFn

WithBatchDelay sets the delay between batches for the backfill operation.

func WithBatchSize 

func WithBatchSize(batchSize int) OptionFn

WithBatchSize sets the batch size for the backfill operation.

type Task added in v0.14.0 

type Task struct {
	// contains filtered or unexported fields
}

Task represents a backfill task for a specific table from an operation.

func NewTask added in v0.14.0 

func NewTask(table *schema.Table, triggers ...OperationTrigger) *Task

func (*Task) AddTriggers added in v0.14.0 

func (t *Task) AddTriggers(other *Task)

type TriggerDirection added in v0.14.0 

type TriggerDirection string
const (
	TriggerDirectionUp   TriggerDirection = "up"
	TriggerDirectionDown TriggerDirection = "down"
)

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.