copier

package

v0.10.1 Latest Latest Go to latest Published: Jan 19, 2026 License: Apache-2.0 Imports: 17 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/block/spirit

Links

Open Source Insights

Documentation ¶

Overview ¶

Package copier copies rows from one table to another. it makes use of tableinfo.Chunker, and does the parallelism and retries here. It fails on the first error.

Index ¶

type Copier
- func NewCopier(db *sql.DB, chunker table.Chunker, config *CopierConfig) (Copier, error)
type CopierConfig
- func NewCopierDefaultConfig() *CopierConfig
type Unbuffered

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

This section is empty.

Types ¶

type Copier ¶

type Copier interface {
	Run(ctx context.Context) error
	GetETA() string
	GetChunker() table.Chunker
	SetThrottler(throttler throttler.Throttler)
	GetThrottler() throttler.Throttler
	StartTime() time.Time
	GetProgress() string
}

Copier is the interface which copiers use. Currently we only have one implementation, which we call unbuffered because it uses INSERT .. SELECT without any intermediate buffering in spirit. In future we may have another implementation, see: https://github.com/block/spirit/issues/451

func NewCopier ¶

func NewCopier(db *sql.DB, chunker table.Chunker, config *CopierConfig) (Copier, error)

NewCopier creates a new copier object with the provided chunker. The chunker could have been opened at a watermark, we are agnostic to that. It could also return different tables on each Next() call in future, so we don't save any fields related to the table.

type CopierConfig ¶

type CopierConfig struct {
	Concurrency                   int
	TargetChunkTime               time.Duration
	Throttler                     throttler.Throttler
	Logger                        *slog.Logger
	MetricsSink                   metrics.Sink
	DBConfig                      *dbconn.DBConfig
	UseExperimentalBufferedCopier bool
	Applier                       applier.Applier
}

func NewCopierDefaultConfig ¶

func NewCopierDefaultConfig() *CopierConfig

NewCopierDefaultConfig returns a default config for the copier.

type Unbuffered ¶

type Unbuffered struct {
	sync.Mutex
	// contains filtered or unexported fields
}

func (*Unbuffered) CopyChunk ¶

func (c *Unbuffered) CopyChunk(ctx context.Context, chunk *table.Chunk) error

CopyChunk copies a chunk from the table to the newTable. it is public so it can be used in tests incrementally.

func (*Unbuffered) GetChunker ¶

func (c *Unbuffered) GetChunker() table.Chunker

GetChunker returns the chunker for accessing progress information

func (*Unbuffered) GetETA ¶

func (c *Unbuffered) GetETA() string

func (*Unbuffered) GetProgress ¶

func (c *Unbuffered) GetProgress() string

GetProgress returns the progress of the copier

func (*Unbuffered) GetThrottler ¶

func (c *Unbuffered) GetThrottler() throttler.Throttler

func (*Unbuffered) Run ¶

func (c *Unbuffered) Run(ctx context.Context) error

func (*Unbuffered) SetThrottler ¶

func (c *Unbuffered) SetThrottler(throttler throttler.Throttler)

func (*Unbuffered) StartTime ¶

func (c *Unbuffered) StartTime() time.Time

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL