s3

package
v0.3.4-alpha20 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 17, 2021 License: GPL-3.0 Imports: 24 Imported by: 5

Documentation

Index

Constants

View Source
const (
	DefaultSyncInterval = 10 * time.Second

	DefaultRetention = 24 * time.Hour

	DefaultRetentionCheckInterval = 1 * time.Hour
)

S3 replica default settings.

View Source
const MaxKeys = 1000

MaxKeys is the number of keys S3 can operate on per batch.

Variables

This section is empty.

Functions

func ParseHost added in v0.3.3

func ParseHost(s string) (bucket, region, endpoint string, forcePathStyle bool)

ParseHost extracts data from a hostname depending on the service provider.

Types

type Replica

type Replica struct {

	// AWS authentication keys.
	AccessKeyID     string
	SecretAccessKey string

	// S3 bucket information
	Region         string
	Bucket         string
	Path           string
	Endpoint       string
	ForcePathStyle bool
	SkipVerify     bool

	// Time between syncs with the shadow WAL.
	SyncInterval time.Duration

	// Frequency to create new snapshots.
	SnapshotInterval time.Duration

	// Time to keep snapshots and related WAL files.
	// Database is snapshotted after interval and older WAL files are discarded.
	Retention time.Duration

	// Time between retention checks.
	RetentionCheckInterval time.Duration

	// Time between validation checks.
	ValidationInterval time.Duration

	// If true, replica monitors database for changes automatically.
	// Set to false if replica is being used synchronously (such as in tests).
	MonitorEnabled bool
	// contains filtered or unexported fields
}

Replica is a replica that replicates a DB to an S3 bucket.

func NewReplica

func NewReplica(db *litestream.DB, name string) *Replica

NewReplica returns a new instance of Replica.

func (*Replica) CalcPos

func (r *Replica) CalcPos(ctx context.Context, generation string) (pos litestream.Pos, err error)

CalcPos returns the position for the replica for the current generation. Returns a zero value if there is no active generation.

func (*Replica) DB

func (r *Replica) DB() *litestream.DB

DB returns the parent database reference.

func (*Replica) EnforceRetention

func (r *Replica) EnforceRetention(ctx context.Context) (err error)

EnforceRetention forces a new snapshot once the retention interval has passed. Older snapshots and WAL files are then removed.

func (*Replica) GenerationDir

func (r *Replica) GenerationDir(generation string) string

GenerationDir returns the path to a generation's root directory.

func (*Replica) GenerationStats

func (r *Replica) GenerationStats(ctx context.Context, generation string) (stats litestream.GenerationStats, err error)

GenerationStats returns stats for a generation.

func (*Replica) Generations

func (r *Replica) Generations(ctx context.Context) ([]string, error)

Generations returns a list of available generation names.

func (*Replica) Init

func (r *Replica) Init(ctx context.Context) (err error)

Init initializes the connection to S3. No-op if already initialized.

func (*Replica) LastPos

func (r *Replica) LastPos() litestream.Pos

LastPos returns the last successfully replicated position.

func (*Replica) MaxSnapshotIndex

func (r *Replica) MaxSnapshotIndex(generation string) (int, error)

MaxSnapshotIndex returns the highest index for the snapshots.

func (*Replica) Name

func (r *Replica) Name() string

Name returns the name of the replica. Returns the type if no name set.

func (*Replica) Snapshot added in v0.3.3

func (r *Replica) Snapshot(ctx context.Context) error

Snapshot copies the entire database to the replica path.

func (*Replica) SnapshotDir

func (r *Replica) SnapshotDir(generation string) string

SnapshotDir returns the path to a generation's snapshot directory.

func (*Replica) SnapshotPath

func (r *Replica) SnapshotPath(generation string, index int) string

SnapshotPath returns the path to a snapshot file.

func (*Replica) SnapshotReader

func (r *Replica) SnapshotReader(ctx context.Context, generation string, index int) (io.ReadCloser, error)

SnapshotReader returns a reader for snapshot data at the given generation/index.

func (*Replica) Snapshots

func (r *Replica) Snapshots(ctx context.Context) ([]*litestream.SnapshotInfo, error)

Snapshots returns a list of available snapshots in the replica.

func (*Replica) Start

func (r *Replica) Start(ctx context.Context) (err error)

Start starts replication for a given generation.

func (*Replica) Stop

func (r *Replica) Stop(hard bool) (err error)

Stop cancels any outstanding replication and blocks until finished.

Performing a hard stop will close the DB file descriptor which could release locks on per-process locks. Hard stops should only be performed when stopping the entire process.

func (*Replica) Sync

func (r *Replica) Sync(ctx context.Context) (err error)

Sync replays data from the shadow WAL and uploads it to S3.

func (*Replica) Type

func (r *Replica) Type() string

Type returns the type of replica.

func (*Replica) WALDir

func (r *Replica) WALDir(generation string) string

WALDir returns the path to a generation's WAL directory

func (*Replica) WALReader

func (r *Replica) WALReader(ctx context.Context, generation string, index int) (io.ReadCloser, error)

WALReader returns a reader for WAL data at the given index. Returns os.ErrNotExist if no matching index is found.

func (*Replica) WALs

func (r *Replica) WALs(ctx context.Context) ([]*litestream.WALInfo, error)

WALs returns a list of available WAL files in the replica.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL