Documentation
¶
Overview ¶
Package shipper detects directories on the local file system and uploads them to a block storage.
Index ¶
- Constants
- Variables
- func WriteMetaFile(logger log.Logger, path string, meta *Meta) error
- type Meta
- type Option
- func WithAllowOutOfOrderUploads(allow bool) Option
- func WithHashFunc(hashFunc metadata.HashFunc) Option
- func WithLabels(lbls func() labels.Labels) Option
- func WithLogger(logger log.Logger) Option
- func WithMetaFileName(name string) Option
- func WithRegisterer(r prometheus.Registerer) Option
- func WithSkipCorruptedBlocks(skip bool) Option
- func WithSource(source metadata.SourceType) Option
- func WithUploadCompacted(upload bool) Option
- func WithUploadConcurrency(concurrency int) Option
- type Shipper
Constants ¶
const ( // DefaultMetaFilename is the default JSON filename for meta information. DefaultMetaFilename = "thanos.shipper.json" // MetaVersion1 represents 1 version of meta. MetaVersion1 = 1 )
Variables ¶
var (
ErrorSyncBlockCorrupted = errors.New("corrupted blocks found")
)
Functions ¶
Types ¶
type Meta ¶
Meta defines the format thanos.shipper.json file that the shipper places in the data directory.
func ReadMetaFile ¶
ReadMetaFile reads the given meta from <dir>/thanos.shipper.json.
type Option ¶ added in v0.39.0
type Option func(*shipperOptions)
func WithAllowOutOfOrderUploads ¶ added in v0.39.0
WithAllowOutOfOrderUploads sets whether to allow out of order uploads.
func WithHashFunc ¶ added in v0.39.0
WithHashFunc sets the hash function.
func WithLabels ¶ added in v0.39.0
WithLabels sets the labels function.
func WithLogger ¶ added in v0.39.0
WithLogger sets the logger.
func WithMetaFileName ¶ added in v0.39.0
WithMetaFileName sets the meta file name.
func WithRegisterer ¶ added in v0.39.0
func WithRegisterer(r prometheus.Registerer) Option
WithRegisterer sets the Prometheus registerer.
func WithSkipCorruptedBlocks ¶ added in v0.39.0
WithSkipCorruptedBlocks sets whether to skip corrupted blocks.
func WithSource ¶ added in v0.39.0
func WithSource(source metadata.SourceType) Option
WithSource sets the metadata source type.
func WithUploadCompacted ¶ added in v0.39.0
WithUploadCompacted sets whether to upload compacted blocks.
func WithUploadConcurrency ¶ added in v0.41.0
WithUploadConcurrency sets the number of goroutines to use when uploading block files.
type Shipper ¶
type Shipper struct {
// contains filtered or unexported fields
}
Shipper watches a directory for matching files and directories and uploads them to a remote data store.
func New ¶
New creates a new shipper that detects new TSDB blocks in dir and uploads them to remote if necessary. It attaches the Thanos metadata section in each meta JSON file. If uploadCompacted is enabled, it also uploads compacted blocks which are already in filesystem.
func (*Shipper) Sync ¶
Sync performs a single synchronization, which ensures all non-compacted local blocks have been uploaded to the object bucket once.
If uploaded.
It is not concurrency-safe, however it is compactor-safe (running concurrently with compactor is ok).
Source Files