Documentation
¶
Overview ¶
Package stream implements a time-series-based storage which is consists of a sequence of element. Each element drops in a arbitrary interval. They are immutable, can not be updated or overwritten.
Index ¶
- Constants
- Variables
- func CreatePartFileReaderFromPath(partPath string, lfs fs.FileSystem) ([]queue.FileInfo, func())
- func CreateTestPartForDump(tmpPath string, fileSystem fs.FileSystem) (string, func())
- func ParsePartMetadata(fileSystem fs.FileSystem, partPath string) (queue.StreamingPartData, error)
- func VisitStreamsInTimeRange(tsdbRootPath string, timeRange timestamp.TimeRange, visitor Visitor, ...) ([]string, error)
- type Filter
- type NodeSelector
- type Query
- type SchemaService
- type Service
- type Stream
- type Visitor
Constants ¶
const PartTypeCore = "core"
PartTypeCore is the type of the core part.
Variables ¶
var ( // ErrStreamNotExist denotes a stream doesn't exist in the metadata repo. ErrStreamNotExist = errors.New("stream doesn't exist") )
Functions ¶
func CreatePartFileReaderFromPath ¶ added in v0.9.0
func CreatePartFileReaderFromPath(partPath string, lfs fs.FileSystem) ([]queue.FileInfo, func())
CreatePartFileReaderFromPath opens all files in a part directory and returns their FileInfo and a cleanup function.
func CreateTestPartForDump ¶ added in v0.9.0
func CreateTestPartForDump(tmpPath string, fileSystem fs.FileSystem) (string, func())
CreateTestPartForDump creates a test stream part for testing the dump tool. It takes a temporary path and a file system as input, generates test elements with various tag types, creates a memory part, flushes it to disk, and returns the path to the created part directory. Parameters:
tmpPath: the base directory where the part will be created. fileSystem: the file system to use for writing the part.
Returns:
The path to the created part directory and a cleanup function.
func ParsePartMetadata ¶ added in v0.9.0
func ParsePartMetadata(fileSystem fs.FileSystem, partPath string) (queue.StreamingPartData, error)
ParsePartMetadata parses the part metadata from the metadata.json file.
func VisitStreamsInTimeRange ¶ added in v0.9.0
func VisitStreamsInTimeRange(tsdbRootPath string, timeRange timestamp.TimeRange, visitor Visitor, segmentInterval storage.IntervalRule) ([]string, error)
VisitStreamsInTimeRange traverses stream segments within the specified time range and calls the visitor methods for series index, parts, and element indexes. This function works directly with the filesystem without requiring a database instance. Returns a list of segment suffixes that were visited.
Types ¶
type Filter ¶ added in v0.10.0
Filter interface provides a unified interface for both BloomFilter and DictionaryFilter.
type NodeSelector ¶ added in v0.9.0
type NodeSelector interface {
Locate(group, name string, shardID, replicaID uint32) (string, error)
fmt.Stringer
}
NodeSelector provides functionality to locate nodes for data distribution.
type Query ¶
type Query interface {
LoadGroup(name string) (schema.Group, bool)
Stream(stream *commonv1.Metadata) (Stream, error)
GetRemovalSegmentsTimeRange(group string) *timestamp.TimeRange
}
Query allow to retrieve elements in a series of streams.
type SchemaService ¶ added in v0.5.0
type SchemaService interface {
Query
Close()
}
SchemaService allows querying schema information.
type Service ¶
type Service interface {
run.PreRunner
run.Config
run.Service
Query
CollectDataInfo(context.Context, string) (*databasev1.DataInfo, error)
CollectLiaisonInfo(context.Context, string) (*databasev1.LiaisonInfo, error)
}
Service allows inspecting the stream elements.
func NewLiaison ¶ added in v0.9.0
func NewLiaison(metadata metadata.Repo, pipeline queue.Server, omr observability.MetricsRegistry, pm protector.Memory, dataNodeSelector node.Selector, tire2Client queue.Client, ) (Service, error)
NewLiaison creates a new stream liaison service with the given dependencies.
func NewReadonlyService ¶ added in v0.8.0
func NewReadonlyService(metadata metadata.Repo, omr observability.MetricsRegistry, pm protector.Memory) (Service, error)
NewReadonlyService returns a new readonly service.
func NewService ¶
func NewService( metadata metadata.Repo, pipeline queue.Server, omr observability.MetricsRegistry, pm protector.Memory, internalWritePipeline queue.Server, ) (Service, error)
NewService returns a new service.
type Stream ¶
type Stream interface {
GetSchema() *databasev1.Stream
GetIndexRules() []*databasev1.IndexRule
Query(ctx context.Context, opts model.StreamQueryOptions) (model.StreamQueryResult, error)
}
Stream allows inspecting elements' details.
type Visitor ¶ added in v0.9.0
type Visitor interface {
// VisitSeries visits the series index directory for a segment.
VisitSeries(segmentTR *timestamp.TimeRange, seriesIndexPath string, shardIDs []common.ShardID) error
// VisitPart visits a part directory within a shard.
VisitPart(segmentTR *timestamp.TimeRange, shardID common.ShardID, partPath string) error
// VisitElementIndex visits the element index directory within a shard.
VisitElementIndex(segmentTR *timestamp.TimeRange, shardID common.ShardID, indexPath string) error
}
Visitor defines the interface for visiting stream components.
Source Files
¶
- block.go
- block_metadata.go
- block_reader.go
- block_scanner.go
- block_writer.go
- elements.go
- flusher.go
- gc.go
- index.go
- introducer.go
- merger.go
- merger_policy.go
- metadata.go
- metrics.go
- part.go
- part_iter.go
- part_metadata.go
- primary_metadata.go
- query.go
- query_by_idx.go
- query_by_ts.go
- snapshot.go
- stream.go
- svc_liaison.go
- svc_standalone.go
- syncer.go
- tag.go
- tag_filter.go
- tag_metadata.go
- test_helper.go
- tracing.go
- tstable.go
- visitor.go
- wqueue.go
- write_data.go
- write_index.go
- write_liaison.go
- write_standalone.go