Documentation
¶
Overview ¶
Package zst implements the reading and writing of ZST storage objects to and from any Zed format. The ZST storage format is described at https://github.com/brimdata/zed/blob/main/docs/formats/zst.md.
A ZST storage object must be seekable (e.g., a local file or S3 object), so, unlike ZNG, streaming of ZST objects is not supported.
The zst/vector package handles reading and writing Zed sequence data to vectors, while the zst package comprises the API used to read and write ZST objects.
Index ¶
- Constants
- func ReadIntVector(segments []vector.Segment, r io.ReaderAt) ([]int32, error)
- type FileMeta
- type Object
- func NewObject(zctx *zed.Context, r io.ReaderAt, size int64) (*Object, error)
- func NewObjectFromPath(ctx context.Context, zctx *zed.Context, engine storage.Engine, path string) (*Object, error)
- func NewObjectFromStorageReader(zctx *zed.Context, r storage.Reader) (*Object, error)
- func NewObjectFromStorageReaderNoCloser(zctx *zed.Context, r storage.Reader) (*Object, error)
- func NewObjectFromURI(ctx context.Context, zctx *zed.Context, engine storage.Engine, ...) (*Object, error)
- type Reader
- type Writer
- type WriterURI
Constants ¶
const ( FileType = "zst" Version = 3 )
const ( MaxSegmentThresh = vector.MaxSegmentThresh MaxSkewThresh = 512 * 1024 * 1024 )
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Object ¶
type Object struct {
// contains filtered or unexported fields
}
func NewObjectFromPath ¶
func NewObjectFromStorageReader ¶ added in v1.3.0
func NewObjectFromStorageReaderNoCloser ¶ added in v1.3.0
func NewObjectFromURI ¶ added in v1.3.0
func (*Object) FetchMetadata ¶ added in v1.3.0
func (*Object) NewReassemblyReader ¶
type Reader ¶
type Reader struct {
*Object
// contains filtered or unexported fields
}
Reader implements zio.Reader and io.Closer. It reads a vector zst object to generate a stream of zed.Records.
func NewReader ¶
NewReader returns a Reader ready to read a zst object as zed.Records. Close() should be called when done. This embeds a zst.Object.
func NewReaderFromPath ¶
func NewReaderFromStorageReader ¶ added in v1.3.0
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
Writer implements the zio.Writer interface. A Writer creates a vector zst object from a stream of zed.Records.
Source Files
Directories