data

package

v1.0.0 Latest Latest Go to latest Published: Mar 26, 2022 License: BSD-3-Clause Imports: 18 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/brimdata/zed

Links

Open Source Insights

Documentation ¶

Index ¶

Constants
func RowObjectName(id ksuid.KSUID) string
func RowObjectPath(path *storage.URI, id ksuid.KSUID) *storage.URI
type FileKind
- func FileMatch(s string) (kind FileKind, id ksuid.KSUID, ok bool)
- func (k FileKind) Description() string
type Meta
type Object
- func NewObject() Object
type ObjectScan
- func NewObjectScan(o Object) *ObjectScan
- func (o *ObjectScan) NewReader(ctx context.Context, engine storage.Engine, path *storage.URI, ...) (*Reader, error)
type Reader
type Writer

Constants ¶

View Source

const (
	DefaultSeekStride = 64 * 1024
	DefaultThreshold  = 500 * 1024 * 1024
)

Variables ¶

This section is empty.

Functions ¶

func RowObjectName ¶

func RowObjectName(id ksuid.KSUID) string

func RowObjectPath ¶

func RowObjectPath(path *storage.URI, id ksuid.KSUID) *storage.URI

Types ¶

type FileKind ¶

type FileKind string

A FileKind is the first part of a file name, used to differentiate files when they are listed from the archive's backing store.

const (
	FileKindUnknown  FileKind = ""
	FileKindData     FileKind = "data"
	FileKindMetadata FileKind = "meta"
	FileKindSeek     FileKind = "seek"
)

func FileMatch ¶

func FileMatch(s string) (kind FileKind, id ksuid.KSUID, ok bool)

XXX this won't work right until we integrate segID

func (FileKind) Description ¶

func (k FileKind) Description() string

type Meta ¶

type Meta struct {
	First zed.Value `zed:"first"`
	Last  zed.Value `zed:"last"`
	Count uint64    `zed:"count"`
	Size  int64     `zed:"size"`
}

type Object ¶

type Object struct {
	ID   ksuid.KSUID `zed:"id"`
	Meta `zed:"meta"`
}

func NewObject ¶

func NewObject() Object

func (*Object) Equal ¶

func (o *Object) Equal(to *Object) bool

func (Object) IsZero ¶

func (o Object) IsZero() bool

func (*Object) NewWriter ¶

func (o *Object) NewWriter(ctx context.Context, engine storage.Engine, path *storage.URI, order order.Which, poolKey field.Path, seekIndexStride int) (*Writer, error)

NewWriter returns a writer for writing the data of a zng-row storage object as well as optionally creating a seek index for the row object when the seekIndexStride is non-zero. We assume all records are non-volatile until Close as zed.Values from the various record bodies are referenced across calls to Write.

func (Object) ObjectPrefix ¶

func (o Object) ObjectPrefix(path *storage.URI) *storage.URI

ObjectPrefix returns a prefix for the various objects that comprise a data object so they can all be deleted with the storage engine's DeleteByPrefix method.

func (Object) Range ¶

func (o Object) Range() string

func (Object) Remove ¶

func (o Object) Remove(ctx context.Context, engine storage.Engine, path *storage.URI) error

Remove deletes the row object and its seek index. Any 'not found' errors are ignored.

func (Object) RowObjectName ¶

func (o Object) RowObjectName() string

func (Object) RowObjectPath ¶

func (o Object) RowObjectPath(path *storage.URI) *storage.URI

func (Object) SeekObjectName ¶

func (o Object) SeekObjectName() string

func (Object) SeekObjectPath ¶

func (o Object) SeekObjectPath(path *storage.URI) *storage.URI

func (Object) Span ¶

func (o Object) Span(order order.Which) *extent.Generic

func (Object) String ¶

func (o Object) String() string

func (Object) StringRange ¶

func (o Object) StringRange() string

type ObjectScan ¶ added in v1.0.0

type ObjectScan struct {
	Object
	ScanRange seekindex.Range
}

func NewObjectScan ¶ added in v1.0.0

func NewObjectScan(o Object) *ObjectScan

func (*ObjectScan) NewReader ¶ added in v1.0.0

func (o *ObjectScan) NewReader(ctx context.Context, engine storage.Engine, path *storage.URI, scanRange extent.Span, cmp expr.CompareFn) (*Reader, error)

NewReader returns a Reader for this data object. If the object has a seek index and if the provided span skips part of the object, the seek index will be used to limit the reading window of the returned reader.

type Reader ¶

type Reader struct {
	io.Reader
	io.Closer
	TotalBytes int64
	ReadBytes  int64
}

type Writer ¶

type Writer struct {
	// contains filtered or unexported fields
}

Writer is a zio.Writer that writes a stream of sorted records into a data object.

func (*Writer) Abort ¶

func (w *Writer) Abort()

Abort is called when an error occurs during write. Errors are ignored because the write error will be more informative and should be returned.

func (*Writer) BytesWritten ¶

func (w *Writer) BytesWritten() int64

func (*Writer) Close ¶

func (w *Writer) Close(ctx context.Context) error

func (*Writer) Object ¶

func (w *Writer) Object() *Object

Object returns the Object written by the writer. This is only valid after Close() has returned a nil error.

func (*Writer) RecordsWritten ¶

func (w *Writer) RecordsWritten() uint64

func (*Writer) Write ¶

func (w *Writer) Write(rec *zed.Value) error

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL