file

package
v0.55.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 10, 2026 License: Apache-2.0 Imports: 17 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CommitWriter 

func CommitWriter(w *Writer) (*block.BlockRef, *block.Cursor, error)

CommitWriter commits any pending writes using a block transaction. Note: the block transaction must match the handle's block cursor.

func FetchToBuffer 

func FetchToBuffer(ctx context.Context, bcs *block.Cursor, buf *bytes.Buffer) error

FetchToBuffer fetches a full File to a buffer.

func FetchToBytes 

func FetchToBytes(ctx context.Context, bcs *block.Cursor) ([]byte, error)

FetchToBytes fetches to a bytes slice.

func NewFileBlock 

func NewFileBlock() block.Block

NewFileBlock builds a new file root block.

func NewRangeSet 

func NewRangeSet(v *[]*Range, bcs *block.Cursor) *sbset.SubBlockSet

NewRangeSet builds a new range set container.

bcs should be located at the range set sub-block.

Types

type File 

type File struct {

	// TotalSize is the total size of the file.
	TotalSize uint64 `protobuf:"varint,1,opt,name=total_size,json=totalSize,proto3" json:"totalSize,omitempty"`
	// RootBlob, if set, contains the entire file in a blob.
	// Used when there is a single Range of data starting at index 0.
	// If unset and len(ranges) == 0, the file is empty (all zeros).
	RootBlob *blob.Blob `protobuf:"bytes,2,opt,name=root_blob,json=rootBlob,proto3" json:"rootBlob,omitempty"`
	// RangeNonce is the next range nonce id to use.
	RangeNonce uint64 `protobuf:"varint,3,opt,name=range_nonce,json=rangeNonce,proto3" json:"rangeNonce,omitempty"`
	// Ranges contains file data ranges.
	// Files are sparse when created.
	// Ranges may overlap.
	Ranges []*Range `protobuf:"bytes,4,rep,name=ranges,proto3" json:"ranges,omitempty"`
	// contains filtered or unexported fields
}

File defines a pattern for storing a block-addressed file. Changes are transactional and deterministic. File write ranges and write mode are considered. Concurrent O_APPEND is implemented. Blobs are addressed as they are added, and periodically compacted. Some operations partially overwrite old blobs without rewriting them. The File object contains metadata and might contain links to sub-objects. Rabin fingerprinting is used to select deterministic chunk sizes.

func BuildFileWithBytes 

func BuildFileWithBytes(
	ctx context.Context,
	bcs *block.Cursor,
	data []byte,
	buildBlobOpts *blob.BuildBlobOpts,
) (*File, error)

BuildFileWithBytes builds a file with data, building the root blob. The new root will be stored at bcs.

func BuildFileWithReader 

func BuildFileWithReader(
	ctx context.Context,
	bcs *block.Cursor,
	rdr io.Reader,
	buildBlobOpts *blob.BuildBlobOpts,
) (*File, error)

BuildFileWithReader builds a file with a reader, building the root blob. The new root will be stored at bcs.

func NewFileWithBlob 

func NewFileWithBlob(rootBlob *blob.Blob) *File

NewFileWithBlob builds a file with a single root blob.

func UnmarshalFile 

func UnmarshalFile(ctx context.Context, bcs *block.Cursor) (*File, error)

UnmarshalFile unmarshals the File block. Returns nil, nil if empty

func (*File) ApplySubBlock 

func (f *File) ApplySubBlock(id uint32, next block.SubBlock) error

ApplySubBlock applies a sub-block change with a field id.

func (*File) BlockAliasIdentity added in v0.52.0 

func (f *File) BlockAliasIdentity() *block.AliasIdentityToken

BlockAliasIdentity returns the in-memory alias token for File.

func (*File) CloneMessageVT 

func (m *File) CloneMessageVT() protobuf_go_lite.CloneMessage

func (*File) CloneVT 

func (m *File) CloneVT() *File

func (*File) EqualMessageVT 

func (this *File) EqualMessageVT(thatMsg any) bool

func (*File) EqualVT 

func (this *File) EqualVT(that *File) bool

func (*File) GetRangeNonce 

func (x *File) GetRangeNonce() uint64

func (*File) GetRanges 

func (x *File) GetRanges() []*Range

func (*File) GetRootBlob 

func (x *File) GetRootBlob() *blob.Blob

func (*File) GetSubBlockCtor 

func (f *File) GetSubBlockCtor(id uint32) block.SubBlockCtor

GetSubBlockCtor returns a function which creates or returns the existing sub-block at reference id. Can return nil to indicate invalid reference id.

func (*File) GetSubBlocks 

func (f *File) GetSubBlocks() map[uint32]block.SubBlock

GetSubBlocks returns all constructed sub-blocks by ID. May return nil, and values may also be nil.

func (*File) GetTotalSize 

func (x *File) GetTotalSize() uint64

func (*File) IsNil 

func (f *File) IsNil() bool

IsNil returns if the object is nil.

func (*File) MarshalBlock 

func (f *File) MarshalBlock() ([]byte, error)

MarshalBlock marshals the block to binary. This is the initial step of marshaling, before transformations.

func (*File) MarshalJSON 

func (x *File) MarshalJSON() ([]byte, error)

MarshalJSON marshals the File to JSON.

func (*File) MarshalProtoJSON 

func (x *File) MarshalProtoJSON(s *json.MarshalState)

MarshalProtoJSON marshals the File message to JSON.

func (*File) MarshalProtoText 

func (x *File) MarshalProtoText() string

func (*File) MarshalToSizedBufferVT 

func (m *File) MarshalToSizedBufferVT(dAtA []byte) (int, error)

func (*File) MarshalToVT 

func (m *File) MarshalToVT(dAtA []byte) (int, error)

func (*File) MarshalVT 

func (m *File) MarshalVT() (dAtA []byte, err error)

func (*File) ProtoMessage 

func (*File) ProtoMessage()

func (*File) Reset 

func (x *File) Reset()

func (*File) SizeVT 

func (m *File) SizeVT() (n int)

func (*File) String 

func (x *File) String() string

func (*File) UnmarshalBlock 

func (f *File) UnmarshalBlock(data []byte) error

UnmarshalBlock unmarshals the block to the object. This is the final step of decoding, after transformations.

func (*File) UnmarshalJSON 

func (x *File) UnmarshalJSON(b []byte) error

UnmarshalJSON unmarshals the File from JSON.

func (*File) UnmarshalProtoJSON 

func (x *File) UnmarshalProtoJSON(s *json.UnmarshalState)

UnmarshalProtoJSON unmarshals the File message from JSON.

func (*File) UnmarshalVT 

func (m *File) UnmarshalVT(dAtA []byte) error

func (*File) Validate 

func (f *File) Validate() error

Validate performs cursory validation of the file object.

type Handle 

type Handle struct {
	// contains filtered or unexported fields
}

Handle is a open file handle, using a block transaction and a Not concurrency safe.

func NewHandle 

func NewHandle(
	ctx context.Context,
	bcs *block.Cursor,
	root *File,
) *Handle

NewHandle constructs a new reader. bcs is located at the root of the file.

func (*Handle) Close 

func (r *Handle) Close() error

Close closes the reader, canceling the context. Not concurrency safe.

func (*Handle) ComputeStorageSize 

func (r *Handle) ComputeStorageSize(ctx context.Context) (uint64, error)

ComputeStorageSize computes the total size of all blocks making up the File.

note: not accurate until the btx has been committed.

func (*Handle) GetCursor 

func (r *Handle) GetCursor() *block.Cursor

GetCursor returns the underlying cursor pointing to *File.

func (*Handle) GetRef 

func (r *Handle) GetRef() *block.BlockRef

GetRef returns the root reference.

func (*Handle) Read 

func (r *Handle) Read(p []byte) (n int, err error)

Read implements the reader interface. Read and Seek are not concurrent safe. It fills p across range boundaries until p is full, EOF is reached, or a selected blob reader stops making progress.

func (*Handle) Seek 

func (r *Handle) Seek(offset int64, whence int) (int64, error)

Seek implements the seeking interface. Seek sets the offset for the next Read or Write to offset, interpreted according to whence: SeekStart means relative to the start of the file, SeekCurrent means relative to the current offset, and SeekEnd means relative to the end. Seek returns the new offset relative to the start of the file and an error, if any.

Seeking to an offset before the start of the file is an error. Seeking to any positive offset is legal, but the behavior of subsequent Read and Seek are not concurrent safe.

func (*Handle) Size 

func (r *Handle) Size() uint64

Size returns the total size of the file.

type HandleRangeSlice 

type HandleRangeSlice struct {
	// contains filtered or unexported fields
}

HandleRangeSlice is a sortable slice of ranges.

func NewHandleRangeSlice 

func NewHandleRangeSlice(h *Handle) *HandleRangeSlice

NewHandleRangeSlice builds a slice with a handle for sorting the block graph.

func (*HandleRangeSlice) Len 

func (r *HandleRangeSlice) Len() int

Len is the number of elements in the collection.

func (*HandleRangeSlice) Less 

func (r *HandleRangeSlice) Less(i, j int) bool

Less reports whether the element with index i should sort before the element with index j.

func (HandleRangeSlice) Swap 

func (r HandleRangeSlice) Swap(i, j int)

Swap swaps the elements with indexes i and j.

type Range 

type Range struct {

	// Nonce is the incrementing nonce of the range.
	// Ranges with a higher nonce overwrite lower nonce.
	Nonce uint64 `protobuf:"varint,1,opt,name=nonce,proto3" json:"nonce,omitempty"`
	// Start contains the starting index of the range.
	Start uint64 `protobuf:"varint,2,opt,name=start,proto3" json:"start,omitempty"`
	// Length contains the len of data in the range.
	// Start + length = end index + 1.
	Length uint64 `protobuf:"varint,3,opt,name=length,proto3" json:"length,omitempty"`
	// Ref contains the blob ref.
	// If the ref is empty, the range represents a hole (zeros).
	Ref *block.BlockRef `protobuf:"bytes,4,opt,name=ref,proto3" json:"ref,omitempty"`
	// contains filtered or unexported fields
}

Range contains a chunk of a file. Ranges are sorted by start, then nonce (ascending).

func (*Range) ApplyBlockRef 

func (r *Range) ApplyBlockRef(id uint32, ptr *block.BlockRef) error

ApplyBlockRef applies a ref change with a field id. The reference may be nil if the child block is nil.

func (*Range) BlockAliasIdentity added in v0.52.0 

func (r *Range) BlockAliasIdentity() *block.AliasIdentityToken

BlockAliasIdentity returns the in-memory alias token for Range.

func (*Range) CloneMessageVT 

func (m *Range) CloneMessageVT() protobuf_go_lite.CloneMessage

func (*Range) CloneVT 

func (m *Range) CloneVT() *Range

func (*Range) EqualMessageVT 

func (this *Range) EqualMessageVT(thatMsg any) bool

func (*Range) EqualVT 

func (this *Range) EqualVT(that *Range) bool

func (*Range) FollowBlob 

func (r *Range) FollowBlob(bcs *block.Cursor) *block.Cursor

FollowBlob follows the blob reference.

func (*Range) GetBlockRefCtor 

func (r *Range) GetBlockRefCtor(id uint32) block.Ctor

GetBlockRefCtor returns the constructor for the block at the ref id. Return nil to indicate invalid ref ID or unknown.

func (*Range) GetBlockRefs 

func (r *Range) GetBlockRefs() (map[uint32]*block.BlockRef, error)

GetBlockRefs returns all block references by ID. May return nil, and values may also be nil. Note: this does not include pending references (in a cursor)

func (*Range) GetLength 

func (x *Range) GetLength() uint64

func (*Range) GetNonce 

func (x *Range) GetNonce() uint64

func (*Range) GetRef 

func (x *Range) GetRef() *block.BlockRef

func (*Range) GetStart 

func (x *Range) GetStart() uint64

func (*Range) IsNil 

func (r *Range) IsNil() bool

IsNil checks if the object is nil.

func (*Range) LessThanRange 

func (r *Range) LessThanRange(other *Range) bool

LessThanRange compares to ranges, returning true if r < other.

func (*Range) MarshalJSON 

func (x *Range) MarshalJSON() ([]byte, error)

MarshalJSON marshals the Range to JSON.

func (*Range) MarshalProtoJSON 

func (x *Range) MarshalProtoJSON(s *json.MarshalState)

MarshalProtoJSON marshals the Range message to JSON.

func (*Range) MarshalProtoText 

func (x *Range) MarshalProtoText() string

func (*Range) MarshalToSizedBufferVT 

func (m *Range) MarshalToSizedBufferVT(dAtA []byte) (int, error)

func (*Range) MarshalToVT 

func (m *Range) MarshalToVT(dAtA []byte) (int, error)

func (*Range) MarshalVT 

func (m *Range) MarshalVT() (dAtA []byte, err error)

func (*Range) ProtoMessage 

func (*Range) ProtoMessage()

func (*Range) Reset 

func (x *Range) Reset()

func (*Range) SizeVT 

func (m *Range) SizeVT() (n int)

func (*Range) String 

func (x *Range) String() string

func (*Range) UnmarshalJSON 

func (x *Range) UnmarshalJSON(b []byte) error

UnmarshalJSON unmarshals the Range from JSON.

func (*Range) UnmarshalProtoJSON 

func (x *Range) UnmarshalProtoJSON(s *json.UnmarshalState)

UnmarshalProtoJSON unmarshals the Range message from JSON.

func (*Range) UnmarshalVT 

func (m *Range) UnmarshalVT(dAtA []byte) error

type RangeSlice 

type RangeSlice []*Range

RangeSlice is a sortable slice of ranges.

func (RangeSlice) Len 

func (r RangeSlice) Len() int

Len is the number of elements in the collection.

func (RangeSlice) Less 

func (r RangeSlice) Less(i, j int) bool

Less reports whether the element with index i should sort before the element with index j.

func (RangeSlice) LocatePosition 

func (r RangeSlice) LocatePosition(pos int) (*Range, int, bool)

LocatePosition locates the range covering a position pos. returns the index of that range returns nil, 0, false if no range covering pos is located.

func (RangeSlice) Swap 

func (r RangeSlice) Swap(i, j int)

Swap swaps the elements with indexes i and j.

type Writer 

type Writer struct {
	*Handle
	// contains filtered or unexported fields
}

Writer is a handle that can write to a handle.

func NewWriter 

func NewWriter(
	h *Handle,
	btx *block.Transaction,
	buildBlobOpts *blob.BuildBlobOpts,
) *Writer

NewWriter builds a new writer handle. btx can be nil buildBlobOpts can be nil

func (*Writer) Reset 

func (w *Writer) Reset()

Reset completely clears the contents of the file.

func (*Writer) Truncate 

func (w *Writer) Truncate(size uint64) error

Truncate shrinks or extends the file handle to the given size.

func (*Writer) Write 

func (w *Writer) Write(p []byte) (n int, err error)

Write writes to the handle, immediately flushing if btx is set.

func (*Writer) WriteBlob 

func (w *Writer) WriteBlob(index, size uint64, ref *block.BlockRef) error

WriteBlob writes a blob to an index in a new range. Implies removing any ranges which are completely occluded.

func (*Writer) WriteBytes 

func (w *Writer) WriteBytes(index uint64, data []byte) error

WriteBytes writes bytes to a blob and then to an index.

func (*Writer) WriteFrom 

func (w *Writer) WriteFrom(index uint64, dataLen int64, dataRdr io.Reader) error

WriteFrom writes data from a reader to a blob and then an index.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.