extractor

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 27, 2025 License: Apache-2.0, MIT Imports: 15 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func IsMissing 

func IsMissing(err error) ([]cid.Cid, bool)

IsMissing returns the missing CIDs if error is IncompleteError.

Types 

type ChildLink struct {
	Cid  cid.Cid
	Name string // empty for file chunks
}

ChildLink represents a link to a child block with its name (for directories)

type ExtractingCarReader 

type ExtractingCarReader struct {
	// contains filtered or unexported fields
}

ExtractingCarReader reads a CAR stream and extracts UnixFS content directly to disk without storing blocks in memory. It uses a frontier-based approach to track expected blocks and verify the DAG is complete.

func NewExtractingCarReader 

func NewExtractingCarReader(ext *Extractor, rootCid cid.Cid) *ExtractingCarReader

NewExtractingCarReader creates a reader that extracts from a CAR stream.

func (*ExtractingCarReader) OnBlock 

func (r *ExtractingCarReader) OnBlock(cb func(int))

OnBlock registers a callback invoked for each block processed.

func (*ExtractingCarReader) ReadAndExtract 

func (r *ExtractingCarReader) ReadAndExtract(ctx context.Context, rdr io.Reader) (uint64, uint64, error)

ReadAndExtract reads the CAR stream and extracts content to disk. Returns the number of blocks and bytes processed.

type Extractor 

type Extractor struct {
	// contains filtered or unexported fields
}

Extractor handles streaming extraction of UnixFS content to disk. It processes blocks one at a time and writes file/directory content as it arrives, without buffering entire files in memory.

func New 

func New(outputDir string) (*Extractor, error)

New creates a new Extractor that writes to the given output directory.

func (*Extractor) Close 

func (e *Extractor) Close() error

Close closes any open file handles. Call when extraction is complete.

func (*Extractor) IsProcessed 

func (e *Extractor) IsProcessed(c cid.Cid) bool

IsProcessed returns true if the block has already been processed.

func (*Extractor) OnFileComplete 

func (e *Extractor) OnFileComplete(cb FileProgressCallback)

OnFileComplete sets a callback invoked when a file extraction finishes.

func (*Extractor) OnFileProgress 

func (e *Extractor) OnFileProgress(cb FileProgressCallback)

OnFileProgress sets a callback invoked when file data is written.

func (*Extractor) OnFileStart 

func (e *Extractor) OnFileStart(cb FileProgressCallback)

OnFileStart sets a callback invoked when a file extraction begins.

func (*Extractor) ProcessBlock 

func (e *Extractor) ProcessBlock(ctx context.Context, block blocks.Block) ([]cid.Cid, error)

ProcessBlock decodes a block and processes it according to its UnixFS type. Returns child CIDs that should be fetched next.

func (*Extractor) SetRootPath 

func (e *Extractor) SetRootPath(rootCid cid.Cid, name string)

SetRootPath sets the path context for the root CID. Call this before processing to set the base name for single-file extractions. The name is sanitized to prevent path traversal.

type FileProgressCallback 

type FileProgressCallback func(path string, written, total int64)

FileProgressCallback is called when file extraction progress is made. path is relative to output dir, written is bytes written so far, total is expected size (-1 if unknown).

type IncompleteError 

type IncompleteError struct {
	Missing []cid.Cid
}

IncompleteError indicates the CAR stream ended before all expected blocks were received. The Missing field contains CIDs that were not found.

func (*IncompleteError) Error 

func (e *IncompleteError) Error() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.