io

package
v0.42.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 22, 2026 License: Apache-2.0, MIT Imports: 19 Imported by: 41

Documentation

Overview

Package io implements convenience objects for working with the IPFS UnixFS data format.

Directory Types

The package provides three directory implementations:

  • BasicDirectory: A flat directory storing all entries in a single node.
  • HAMTDirectory: A sharded directory using HAMT (Hash Array Mapped Trie) for large directories.
  • DynamicDirectory: Automatically switches between Basic and HAMT based on size thresholds.

HAMT Sharding

Directories can automatically convert between basic (flat) and HAMT (sharded) formats based on configurable thresholds:

Use WithMaxLinks, WithMaxHAMTFanout, and WithSizeEstimationMode to configure directories at creation time.

File Reading

DagReader provides io.Reader and io.Seeker interfaces for reading UnixFS file content from the DAG.

CID Profiles (IPIP-499)

UnixFSProfile defines import settings for deterministic CID generation. Predefined profiles ensure cross-implementation compatibility:

See https://specs.ipfs.tech/ipips/ipip-0499/ for specification details.

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrIsDir            = errors.New("this dag node is a directory")
	ErrCantReadSymlinks = errors.New("cannot currently read symlinks")
	ErrUnkownNodeType   = errors.New("unknown node type")
	ErrSeekNotSupported = errors.New("file does not support seeking")
)

Common errors

View Source 
var (
	// UnixFS_v0_2015 matches the unixfs-v0-2015 profile from IPIP-499.
	// Documents default UnixFS DAG construction parameters used by Kubo
	// through version 0.39 when producing CIDv0. Use when compatibility
	// with historical CIDv0 references is required.
	//
	// Historical note on FileDAGWidth=174:
	//
	//	var roughLinkBlockSize = 1 << 13 // 8KB
	//	var roughLinkSize = 34 + 8 + 5   // sha256 multihash + size + no name
	//	                                 // + protobuf framing
	//	var DefaultLinksPerBlock = (roughLinkBlockSize / roughLinkSize)
	//	                         = ( 8192 / 47 )
	//	                         = (approximately) 174
	UnixFS_v0_2015 = UnixFSProfile{
		CIDVersion:         0,
		MhType:             mh.SHA2_256,
		ChunkSize:          int64(256 * unitKiB),
		FileDAGWidth:       174,
		RawLeaves:          false,
		HAMTShardingSize:   int(256 * unitKiB),
		HAMTSizeEstimation: SizeEstimationLinks,
		HAMTShardWidth:     256,
	}

	// UnixFS_v1_2025 matches the unixfs-v1-2025 profile from IPIP-499.
	// Opinionated profile for deterministic CID generation with CIDv1.
	// Use for cross-implementation CID determinism with modern settings.
	UnixFS_v1_2025 = UnixFSProfile{
		CIDVersion:         1,
		MhType:             mh.SHA2_256,
		ChunkSize:          int64(1 * unitMiB),
		FileDAGWidth:       1024,
		RawLeaves:          true,
		HAMTShardingSize:   int(256 * unitKiB),
		HAMTSizeEstimation: SizeEstimationBlock,
		HAMTShardWidth:     256,
	}
)

Predefined profiles matching IPIP-499 specifications.

View Source 
var DefaultShardWidth = 256

DefaultShardWidth is the default value used for hamt sharding width. Needs to be a power of two (shard entry size) and multiple of 8 (bitfield size).

Thread safety: this global is not safe for concurrent modification. Set it once during program initialization, before starting any imports.

View Source 
var ErrInvalidHAMTFanout = errors.New("HAMT fanout must be a positive power of 2 and multiple of 8")

ErrInvalidHAMTFanout is returned when an invalid HAMT fanout value is provided. Valid values must be a positive power of 2 and a multiple of 8 (e.g., 8, 16, 32, 64, 128, 256).

View Source 
var ErrNotADir = errors.New("merkledag node was not a directory or shard")

ErrNotADir implies that the given node was not a unixfs directory

View Source 
var HAMTShardingSize = int(256 * unitKiB)

HAMTShardingSize is a global option that allows switching to a HAMTDirectory when the BasicDirectory grows above the size (in bytes) signalled by this flag. The default size of 0 disables the option.

Size estimation depends on HAMTSizeEstimation mode:

  • SizeEstimationLinks (default): estimates using link name + CID byte lengths, ignoring Tsize, protobuf overhead, and UnixFS Data field (mode/mtime).
  • SizeEstimationBlock: computes exact serialized dag-pb block size arithmetically, including the UnixFS Data field and all protobuf encoding overhead.
  • SizeEstimationDisabled: ignores size entirely, uses only MaxLinks.

Threshold behavior: directory converts to HAMT when estimatedSize > HAMTShardingSize. A directory exactly at the threshold stays basic (threshold value is NOT included).

Thread safety: this global is not safe for concurrent modification. Set it once during program initialization, before starting any imports.

View Source 
var HAMTSizeEstimation = SizeEstimationLinks

HAMTSizeEstimation controls which method is used to estimate directory size for HAMT sharding decisions. Default is SizeEstimationLinks for backward compatibility. Modern software should set this to SizeEstimationBlock for accurate estimation.

Thread safety: this global is not safe for concurrent modification. Set it once during program initialization, before starting any imports.

Functions

func ResolveUnixfsOnce 

func ResolveUnixfsOnce(ctx context.Context, ds ipld.NodeGetter, nd ipld.Node, names []string) (*ipld.Link, []string, error)

ResolveUnixfsOnce resolves a single hop of a path through a graph in a unixfs context. This includes handling traversing sharded directories.

Types

type BasicDirectory 

type BasicDirectory struct {
	// contains filtered or unexported fields
}

BasicDirectory is the basic implementation of `Directory`. All the entries are stored in a single node.

func NewBasicDirectory added in v0.30.0 

func NewBasicDirectory(dserv ipld.DAGService, opts ...DirectoryOption) (*BasicDirectory, error)

NewBasicDirectory creates an empty basic directory with the given options.

func NewBasicDirectoryFromNode added in v0.30.0 

func NewBasicDirectoryFromNode(dserv ipld.DAGService, node *mdag.ProtoNode) *BasicDirectory

NewBasicDirectoryFromNode creates a basic directory wrapping the given ProtoNode.

func (*BasicDirectory) AddChild 

func (d *BasicDirectory) AddChild(ctx context.Context, name string, node ipld.Node) error

AddChild implements the `Directory` interface. It adds (or replaces) a link to the given `node` under `name`.

func (*BasicDirectory) EnumLinksAsync 

func (d *BasicDirectory) EnumLinksAsync(ctx context.Context) <-chan format.LinkResult

EnumLinksAsync returns a channel which will receive Links in the directory as they are enumerated, where order is not guaranteed

func (*BasicDirectory) Find 

func (d *BasicDirectory) Find(ctx context.Context, name string) (ipld.Node, error)

Find implements the `Directory` interface. 

func (d *BasicDirectory) ForEachLink(_ context.Context, f func(*ipld.Link) error) error

ForEachLink implements the `Directory` interface.

func (*BasicDirectory) GetCidBuilder 

func (d *BasicDirectory) GetCidBuilder() cid.Builder

GetCidBuilder implements the `Directory` interface.

func (*BasicDirectory) GetHAMTShardingSize added in v0.37.0 

func (d *BasicDirectory) GetHAMTShardingSize() int

GetHAMTShardingSize returns the per-directory threshold for HAMT sharding. If not set (0), the global HAMTShardingSize is used.

func (*BasicDirectory) GetMaxHAMTFanout added in v0.30.0 

func (d *BasicDirectory) GetMaxHAMTFanout() int

GetMaxLinks returns the configured HAMTFanout. 

func (d *BasicDirectory) GetMaxLinks() int

GetMaxLinks returns the configured MaxLinks.

func (*BasicDirectory) GetNode 

func (d *BasicDirectory) GetNode() (ipld.Node, error)

GetNode implements the `Directory` interface.

func (*BasicDirectory) GetSizeEstimationMode added in v0.37.0 

func (d *BasicDirectory) GetSizeEstimationMode() SizeEstimationMode

GetSizeEstimationMode returns the method used to estimate serialized dag-pb block size for BasicDirectory to HAMTDirectory conversion decisions. Returns the instance-specific mode if set, otherwise the global HAMTSizeEstimation. 

func (d *BasicDirectory) Links(ctx context.Context) ([]*ipld.Link, error)

Links implements the `Directory` interface.

func (*BasicDirectory) RemoveChild 

func (d *BasicDirectory) RemoveChild(ctx context.Context, name string) error

RemoveChild implements the `Directory` interface.

func (*BasicDirectory) SetCidBuilder 

func (d *BasicDirectory) SetCidBuilder(builder cid.Builder)

SetCidBuilder implements the `Directory` interface.

func (*BasicDirectory) SetHAMTShardingSize added in v0.37.0 

func (d *BasicDirectory) SetHAMTShardingSize(size int)

SetHAMTShardingSize sets the per-directory threshold for HAMT sharding. Used when inheriting settings from a parent directory after loading from disk.

func (*BasicDirectory) SetMaxHAMTFanout added in v0.30.0 

func (d *BasicDirectory) SetMaxHAMTFanout(n int)

SetMaxHAMTFanout sets the HAMT fanout for use during Basic->HAMT conversion. The value is validated when the directory is created via NewBasicDirectory. Valid values: positive power of 2 and multiple of 8. Use 0 for default. 

func (d *BasicDirectory) SetMaxLinks(n int)

SetMaxLinks sets the maximum number of links for the Directory, but has no side effects until new children are added (in which case the new limit applies).

func (*BasicDirectory) SetSizeEstimationMode added in v0.37.0 

func (d *BasicDirectory) SetSizeEstimationMode(mode SizeEstimationMode)

SetSizeEstimationMode sets the method used to estimate serialized dag-pb block size. Used when inheriting settings from a parent directory after loading from disk. Note: This only recomputes estimatedSize, not totalLinks, since link count is independent of the estimation mode.

func (*BasicDirectory) SetStat added in v0.30.0 

func (d *BasicDirectory) SetStat(mode os.FileMode, mtime time.Time)

SetStat stores mode and/or mtime values for use during Basic<->HAMT conversions. Pass zero for mode or zero time for mtime to leave that field unchanged. Note: clearing previously set values is not supported; to clear mode/mtime, create a new directory without them.

This method does NOT modify the underlying node's Data field or cachedBlockSize. The stored values are only applied when creating a new directory during conversion (via WithStat option). If you need to update an existing node's mode/mtime, modify the node directly and replace the directory, or use MFS which handles this for you.

type DagReader 

type DagReader interface {
	ReadSeekCloser
	Size() uint64
	Mode() os.FileMode
	ModTime() time.Time
	CtxReadFull(context.Context, []byte) (int, error)
}

A DagReader provides read-only read and seek acess to a unixfs file. Different implementations of readers are used for the different types of unixfs/protobuf-encoded nodes.

func NewDagReader 

func NewDagReader(ctx context.Context, n ipld.Node, serv ipld.NodeGetter) (DagReader, error)

NewDagReader creates a new reader object that reads the data represented by the given node, using the passed in DAGService for data retrieval.

type Directory 

type Directory interface {
	// SetCidBuilder sets the CID Builder of the root node.
	SetCidBuilder(cid.Builder)

	// AddChild adds a (name, key) pair to the root node.
	AddChild(context.Context, string, ipld.Node) error

	// ForEachLink applies the given function to Links in the directory.
	ForEachLink(context.Context, func(*ipld.Link) error) error

	// EnumLinksAsync returns a channel which will receive Links in the directory
	// as they are enumerated, where order is not guaranteed
	EnumLinksAsync(context.Context) <-chan format.LinkResult

	// Links returns the all the links in the directory node.
	Links(context.Context) ([]*ipld.Link, error)

	// Find returns the root node of the file named 'name' within this directory.
	// In the case of HAMT-directories, it will traverse the tree.
	//
	// Returns os.ErrNotExist if the child does not exist.
	Find(context.Context, string) (ipld.Node, error)

	// RemoveChild removes the child with the given name.
	//
	// Returns os.ErrNotExist if the child doesn't exist.
	RemoveChild(context.Context, string) error

	// GetNode returns the root of this directory.
	GetNode() (ipld.Node, error)

	// GetCidBuilder returns the CID Builder used.
	GetCidBuilder() cid.Builder

	// GetMaxLinks returns the configured value for MaxLinks.
	GetMaxLinks() int

	// SetMaxLinks sets the number of links for the directory. Used when converting
	// between Basic and HAMT.
	SetMaxLinks(n int)

	// GetMaxHAMTFanout returns the configured value for MaxHAMTFanout.
	GetMaxHAMTFanout() int

	// SetMaxHAMTFanout sets the max width of shards when using a HAMT.
	// It must be a power of 2 and multiple of 8. Used when converting
	// between Basic and HAMT.
	SetMaxHAMTFanout(n int)

	// SetStat sets the stat information for the directory. Used when
	// converting between Basic and HAMT.
	SetStat(os.FileMode, time.Time)

	// GetSizeEstimationMode returns the method used to estimate serialized dag-pb block size
	// for directory type conversion decisions.
	// Returns the instance-specific mode if set, otherwise the global HAMTSizeEstimation.
	GetSizeEstimationMode() SizeEstimationMode

	// SetSizeEstimationMode sets the method used to estimate serialized dag-pb block size.
	// Used when inheriting settings from a parent directory after loading from disk.
	SetSizeEstimationMode(SizeEstimationMode)

	// GetHAMTShardingSize returns the per-directory threshold for HAMT sharding.
	// If not set (0), the global HAMTShardingSize is used.
	GetHAMTShardingSize() int

	// SetHAMTShardingSize sets the per-directory threshold for HAMT sharding.
	// Used when inheriting settings from a parent directory after loading from disk.
	SetHAMTShardingSize(size int)
}

Directory defines a UnixFS directory. It is used for creating, reading and editing directories. It allows to work with different directory schemes, like the basic or the HAMT implementation.

It just allows to perform explicit edits on a single directory, working with directory trees is out of its scope, they are managed by the MFS layer (which is the main consumer of this interface).

func NewDirectory 

func NewDirectory(dserv ipld.DAGService, opts ...DirectoryOption) (Directory, error)

NewDirectory returns a Directory implemented by DynamicDirectory containing a BasicDirectory that automatically converts to a from a HAMTDirectory based on HAMTShardingSize, MaxLinks and MaxHAMTFanout (when set).

func NewDirectoryFromNode 

func NewDirectoryFromNode(dserv ipld.DAGService, node ipld.Node) (Directory, error)

NewDirectoryFromNode loads a unixfs directory from the given IPLD node and DAGService.

type DirectoryOption added in v0.30.0 

type DirectoryOption func(d Directory)

A DirectoryOption can be used to initialize directories.

func WithCidBuilder added in v0.30.0 

func WithCidBuilder(cb cid.Builder) DirectoryOption

WithCidBuilder sets the CidBuilder for new Directories.

func WithMaxHAMTFanout added in v0.30.0 

func WithMaxHAMTFanout(n int) DirectoryOption

WithMaxHAMTFanout establishes the maximum fanout factor (number of links) for a HAMT directory or a Dynamic directory with an underlying HAMT directory:

  • On Dynamic directories, it specifies the HAMT fanout when a HAMT is used. When unset, DefaultShardWidth applies.
  • On pure HAMT directories, it sets the ShardWidth, otherwise DefaultShardWidth is used.

Valid values must be a positive power of 2 and a multiple of 8 (e.g., 8, 16, 32, 64, 128, 256, 512, 1024).

If an invalid value is provided, NewBasicDirectory and NewHAMTDirectory will return ErrInvalidHAMTFanout. Use 0 or omit this option to use DefaultShardWidth. 

func WithMaxLinks(n int) DirectoryOption

WithMaxLinks stablishes the max number of links allowed for a Basic directory or a Dynamic directory with an underlying Basic directory:

- On Dynamic directories using a BasicDirectory, it can trigger conversion to HAMT when set and exceeded. The subsequent HAMT nodes will use MaxHAMTFanout as ShardWidth when set, or DefaultShardWidth otherwise. Conversion can be triggered too based on HAMTShardingSize.

- On Dynamic directories using a HAMTDirectory, it can trigger conversion to BasicDirectory when the number of directory entries is below MaxLinks (and HAMTShardingSize allows).

- On pure Basic directories, it causes an error when adding more than MaxLinks children.

func WithSizeEstimationMode added in v0.37.0 

func WithSizeEstimationMode(mode SizeEstimationMode) DirectoryOption

WithSizeEstimationMode sets the method used to estimate serialized dag-pb block size when deciding whether to convert between BasicDirectory and HAMTDirectory. This must be set at directory creation time; mode cannot be changed afterwards.

func WithStat added in v0.30.0 

func WithStat(mode os.FileMode, mtime time.Time) DirectoryOption

WithStat can be used to set the empty directory node permissions.

type DynamicDirectory 

type DynamicDirectory struct {
	Directory
}

DynamicDirectory wraps a Directory interface and provides extra logic to switch from BasicDirectory to HAMTDirectory and backwards based on size.

func (*DynamicDirectory) AddChild 

func (d *DynamicDirectory) AddChild(ctx context.Context, name string, nd ipld.Node) error

AddChild implements the `Directory` interface. We check when adding new entries if we should switch to HAMTDirectory according to global option(s).

func (*DynamicDirectory) RemoveChild 

func (d *DynamicDirectory) RemoveChild(ctx context.Context, name string) error

RemoveChild implements the `Directory` interface. Used in the case where we wrap a HAMTDirectory that might need to be downgraded to a BasicDirectory. The upgrade path is in AddChild.

func (*DynamicDirectory) SetMaxHAMTFanout added in v0.30.0 

func (d *DynamicDirectory) SetMaxHAMTFanout(n int)

SetMaxHAMTFanout sets the maximum shard width for HAMT directories. This operation does not produce any side effect and only takes effect on a conversion from Basic to HAMT directory. 

func (d *DynamicDirectory) SetMaxLinks(n int)

SetMaxLinks sets the maximum number of links for the underlying Basic directory when used. This operation does not produce any side effects, but the new value may trigger Basic-to-HAMT conversions when adding new children to Basic directories, or HAMT-to-Basic conversion when operating with HAMT directories.

func (*DynamicDirectory) SetStat added in v0.30.0 

func (d *DynamicDirectory) SetStat(mode os.FileMode, mtime time.Time)

SetStat stores mode and/or mtime values for use during Basic<->HAMT conversions. Pass zero for mode or zero time for mtime to leave that field unchanged. See BasicDirectory.SetStat for full documentation.

type HAMTDirectory 

type HAMTDirectory struct {
	// contains filtered or unexported fields
}

HAMTDirectory is the HAMT implementation of `Directory`. (See package `hamt` for more information.)

func NewHAMTDirectory added in v0.30.0 

func NewHAMTDirectory(dserv ipld.DAGService, sizeChange int, opts ...DirectoryOption) (*HAMTDirectory, error)

NewHAMTDirectory creates an empty HAMT directory with the given options.

func NewHAMTDirectoryFromNode added in v0.30.0 

func NewHAMTDirectoryFromNode(dserv ipld.DAGService, node ipld.Node) (*HAMTDirectory, error)

NewHAMTDirectoryFromNode creates a HAMT directory from the given node, which must correspond to an existing HAMT.

func (*HAMTDirectory) AddChild 

func (d *HAMTDirectory) AddChild(ctx context.Context, name string, nd ipld.Node) error

AddChild implements the `Directory` interface.

func (*HAMTDirectory) EnumLinksAsync 

func (d *HAMTDirectory) EnumLinksAsync(ctx context.Context) <-chan format.LinkResult

EnumLinksAsync returns a channel which will receive Links in the directory as they are enumerated, where order is not guaranteed

func (*HAMTDirectory) Find 

func (d *HAMTDirectory) Find(ctx context.Context, name string) (ipld.Node, error)

Find implements the `Directory` interface. It will traverse the tree. 

func (d *HAMTDirectory) ForEachLink(ctx context.Context, f func(*ipld.Link) error) error

ForEachLink implements the `Directory` interface.

func (*HAMTDirectory) GetCidBuilder 

func (d *HAMTDirectory) GetCidBuilder() cid.Builder

GetCidBuilder implements the `Directory` interface.

func (*HAMTDirectory) GetHAMTShardingSize added in v0.37.0 

func (d *HAMTDirectory) GetHAMTShardingSize() int

GetHAMTShardingSize returns the per-directory threshold for HAMT sharding. If not set (0), the global HAMTShardingSize is used.

func (*HAMTDirectory) GetMaxHAMTFanout added in v0.30.0 

func (d *HAMTDirectory) GetMaxHAMTFanout() int

GetMaxHAMTFanout returns the maxHAMTFanout value from a HAMTDirectory. 

func (d *HAMTDirectory) GetMaxLinks() int

GetMaxLinks returns the maxLinks value from a HAMTDirectory.

func (*HAMTDirectory) GetNode 

func (d *HAMTDirectory) GetNode() (ipld.Node, error)

GetNode implements the `Directory` interface.

func (*HAMTDirectory) GetSizeEstimationMode added in v0.37.0 

func (d *HAMTDirectory) GetSizeEstimationMode() SizeEstimationMode

GetSizeEstimationMode returns the method used to estimate serialized dag-pb block size for HAMTDirectory to BasicDirectory conversion decisions. Returns the instance-specific mode if set, otherwise the global HAMTSizeEstimation. 

func (d *HAMTDirectory) Links(ctx context.Context) ([]*ipld.Link, error)

Links implements the `Directory` interface.

func (*HAMTDirectory) RemoveChild 

func (d *HAMTDirectory) RemoveChild(ctx context.Context, name string) error

RemoveChild implements the `Directory` interface.

func (*HAMTDirectory) SetCidBuilder 

func (d *HAMTDirectory) SetCidBuilder(builder cid.Builder)

SetCidBuilder implements the `Directory` interface.

func (*HAMTDirectory) SetHAMTShardingSize added in v0.37.0 

func (d *HAMTDirectory) SetHAMTShardingSize(size int)

SetHAMTShardingSize sets the per-directory threshold for HAMT sharding. Used when inheriting settings from a parent directory after loading from disk.

func (*HAMTDirectory) SetMaxHAMTFanout added in v0.30.0 

func (d *HAMTDirectory) SetMaxHAMTFanout(n int)

SetMaxHAMTFanout sets the HAMT fanout. The value is validated when the directory is created via NewHAMTDirectory. Valid values: positive power of 2 and multiple of 8. Use 0 for default. 

func (d *HAMTDirectory) SetMaxLinks(n int)

SetMaxLinks has no effect and only exists to support Dynamic directories.

func (*HAMTDirectory) SetSizeEstimationMode added in v0.37.0 

func (d *HAMTDirectory) SetSizeEstimationMode(mode SizeEstimationMode)

SetSizeEstimationMode sets the method used to estimate serialized dag-pb block size. Used when inheriting settings from a parent directory after loading from disk.

func (*HAMTDirectory) SetStat added in v0.30.0 

func (d *HAMTDirectory) SetStat(mode os.FileMode, mtime time.Time)

SetStat stores mode and/or mtime values for use during HAMT->Basic conversions and also propagates them to the underlying shard for inclusion in GetNode(). Pass zero for mode or zero time for mtime to leave that field unchanged. See BasicDirectory.SetStat for full documentation.

type ReadSeekCloser 

type ReadSeekCloser interface {
	io.Reader
	io.Seeker
	io.Closer
	io.WriterTo
}

A ReadSeekCloser implements interfaces to read, copy, seek and close.

type SizeEstimationMode added in v0.37.0 

type SizeEstimationMode int

SizeEstimationMode defines how directory size is estimated for HAMT sharding decisions. If unsure which mode to use, prefer SizeEstimationBlock for accurate estimation. 

const (
	// SizeEstimationLinks estimates size using link names + CID byte lengths.
	// This is the legacy behavior: sum(len(link.Name) + len(link.Cid.Bytes()))
	// This mode ignores Tsize, protobuf overhead, and optional metadata fields
	// (mode, mtime), which may lead to underestimation and delayed HAMT conversion.
	// Use only when compatibility with legacy DAGs and software is required.
	SizeEstimationLinks SizeEstimationMode = iota

	// SizeEstimationBlock estimates size using full serialized dag-pb block size.
	// This correctly accounts for all fields including Tsize, protobuf varints,
	// and optional metadata (mode, mtime). Use this mode for accurate HAMT
	// threshold decisions and cross-implementation CID determinism.
	SizeEstimationBlock

	// SizeEstimationDisabled disables size-based HAMT threshold entirely.
	// When set, the decision to convert between BasicDirectory and HAMTDirectory
	// is based solely on the number of links, controlled by MaxLinks option
	// (set via WithMaxLinks). HAMTShardingSize is ignored. Use this mode when
	// you want explicit control over directory sharding based on entry count
	// rather than serialized size.
	SizeEstimationDisabled
)

type UnixFSProfile added in v0.37.0 

type UnixFSProfile struct {
	// CIDVersion specifies the CID version (0 or 1).
	// CIDv0 only supports dag-pb codec, CIDv1 supports multiple codecs.
	CIDVersion int

	// MhType is the multihash function code that determines the hash algorithm.
	// Historical default is mh.SHA2_256 (0x12).
	MhType uint64

	// ChunkSize is the maximum size of file chunks (in bytes).
	// Common values: 256 KiB (legacy), 1 MiB (modern).
	ChunkSize int64

	// FileDAGWidth is the maximum number of links per file DAG node.
	// Common values: 174 (legacy), 1024 (modern).
	FileDAGWidth int

	// RawLeaves controls whether file leaf nodes use raw codec (true)
	// or dag-pb wrapped UnixFS nodes (false).
	// Raw leaves require CIDv1; CIDv0 only supports dag-pb leaves.
	RawLeaves bool

	// HAMTShardingSize is the threshold (in bytes) for switching to HAMT directories.
	// 0 disables HAMT sharding entirely.
	HAMTShardingSize int

	// HAMTSizeEstimation controls how directory size is estimated for HAMT threshold.
	// Use SizeEstimationBlock for accurate estimation, SizeEstimationLinks for legacy
	// behavior, or SizeEstimationDisabled to rely solely on link count.
	HAMTSizeEstimation SizeEstimationMode

	// HAMTShardWidth is the fanout for HAMT directory nodes.
	// Must be a power of 2 and multiple of 8.
	HAMTShardWidth int
}

UnixFSProfile defines a set of UnixFS import settings for CID determinism. Profiles ensure that different implementations produce the same CID for the same input when using the same profile.

See IPIP-499 for details: https://github.com/ipfs/specs/pull/499

func (UnixFSProfile) ApplyGlobals added in v0.37.0 

func (p UnixFSProfile) ApplyGlobals()

ApplyGlobals sets the global variables to match this profile's settings. This affects all subsequent file and directory import operations. Note: RawLeaves and CidBuilder are not globals; pass them to DAG builder options.

Thread safety: this function modifies global variables and is not safe for concurrent use. Call it once during program initialization, before starting any imports. Do not call from multiple goroutines.

func (UnixFSProfile) CidBuilder added in v0.37.0 

func (p UnixFSProfile) CidBuilder() cid.Builder

CidBuilder returns a cid.Builder configured for this profile. Pass this to DagBuilderParams.CidBuilder when importing files.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.