mfs

package
v0.38.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 9, 2026 License: Apache-2.0, MIT Imports: 19 Imported by: 30

Documentation

Overview

Package mfs implements an in-memory model of a mutable IPFS filesystem.

The filesystem is rooted at a Root which contains a tree of Directory and File nodes. Changes to files and directories are accumulated in memory and flushed to the underlying DAG service.

Structure

  • Root: Top-level entry point, created via NewRoot, with optional republishing of the root CID on changes
  • Directory: A mutable directory that maps names to child nodes
  • File: A mutable file backed by a UnixFS DAG

Filesystem Operations

Top-level functions Mv, Lookup, and FlushPath operate on paths within the MFS tree. Directories support adding, removing, and listing entries. Files support reading and writing through FileDescriptor.

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrNotYetImplemented = errors.New("not yet implemented")
	ErrInvalidChild      = errors.New("invalid child node")
	ErrDirExists         = errors.New("directory already has entry by that name")
)
View Source 
var (
	ErrNotExist = errors.New("no such rootfs")
	ErrClosed   = errors.New("file closed")
)

TODO: Remove if not used.

View Source 
var ErrIsDirectory = errors.New("error: is a directory")

TODO: Remove if not used.

Functions

func Chmod added in v0.23.0 

func Chmod(rt *Root, pth string, mode os.FileMode) error

func FlushPath 

func FlushPath(ctx context.Context, rt *Root, pth string) (ipld.Node, error)

TODO: Document this function and link its functionality with the republisher.

func IsDir 

func IsDir(fsn FSNode) bool

IsDir checks whether the FSNode is dir type

func IsFile 

func IsFile(fsn FSNode) bool

IsFile checks whether the FSNode is file type

func Mkdir 

func Mkdir(r *Root, pth string, opts MkdirOpts, dirOpts ...Option) error

Mkdir creates a directory at 'path' under the root. Any Option values not explicitly provided are inherited from the root directory's current settings.

func Mv 

func Mv(r *Root, src, dst string) error

Mv moves the file or directory at 'src' to 'dst' TODO: Document what the strings 'src' and 'dst' represent.

func PutNode 

func PutNode(r *Root, path string, nd ipld.Node) error

PutNode inserts 'nd' at 'path' in the given mfs TODO: Rename or clearly document that this is not about nodes but actually MFS files/directories (that in the underlying representation can be considered as just nodes). TODO: Document why are we handling IPLD nodes in the first place when we are actually referring to files/directories (that is, it can't be any node, it has to have a specific format). TODO: Can this function add directories or just files? What would be the difference between adding a directory with this method and creating it with `Mkdir`.

func Touch added in v0.23.0 

func Touch(rt *Root, pth string, ts time.Time) error

Types

type Directory 

type Directory struct {
	// contains filtered or unexported fields
}

TODO: There's too much functionality associated with this structure, let's organize it (and if possible extract part of it elsewhere) and document the main features of `Directory` here.

func NewDirectory 

func NewDirectory(ctx context.Context, name string, node ipld.Node, parent parent, dserv ipld.DAGService, prov provider.MultihashProvider) (*Directory, error)

NewDirectory constructs a new MFS directory.

You probably don't want to call this directly. Instead, construct a new root using NewRoot.

func NewEmptyDirectory added in v0.30.0 

func NewEmptyDirectory(ctx context.Context, name string, p parent, dserv ipld.DAGService, prov provider.MultihashProvider, opts ...Option) (*Directory, error)

NewEmptyDirectory creates an empty MFS directory with the given [Option]s. The directory is added to the DAGService. To create a new MFS root use NewEmptyRoot instead.

func (*Directory) AddChild 

func (d *Directory) AddChild(name string, nd ipld.Node) error

AddChild adds the node 'nd' under this directory giving it the name 'name'

func (*Directory) Child 

func (d *Directory) Child(name string) (FSNode, error)

Child returns the child of this directory by the given name

func (*Directory) Flush 

func (d *Directory) Flush() error

func (*Directory) ForEachEntry 

func (d *Directory) ForEachEntry(ctx context.Context, f func(NodeListing) error) error

func (*Directory) GetCidBuilder 

func (d *Directory) GetCidBuilder() cid.Builder

GetCidBuilder gets the CID builder of the root node

func (*Directory) GetNode 

func (d *Directory) GetNode() (ipld.Node, error)

func (*Directory) List 

func (d *Directory) List(ctx context.Context) ([]NodeListing, error)

func (*Directory) ListNames 

func (d *Directory) ListNames(ctx context.Context) ([]string, error)

func (*Directory) Mkdir 

func (d *Directory) Mkdir(name string) (*Directory, error)

Mkdir creates a child directory that inherits settings from this directory.

func (*Directory) MkdirWithOpts added in v0.23.0 

func (d *Directory) MkdirWithOpts(name string, opts ...Option) (*Directory, error)

MkdirWithOpts creates a child directory with explicit [Option]s.

func (*Directory) ModTime added in v0.38.0 

func (d *Directory) ModTime() (time.Time, error)

ModTime returns the directory's last modification time from UnixFS metadata. Returns zero time when no mtime is stored.

func (*Directory) Mode added in v0.38.0 

func (d *Directory) Mode() (os.FileMode, error)

Mode returns the directory's POSIX permission bits from UnixFS metadata. Returns 0 when no mode is stored.

func (*Directory) Path 

func (d *Directory) Path() string

func (*Directory) SetCidBuilder 

func (d *Directory) SetCidBuilder(b cid.Builder)

SetCidBuilder sets the CID builder

func (*Directory) SetModTime added in v0.23.0 

func (d *Directory) SetModTime(ts time.Time) error

func (*Directory) SetMode added in v0.23.0 

func (d *Directory) SetMode(mode os.FileMode) error

func (*Directory) Type 

func (d *Directory) Type() NodeType

func (*Directory) Uncache 

func (d *Directory) Uncache(name string)
func (d *Directory) Unlink(name string) error

type FSNode 

type FSNode interface {
	GetNode() (ipld.Node, error)

	Flush() error
	Type() NodeType
	SetModTime(ts time.Time) error
	SetMode(mode os.FileMode) error
}

FSNode abstracts the `Directory` and `File` structures, it represents any child node in the MFS (i.e., all the nodes besides the `Root`). It is the counterpart of the `parent` interface which represents any parent node in the MFS (`Root` and `Directory`). (Not to be confused with the `unixfs.FSNode`.)

func DirLookup 

func DirLookup(d *Directory, pth string) (FSNode, error)

DirLookup will look up a file or directory at the given path under the directory 'd'

func Lookup 

func Lookup(r *Root, path string) (FSNode, error)

Lookup extracts the root directory and performs a lookup under it. TODO: Now that the root is always a directory, can this function be collapsed with `DirLookup`? Or at least be made a method of `Root`?

type File 

type File struct {
	RawLeaves bool
	// contains filtered or unexported fields
}

File represents a file in the MFS, its logic its mainly targeted to coordinating (potentially many) `FileDescriptor`s pointing to it.

func NewFile 

func NewFile(name string, node ipld.Node, parent parent, dserv ipld.DAGService, providing provider.MultihashProvider) (*File, error)

NewFile returns a NewFile object with the given parameters. If the CID version is non-zero RawLeaves will be enabled. A nil providing parameter means that MFS will not provide content to the routing system.

func (*File) Flush 

func (fi *File) Flush() error

TODO: Tight coupling with the `FileDescriptor`, at the very least this should be an independent function that takes a `File` argument and automates the open/flush/close operations.

TODO: Why do we need to flush a file that isn't opened? (the `OpenWriteOnly` seems to implicitly be targeting a closed file, a file we forgot to flush? can we close a file without flushing?)

func (*File) GetNode 

func (fi *File) GetNode() (ipld.Node, error)

GetNode returns the dag node associated with this file

TODO: Use this method and do not access the `nodeLock` directly anywhere else.

func (*File) ModTime added in v0.23.0 

func (fi *File) ModTime() (time.Time, error)

ModTime returns the files' last modification time.

func (*File) Mode added in v0.23.0 

func (fi *File) Mode() (os.FileMode, error)

func (*File) Open 

func (fi *File) Open(flags Flags) (_ FileDescriptor, _retErr error)

func (*File) SetModTime added in v0.23.0 

func (fi *File) SetModTime(ts time.Time) error

SetModTime sets the files' last modification time.

func (*File) SetMode added in v0.23.0 

func (fi *File) SetMode(mode os.FileMode) error

func (*File) Size 

func (fi *File) Size() (int64, error)

Size returns the size of this file

TODO: Should we be providing this API?

TODO: There's already a `FileDescriptor.Size()` that through the `DagModifier`'s `fileSize` function is doing pretty much the same thing as here, we should at least call that function and wrap the `ErrNotUnixfs` with an MFS text.

func (*File) Sync 

func (fi *File) Sync() error

func (*File) Type 

func (fi *File) Type() NodeType

Type returns the type FSNode this is.

type FileDescriptor 

type FileDescriptor interface {
	io.Reader
	CtxReadFull(context.Context, []byte) (int, error)

	io.Writer
	io.WriterAt

	io.Closer
	io.Seeker

	Truncate(int64) error
	Size() (int64, error)
	Flush() error
}

One `File` can have many `FileDescriptor`s associated to it (only one if it's RW, many if they are RO, see `File.desclock`). A `FileDescriptor` contains the "view" of the file (through an instance of a `DagModifier`), that's why it (and not the `File`) has the responsibility to `Flush` (which crystallizes that view in the `File`'s `Node`).

type Flags 

type Flags struct {
	Read  bool
	Write bool
	Sync  bool
}

type MkdirOpts 

type MkdirOpts struct {
	Mkparents bool // create intermediate directories as needed
	Flush     bool // flush the final directory after creation
}

MkdirOpts holds operation flags for Mkdir.

type NodeListing 

type NodeListing struct {
	Name string
	Type int
	Size int64
	Hash string
}

type NodeType 

type NodeType int
const (
	TFile NodeType = iota
	TDir
)

type Option added in v0.38.0 

type Option func(*options)

Option configures MFS root and directory creation.

Options follow the functional options pattern used by the underlying uio.DirectoryOption in ipld/unixfs/io. Most settings are inherited from the parent directory when not explicitly set.

Chunker is the only setting that uses a different inheritance path: it is stored per-directory and accessed by files via the [parent] interface's getChunker method, rather than being copied through options. This is because chunker is consumed by File.Open (not by the unixfs directory layer), and a single interface method is a cleaner abstraction than explicit copying for a value that never changes after root creation.

func WithChunker added in v0.37.0 

func WithChunker(c chunker.SplitterGen) Option

WithChunker sets the chunker factory for files created under this MFS root. If not set, chunker.DefaultSplitter is used.

Unlike other options, the chunker is not propagated through [options] but through the [parent] interface (see Option for details). This option only takes effect when passed to NewRoot or NewEmptyRoot.

func WithCidBuilder added in v0.38.0 

func WithCidBuilder(b cid.Builder) Option

WithCidBuilder sets the CID builder (version, codec, hash function) for new directories. If not set, the parent directory's builder is used.

func WithHAMTShardingSize added in v0.37.0 

func WithHAMTShardingSize(size int) Option

WithHAMTShardingSize sets the per-directory serialized block size threshold in bytes for converting to a HAMT shard. If not set, the global uio.HAMTShardingSize is used.

func WithMaxHAMTFanout added in v0.37.0 

func WithMaxHAMTFanout(n int) Option

WithMaxHAMTFanout sets the maximum fanout (bucket width) for HAMT sharded directories. Must be a power of 2 and a multiple of 8. 

func WithMaxLinks(n int) Option

WithMaxLinks sets the maximum number of directory entries before the directory is converted to a HAMT shard.

func WithModTime added in v0.38.0 

func WithModTime(t time.Time) Option

WithModTime sets the modification time on the created directory.

func WithMode added in v0.38.0 

func WithMode(mode os.FileMode) Option

WithMode sets the Unix permission bits on the created directory.

func WithSizeEstimationMode added in v0.37.0 

func WithSizeEstimationMode(mode uio.SizeEstimationMode) Option

WithSizeEstimationMode sets the method used to estimate directory size for HAMT sharding threshold decisions.

type PubFunc 

type PubFunc func(context.Context, cid.Cid) error

PubFunc is the user-defined function that determines exactly what logic entails "publishing" a `Cid` value.

type Republisher 

type Republisher struct {
	// contains filtered or unexported fields
}

Republisher manages when to publish a given entry.

func NewRepublisher 

func NewRepublisher(pf PubFunc, tshort, tlong time.Duration, lastPublished cid.Cid) *Republisher

NewRepublisher creates a new Republisher object to republish the given root using the given short and long time intervals.

func (*Republisher) Close 

func (rp *Republisher) Close() error

Close tells the republisher to stop and waits for it to stop.

func (*Republisher) Update 

func (rp *Republisher) Update(c cid.Cid)

Update the current value. The value will be published after a delay but each consecutive call to Update may extend this delay up to TimeoutLong.

func (*Republisher) WaitPub 

func (rp *Republisher) WaitPub(ctx context.Context) error

WaitPub waits for the current value to be published (or returns early if it already has).

type Root 

type Root struct {
	// contains filtered or unexported fields
}

Root represents the root of a filesystem tree.

func NewEmptyRoot added in v0.30.0 

func NewEmptyRoot(ctx context.Context, ds ipld.DAGService, pf PubFunc, prov provider.MultihashProvider, opts ...Option) (*Root, error)

NewEmptyRoot creates an empty Root directory with the given [Option]s. A republisher is created if PubFunc is not nil.

func NewRoot 

func NewRoot(ctx context.Context, ds ipld.DAGService, node *dag.ProtoNode, pf PubFunc, prov provider.MultihashProvider, opts ...Option) (*Root, error)

NewRoot creates a new Root from an existing DAG node and starts a republisher routine for it. The provided [Option]s configure the root directory's DAG-shape settings (CidBuilder, MaxLinks, etc.).

func (*Root) Close 

func (kr *Root) Close() error

func (*Root) Flush 

func (kr *Root) Flush() error

Flush signals that an update has occurred since the last publish, and updates the Root republisher. TODO: We are definitely abusing the "flush" terminology here.

func (*Root) FlushMemFree 

func (kr *Root) FlushMemFree(ctx context.Context) error

FlushMemFree flushes the root directory and then uncaches all of its links. This has the effect of clearing out potentially stale references and allows them to be garbage collected. CAUTION: Take care not to ever call this while holding a reference to any child directories. Those directories will be bad references and using them may have unintended racy side effects. A better implemented mfs system (one that does smarter internal caching and refcounting) shouldnt need this method. TODO: Review the motivation behind this method once the cache system is refactored.

func (*Root) GetChunker added in v0.37.0 

func (kr *Root) GetChunker() chunker.SplitterGen

GetChunker returns the chunker factory, or nil if using default.

func (*Root) GetDirectory 

func (kr *Root) GetDirectory() *Directory

GetDirectory returns the root directory.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.