filesystem

package
v0.1.147 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 7, 2026 License: AGPL-3.0, AGPL-3.0-or-later Imports: 35 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	BackendFUSE = "fuse"
	BackendNFS  = "nfs"
)

Backend name constants.

Variables

View Source 
var (
	// ErrFUSEUnavailable indicates that the runtime cannot mount via FUSE.
	ErrFUSEUnavailable = errors.New("fuse unavailable")
	// ErrNFSHelperMissing indicates mount(8) could not find mount.nfs helper.
	ErrNFSHelperMissing = errors.New("nfs helper missing")
)
View Source 
var (
	ErrNotFound     = fs.ErrNotExist
	ErrPermission   = fs.ErrPermission
	ErrExist        = fs.ErrExist
	ErrNotDir       = syscall.ENOTDIR
	ErrIsDir        = syscall.EISDIR
	ErrNotEmpty     = syscall.ENOTEMPTY
	ErrReadOnly     = syscall.EROFS
	ErrInvalid      = fs.ErrInvalid
	ErrIO           = syscall.EIO
	ErrNoSpace      = syscall.ENOSPC
	ErrNotSupported = syscall.ENOTSUP
	ErrNoAttr       = syscall.ENODATA // ENOATTR on macOS maps to ENODATA
)

Functions

func GenerateDirectoryID 

func GenerateDirectoryID(parentID, name string, version int) string

func MountAccessUnaryInterceptor added in v0.1.47 

func MountAccessUnaryInterceptor(session string) grpc.UnaryClientInterceptor

MountAccessUnaryInterceptor attaches mount telemetry metadata to every unary RPC so access events can be grouped by session and marked as mount-origin.

Types

type AccessCollector added in v0.1.47 

type AccessCollector struct {
	// contains filtered or unexported fields
}

AccessCollector buffers logical read events and periodically flushes them to the gateway AccessLogService. Record() is non-blocking; events are dropped if the local buffer is full.

func NewAccessCollector added in v0.1.47 

func NewAccessCollector(client pb.AccessLogServiceClient, cfg AccessCollectorConfig) *AccessCollector

func NewAccessCollectorWithToken added in v0.1.47 

func NewAccessCollectorWithToken(client pb.AccessLogServiceClient, token string) *AccessCollector

NewAccessCollectorWithToken is a convenience wrapper that uses default buffering/flush behavior while attaching bearer auth for ingest RPCs.

func (*AccessCollector) Close added in v0.1.47 

func (c *AccessCollector) Close()

func (*AccessCollector) Record added in v0.1.47 

func (c *AccessCollector) Record(event *pb.AccessLogEvent)

type AccessCollectorConfig added in v0.1.47 

type AccessCollectorConfig struct {
	BufferSize    int
	BatchSize     int
	FlushInterval time.Duration
	AuthToken     string // optional bearer token for AccessLogService auth
}

AccessCollectorConfig configures client-side batched access log ingestion.

type BackPointer 

type BackPointer struct {
	BirthParentID string `json:"birthParentId"`
	NameVersion   int    `json:"nameVersion"`
}

type Config 

type Config struct {
	MountPoint  string
	GatewayAddr string
	Token       string
	Verbose     bool
	Uid         *uint32 // File owner uid (nil = use current user, 0 = root)
	Gid         *uint32 // File owner gid (nil = use current user, 0 = root)
	Backend     string  // "fuse", "nfs", or "" for platform auto-detect
	Compression string  // compression strategy: "strip", "distill", "chain", or "" (disabled)
	Session     string  // custom access session ID; defaults to workspace ID if empty
	AccessLog   bool    // enable access logging; when false, no session header is sent
}

Config configures the filesystem mount

type DirEntry 

type DirEntry struct {
	Name  string
	Mode  uint32
	Ino   uint64
	Size  int64 // File size (0 for directories)
	Mtime int64 // Unix timestamp (0 = use current time)
}

type DirectoryAccessMetadata 

type DirectoryAccessMetadata struct {
	PID         string            `json:"pid"`
	ID          string            `json:"id"`
	Permission  uint32            `json:"permission"`
	RenameList  map[string]string `json:"renameList,omitempty"`
	BackPointer *BackPointer      `json:"backPointer,omitempty"`
}

type DirectoryContentMetadata 

type DirectoryContentMetadata struct {
	Id         string               `json:"id"`
	EntryList  []string             `json:"entryList"`
	Timestamps map[string]time.Time `json:"timestamps"`
}

type FileHandle 

type FileHandle uint64

type FileInfo 

type FileInfo struct {
	Ino   uint64
	Size  int64
	Mode  uint32
	Nlink uint32
	Uid   uint32
	Gid   uint32
	Atime time.Time
	Mtime time.Time
	Ctime time.Time
}

func (*FileInfo) IsDir 

func (fi *FileInfo) IsDir() bool

func (*FileInfo) IsRegular 

func (fi *FileInfo) IsRegular() bool
func (fi *FileInfo) IsSymlink() bool

type FileMetadata 

type FileMetadata struct {
	ID       string `json:"id"`
	PID      string `json:"pid"`
	Name     string `json:"name"`
	FileData []byte `json:"fileData"`
}

type Filesystem 

type Filesystem struct {
	// contains filtered or unexported fields
}

Filesystem connects to the gateway via gRPC and exposes a virtual filesystem. It can be mounted via FUSE or NFS depending on the configured backend.

func NewFilesystem 

func NewFilesystem(cfg Config) (*Filesystem, error)

NewFilesystem creates a new Filesystem that connects to the gateway via gRPC. All filesystem operations go through the gateway which handles Redis/S3.

func (*Filesystem) Chmod 

func (f *Filesystem) Chmod(path string, mode uint32) error

func (*Filesystem) Chown 

func (f *Filesystem) Chown(path string, uid, gid uint32) error

func (*Filesystem) Create 

func (f *Filesystem) Create(path string, flags int, mode uint32) (FileHandle, error)

Write operations - delegate to vnodes or fallback storage

func (*Filesystem) Destroy 

func (f *Filesystem) Destroy()

func (*Filesystem) Flush 

func (f *Filesystem) Flush(path string, fh FileHandle) error

Flush and Fsync

Flush is called on every close() of a file descriptor. We intentionally do NOT force-sync to S3 here — that would block every close() for ~300ms. Data safety is ensured by:

  • Release (last fd close): ForceFlush to S3
  • Fsync (explicit sync): ForceFlush to S3
  • AsyncWriter debounce timer: uploads within 500ms
  • Read path: serves dirty data from asyncWriter memory

func (*Filesystem) Fsync 

func (f *Filesystem) Fsync(path string, datasync bool, fh FileHandle) error

func (*Filesystem) Getattr 

func (f *Filesystem) Getattr(path string) (*FileInfo, error)

func (*Filesystem) Getxattr 

func (f *Filesystem) Getxattr(path, name string) ([]byte, error)

Getxattr returns empty data for all xattrs (we do not store them).

func (*Filesystem) Init 

func (f *Filesystem) Init() error

func (*Filesystem) IsDestroyed 

func (f *Filesystem) IsDestroyed() bool

func (*Filesystem) IsMounted 

func (f *Filesystem) IsMounted() bool
func (f *Filesystem) Link(oldpath, newpath string) error

func (*Filesystem) Listxattr 

func (f *Filesystem) Listxattr(path string) ([]string, error)

func (*Filesystem) Mkdir 

func (f *Filesystem) Mkdir(path string, mode uint32) error

func (*Filesystem) Mount 

func (f *Filesystem) Mount() error

func (*Filesystem) Open 

func (f *Filesystem) Open(path string, flags int) (FileHandle, error)

func (*Filesystem) Opendir 

func (f *Filesystem) Opendir(path string) (FileHandle, error)

func (*Filesystem) Read 

func (f *Filesystem) Read(path string, buf []byte, off int64, fh FileHandle) (int, *vnode.ReadAttribution, error)

func (*Filesystem) Readdir 

func (f *Filesystem) Readdir(path string) ([]DirEntry, error)
func (f *Filesystem) Readlink(path string) (string, error)

Symlink operations

func (*Filesystem) RegisterVNode 

func (f *Filesystem) RegisterVNode(node vnode.VirtualNode)

RegisterVNode registers a virtual node handler for a path prefix

func (*Filesystem) Release 

func (f *Filesystem) Release(path string, fh FileHandle) error

func (*Filesystem) Releasedir 

func (f *Filesystem) Releasedir(path string, fh FileHandle) error

func (*Filesystem) Removexattr 

func (f *Filesystem) Removexattr(path, name string) error

Removexattr silently succeeds since we don't store xattrs.

func (*Filesystem) Rename 

func (f *Filesystem) Rename(oldpath, newpath string) error

func (*Filesystem) Rmdir 

func (f *Filesystem) Rmdir(path string) error

func (*Filesystem) SetAccessCollector added in v0.1.47 

func (f *Filesystem) SetAccessCollector(collector *AccessCollector)

SetAccessCollector sets the mount-side access collector used to flush logical read events to the gateway.

func (*Filesystem) SetStorageFallback 

func (f *Filesystem) SetStorageFallback(node vnode.VirtualNode)

SetStorageFallback sets the fallback vnode for unmatched storage paths

func (*Filesystem) Setxattr 

func (f *Filesystem) Setxattr(path, name string, value []byte, flags int) error

Setxattr silently accepts and discards extended attributes.

func (*Filesystem) Statfs 

func (f *Filesystem) Statfs() (*StatInfo, error)
func (f *Filesystem) Symlink(target, newpath string) error

func (*Filesystem) Truncate 

func (f *Filesystem) Truncate(path string, size int64, fh FileHandle) error
func (f *Filesystem) Unlink(path string) error

func (*Filesystem) Unmount 

func (f *Filesystem) Unmount() error

func (*Filesystem) Utimens 

func (f *Filesystem) Utimens(path string, atime, mtime *int64) error

func (*Filesystem) Write 

func (f *Filesystem) Write(path string, buf []byte, off int64, fh FileHandle) (int, error)

type FuseBackend added in v0.1.36 

type FuseBackend struct {
	// contains filtered or unexported fields
}

FuseBackend mounts the filesystem using FUSE (via cgofuse). This is the default backend on all platforms where FUSE is available. On macOS it uses fuse-t; on Linux it uses libfuse.

func NewFuseBackend added in v0.1.36 

func NewFuseBackend() *FuseBackend

NewFuseBackend creates a new FUSE mount backend.

func (*FuseBackend) Mount added in v0.1.36 

func (b *FuseBackend) Mount(fs *Filesystem, mountPoint string) (err error)

func (*FuseBackend) Unmount added in v0.1.36 

func (b *FuseBackend) Unmount() error

type FuseTrace 

type FuseTrace struct {
	// contains filtered or unexported fields
}

FuseTrace is an opt-in, low-overhead tracer for the FUSE layer.

Enable with: 

AIRSTORE_FUSE_TRACE=1

Optional: 

AIRSTORE_FUSE_TRACE_INTERVAL=2s   (default: 2s)
AIRSTORE_FUSE_TRACE_SLOW_MS=50    (default: 50ms; 0 disables slow-op logging)

type GRPCConfig 

type GRPCConfig struct {
	GatewayAddr string
	Token       string
}

GRPCConfig holds configuration for connecting to the gateway.

type GRPCMetadataEngine 

type GRPCMetadataEngine struct {
	// contains filtered or unexported fields
}

GRPCMetadataEngine implements MetadataEngine and LegacyMetadataEngine via gRPC.

func NewGRPCMetadataEngine 

func NewGRPCMetadataEngine(cfg GRPCConfig) (*GRPCMetadataEngine, error)

NewGRPCMetadataEngine creates a new gRPC-based metadata engine.

func (*GRPCMetadataEngine) Close 

func (m *GRPCMetadataEngine) Close() error

Close closes the gRPC connection.

func (*GRPCMetadataEngine) Conn 

func (m *GRPCMetadataEngine) Conn() *grpc.ClientConn

Conn returns the underlying gRPC connection for sharing with other services.

func (*GRPCMetadataEngine) DeleteDirectory 

func (m *GRPCMetadataEngine) DeleteDirectory(parentID, name string, version int) error

func (*GRPCMetadataEngine) GetDirectoryAccessMetadata 

func (m *GRPCMetadataEngine) GetDirectoryAccessMetadata(pid, name string) (*DirectoryAccessMetadata, error)

func (*GRPCMetadataEngine) GetDirectoryContentMetadata 

func (m *GRPCMetadataEngine) GetDirectoryContentMetadata(id string) (*DirectoryContentMetadata, error)

func (*GRPCMetadataEngine) GetFileMetadata 

func (m *GRPCMetadataEngine) GetFileMetadata(pid, name string) (*FileMetadata, error)

func (*GRPCMetadataEngine) ListDirectory 

func (m *GRPCMetadataEngine) ListDirectory(path string) []DirEntry

func (*GRPCMetadataEngine) RenameDirectory 

func (m *GRPCMetadataEngine) RenameDirectory(oldPID, oldName, newPID, newName string, version int) error

func (*GRPCMetadataEngine) SaveDirectoryAccessMetadata 

func (m *GRPCMetadataEngine) SaveDirectoryAccessMetadata(meta *DirectoryAccessMetadata) error

func (*GRPCMetadataEngine) SaveDirectoryContentMetadata 

func (m *GRPCMetadataEngine) SaveDirectoryContentMetadata(meta *DirectoryContentMetadata) error

func (*GRPCMetadataEngine) SaveFileMetadata 

func (m *GRPCMetadataEngine) SaveFileMetadata(meta *FileMetadata) error

type LegacyMetadataEngine 

type LegacyMetadataEngine interface {
	GetDirectoryContentMetadata(id string) (*DirectoryContentMetadata, error)
	GetDirectoryAccessMetadata(pid, name string) (*DirectoryAccessMetadata, error)
	GetFileMetadata(pid, name string) (*FileMetadata, error)
	SaveDirectoryContentMetadata(meta *DirectoryContentMetadata) error
	SaveDirectoryAccessMetadata(meta *DirectoryAccessMetadata) error
	SaveFileMetadata(meta *FileMetadata) error
	ListDirectory(path string) []DirEntry
	RenameDirectory(oldPID, oldName, newPID, newName string, version int) error
	DeleteDirectory(parentID, name string, version int) error
}

LegacyMetadataEngine provides filesystem metadata operations via gRPC. This interface is for backward compatibility with the old FUSE implementation. New code should use the path-based MetadataEngine interface.

type MountBackend added in v0.1.36 

type MountBackend interface {
	// Mount presents the filesystem at the given mount point.
	// This call blocks until the filesystem is unmounted.
	Mount(fs *Filesystem, mountPoint string) error

	// Unmount requests the filesystem to be unmounted.
	Unmount() error
}

MountBackend abstracts how the filesystem is presented to the OS. Supported backends: "fuse" (default) and "nfs" (Linux fallback when FUSE is unavailable).

func NewBackend added in v0.1.36 

func NewBackend(name string) MountBackend

NewBackend creates a MountBackend by name. Callers should resolve "" to a concrete name via defaultBackend() before calling.

type NFSBackend added in v0.1.36 

type NFSBackend struct {
	// contains filtered or unexported fields
}

NFSBackend mounts the filesystem by running a userspace NFSv3 server on loopback and mounting it via the kernel NFS client. This is the fallback for Linux environments where FUSE (/dev/fuse) is not available.

func NewNFSBackend added in v0.1.36 

func NewNFSBackend() *NFSBackend

NewNFSBackend creates a new NFS mount backend.

func (*NFSBackend) Mount added in v0.1.36 

func (b *NFSBackend) Mount(fs *Filesystem, mountPoint string) error

func (*NFSBackend) Unmount added in v0.1.36 

func (b *NFSBackend) Unmount() error

type NodeInfo 

type NodeInfo struct {
	Path string
	ID   string
	PID  string
}

type StatInfo 

type StatInfo struct {
	Bsize   uint64
	Blocks  uint64
	Bfree   uint64
	Bavail  uint64
	Files   uint64
	Ffree   uint64
	Namemax uint64
}

Source Files

View all Source files

Directories

Path Synopsis
Package vnode provides virtual filesystem nodes for the FUSE layer.
 Package vnode provides virtual filesystem nodes for the FUSE layer.
embed

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.