README
¶
s3fs
created by Austin Poor
An implementation of the go-billy Filesystem for treating an S3 bucket like a filesystem.
Documentation
¶
Index ¶
- Constants
- Variables
- func NewS3FS(client *storage.Client, bucket string, opts ...Option) (billy.Filesystem, error)
- type FileStat
- type Option
- type S3FS
- func (fs3 *S3FS) Capabilities() billy.Capability
- func (fs3 *S3FS) Chroot(path string) (billy.Filesystem, error)
- func (fs3 *S3FS) Create(filename string) (billy.File, error)
- func (fs3 *S3FS) Join(elem ...string) string
- func (fs3 *S3FS) Lstat(filename string) (os.FileInfo, error)
- func (fs3 *S3FS) MkdirAll(filename string, perm os.FileMode) error
- func (fs3 *S3FS) Open(filename string) (billy.File, error)
- func (fs3 *S3FS) OpenFile(filename string, flag int, perm os.FileMode) (billy.File, error)
- func (fs3 *S3FS) ReadDir(dir string) ([]fs.DirEntry, error)
- func (fs3 *S3FS) Readlink(link string) (string, error)
- func (fs3 *S3FS) Remove(filename string) error
- func (fs3 *S3FS) Rename(oldpath, newpath string) error
- func (fs3 *S3FS) Root() string
- func (fs3 *S3FS) Stat(filename string) (os.FileInfo, error)
- func (fs3 *S3FS) Symlink(target, link string) error
- func (fs3 *S3FS) TempFile(dir, prefix string) (billy.File, error)
Constants ¶
const ( O_RDONLY int = os.O_RDONLY // open the file read-only. O_WRONLY int = os.O_WRONLY // open the file write-only. O_WRMULTIPART int = 0x4 // open the file for write-only using multipart upload. SupportedOFlags = O_RDONLY | O_WRONLY | O_WRMULTIPART // supported open flags for s3fs )
const (
DefaultSeparator = "/"
)
const (
ModeMultipartUpload os.FileMode = fs.ModePerm + 1 // Custom os.FileMode for S3 multipart upload
)
Variables ¶
var ( ErrNotImplemented = errors.New("not implemented") ErrLockNotSupported = errors.New("lock not supported by s3") ErrTruncateNotSupported = errors.New("truncate not supported by s3") ErrFileClosed = errors.New("file is closed") ErrCantWriteToReadOnly = errors.New("can't write to read-only file") ErrCantReadFromWriteOnly = errors.New("can't read from write-only file") )
var (
ErrOpenFlagNotSupported = errors.New("open flag not supported")
)
var (
ErrSymLinkNotSupported = errors.New("symlink not supported by s3")
)
Functions ¶
Types ¶
type FileStat ¶
type FileStat struct {
UID, GID uint32
}
FileStat is the value returned by simpleFileInfo.Sys() when the Unix-metadata feature is enabled. It carries the raw numeric owner and group so consumers can resolve them to names however they like.
type Option ¶
type Option func(*S3FS)
Option configures an S3FS at construction time.
func WithUnixMetadata ¶
WithUnixMetadata enables storing and reading POSIX file attributes as S3 user metadata (see the unixmeta package). uid and gid are the numeric owner/group recorded on newly written objects; callers resolve any names to numbers themselves. umask is applied to the default mode (0666 for files) when writing. Without this option the filesystem stores no attributes.
type S3FS ¶
type S3FS struct {
// contains filtered or unexported fields
}
func (*S3FS) Capabilities ¶
func (fs3 *S3FS) Capabilities() billy.Capability
Capabilities returns the filesystem capabilities.
func (*S3FS) Chroot ¶
func (fs3 *S3FS) Chroot(path string) (billy.Filesystem, error)
Chroot returns a new filesystem from the same type where the new root is the given path. Files outside of the designated directory tree cannot be accessed.
func (*S3FS) Create ¶
Create implements billy.Basic Create creates the named file with mode 0666 (before umask), truncating it if it already exists. If successful, methods on the returned File can be used for I/O; the associated file descriptor has mode O_RDWR.
func (*S3FS) Lstat ¶
Lstat returns a FileInfo describing the named file. S3 has no symlinks, so Lstat is equivalent to Stat. We still implement it so callers that type-assert billy.Symlink (rm, du, cp, ls, touch, file, billyfs, billysh) get a usable FileInfo instead of ErrSymLinkNotSupported.
func (*S3FS) MkdirAll ¶
MkdirAll creates a directory named path, along with any necessary parents, and returns nil, or else returns an error. The permission bits perm are used for all directories that MkdirAll creates. If path is/ already a directory, MkdirAll does nothing and returns nil.
func (*S3FS) Open ¶
Open opens the named file for reading. If successful, methods on the returned file can be used for reading; the associated file descriptor has mode O_RDONLY.
func (*S3FS) OpenFile ¶
OpenFile is the generalized open call; most users will use Open or Create instead. It opens the named file with specified flag (O_RDONLY etc.) and perm, (0666 etc.) if applicable. If successful, methods on the returned File can be used for I/O.
func (*S3FS) ReadDir ¶
ReadDir reads the directory named by dirname and returns a list of directory entries sorted by filename.
func (*S3FS) Readlink ¶
Readlink returns the target path of link.
NOTE: Readlink is not supported by s3. It always returns an error. (This may be revised in the future.)
func (*S3FS) Remove ¶
Remove removes the named file or directory. In-memory TempFile entries are dropped from the registry without an S3 call.
func (*S3FS) Rename ¶
Rename renames (moves) oldpath to newpath. If oldpath refers to an in-memory TempFile, its buffer is uploaded to S3 under newpath and the registry entry is dropped — this is how PackWriter's "tmp_pack_… → pack-<sha>.pack" promotion lands the final pack in the bucket. Otherwise Rename uses Tigris's in-place RenameObject extension.
func (*S3FS) Symlink ¶
Symlink creates a symbolic-link from link to target. target may be an absolute or relative path, and need not refer to an existing node. Parent directories of link are created as necessary.
NOTE: Symlink is not supported by s3. It always returns an error.
func (*S3FS) TempFile ¶
TempFile creates a uniquely named file under dir whose name begins with prefix and returns a write handle to it. The bytes live in an in-memory buffer registered against the filesystem; a subsequent Open of the same path returns a reader over that same buffer (needed by go-git's streaming PackWriter, which reads the temp pack back as it is written). The buffer is uploaded to S3 only when the caller renames the path to its final location; Remove discards it.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package unixmeta encodes and decodes POSIX file attributes (owner, group, permissions, timestamps) as S3 user-defined metadata (x-amz-meta-* headers).
|
Package unixmeta encodes and decodes POSIX file attributes (owner, group, permissions, timestamps) as S3 user-defined metadata (x-amz-meta-* headers). |