Documentation
¶
Index ¶
- Constants
- Variables
- func AzAccountContainer(p string) (account, container string, err error)
- func BaseName(p string) string
- func CanCopyServerSide(src, dst string) bool
- func ChildPath(parent, child string) string
- func CopyServerSide(ctx context.Context, src, dst string, concurrency int, sizeHint int64, ...) error
- func Delete(ctx context.Context, p string) error
- func DeletePrefix(ctx context.Context, p string) error
- func Exists(ctx context.Context, p string) (bool, error)
- func ExistsAsBlob(ctx context.Context, p string) (bool, error)
- func IsAz(path string) bool
- func IsDirLike(ctx context.Context, p string) (bool, error)
- func IsDirLikeFromPath(p string) bool
- func IsHF(path string) bool
- func IsNonRetryableHTTPErr(err error) bool
- func IsRemote(path string) bool
- func ListFilesFlat(ctx context.Context, p string) ([]string, error)
- func ListRecursive(ctx context.Context, root string) <-chan ListResult
- func ListRecursiveWithSizeStream(ctx context.Context, p string, emit func(Entry) error) error
- func ListStream(ctx context.Context, p string, fn func(Entry) error) error
- func MkDir(ctx context.Context, p string) error
- func ParseShareInfo(p string) (portal, direct string, err error)
- func PreAuthenticateAz(ctx context.Context, paths ...string) error
- func Register(provider FS)
- func RegisterAzAccountRoles(srcPaths, dstPaths []string)
- func ResolveDstPath(dst, base string, mustBeDir bool) (string, error)
- func ScanConcurrency(ctx context.Context) int
- func Touch(ctx context.Context, p string) error
- func UploadConcurrency(ctx context.Context) int
- func WithScanConcurrency(ctx context.Context, n int) context.Context
- func WithUploadConcurrency(ctx context.Context, n int) context.Context
- type CopyProgress
- type Entry
- type FS
- type ListResult
Constants ¶
const HFScheme = "hf://"
HFScheme is the scheme prefix for Hugging Face paths.
Variables ¶
var ErrWriteUnsupported = errors.New("bbbfs: write not supported")
ErrWriteUnsupported indicates that a backend does not support writes.
Functions ¶
func AzAccountContainer ¶ added in v0.1.18
AzAccountContainer returns the account and container from an Azure path. Returns empty strings for non-Azure paths.
func CanCopyServerSide ¶ added in v0.1.18
CanCopyServerSide returns true when both src and dst can use server-side copy.
func ChildPath ¶ added in v0.1.18
ChildPath joins child to parent using the backend-specific separator.
func CopyServerSide ¶ added in v0.1.18
func CopyServerSide(ctx context.Context, src, dst string, concurrency int, sizeHint int64, onProgress CopyProgress) error
CopyServerSide performs an optimised server-side copy (e.g. Azure→Azure). sizeHint, when > 0, avoids a HeadBlob round-trip for the source size. Returns an error if the backends do not support server-side copy.
func DeletePrefix ¶ added in v0.1.18
DeletePrefix deletes all files under the prefix. Only Azure supports this natively.
func ExistsAsBlob ¶ added in v0.1.18
ExistsAsBlob checks whether the path points to an existing non-directory file. Returns false if the path is directory-like or does not exist.
func IsDirLike ¶ added in v0.1.18
IsDirLike checks whether the path is directory-like. For remote paths this is determined from the path structure; for local paths os.Stat is used.
func IsDirLikeFromPath ¶ added in v0.1.18
IsDirLikeFromPath checks if a path is directory-like without making any network calls. Uses path structure only.
func IsNonRetryableHTTPErr ¶ added in v0.1.18
IsNonRetryableHTTPErr returns true when err is an HTTP 401, 403, or 404 from any supported backend, indicating a non-retryable failure.
func IsRemote ¶ added in v0.1.18
IsRemote returns true if the path targets a remote (non-local) backend.
func ListFilesFlat ¶ added in v0.1.18
ListFilesFlat returns a flat list of relative file names under the path.
func ListRecursive ¶
func ListRecursive(ctx context.Context, root string) <-chan ListResult
ListRecursive returns a channel that streams all files under the path. Entries are emitted as they are discovered; any listing error is sent as a ListResult with Err set. The channel is closed when listing completes or the context is cancelled. Callers should cancel the context if they stop consuming the channel early.
func ListRecursiveWithSizeStream ¶ added in v0.1.18
ListRecursiveWithSizeStream streams all entries recursively with their sizes via a callback. If the backend does not implement streaming, it falls back to collecting all entries and emitting them one by one.
func ListStream ¶ added in v0.1.18
ListStream lists entries via a streaming callback. Falls back to List when the backend does not provide a streaming implementation.
func ParseShareInfo ¶ added in v0.1.18
ParseShareInfo returns shareable links for the path.
func PreAuthenticateAz ¶ added in v0.1.19
PreAuthenticateAz eagerly authenticates to the storage accounts referenced by the given az:// paths. Call this before spawning parallel workers so that any interactive login popups happen sequentially.
func RegisterAzAccountRoles ¶ added in v0.1.19
func RegisterAzAccountRoles(srcPaths, dstPaths []string)
RegisterAzAccountRoles tags source and destination storage accounts with their roles so that SRC_AZURE_* / DST_AZURE_* environment variables are used for authentication in multi-tenant environments.
If the same account appears in both srcPaths and dstPaths it is not tagged with any role, so the normal (non-role-scoped) credential flow is used.
func ResolveDstPath ¶ added in v0.1.18
ResolveDstPath computes the final destination file path for a copy operation. If the destination is directory-like, base is appended.
func ScanConcurrency ¶ added in v0.1.18
ScanConcurrency returns the scan concurrency stored in ctx, or 1 if unset.
func UploadConcurrency ¶ added in v0.1.19
UploadConcurrency returns the upload concurrency stored in ctx, or 1 if unset.
func WithScanConcurrency ¶ added in v0.1.18
WithScanConcurrency returns a context that carries the scan (listing) concurrency hint. Backends that support parallel prefix walking (e.g. Azure) use this to bound the number of concurrent listing goroutines.
Types ¶
type CopyProgress ¶ added in v0.1.18
type CopyProgress = func(copied, total int64)
CopyProgress is the callback type for copy progress reporting.
type FS ¶
type FS interface {
Match(path string) bool
Read(ctx context.Context, path string) (io.ReadCloser, error)
Write(ctx context.Context, path string, r io.Reader) error
List(ctx context.Context, path string) ([]Entry, error)
Stat(ctx context.Context, path string) (Entry, error)
}
FS provides abstract access for supported backends.
type ListResult ¶
ListResult wraps an Entry with an optional error from a listing operation.
Source Files