Documentation
¶
Overview ¶
Package filecas is the content-addressed store for tenant FILES/ assets. The bytes live here keyed by their sha256; the runtime DB holds only the fingerprint (stack_files.content_hash). This keeps tenant file bytes out of the in-memory runtime DB on every node and lets identical content dedup across tenants.
Like artifact/continuation it is a generic, swappable seam: the backend is selected by name through a registry, so an S3-compatible backend can register itself (fleet overlay) without changing this package or its callers. The bundled "file" backend (sub-package filestore) is disk-only and self-registers via a blank import.
Integrity: the key is the content's sha256, and every backend's Put MUST Verify(hash, data) before writing — a key is always a truthful digest of the bytes, never merely the caller's claim.
Immutability: values are content-addressed and therefore immutable. Callers MUST treat bytes returned by Get as read-only; the LRU wrapper (cached.go) hands back copies, but a bare backend may return a slice it does not expect to be mutated.
Index ¶
- Variables
- func BlobPath(s Store, hash string) (string, bool)
- func GetReader(ctx context.Context, s Store, hash string) (io.ReadCloser, int64, error)
- func PutReader(ctx context.Context, s Store, hash string, r io.Reader, size int64) error
- func Register(name string, c Constructor)
- func ShardKey(hash string) (key string, ok bool)
- func Verify(hash string, data []byte) error
- type Constructor
- type PathProvider
- type ReaderGetter
- type ReaderPutter
- type Store
- type StoreConfig
Constants ¶
This section is empty.
Variables ¶
var ErrHashMismatch = errors.New("filecas: data does not match hash")
ErrHashMismatch is returned by Put when sha256(data) != hash.
var ErrNotFound = errors.New("filecas: hash not found")
ErrNotFound is returned by Get/Exists when a hash is absent.
Functions ¶
func BlobPath ¶ added in v0.2.19
BlobPath reports the local filesystem path for hash when the backend has one (the bundled file backend). No fallback — absence means "materialise a copy yourself via GetReader".
func GetReader ¶ added in v0.2.19
GetReader streams the blob for hash out of s, via the backend's native reader when available, else a buffered fallback over Get (which may be served from the LRU).
func PutReader ¶ added in v0.2.19
PutReader streams r into s under hash, using the backend's native streaming when available and a buffered fallback (ReadAll → Put, which verifies the hash) otherwise. The fallback holds the whole blob in memory — acceptable for backends that never advertised streaming, since their Put would have buffered anyway.
func Register ¶
func Register(name string, c Constructor)
Register adds a backend constructor. Called from a backend package's init(); the chassis activates a backend with a blank import.
Types ¶
type Constructor ¶
type Constructor func(StoreConfig) (Store, error)
Constructor builds a backend Store from resolved config.
type PathProvider ¶ added in v0.2.19
PathProvider exposes a blob's local filesystem path, for zero-copy consumers (opening a dataset artifact read-only in place rather than materialising a second multi-GB copy). ok=false when the hash is absent or the backend has no local files (S3). The returned path is content-addressed and therefore immutable — callers MUST NOT write to it.
type ReaderGetter ¶ added in v0.2.19
type ReaderGetter interface {
GetReader(ctx context.Context, hash string) (io.ReadCloser, int64, error)
}
ReaderGetter streams content out. Returns the reader, the blob size, and ErrNotFound when absent. The caller owns closing the reader.
type ReaderPutter ¶ added in v0.2.19
type ReaderPutter interface {
PutReader(ctx context.Context, hash string, r io.Reader, size int64) error
}
ReaderPutter streams content in. Implementations MUST hash while streaming and commit create-if-absent ONLY when sha256(stream)==hash — on mismatch they return ErrHashMismatch and store nothing (partial writes must never become visible). size is the exact expected byte count when the caller knows it (Content-Length), or -1; backends that need sizing up front (multipart uploads) may require it.
type Store ¶
type Store interface {
// Put writes data under hash iff sha256(data)==hash (ErrHashMismatch
// otherwise) and the hash is absent (create-if-absent dedup; re-Put of
// identical content is a no-op success).
Put(ctx context.Context, hash string, data []byte) error
// Get returns the bytes for hash. ErrNotFound if absent. The result
// MUST be treated as immutable by the caller.
Get(ctx context.Context, hash string) ([]byte, error)
// Exists reports presence without fetching bytes.
Exists(ctx context.Context, hash string) (bool, error)
// Name is the backend identity (for logs).
Name() string
}
Store is a content-addressed blob store. The key IS the content's lowercase sha256 hex.
type StoreConfig ¶
type StoreConfig struct {
FileDir string
// Reserved for the out-of-tree S3-compatible backend (fleet overlay).
S3Bucket string
S3Prefix string
// CacheBytes is the in-memory LRU budget (bytes) fronting Get. 0
// disables the cache (pure pass-through to the backend).
CacheBytes int64
// MaxEntryBytes is the per-entry cache guard: a blob larger than this
// is served straight from the backend and never cached, so one large
// file can't evict the whole cache. 0 means no per-entry guard.
MaxEntryBytes int64
}
StoreConfig carries backend-selecting options resolved from chassis config. The file backend uses FileDir; the S3 fields are the fleet overlay seam (unused in open core). CacheBytes/MaxEntryBytes configure the LRU that Open wraps around any backend.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package filestore is the local-filesystem filecas backend: the bundled default, content-addressed, maildir-sharded (sha256/<ab>/<cd>/<hash>), zero infrastructure, directly inspectable.
|
Package filestore is the local-filesystem filecas backend: the bundled default, content-addressed, maildir-sharded (sha256/<ab>/<cd>/<hash>), zero infrastructure, directly inspectable. |