Documentation
¶
Index ¶
- Variables
- type CompressedOption
- type CompressedStore
- func (s *CompressedStore) Delete(ctx context.Context, key string) error
- func (s *CompressedStore) Exists(ctx context.Context, key string) (bool, error)
- func (s *CompressedStore) Flush(ctx context.Context) error
- func (s *CompressedStore) Get(ctx context.Context, key string) ([]byte, error)
- func (s *CompressedStore) List(ctx context.Context, prefix string) ([]string, error)
- func (s *CompressedStore) Put(ctx context.Context, key string, data []byte) error
- func (s *CompressedStore) Size(ctx context.Context, key string) (int64, error)
- func (s *CompressedStore) TotalSize(ctx context.Context) (int64, error)
- func (s *CompressedStore) Unwrap() store.ObjectStore
- type EncryptedStore
- type KeyCacheStore
- func (s *KeyCacheStore) Delete(ctx context.Context, key string) error
- func (s *KeyCacheStore) Exists(ctx context.Context, key string) (bool, error)
- func (s *KeyCacheStore) Flush(ctx context.Context) error
- func (s *KeyCacheStore) Get(ctx context.Context, key string) ([]byte, error)
- func (s *KeyCacheStore) List(ctx context.Context, prefix string) ([]string, error)
- func (s *KeyCacheStore) PreloadKeys(ctx context.Context, prefixes ...string) error
- func (s *KeyCacheStore) Put(ctx context.Context, key string, data []byte) error
- func (s *KeyCacheStore) Size(ctx context.Context, key string) (int64, error)
- func (s *KeyCacheStore) TotalSize(ctx context.Context) (int64, error)
- func (s *KeyCacheStore) Unwrap() store.ObjectStore
- type MeteredStore
- func (m *MeteredStore) BytesWritten() int64
- func (m *MeteredStore) Delete(ctx context.Context, key string) error
- func (m *MeteredStore) DeleteReturnSize(ctx context.Context, key string) (int64, error)
- func (m *MeteredStore) Put(ctx context.Context, key string, data []byte) error
- func (m *MeteredStore) Reset()
- func (m *MeteredStore) Unwrap() store.ObjectStore
- type PackEntry
- type PackOption
- type PackStore
- func (s *PackStore) CompactCatalog(ctx context.Context) (int, error)
- func (s *PackStore) Delete(ctx context.Context, key string) error
- func (s *PackStore) Exists(ctx context.Context, key string) (bool, error)
- func (s *PackStore) Flush(ctx context.Context) error
- func (s *PackStore) Get(ctx context.Context, key string) ([]byte, error)
- func (s *PackStore) List(ctx context.Context, prefix string) ([]string, error)
- func (s *PackStore) Put(ctx context.Context, key string, data []byte) error
- func (s *PackStore) RebuildCatalog(ctx context.Context) (recovered int, footerless int, err error)
- func (s *PackStore) Repack(ctx context.Context, maxWastedRatio float64) (int64, int, error)
- func (s *PackStore) Size(ctx context.Context, key string) (int64, error)
- func (s *PackStore) TotalSize(ctx context.Context) (int64, error)
Constants ¶
This section is empty.
Variables ¶
var ErrInvalidFrame = errors.New("store: invalid compression frame")
ErrInvalidFrame reports a framed object whose header or payload is not self-consistent. It is never downgraded to "return the bytes as they are": that is the sniffing behaviour this frame exists to remove.
var ErrPlaintextObject = errors.New("store: unencrypted object in an encrypted repository")
ErrPlaintextObject is returned when an object is not a ciphertext in an encrypted repository, outside what is plaintext by design.
Functions ¶
This section is empty.
Types ¶
type CompressedOption ¶
type CompressedOption func(*CompressedStore)
CompressedOption configures a CompressedStore.
func WithFrameGate ¶
func WithFrameGate(gate func() bool) CompressedOption
WithFrameGate frames writes whenever gate reports true, evaluated per write.
This is how framing tracks the repository format without a second source of truth: the caller owns one format value and points the gate at it, so raising the format turns framing on for every subsequent write at once. Enabling framing partway through a mutation is still the caller's responsibility to avoid — an object written unframed by this build is unframed permanently, since content-addressed objects are never rewritten once stored.
func WithFramedWrites ¶
func WithFramedWrites(enabled bool) CompressedOption
WithFramedWrites is the static form of WithFrameGate, for callers whose framing decision does not change over the store's life (chiefly tests).
type CompressedStore ¶
type CompressedStore struct {
// contains filtered or unexported fields
}
CompressedStore wraps an store.ObjectStore and transparently zstd-compresses on write and decompresses on read.
Whether a write is framed is not a mode this store is told to enter; it is derived, per write, from a gate the caller supplies (WithFrameGate). Framing belongs on only once the repository records a format whose readers understand the frame, and that format is raised during a mutation — so the gate lets the owner of the format flip one value and have every write follow, with no second copy of the decision to keep in sync. See core.FramedCompressionFormat.
Reads accept both framed and unframed objects regardless of the gate: unframed objects exist in every repository indefinitely, because upgrades are opportunistic and permanently partial (docs/compatibility.md).
func NewCompressedStore ¶
func NewCompressedStore(inner store.ObjectStore, opts ...CompressedOption) *CompressedStore
func (*CompressedStore) Delete ¶
func (s *CompressedStore) Delete(ctx context.Context, key string) error
func (*CompressedStore) TotalSize ¶
func (s *CompressedStore) TotalSize(ctx context.Context) (int64, error)
func (*CompressedStore) Unwrap ¶
func (s *CompressedStore) Unwrap() store.ObjectStore
type EncryptedStore ¶
type EncryptedStore struct {
store.ObjectStore
// contains filtered or unexported fields
}
EncryptedStore wraps an store.ObjectStore and transparently encrypts data on Put and decrypts on Get using AES-256-GCM.
Objects under "keys/" are exempt from encryption entirely — they hold the wrapped master key needed to derive the encryption key, so Put never encrypts them and Get never expects ciphertext there. "config", the repository marker read before any key is resolved, is a narrower exemption: Get returns it as-is when it is not a ciphertext (which is how it is always actually written, directly through the raw store), but Put still encrypts it like any other key, and Get still decrypts it if it ever does arrive as one.
A non-ciphertext object anywhere else is refused rather than returned. The passthrough that used to apply unconditionally here — legacy plaintext data returned as-is, documented as a gradual-migration affordance — is what let anyone with write access to the backing store have a client *holding the correct key* read attacker-written plaintext as repository content, with no key and no tampering with config or key slots required.
func NewEncryptedStore ¶
func NewEncryptedStore(inner store.ObjectStore, key []byte) *EncryptedStore
NewEncryptedStore creates an EncryptedStore that encrypts all Put operations and decrypts Get operations. The key must be 32 bytes (AES-256).
func (*EncryptedStore) Unwrap ¶
func (s *EncryptedStore) Unwrap() store.ObjectStore
type KeyCacheStore ¶
type KeyCacheStore struct {
// contains filtered or unexported fields
}
KeyCacheStore wraps an store.ObjectStore and caches key existence from List calls, so that Exists returns immediately for known keys. Thread-safe.
func NewKeyCacheStore ¶
func NewKeyCacheStore(inner store.ObjectStore) *KeyCacheStore
func (*KeyCacheStore) Delete ¶
func (s *KeyCacheStore) Delete(ctx context.Context, key string) error
func (*KeyCacheStore) PreloadKeys ¶
func (s *KeyCacheStore) PreloadKeys(ctx context.Context, prefixes ...string) error
func (*KeyCacheStore) TotalSize ¶
func (s *KeyCacheStore) TotalSize(ctx context.Context) (int64, error)
func (*KeyCacheStore) Unwrap ¶
func (s *KeyCacheStore) Unwrap() store.ObjectStore
type MeteredStore ¶
type MeteredStore struct {
store.ObjectStore
// contains filtered or unexported fields
}
MeteredStore wraps an store.ObjectStore and tracks bytes written/deleted.
func NewMeteredStore ¶
func NewMeteredStore(s store.ObjectStore) *MeteredStore
func (*MeteredStore) BytesWritten ¶
func (m *MeteredStore) BytesWritten() int64
func (*MeteredStore) DeleteReturnSize ¶
func (*MeteredStore) Reset ¶
func (m *MeteredStore) Reset()
func (*MeteredStore) Unwrap ¶
func (m *MeteredStore) Unwrap() store.ObjectStore
type PackOption ¶
type PackOption func(*PackStore)
PackOption configures a PackStore.
func WithPackIndexKey ¶
func WithPackIndexKey(key []byte) PackOption
WithPackIndexKey seals the pack catalog and packfile footers with key.
PackStore sits below EncryptedStore in the store chain, so the objects it writes on its own behalf — index/packs and each pack's footer — never pass through the encryption layer. Without a key they are stored in plaintext, exposing every packed object's key, its exact ciphertext length, and which objects share a pack. Pack *contents* are unaffected either way: they are already ciphertext by the time PackStore sees them.
Pass a key derived with crypto.HKDFInfoPackIndexV1, not the master key. Repositories without encryption pass none and keep plaintext indexes.
func WithPackLogger ¶
func WithPackLogger(w io.Writer) PackOption
WithPackLogger sends this store's debug output to w.
type PackStore ¶
type PackStore struct {
store.ObjectStore
// contains filtered or unexported fields
}
PackStore wraps an store.ObjectStore to aggregate small objects into larger "packfiles". It uses a stateless JSON catalog ("index/packs") to keep track of which pack contains which object.
func NewPackStore ¶
func NewPackStore(inner store.ObjectStore, opts ...PackOption) (*PackStore, error)
NewPackStore initializes a new MicroPackStore over an existing store.ObjectStore.
func (*PackStore) CompactCatalog ¶
CompactCatalog folds every shard, and the legacy monolithic catalog, into a single shard and removes what it replaced.
Shards accumulate one per flush, so without compaction opening a repository costs a request per flush ever made. Callers must hold the repository's exclusive lock: this is the one operation that removes index material, and doing it alongside a concurrent writer could drop a shard written between the merge and the delete.
The consolidated shard is written before anything is deleted. A reader that lists midway sees both it and its inputs, which merge to the same result.
func (*PackStore) Delete ¶
Delete removes an object. For packed objects, it just removes it from the catalog. The actual packfile is not currently garbage collected.
func (*PackStore) Exists ¶
Exists checks the un-flushed buffer, the catalog, or falls back to inner.
func (*PackStore) Flush ¶
Flush ensures any pending small objects are written to a packfile, and uploads the latest JSON catalog.
func (*PackStore) Get ¶
Get retrieves an object from the active buffer, a cached pack, or downloads the pack.
func (*PackStore) List ¶
List returns all keys matching the prefix, merging results from the inner store with the keys currently buffered or indexed in packfiles.
func (*PackStore) Put ¶
Put stores data either in the active packbuffer or directly to the inner store.
func (*PackStore) RebuildCatalog ¶
RebuildCatalog reconstructs the pack catalog by reading every packfile's footer, and merges the result into the in-memory catalog.
This is what demotes index/packs from a single point of failure to a cache: entries recovered here are authoritative, because they come from inside the immutable, content-addressed packfile they describe. Packs written before footers existed contribute nothing and are reported as unrecoverable, since their offsets genuinely exist nowhere but the catalog.
Existing in-memory entries win over recovered ones. Both locate byte-identical content, so the merge is idempotent and order-independent.
func (*PackStore) Repack ¶
Repack analyzes the packfiles and repacks those that have too much wasted space. Wasted space occurs when objects within a packfile are logically deleted (removed from catalog). maxWastedRatio is the threshold (0.0 to 1.0) above which a pack is repacked. For example, 0.3 means a pack is repacked if it is more than 30% empty. Returns the number of bytes reclaimed, number of packs deleted, and error.
Source Files