Documentation
¶
Index ¶
- Constants
- Variables
- func BulkLoadIndexTimeout() time.Duration
- func C1ZFileCheckHeader(f io.ReadSeeker) (bool, error)
- func CleanupSkippedByEnv() bool
- func FinalizeTimeout() time.Duration
- func NewC1ZFileDecoder(f io.Reader, opts ...DecoderOption) (io.ReadCloser, error)
- func NewDecoder(f io.Reader, opts ...DecoderOption) (*decoder, error)
- func NewStore(ctx context.Context, outputFilePath string, opts ...C1ZOption) (c1zstore.Store, error)
- func ReadHeader(reader io.Reader) error
- func RegisterEngine(driver EngineDriver) error
- func ResolveCleanupSyncLimit(callerLimit int, currentSyncOpen bool) int
- func SelectSyncsToDelete(candidates []c1zstore.SyncRun, currentSyncID string, syncLimit int) []string
- type C1ZFormat
- type C1ZOption
- func WithDecoderOptions(opts ...DecoderOption) C1ZOption
- func WithDecoderPool(p *EnvelopeDecoderPool) C1ZOption
- func WithEngine(engine c1zstore.Engine) C1ZOption
- func WithGrantDigestIndex(enabled bool) C1ZOption
- func WithPayloadEncoding(enc c1zstore.PayloadEncoding) C1ZOption
- func WithReadOnly(readOnly bool) C1ZOption
- func WithSkipCleanup(skip bool) C1ZOption
- func WithSyncLimit(limit int) C1ZOption
- func WithTmpDir(tmpDir string) C1ZOption
- type DecoderOption
- type EngineDriver
- type EnvelopeDecoderPool
- type IngestInvariantStore
- type RollbackOption
- type RollbackResult
- type SourceCacheReplayResult
- type SourceCacheStore
- type StoreOptions
Constants ¶
const DefaultBulkLoadIndexTimeout = 6 * time.Hour
DefaultBulkLoadIndexTimeout bounds the detached context for the bulk-load deferred-index rebuild at Close. Building several secondary indexes over a 50M+-row grants table is tens of minutes — much longer than the checkpoint+save FinalizeTimeout covers — so this gets its own generous ceiling. Six hours is a backstop against a wedged build, not an expected duration.
const DefaultFinalizeTimeout = 1 * time.Hour
DefaultFinalizeTimeout bounds the detached context used for c1z finalization (WAL checkpoint, save-to-disk, upload). Large tenants take 5-15 min to upload a 4 GB compressed c1z; one hour gives plenty of headroom while still being a hard ceiling so a wedged upload cannot hold a worker indefinitely.
Variables ¶
var ( ErrInvalidFile = fmt.Errorf("c1z: invalid file") ErrMaxSizeExceeded = fmt.Errorf("c1z: max decoded size exceeded, increase DecoderMaxDecodedSize using %v environment variable", maxDecodedSizeEnvVar) ErrWindowSizeExceeded = fmt.Errorf("c1z: window size exceeded, increase DecoderMaxMemory using %v environment variable", maxDecoderMemorySizeEnv) )
var C1Z3FileHeader = []byte("C1Z3\x00")
C1Z3FileHeader is the magic byte sequence for v3 files.
var C1ZFileHeader = []byte("C1ZF\x00")
var ErrArtifactUnusable = errors.New("sync artifact unusable")
ErrArtifactUnusable classifies STORAGE VERDICTS — the output c1z does not reflect a clean commit of this run's progress — and is the only signal that permits a runner to discard a partial sync artifact (RFC 0009: preserve by default, discard on typed verdict). Producers are the close/finalize paths that had mutations but failed before the c1z was rewritten; both engines save atomically, so a carried verdict means the artifact is STALE, never torn. Failures after a successful save (cleanup, engine teardown) deliberately do not carry it — the artifact is a faithful commit at that point. Consumed via sync.ShouldDiscardSyncArtifact; test with errors.Is.
var ErrEngineNotAvailable = fmt.Errorf("dotc1z: engine not available")
ErrEngineNotAvailable is returned when a caller requests an engine that the binary does not support.
var ErrSyncNotExpanded = errors.New("c1z: sync did not run grant expansion; nothing to roll back")
ErrSyncNotExpanded is returned when a rollback targets a sync that never ran grant expansion: no grant in the sync carries Sources.
var ErrSyncNotFinished = errors.New("c1z: refusing to roll back expansion on an unfinished sync")
ErrSyncNotFinished is returned when a rollback targets a sync that has not finished. Expansion completeness for an in-progress sync lives in the entitlement graph persisted in the sync token, not in the grant rows. Deleting expansion output from such a sync would strand that graph: a later resume would treat the affected edges as already expanded and never regenerate the deleted grants. Pebble wraps pebble.ErrRollbackIncompleteCollection in it, and accepts an unfinished sync whose collection completed.
var ErrV1Unsupported = errors.New("dotc1z: v1 (sqlite) c1z files are not supported; convert with `baton to-pebble` or delete the file to start fresh")
ErrV1Unsupported is returned by NewStore for a v1 (SQLite) .c1z. The SDK no longer reads or writes that format; convert the file with `baton to-pebble` or read it through pkg/dotc1z/sqlite.
Functions ¶
func BulkLoadIndexTimeout ¶ added in v0.12.7
BulkLoadIndexTimeout returns the bound for the detached context that wraps the bulk-load deferred-index rebuild.
func C1ZFileCheckHeader ¶
func C1ZFileCheckHeader(f io.ReadSeeker) (bool, error)
C1ZFileCheckHeader reads len(C1ZFileHeader) bytes from the given io.ReadSeeker and compares them to C1ZFileHeader. Returns true if the header is valid. Returns any errors from Read() or Seek(). If a nil error is returned, the given io.ReadSeeker will be pointing to the first byte of the stream, and is suitable to be passed to NewC1ZFileDecoder.
func CleanupSkippedByEnv ¶ added in v0.10.0
func CleanupSkippedByEnv() bool
CleanupSkippedByEnv reports whether BATON_SKIP_CLEANUP is set to a truthy value. See c1zstore.CleanupSkippedByEnv.
func FinalizeTimeout ¶ added in v0.10.0
FinalizeTimeout returns the bound for the detached context that wraps c1z finalize-and-upload tails.
func NewC1ZFileDecoder ¶
func NewC1ZFileDecoder(f io.Reader, opts ...DecoderOption) (io.ReadCloser, error)
NewC1ZFileDecoder wraps a given .c1z io.Reader that validates the .c1z and decompresses/decodes the underlying file. Defaults: 128MiB max memory and 3GiB max decoded size You must close the resulting io.ReadCloser when you are done, do not forget to close the given io.Reader if necessary.
func NewDecoder ¶ added in v0.0.24
func NewDecoder(f io.Reader, opts ...DecoderOption) (*decoder, error)
NewDecoder wraps a given .c1z file io.Reader and returns an io.Reader for the underlying decoded/uncompressed file.
func NewStore ¶ added in v0.10.0
func NewStore(ctx context.Context, outputFilePath string, opts ...C1ZOption) (c1zstore.Store, error)
NewStore opens outputFilePath through the registered engine registry.
func ReadHeader ¶ added in v0.0.24
ReadHeader reads len(C1ZFileHeader) bytes from the given io.Reader and compares them to C1ZFileHeader, returning an error if they don't match. If possible, ReadHeader will Seek() to the start of the stream before checking the header bytes. On return, the reader will be pointing to the first byte after the header.
func RegisterEngine ¶ added in v0.10.0
func RegisterEngine(driver EngineDriver) error
RegisterEngine registers a storage engine driver with the process-global dotc1z engine registry.
func ResolveCleanupSyncLimit ¶ added in v0.10.0
ResolveCleanupSyncLimit resolves the effective retention limit. See c1zstore.ResolveCleanupSyncLimit.
func SelectSyncsToDelete ¶ added in v0.10.0
func SelectSyncsToDelete(candidates []c1zstore.SyncRun, currentSyncID string, syncLimit int) []string
SelectSyncsToDelete applies the SDK retention policy to a snapshot of sync runs and returns the IDs whose data should be deleted. See c1zstore.SelectSyncsToDelete for the policy details.
Types ¶
type C1ZFormat ¶ added in v0.10.0
type C1ZFormat int
C1ZFormat identifies the on-disk format of a .c1z file. The format byte is the first 5 bytes of the file; see ReadHeaderFormat.
const ( // C1ZFormatUnknown is the zero value. Returned when the header bytes // match neither the v1 nor the v3 magic, or when the read failed. C1ZFormatUnknown C1ZFormat = iota // C1ZFormatV1 is the original .c1z format: 5-byte magic "C1ZF\x00" // followed by a zstd-compressed SQLite database. C1ZFormatV1 // C1ZFormatV3 is the v3 format introduced by the storage-engine-v4 // RFC: 5-byte magic "C1Z3\x00", a length-prefixed proto manifest, // and a zstd-tar payload of a Pebble Checkpoint directory. C1ZFormatV3 )
func ReadHeaderFormat ¶ added in v0.10.0
ReadHeaderFormat reads the first 5 bytes of reader and returns the detected format. On return, the reader is positioned immediately after the header bytes. If reader is also an io.Seeker, it is rewound to offset 0 before reading.
Returns:
- C1ZFormatV1, nil — file starts with "C1ZF\x00".
- C1ZFormatV3, nil — file starts with "C1Z3\x00".
- C1ZFormatUnknown, ErrInvalidFile — header matched no known magic.
- C1ZFormatUnknown, err — underlying read error.
type C1ZOption ¶ added in v0.1.8
type C1ZOption func(*c1zOptions)
func WithDecoderOptions ¶ added in v0.3.50
func WithDecoderOptions(opts ...DecoderOption) C1ZOption
func WithDecoderPool ¶ added in v0.13.2
func WithDecoderPool(p *EnvelopeDecoderPool) C1ZOption
WithDecoderPool threads a caller-owned envelope decoder pool through a store open. Useful only for callers that open MANY v3 c1z files in one operation (the compactor opens one envelope per source per merge); each open draws its zstd payload decoder from the pool instead of constructing a fresh one. The caller must Close the pool when the operation finishes — that deterministic release is the point of scoping the pool instead of sharing one process-wide. Harmless when nil.
func WithEngine ¶ added in v0.10.0
WithEngine sets the storage engine for newly created .c1z files. EnginePebble is the default and the only registered engine; any other value makes NewStore return ErrEngineNotAvailable.
func WithGrantDigestIndex ¶ added in v0.18.3
WithGrantDigestIndex toggles the Pebble engine's seal-time build of the by_entitlement_principal_hash index and per-entitlement grant digests (the index cross-file grant diffing reads). Default true.
Pass false for files that will never be grant-diffed (local CLI syncs, connector development) to skip the seal-time derivation pass. Safe to toggle: a file sealed with this off stores no digest roots, which readers treat as "missing — recalculate", never as "no grants".
func WithPayloadEncoding ¶ added in v0.10.0
func WithPayloadEncoding(enc c1zstore.PayloadEncoding) C1ZOption
WithPayloadEncoding selects the c1z v3 envelope payload encoding for newly created files. Default is PayloadEncodingTarZstd. PayloadEncodingTar skips the outer zstd compression — useful when Pebble's L5/L6 SSTs are already zstd-compressed at the engine layer, or when the storage target (S3 with Content-Encoding negotiation, etc.) compresses in transit.
func WithReadOnly ¶ added in v0.6.6
WithReadOnly opens the c1z file in read only mode. Modifying the c1z will result in an error on close.
func WithSkipCleanup ¶ added in v0.8.22
WithSkipCleanup skips cleanup of old syncs when set to true.
func WithSyncLimit ¶ added in v0.7.0
WithSyncLimit sets the number of syncs to keep during cleanup. If not set, defaults to 2 (or BATON_KEEP_SYNC_COUNT env var if set).
func WithTmpDir ¶ added in v0.1.8
WithTmpDir sets the temporary directory to extract the c1z file to. If not provided, os.TempDir() will be used.
type DecoderOption ¶ added in v0.0.24
type DecoderOption func(*decoderOptions) error
DecoderOption is an option for creating a decoder.
func WithContext ¶
func WithContext(ctx context.Context) DecoderOption
WithContext sets a context, when cancelled, will cause subequent calls to Read() to return ctx.Error().
func WithDecoderConcurrency ¶ added in v0.3.50
func WithDecoderConcurrency(n int) DecoderOption
WithDecoderConcurrency sets the number of created decoders. Default is 1, which disables async decoding/concurrency. 0 uses GOMAXPROCS. -1 uses GOMAXPROCS or 4, whichever is lower.
func WithDecoderMaxDecodedSize ¶
func WithDecoderMaxDecodedSize(n uint64) DecoderOption
WithDecoderMaxDecodedSize sets the maximum size of the decoded stream. This can be used to cap the resulting decoded stream size. Maximum is 1 << 63 bytes. Default is 1GiB.
func WithDecoderMaxMemory ¶
func WithDecoderMaxMemory(n uint64) DecoderOption
WithDecoderMaxMemory sets the maximum window size for streaming operations. This can be used to control memory usage of potentially hostile content. Maximum is 1 << 63 bytes. Default is 128MiB.
type EngineDriver ¶ added in v0.10.0
type EngineDriver interface {
Engine() c1zstore.Engine
Format() C1ZFormat
OpenStore(ctx context.Context, outputFilePath string, opts StoreOptions) (c1zstore.Store, error)
}
EngineDriver opens a .c1z file for a specific storage engine. The Pebble driver is registered statically by this package; RegisterEngine exists for additional engines.
func EngineDriverFor ¶ added in v0.10.0
func EngineDriverFor(engine c1zstore.Engine) (EngineDriver, bool)
EngineDriverFor returns the registered driver for engine.
type EnvelopeDecoderPool ¶ added in v0.13.2
type EnvelopeDecoderPool = formatv3.DecoderPool
EnvelopeDecoderPool re-exports the v3 envelope payload decoder pool so callers (e.g. the sync compactor) can scope decoder reuse to one operation without importing the format package. See formatv3.DecoderPool for semantics.
func NewEnvelopeDecoderPool ¶ added in v0.13.2
func NewEnvelopeDecoderPool() *EnvelopeDecoderPool
NewEnvelopeDecoderPool returns an empty pool; the caller owns its lifetime and must Close it when the operation completes.
type IngestInvariantStore ¶ added in v0.20.2
type IngestInvariantStore interface {
// ForEachDistinctGrantEntitlementResource visits each distinct
// entitlement resource referenced by any grant: one seek per
// distinct resource, never O(grants).
ForEachDistinctGrantEntitlementResource(ctx context.Context, visit func(resourceTypeID, resourceID string) error) error
// GrantsForEntResourceCarryInsertFact reports whether any grant
// under the entitlement resource carries InsertResourceGrants.
// Value-reading; reserved for dangling-reference probes.
GrantsForEntResourceCarryInsertFact(ctx context.Context, resourceTypeID, resourceID string) (bool, error)
// GrantsForEntitlementAllCarryInsertFact reports whether every grant
// under the entitlement identity carries InsertResourceGrants — the
// per-grant side of I8's InsertResourceGrants exemption.
// Value-reading; reserved for dangling-reference probes.
GrantsForEntitlementAllCarryInsertFact(ctx context.Context, entitlementID, entResourceTypeID, entResourceID string) (bool, error)
// HasResourceRecord reports whether a resource row exists.
HasResourceRecord(ctx context.Context, resourceTypeID, resourceID string) (bool, error)
// ForEachDistinctEntitlementResource visits each distinct resource
// referenced by any entitlement row: one seek per distinct
// resource, never O(entitlements).
ForEachDistinctEntitlementResource(ctx context.Context, visit func(resourceTypeID, resourceID string) error) error
// ForEachDanglingGrantEntitlement visits each distinct entitlement
// referenced by grants that has no entitlement row: one seek plus
// one point probe per distinct entitlement, never O(grants).
ForEachDanglingGrantEntitlement(ctx context.Context, visit func(entitlementID, resourceTypeID, resourceID string) error) error
// EnsureGrantIndexes forces a pending deferred grant-index build to
// run now, making by_principal complete for the dangling-principal
// scan. Near-free: the same build would run at EndSync.
EnsureGrantIndexes(ctx context.Context) error
// ForEachDanglingGrantPrincipal visits each distinct principal
// referenced by grants that has no resource row. matchAnnotatedOnly
// reports that every grant of the principal is an unprocessed
// ExternalResourceMatch* carrier (exempt shape); carrierGrants is
// the per-grant count of that population.
ForEachDanglingGrantPrincipal(ctx context.Context, visit func(principalRT, principalID string, matchAnnotatedOnly bool, carrierGrants int64) error) error
}
IngestInvariantStore is the optional store capability backing the syncer's post-collection ingestion invariants (pkg/sync ingest_invariants.go): the ordered-key scans the referential invariants (I3/I7/I8/I9) ride, plus the point probes their verdicts need. Pebble-only; the syncer degrades per-invariant when the store lacks it (SQLite artifacts are never scanned).
This is deliberately the READ-ONLY half of the invariant surface: the default-mode drop mechanics (DeleteGrantsForPrincipal and friends) arrive with the dangling-reference drop policy, and the replay-specific facts (external-match existence bits, source-cache scope scans) arrive with source-cache replay.
type RollbackOption ¶ added in v0.12.2
type RollbackOption func(*rollbackConfig)
RollbackOption configures RollbackExpansion.
func WithPreserveSuspectGrants ¶ added in v0.12.2
func WithPreserveSuspectGrants() RollbackOption
WithPreserveSuspectGrants makes rollback KEEP suspect connector-sourced grants (Sources present, no self-source, no GrantImmutable) instead of deleting them. Default OFF: a suspect grant is an expander artifact in the common case and is deleted, but because Grant.Sources is public connector data and replay runs an empty connector, a genuinely connector-set grant of that shape would be permanently dropped — this option trades a possibly-stale expander grant left in the output for never silently dropping real connector data.
func WithSyncTokenRewrite ¶ added in v0.12.6
func WithSyncTokenRewrite(fn func(token string) (string, error)) RollbackOption
WithSyncTokenRewrite supplies how the sync's persisted token is rewritten once the grants are rolled back. The function receives the current token and returns the replacement; rollback writes the result verbatim. Without it, rollback clears the token to empty, which makes a resumed syncer seed a fresh run. A caller that wants to preserve the token's other state passes a rewrite that re-marks the sync for expansion instead. Ignored on a dry run.
type RollbackResult ¶ added in v0.12.2
type RollbackResult struct {
SyncID string
// GrantsDeleted is the number of purely expander-derived grants
// removed: their Sources carried only foreign entitlement keys, never
// a self-source for their own entitlement.
GrantsDeleted int
// SourcesCleared is the number of surviving direct grants whose
// Sources were reset to empty, restoring the pre-expansion shape.
SourcesCleared int
// SuspectConnectorSourced counts grants that had Sources but no
// self-source AND lacked the GrantImmutable annotation that every
// expander-created grant carries — a shape that looks connector-set
// rather than expander-set. By default these are deleted (and counted
// in GrantsDeleted); with WithPreserveSuspectGrants they are kept
// instead (counted in SuspectPreserved, not GrantsDeleted). Either way
// the count is surfaced so a real connector-sourced grant is visible.
// Zero in the common case.
SuspectConnectorSourced int
// SuspectPreserved counts suspect grants kept rather than deleted
// because WithPreserveSuspectGrants was set. Always 0 in the default
// (delete) mode; a subset of SuspectConnectorSourced when set.
SuspectPreserved int
DryRun bool
}
RollbackResult reports what a rollback changed, or — for a dry run — what it would change.
type SourceCacheReplayResult ¶ added in v0.26.0
type SourceCacheReplayResult = pebble.SourceCacheReplayResult
SourceCacheReplayResult reports what one scope's replay copied.
type SourceCacheStore ¶ added in v0.26.0
type SourceCacheStore interface {
// LookupSourceCacheEntry returns this store's manifest entry for
// (kind, scopeKey). Backs the connector-facing lookup when this
// store is the previous sync.
LookupSourceCacheEntry(ctx context.Context, kind sourcecache.RowKind, scopeKey string) (sourcecache.Entry, bool, error)
// PutSourceCacheEntry writes the current sync's manifest entry for
// (kind, scopeKey). Zero-row scopes still get entries.
PutSourceCacheEntry(ctx context.Context, kind sourcecache.RowKind, scopeKey string, cacheValidator string) error
// ReplaySourceCache copies every row stamped with scopeKey from prev
// (the previous sync's store, opened read-only) into this store. prev
// must be a Pebble store. Does NOT write the manifest entry — the
// caller writes it after the scope's overlay/deletes complete, so a
// failed replay can't leave a phantom hit for the next sync.
//
// On error, the result's Rows reports rows whose bounded intermediate
// commits landed (retry converges; matching the delete siblings
// below), and NeedsExpansion may overreport a staged-but-uncommitted
// row — the safe direction, since arming expansion is idempotent and
// add-only.
ReplaySourceCache(ctx context.Context, prev connectorstore.Reader, kind sourcecache.RowKind, scopeKey string) (SourceCacheReplayResult, error)
// DeleteSourceCacheRows removes rows by public canonical ID from the
// current sync, after replay + overlay (delta-query tombstones).
// ID formats per kind: grants and entitlements use their canonical
// IDs; resources use Baton resource BIDs ("bid:r:...").
//
// scopeKey is the scope whose delta reported the tombstones. The
// deletes act FOR that scope: removing its own rows is the
// legitimate shrink flow and stages no poison, while removing a row
// stamped with any OTHER scope is a row-partition violation that
// durably poisons the stamped scope (CO-015 — it becomes a lookup
// miss and is refused as a replay source).
//
// Grant resolution is BOUNDED: candidate probing only, never the
// O(all grants) stored-external-id scan. Grants stored under
// connector-custom ids are unreachable here (delete no-ops); such
// connectors use DeleteSourceCacheRowsInScope instead. Deletes commit
// in bounded chunks; deletion is idempotent, so retry converges.
DeleteSourceCacheRows(ctx context.Context, kind sourcecache.RowKind, scopeKey string, ids []string) error
// DeleteSourceCacheRowsInScope removes rows stamped with scopeKey by
// bare object id — grants by principal id (no principal type, no
// canonical-id reconstruction), resources by resource id (any type).
// One index scan of the scope per call; a page's tombstones are
// batched into one call. Deletes commit in bounded chunks; on error,
// the returned count reports rows already committed and retry converges.
// Ids with no matching rows are no-ops.
// Not supported for entitlements.
DeleteSourceCacheRowsInScope(ctx context.Context, kind sourcecache.RowKind, scopeKey string, ids []string) (int64, error)
// DeleteSourceCacheGrantsByIDInScope removes grant rows stamped with
// scopeKey whose STORED grant id is in ids — works for
// connector-custom grant-id shapes that the global bounded delete
// cannot resolve, and stays bounded by the scope's row count. Ids with
// no matching rows are no-ops. Deletes commit in bounded chunks; on
// error, the returned count reports rows already committed.
DeleteSourceCacheGrantsByIDInScope(ctx context.Context, scopeKey string, ids []string) (int64, error)
}
SourceCacheStore is the optional store capability backing source-cache replay (see proto/c1/connector/v2/annotation_source_cache.proto). It is implemented ONLY by the Pebble engine; the syncer type-asserts for it and treats a store without it as "source cache unsupported" (no-op lookup, no replay). It is deliberately NOT part of c1zstore.Store.
Advanced: this interface exists for the SDK's replay orchestration, not for direct use. The correctness obligations (preflight, scope poisoning, compat validation) live in the callers; see pkg/sourcecache for the connector-facing contract.
type StoreOptions ¶ added in v0.10.0
type StoreOptions struct {
TmpDir string
DecoderOptions []DecoderOption
ReadOnly bool
SyncLimit int
SkipCleanup bool
// DisableGrantDigestIndex turns off the Pebble engine's seal-time
// build of the by_entitlement_principal_hash index + grant digests.
// Inverted so the zero value keeps the build on (current behavior).
DisableGrantDigestIndex bool
Engine c1zstore.Engine
// PayloadEncoding selects the v3 envelope payload framing for
// engines that produce a v3 envelope (currently Pebble). Zero
// value means "engine default" (PayloadEncodingIndexedZstd for
// Pebble).
PayloadEncoding c1zstore.PayloadEncoding
// DecoderPool optionally scopes v3 payload-decoder reuse to the
// caller's operation (see WithDecoderPool). Nil means a one-shot
// decoder per open.
DecoderPool *EnvelopeDecoderPool
// MaxDecodedPayloadBytes optionally caps v3 envelope payload extraction.
// Zero means use BATON_DECODER_MAX_DECODED_SIZE_MB or the v3 default.
MaxDecodedPayloadBytes uint64
// MaxDecoderMemoryBytes optionally caps zstd decoder memory for v3 envelope
// payload extraction. Zero means use BATON_DECODER_MAX_MEMORY_MB or the v3
// default.
MaxDecoderMemoryBytes uint64
}
StoreOptions is the engine-neutral form of C1ZOption values passed to a registered engine driver.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package c1zstore defines the engine-neutral storage contract for .c1z files: the Store interface and its sub-stores (GrantStore, SyncMeta, FileOps), the row shapes they exchange, and the SDK-wide sync retention policy.
|
Package c1zstore defines the engine-neutral storage contract for .c1z files: the Store interface and its sub-stores (GrantStore, SyncMeta, FileOps), the row shapes they exchange, and the SDK-wide sync retention policy. |
|
engine
|
|
|
equivalence
Package equivalence runs the same workload through two storage backends and asserts they produce equivalent results across every reader method.
|
Package equivalence runs the same workload through two storage backends and asserts they produce equivalent results across every reader method. |
|
pebble
Package pebble is the v3 storage engine for baton-sdk.
|
Package pebble is the v3 storage engine for baton-sdk. |
|
pebble/codec
Package codec implements the v3 storage engine's record codec layer.
|
Package codec implements the v3 storage engine's record codec layer. |
|
pebble/internal/rawdb
The v3 keyspace ABI the choke point enforces: the type/index discriminator bytes, the grant primary-key splices, the resource value scanners, and the digest invalidation key builders.
|
The v3 keyspace ABI the choke point enforces: the type/index discriminator bytes, the grant primary-key splices, the resource value scanners, and the digest invalidation key builders. |
|
pebble/microtests
Package microtests holds the 5 risk-validation tests that were written to prove the material design choices in RFC 0004 (storage engine v4) actually hold under Pebble's real behavior.
|
Package microtests holds the 5 risk-validation tests that were written to prove the material design choices in RFC 0004 (storage engine v4) actually hold under Pebble's real behavior. |
|
format
|
|
|
v3
Package v3 implements the C1Z3 envelope format: 5-byte magic, a length-prefixed proto manifest, and a payload (typically a zstd-tar of a Pebble directory).
|
Package v3 implements the C1Z3 envelope format: 5-byte magic, a length-prefixed proto manifest, and a payload (typically a zstd-tar of a Pebble directory). |
|
Package sqlite reads v1 c1z files without modifying their SQLite database.
|
Package sqlite reads v1 c1z files without modifying their SQLite database. |