dotc1z

package
v0.13.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 9, 2026 License: Apache-2.0 Imports: 50 Imported by: 2

Documentation

Index

Constants

View Source 
const (
	// EngineSQLite is the default engine: the v1 .c1z format backed by
	// a zstd-compressed SQLite database. Connectors use this; backend
	// infra can opt out.
	EngineSQLite = c1zstore.EngineSQLite

	// EnginePebble is the v3 engine: a Pebble LSM wrapped in the v3
	// envelope.
	EnginePebble = c1zstore.EnginePebble
)
View Source 
const (
	// PayloadEncodingUnspecified is the zero value. Means "use the
	// engine's default" — TarZstd for Pebble.
	PayloadEncodingUnspecified = c1zstore.PayloadEncodingUnspecified

	// PayloadEncodingTarZstd is the default Pebble v3 envelope
	// encoding: tar of the Pebble directory, compressed with zstd.
	PayloadEncodingTarZstd = c1zstore.PayloadEncodingTarZstd

	// PayloadEncodingTar is uncompressed tar. Useful when Pebble's
	// L5/L6 SSTs are already zstd-compressed at the engine layer
	// (avoids double-compression CPU), or when the storage target
	// compresses in transit.
	PayloadEncodingTar = c1zstore.PayloadEncodingTar
)
View Source 
const DefaultBulkLoadIndexTimeout = 6 * time.Hour

DefaultBulkLoadIndexTimeout bounds the detached context for the bulk-load deferred-index rebuild at Close. Building several secondary indexes over a 50M+-row grants table is tens of minutes — much longer than the checkpoint+save FinalizeTimeout covers — so this gets its own generous ceiling. Six hours is a backstop against a wedged build, not an expected duration.

View Source 
const DefaultFinalizeTimeout = 1 * time.Hour

DefaultFinalizeTimeout bounds the detached context used for c1z finalization (WAL checkpoint, save-to-disk, upload). Large tenants take 5-15 min to upload a 4 GB compressed c1z; one hour gives plenty of headroom while still being a hard ceiling so a wedged upload cannot hold a worker indefinitely.

Variables

View Source 
var (
	ErrInvalidFile        = fmt.Errorf("c1z: invalid file")
	ErrMaxSizeExceeded    = fmt.Errorf("c1z: max decoded size exceeded, increase DecoderMaxDecodedSize using %v environment variable", maxDecodedSizeEnvVar)
	ErrWindowSizeExceeded = fmt.Errorf("c1z: window size exceeded, increase DecoderMaxMemory using %v  environment variable", maxDecoderMemorySizeEnv)
)
View Source 
var C1Z3FileHeader = []byte("C1Z3\x00")

C1Z3FileHeader is the magic byte sequence for v3 files.

View Source 
var C1ZFileHeader = []byte("C1ZF\x00")
View Source 
var ErrDbNotOpen = errors.New("c1file: database has not been opened")
View Source 
var ErrEngineNotAvailable = fmt.Errorf("dotc1z: engine not available")

ErrEngineNotAvailable is returned when a caller requests an engine that the binary does not support.

View Source 
var ErrReadOnly = errors.New("c1z: read only mode")
View Source 
var ErrSyncNotExpanded = errors.New("c1z: sync did not run grant expansion (no supports_diff marker); nothing to roll back")

ErrSyncNotExpanded is returned when a rollback targets a finished sync that never ran grant expansion (no supports_diff marker). There is nothing to roll back, and a missing marker also means a resumable graph was never produced.

View Source 
var ErrSyncNotFinished = errors.New("c1z: refusing to roll back expansion on an unfinished sync")

ErrSyncNotFinished is returned when a rollback targets a sync that has not finished. Expansion completeness for an in-progress sync lives in the entitlement graph persisted in the sync token, not in the grant rows. Deleting expansion output from such a sync would strand that graph: a later resume would treat the affected edges as already expanded and never regenerate the deleted grants. Rollback only operates on finished syncs.

Functions

func BulkLoadIndexTimeout added in v0.12.7 

func BulkLoadIndexTimeout() time.Duration

BulkLoadIndexTimeout returns the bound for the detached context that wraps the bulk-load deferred-index rebuild.

func C1ZFileCheckHeader 

func C1ZFileCheckHeader(f io.ReadSeeker) (bool, error)

C1ZFileCheckHeader reads len(C1ZFileHeader) bytes from the given io.ReadSeeker and compares them to C1ZFileHeader. Returns true if the header is valid. Returns any errors from Read() or Seek(). If a nil error is returned, the given io.ReadSeeker will be pointing to the first byte of the stream, and is suitable to be passed to NewC1ZFileDecoder.

func CleanupSkippedByEnv added in v0.10.0 

func CleanupSkippedByEnv() bool

CleanupSkippedByEnv reports whether BATON_SKIP_CLEANUP is set to a truthy value. See c1zstore.CleanupSkippedByEnv.

func FinalizeTimeout added in v0.10.0 

func FinalizeTimeout() time.Duration

FinalizeTimeout returns the bound for the detached context that wraps c1z finalize-and-upload tails.

func NewC1FileReader 

func NewC1FileReader(ctx context.Context, dbFilePath string, opts ...C1FOption) (connectorstore.Reader, error)

NewC1FileReader returns a connectorstore.Reader implementation for the given sqlite db file path.

func NewC1ZFileDecoder 

func NewC1ZFileDecoder(f io.Reader, opts ...DecoderOption) (io.ReadCloser, error)

NewC1ZFileDecoder wraps a given .c1z io.Reader that validates the .c1z and decompresses/decodes the underlying file. Defaults: 128MiB max memory and 3GiB max decoded size You must close the resulting io.ReadCloser when you are done, do not forget to close the given io.Reader if necessary.

func NewDecoder added in v0.0.24 

func NewDecoder(f io.Reader, opts ...DecoderOption) (*decoder, error)

NewDecoder wraps a given .c1z file io.Reader and returns an io.Reader for the underlying decoded/uncompressed file.

func NewExternalC1FileReader added in v0.2.84 

func NewExternalC1FileReader(ctx context.Context, tmpDir string, externalResourceC1ZPath string) (connectorstore.Reader, error)

func ReadHeader added in v0.0.24 

func ReadHeader(reader io.Reader) error

ReadHeader reads len(C1ZFileHeader) bytes from the given io.Reader and compares them to C1ZFileHeader, returning an error if they don't match. If possible, ReadHeader will Seek() to the start of the stream before checking the header bytes. On return, the reader will be pointing to the first byte after the header.

func RegisterEngine added in v0.10.0 

func RegisterEngine(driver EngineDriver) error

RegisterEngine registers a storage engine driver with the process-global dotc1z engine registry.

func ResolveCleanupSyncLimit added in v0.10.0 

func ResolveCleanupSyncLimit(callerLimit int, currentSyncOpen bool) int

ResolveCleanupSyncLimit resolves the effective retention limit. See c1zstore.ResolveCleanupSyncLimit.

func SelectSyncsToDelete added in v0.10.0 

func SelectSyncsToDelete(candidates []SyncRun, currentSyncID string, syncLimit int) []string

SelectSyncsToDelete applies the SDK retention policy to a snapshot of sync runs and returns the IDs whose data should be deleted. See c1zstore.SelectSyncsToDelete for the policy details.

Types

type C1FOption added in v0.1.8 

type C1FOption func(*C1File)

func WithC1FBulkLoad added in v0.12.7 

func WithC1FBulkLoad(enabled bool) C1FOption

WithC1FBulkLoad enables deferred secondary-index creation for a freshly-created destination. See C1File.bulkLoad. Intended for a net-new, single-writer output (e.g. c1z sanitize); never use it to reopen a file that other readers/writers touch concurrently.

Safe-by-construction guard: per table, indexes are deferred only when the table is EMPTY. A non-empty table keeps its indexes live and that table degrades to normal mode (with a warning) — so setting this on a populated file never drops indexes on a large table.

Implies skip-cleanup (Cleanup's old-sync deletes would scan without the deferred indexes). Pair with WithC1FSkipVacuum(true) too. The deferred rebuild at Close sorts through SQLite temp storage: provision SQLITE_TMPDIR / temp_store and cache_size for roughly the total index size; under journal_mode=WAL the WAL can grow by about the index size before checkpoint.

func WithC1FEncoderConcurrency added in v0.6.6 

func WithC1FEncoderConcurrency(concurrency int) C1FOption

WithC1FEncoderConcurrency sets the number of created encoders. Default is 1, which disables async encoding/concurrency. 0 uses GOMAXPROCS.

func WithC1FEngine added in v0.10.0 

func WithC1FEngine(engine Engine) C1FOption

WithC1FEngine selects the storage engine for new .c1z files. The default is EngineSQLite, which keeps the legacy v1 file format and behavior. EnginePebble selects the v3 engine introduced by the storage-engine-v4 RFC.

Engine selection only affects newly created files. Existing files dispatch on their magic byte; readers handle both v1 and v3 regardless of this option.

func WithC1FPayloadEncoding added in v0.10.0 

func WithC1FPayloadEncoding(enc PayloadEncoding) C1FOption

WithC1FPayloadEncoding selects the v3 envelope payload encoding (TAR_ZSTD default, TAR uncompressed). No-op for SQLite engines.

func WithC1FPragma added in v0.1.22 

func WithC1FPragma(name string, value string) C1FOption

WithC1FPragma sets a sqlite pragma for the c1z file.

func WithC1FReadOnly added in v0.6.6 

func WithC1FReadOnly(readOnly bool) C1FOption

WithC1FReadOnly opens the c1file in read only mode. Write operations will return an error. Read only mode is faster, as it disables journaling and synchronous writes.

func WithC1FSkipCleanup added in v0.8.22 

func WithC1FSkipCleanup(skip bool) C1FOption

WithC1FSkipCleanup skips cleanup of old syncs when set to true.

func WithC1FSkipVacuum added in v0.12.1 

func WithC1FSkipVacuum(skip bool) C1FOption

WithC1FSkipVacuum skips the VACUUM step at the end of Cleanup() when set to true. The old-sync delete and WAL truncate inside Cleanup still run.

Low-level option for callers operating on a C1File directly. Most callers should use WithSkipVacuum on NewC1ZFile instead, which propagates this setting through the C1Z constructor.

Trade-off: skipping VACUUM leaves freed pages on the SQLite freelist instead of reclaiming them, so the c1z file on disk grows across syncs with no upper bound until a real VACUUM runs. Use when the file is consumed immediately and re-encoded (e.g. iterative compaction); avoid when the c1z is intended to sit at rest or be read repeatedly. See also WithC1FSkipCleanup to skip the whole Cleanup step instead.

func WithC1FSyncCountLimit added in v0.7.0 

func WithC1FSyncCountLimit(limit int) C1FOption

WithC1FSyncCountLimit sets the number of syncs to keep during cleanup. If not set, defaults to 2 (or BATON_KEEP_SYNC_COUNT env var if set).

func WithC1FTmpDir added in v0.1.8 

func WithC1FTmpDir(tempDir string) C1FOption

WithC1FTmpDir sets the temporary directory to use when cloning a sync. If not provided, os.TempDir() will be used.

func WithC1FV2GrantsWriter added in v0.9.9 

func WithC1FV2GrantsWriter(enabled bool) C1FOption

WithC1FV2GrantsWriter strips Grant.Entitlement and Grant.Principal from the serialized data blob at write time. Readers rebuild them as identity-only stubs (Id + nested Resource.Id) from the grants row's columns. Stubs carry no DisplayName, Annotations, Purpose, Slug, or traits; callers that need those must fetch the Entitlement or Resource directly. Readers accept both shapes regardless of this flag, so old and new rows coexist. Default false.

Per-grant escape hatch: see unsafeForSlim. Grants carrying InsertResourceGrants or any ExternalResourceMatch* annotation stay full-blob — those code paths read non-identity fields off the embedded Resource / Principal.

type C1File added in v0.0.24 

type C1File struct {
	// contains filtered or unexported fields
}

func AsSQLiteStore added in v0.8.29 

func AsSQLiteStore(s C1ZStore) (*C1File, bool)

AsSQLiteStore type-asserts a C1ZStore to the concrete *C1File. It is an escape hatch for callers that legitimately need SQLite-specific primitives (today: the attached compactor in pkg/synccompactor/attached, which uses SQL ATTACH for cross-file merge). Returns (nil, false) when the store is not backed by *C1File OR when the underlying *C1File is nil.

Avoid using this outside pkg/synccompactor. If you find yourself reaching for it, prefer adding a named method to C1ZStore that expresses what you need; sqlite-specific leak-through is a smell. See RFC 0002 §4.4.

func NewC1File added in v0.0.24 

func NewC1File(ctx context.Context, dbFilePath string, opts ...C1FOption) (*C1File, error)

Returns a C1File instance for the given db filepath.

func NewC1ZFile added in v0.0.24 

func NewC1ZFile(ctx context.Context, outputFilePath string, opts ...C1ZOption) (*C1File, error)

Returns a new C1File instance with its state stored at the provided filename.

func (*C1File) AttachFile added in v0.3.35 

func (c *C1File) AttachFile(other *C1File, dbName string) (*C1FileAttached, error)

func (*C1File) CheckpointSync added in v0.0.24 

func (c *C1File) CheckpointSync(ctx context.Context, syncToken string) error

func (*C1File) Cleanup added in v0.0.24 

func (c *C1File) Cleanup(ctx context.Context) error

func (*C1File) Clear added in v0.5.0 

func (c *C1File) Clear(ctx context.Context, opt ...sessions.SessionStoreOption) error

Clear implements types.SessionStore.

func (*C1File) CloneSync added in v0.0.24 

func (c *C1File) CloneSync(ctx context.Context, outPath string, syncID string, opts ...C1FOption) error

CloneSync uses sqlite hackery to directly copy the pertinent rows into a new database. 1. Create a new empty sqlite database in a temp file 2. Open the c1z that we are cloning to get a db handle 3. Execute an ATTACH query to bring our empty sqlite db into the context of our db connection 4. Select directly from the cloned db and insert directly into the new database. 5. Close and save the new database as a c1z at the configured path.

func (*C1File) Close added in v0.0.24 

func (c *C1File) Close(ctx context.Context) (retErr error)

Close ensures that the sqlite database is flushed to disk, and if any changes were made we update the original database with our changes.

When there is real finalize work to do (rawDb open, dbUpdated, not read-only), the WAL checkpoint, raw-db close, and saveC1z all run on a detached context bounded by FinalizeTimeout(). Caller cancellation cannot truncate the c1z mid-finalize; the new-root finalize span links back to the caller's trace for navigability without bloating the parent trace.

func (*C1File) CurrentDBSizeBytes added in v0.8.27 

func (c *C1File) CurrentDBSizeBytes() (int64, error)

CurrentDBSizeBytes returns the current total on-disk size of the underlying uncompressed sqlite database, including the write-ahead log if present. Used by operational tooling (e.g. the grant-expansion progress logger) to observe c1z growth during long in-process writes without waiting for saveC1z to land a new frame.

The WAL file holds writes that have not yet been checkpointed into the main database file; with journal_mode=WAL the main file may stay stable for long stretches while the WAL grows into the hundreds of MB. Summing both gives a representative "bytes written so far" figure during active expansion.

This is the *uncompressed* size. The post-saveC1z c1z file size (compressed) is smaller; for that, see the `c1z: saved` log line emitted by saveC1z.

func (*C1File) CurrentSyncStep added in v0.0.24 

func (c *C1File) CurrentSyncStep(ctx context.Context) (string, error)

func (*C1File) Delete added in v0.5.0 

func (c *C1File) Delete(ctx context.Context, key string, opt ...sessions.SessionStoreOption) error

Delete implements types.SessionStore.

func (*C1File) DeleteGrant added in v0.2.84 

func (c *C1File) DeleteGrant(ctx context.Context, grantId string) error

func (*C1File) DeleteSyncRun added in v0.0.24 

func (c *C1File) DeleteSyncRun(ctx context.Context, syncID string) error

DeleteSyncRun removes all the objects with a given syncID from the database.

func (*C1File) EndSync added in v0.0.24 

func (c *C1File) EndSync(ctx context.Context) error

EndSync updates the current sync_run row with the end time, and removes any other objects that don't have the current sync ID.

func (*C1File) FileOps added in v0.8.29 

func (c *C1File) FileOps() FileOps

FileOps returns the file-operations slice of this c1z.

func (*C1File) GenerateSyncDiff added in v0.3.2 

func (c *C1File) GenerateSyncDiff(ctx context.Context, baseSyncID string, appliedSyncID string) (string, error)

func (*C1File) Get added in v0.5.0 

func (c *C1File) Get(ctx context.Context, key string, opt ...sessions.SessionStoreOption) ([]byte, bool, error)

Get implements types.SessionCache.

func (*C1File) GetAll added in v0.5.0 

func (c *C1File) GetAll(ctx context.Context, pageToken string, opt ...sessions.SessionStoreOption) (map[string][]byte, string, error)

GetAll implements types.SessionStore.

func (*C1File) GetAsset added in v0.0.24 

func (c *C1File) GetAsset(ctx context.Context, request *v2.AssetServiceGetAssetRequest) (string, io.Reader, error)

GetAsset fetches the specified asset from the database, and returns the content type and an io.Reader for the caller to read the asset from.

func (*C1File) GetEntitlement added in v0.0.24 

func (c *C1File) GetEntitlement(ctx context.Context, request *reader_v2.EntitlementsReaderServiceGetEntitlementRequest) (*reader_v2.EntitlementsReaderServiceGetEntitlementResponse, error)

func (*C1File) GetGrant added in v0.0.24 

func (c *C1File) GetGrant(ctx context.Context, request *reader_v2.GrantsReaderServiceGetGrantRequest) (*reader_v2.GrantsReaderServiceGetGrantResponse, error)

func (*C1File) GetLatestFinishedSync added in v0.2.83 

func (c *C1File) GetLatestFinishedSync(
	ctx context.Context, request *reader_v2.SyncsReaderServiceGetLatestFinishedSyncRequest,
) (*reader_v2.SyncsReaderServiceGetLatestFinishedSyncResponse, error)

func (*C1File) GetMany added in v0.5.0 

func (c *C1File) GetMany(ctx context.Context, keys []string, opt ...sessions.SessionStoreOption) (map[string][]byte, []string, error)

GetMany implements types.SessionStore.

func (*C1File) GetResource added in v0.0.24 

func (c *C1File) GetResource(ctx context.Context, request *reader_v2.ResourcesReaderServiceGetResourceRequest) (*reader_v2.ResourcesReaderServiceGetResourceResponse, error)

func (*C1File) GetResourceType added in v0.0.24 

func (c *C1File) GetResourceType(ctx context.Context, request *reader_v2.ResourceTypesReaderServiceGetResourceTypeRequest) (*reader_v2.ResourceTypesReaderServiceGetResourceTypeResponse, error)

func (*C1File) GetSync added in v0.2.83 

func (c *C1File) GetSync(ctx context.Context, request *reader_v2.SyncsReaderServiceGetSyncRequest) (*reader_v2.SyncsReaderServiceGetSyncResponse, error)

func (*C1File) GrantSourcesForSync deprecated added in v0.12.2 

func (c *C1File) GrantSourcesForSync(ctx context.Context, syncID string) (map[string]string, error)

GrantSourcesForSync returns, for every grant in the sync, a canonical string form of its GrantSources keyed by the grant's id.

Deprecated: the rollback subcommand now computes this in the command layer. This method is retained for external callers that depended on it and will be removed in a future minor release. It streams the grants one at a time rather than buffering the table.

func (*C1File) Grants added in v0.8.29 

func (c *C1File) Grants() GrantStore

Grants returns the grant-store slice of this c1z.

func (*C1File) InitTables added in v0.6.5 

func (c *C1File) InitTables(ctx context.Context) (bool, error)

InitTables initializes the tables in the database. Returns true if the any migrations were run, false otherwise.

func (*C1File) LatestFinishedSyncID added in v0.3.56 

func (c *C1File) LatestFinishedSyncID(ctx context.Context, syncType connectorstore.SyncType) (string, error)

func (*C1File) LatestSyncID added in v0.0.24 

func (c *C1File) LatestSyncID(ctx context.Context, syncType connectorstore.SyncType) (string, error)

func (*C1File) ListEntitlements added in v0.0.24 

func (c *C1File) ListEntitlements(ctx context.Context, request *v2.EntitlementsServiceListEntitlementsRequest) (*v2.EntitlementsServiceListEntitlementsResponse, error)

func (*C1File) ListEntitlementsByIds added in v0.10.0 

func (c *C1File) ListEntitlementsByIds(
	ctx context.Context,
	req *reader_v2.EntitlementsReaderServiceListEntitlementsByIdsRequest,
) (*reader_v2.EntitlementsReaderServiceListEntitlementsByIdsResponse, error)

ListEntitlementsByIds returns the entitlements for the requested ids in any order. Missing rows are silently omitted; callers detect partial misses by length / id comparison.

func (*C1File) ListGrants added in v0.0.24 

func (c *C1File) ListGrants(ctx context.Context, request *v2.GrantsServiceListGrantsRequest) (*v2.GrantsServiceListGrantsResponse, error)

func (*C1File) ListGrantsForEntitlement added in v0.0.24 

func (c *C1File) ListGrantsForEntitlement(
	ctx context.Context,
	request *reader_v2.GrantsReaderServiceListGrantsForEntitlementRequest,
) (*reader_v2.GrantsReaderServiceListGrantsForEntitlementResponse, error)

func (*C1File) ListGrantsForEntitlements added in v0.10.0 

func (c *C1File) ListGrantsForEntitlements(
	ctx context.Context,
	req *reader_v2.GrantsReaderServiceListGrantsForEntitlementsRequest,
) (*reader_v2.GrantsReaderServiceListGrantsForEntitlementsResponse, error)

ListGrantsForEntitlements batches K-entitlement grant fetches (RFC §A4). Single SQL query per page with entitlement_id IN (?, ?, ...) — uses the (entitlement_id) index on the grants table.

Cursor encoding mirrors the Pebble adapter: 

varint(entitlement_index) || varint(intra_cursor_len) ||
intra_cursor_bytes || crc32(sorted-ent-IDs)

where intra_cursor here is the row-id of the last returned grant (encoded as a decimal string for compatibility with the row-id page-token used elsewhere in this package).

func (*C1File) ListGrantsForPrincipal added in v0.1.6 

func (c *C1File) ListGrantsForPrincipal(
	ctx context.Context,
	request *reader_v2.GrantsReaderServiceListGrantsForEntitlementRequest,
) (*reader_v2.GrantsReaderServiceListGrantsForEntitlementResponse, error)

func (*C1File) ListGrantsForResourceType added in v0.0.24 

func (c *C1File) ListGrantsForResourceType(
	ctx context.Context,
	request *reader_v2.GrantsReaderServiceListGrantsForResourceTypeRequest,
) (*reader_v2.GrantsReaderServiceListGrantsForResourceTypeResponse, error)

func (*C1File) ListResourceTypes added in v0.0.24 

func (c *C1File) ListResourceTypes(ctx context.Context, request *v2.ResourceTypesServiceListResourceTypesRequest) (*v2.ResourceTypesServiceListResourceTypesResponse, error)

func (*C1File) ListResources added in v0.0.24 

func (c *C1File) ListResources(ctx context.Context, request *v2.ResourcesServiceListResourcesRequest) (*v2.ResourcesServiceListResourcesResponse, error)

func (*C1File) ListResourcesByIds added in v0.10.0 

func (c *C1File) ListResourcesByIds(
	ctx context.Context,
	req *reader_v2.ResourcesReaderServiceListResourcesByIdsRequest,
) (*reader_v2.ResourcesReaderServiceListResourcesByIdsResponse, error)

ListResourcesByIds returns the resources for the supplied ResourceId pairs (resource_type:resource composite externals). Missing rows are silently omitted.

func (*C1File) ListStaticEntitlements added in v0.5.21 

func (c *C1File) ListStaticEntitlements(ctx context.Context, request *v2.EntitlementsServiceListStaticEntitlementsRequest) (*v2.EntitlementsServiceListStaticEntitlementsResponse, error)

func (*C1File) ListSyncRuns added in v0.0.24 

func (c *C1File) ListSyncRuns(ctx context.Context, pageToken string, pageSize uint32) ([]*SyncRun, string, error)

func (*C1File) ListSyncs added in v0.2.83 

func (c *C1File) ListSyncs(ctx context.Context, request *reader_v2.SyncsReaderServiceListSyncsRequest) (*reader_v2.SyncsReaderServiceListSyncsResponse, error)

func (*C1File) Metadata added in v0.10.0 

func (c *C1File) Metadata() connectorstore.StoreMetadata

Metadata describes the storage backing this c1z. C1File is the SQLite-engine implementation; the v1 file format is implied by the engine. No I/O is performed.

func (*C1File) OutputFilepath added in v0.3.3 

func (c *C1File) OutputFilepath() (string, error)

func (*C1File) PreviousSyncID added in v0.0.24 

func (c *C1File) PreviousSyncID(ctx context.Context, syncType connectorstore.SyncType) (string, error)

func (*C1File) PutAsset added in v0.0.24 

func (c *C1File) PutAsset(ctx context.Context, assetRef *v2.AssetRef, contentType string, data []byte) error

PutAsset stores the given asset in the database.

func (*C1File) PutEntitlements added in v0.1.46 

func (c *C1File) PutEntitlements(ctx context.Context, entitlementObjs ...*v2.Entitlement) error

func (*C1File) PutGrants added in v0.1.46 

func (c *C1File) PutGrants(ctx context.Context, bulkGrants ...*v2.Grant) error

PutGrants is the connector-facing write method on connectorstore.Writer. It replaces any conflicting row and re-extracts expansion metadata from the grant payload.

func (*C1File) PutResourceTypes added in v0.1.46 

func (c *C1File) PutResourceTypes(ctx context.Context, resourceTypesObjs ...*v2.ResourceType) error

func (*C1File) PutResources added in v0.1.46 

func (c *C1File) PutResources(ctx context.Context, resourceObjs ...*v2.Resource) error

func (*C1File) ResumeSync added in v0.3.56 

func (c *C1File) ResumeSync(ctx context.Context, syncType connectorstore.SyncType, syncID string) (string, error)

func (*C1File) RollbackExpansion added in v0.12.2 

func (c *C1File) RollbackExpansion(ctx context.Context, syncID string, dryRun bool, opts ...RollbackOption) (*RollbackResult, error)

RollbackExpansion restores a finished, expanded sync to its pre-grant-expansion shape so the expansion can be replayed and timed.

It is driven entirely by each grant's Sources map (the GrantSources provenance that already exists on synced c1z files), so it needs no schema change and no sync-write change — it is a subcommand-only operation over an existing c1z.

For every grant in the target sync:

  • No Sources at all → a plain direct grant the expander never touched. Left untouched.
  • Sources present, but none keyed to the grant's own entitlement → a purely expander-derived grant. Deleted.
  • Sources present and including a self-source for the grant's own entitlement → a direct grant the expander touched on first use. Kept, but its Sources are cleared to empty: pre-expansion these grants had no Sources, so clearing restores the faithful baseline.

The grant's own entitlement id is read from the entitlement_id column rather than from the data blob, because the blob is slimmed on write (slimGrantForWrite nils Entitlement/Principal) while the column is populated pre-slim. Sources are not slimmed, so they are read from the blob.

The grant mutations run one transaction per page rather than one transaction for the whole sync, so the SQLite rollback journal never has to hold every changed grant at once — on a large sync the expander-derived rows can be most of the table. Writes always go to a fresh clone of the sync (the caller never mutates the input), so a failure partway through leaves a discardable output rather than a half-rolled-back source file.

func (*C1File) Set added in v0.5.0 

func (c *C1File) Set(ctx context.Context, key string, value []byte, opt ...sessions.SessionStoreOption) error

Set implements types.SessionStore.

func (*C1File) SetCurrentSync added in v0.3.8 

func (c *C1File) SetCurrentSync(ctx context.Context, syncID string) error

func (*C1File) SetMany added in v0.5.0 

func (c *C1File) SetMany(ctx context.Context, values map[string][]byte, opt ...sessions.SessionStoreOption) error

SetMany implements types.SessionStore.

func (*C1File) SetSupportsDiff added in v0.7.22 

func (c *C1File) SetSupportsDiff(ctx context.Context, syncID string) error

SetSupportsDiff marks the given sync as supporting diff operations. This indicates the sync has SQL-layer grant metadata (is_expandable) properly populated.

func (*C1File) SetSyncID added in v0.6.8 

func (c *C1File) SetSyncID(_ context.Context, syncID string) error

SetSyncID sets the current sync ID. This is only intended for testing. 

func (c *C1File) SetSyncLink(ctx context.Context, syncID string, linkedSyncID string) error

SetSyncLink sets the linked_sync_id of an existing sync run. Diff sync pairs (partial_upserts ↔ partial_deletions) reference each other bidirectionally; a writer rebuilding such a pair cannot supply the link at StartNewSync time because the partner's id is minted by the store, so the pairing is applied after both runs exist.

func (*C1File) StartNewSync added in v0.2.15 

func (c *C1File) StartNewSync(ctx context.Context, syncType connectorstore.SyncType, parentSyncID string) (string, error)

func (*C1File) StartOrResumeSync added in v0.3.52 

func (c *C1File) StartOrResumeSync(ctx context.Context, syncType connectorstore.SyncType, syncID string) (string, bool, error)

StartOrResumeSync checks if a sync is already running and resumes it if it is. If no sync is running, it starts a new sync. It returns the sync ID and a boolean indicating if a new sync was started.

func (*C1File) Stats added in v0.0.24 

func (c *C1File) Stats(ctx context.Context, syncType connectorstore.SyncType, syncId string) (map[string]int64, error)

Stats introspects the database and returns the count of objects for the given sync run. If syncId is empty, it will use the latest sync run of the given type.

func (*C1File) StoreExpandedGrants added in v0.8.29 

func (c *C1File) StoreExpandedGrants(ctx context.Context, grants ...*v2.Grant) error

StoreExpandedGrants persists grants produced by the expander. It strips any residual GrantExpandable annotation from each grant payload and then delegates to the internal UpsertGrants path using PreserveExpansion mode so existing expansion/needs_expansion columns are left untouched.

The strip step is defensive: the expander's upstream callers consume the annotation before writing, but a residual annotation on the payload would disagree with the stored expansion columns. Stripping makes the method total regardless of caller discipline.

This method is exposed on *C1File (not just via GrantStore) because test helpers in pkg/sync/expand construct a *C1File directly and pass it to NewExpander; putting it at the top level keeps those tests free of sub-store wiring.

func (*C1File) StreamEntitlements added in v0.10.0 

func (c *C1File) StreamEntitlements(
	ctx context.Context,
	syncID string,
) iter.Seq2[*v2.Entitlement, error]

StreamEntitlements yields entitlements for the active (or requested) sync.

func (*C1File) StreamGrants added in v0.10.0 

func (c *C1File) StreamGrants(
	ctx context.Context,
	syncID string,
	opts connectorstore.StreamGrantsOptions,
) iter.Seq2[*v2.Grant, error]

StreamGrants yields grants for the active (or requested) sync. Implements connectorstore.StreamingReader. SQLite path uses Rows.Next() as the streaming pump — no client-side pagination buffer.

func (*C1File) StreamResources added in v0.10.0 

func (c *C1File) StreamResources(
	ctx context.Context,
	syncID string,
	opts connectorstore.StreamResourcesOptions,
) iter.Seq2[*v2.Resource, error]

StreamResources yields resources optionally filtered by RT.

func (*C1File) SyncMeta added in v0.8.29 

func (c *C1File) SyncMeta() SyncMeta

SyncMeta returns the sync-metadata slice of this c1z.

func (*C1File) ToPebble added in v0.12.2 

func (c *C1File) ToPebble(ctx context.Context, outPath string, syncID string, opts ...ConvertOption) (*ConvertStats, error)

ToPebble converts a single finished sync from this SQLite store into a new v3/Pebble .c1z written to outPath, which must not already exist. It streams the sync's data (resource types, resources, entitlements, grants, assets) into a fresh SyncTypeFull on the destination, using the engine's fresh-sync fast write path (NoSync per commit, single fsync at EndSync).

syncID selects the source sync to convert; "" uses the latest finished full sync. ToPebble does NOT validate that the sync is a complete snapshot — the caller owns that decision (e.g. compacting targeted-partial/diff syncs into a complete snapshot beforehand). It only requires the sync to exist and be ended, mirroring CloneSync.

The Pebble engine is registered statically with dotc1z; no extra imports are needed before calling.

func (*C1File) Vacuum added in v0.1.8 

func (c *C1File) Vacuum(ctx context.Context) error

Vacuum runs a VACUUM on the database to reclaim space.

func (*C1File) ViewSync added in v0.0.24 

func (c *C1File) ViewSync(ctx context.Context, syncID string) error

type C1FileAttached added in v0.3.35 

type C1FileAttached struct {
	// contains filtered or unexported fields
}

func (*C1FileAttached) CompactEntitlements added in v0.3.35 

func (c *C1FileAttached) CompactEntitlements(ctx context.Context, baseSyncID string, appliedSyncID string) error

func (*C1FileAttached) CompactGrants added in v0.3.35 

func (c *C1FileAttached) CompactGrants(ctx context.Context, baseSyncID string, appliedSyncID string) error

func (*C1FileAttached) CompactResourceTypes added in v0.3.35 

func (c *C1FileAttached) CompactResourceTypes(ctx context.Context, baseSyncID string, appliedSyncID string) error

func (*C1FileAttached) CompactResources added in v0.3.35 

func (c *C1FileAttached) CompactResources(ctx context.Context, baseSyncID string, appliedSyncID string) error

func (*C1FileAttached) CompactTable added in v0.3.35 

func (c *C1FileAttached) CompactTable(ctx context.Context, baseSyncID string, appliedSyncID string, tableName string) error

func (*C1FileAttached) DetachFile added in v0.3.35 

func (c *C1FileAttached) DetachFile(dbName string) (*C1FileAttached, error)

func (*C1FileAttached) GenerateSyncDiffFromFile added in v0.7.0 

func (c *C1FileAttached) GenerateSyncDiffFromFile(ctx context.Context, oldSyncID string, newSyncID string) (string, string, error)

GenerateSyncDiffFromFile compares the old sync (in attached) with the new sync (in main) and generates two new syncs in the main database.

IMPORTANT: This assumes main=NEW/compacted and attached=OLD/base: - diffTableFromAttached: items in attached (OLD) not in main (NEW) = deletions - diffTableFromMain: items in main (NEW) not in attached (OLD) = upserts (additions)

Parameters: - oldSyncID: the sync ID in the attached database (OLD/base state) - newSyncID: the sync ID in the main database (NEW/compacted state)

Returns (upsertsSyncID, deletionsSyncID, error).

func (*C1FileAttached) UpdateSync added in v0.6.11 

func (c *C1FileAttached) UpdateSync(ctx context.Context, baseSync *reader_v2.SyncRun, appliedSync *reader_v2.SyncRun) error

type C1ZFormat added in v0.10.0 

type C1ZFormat int

C1ZFormat identifies the on-disk format of a .c1z file. The format byte is the first 5 bytes of the file; see ReadHeaderFormat. 

const (
	// C1ZFormatUnknown is the zero value. Returned when the header bytes
	// match neither the v1 nor the v3 magic, or when the read failed.
	C1ZFormatUnknown C1ZFormat = iota

	// C1ZFormatV1 is the original .c1z format: 5-byte magic "C1ZF\x00"
	// followed by a zstd-compressed SQLite database.
	C1ZFormatV1

	// C1ZFormatV3 is the v3 format introduced by the storage-engine-v4
	// RFC: 5-byte magic "C1Z3\x00", a length-prefixed proto manifest,
	// and a zstd-tar payload of a Pebble Checkpoint directory.
	C1ZFormatV3
)

func ReadHeaderFormat added in v0.10.0 

func ReadHeaderFormat(reader io.Reader) (C1ZFormat, error)

ReadHeaderFormat reads the first 5 bytes of reader and returns the detected format. On return, the reader is positioned immediately after the header bytes. If reader is also an io.Seeker, it is rewound to offset 0 before reading.

Returns:

  • C1ZFormatV1, nil — file starts with "C1ZF\x00".
  • C1ZFormatV3, nil — file starts with "C1Z3\x00".
  • C1ZFormatUnknown, ErrInvalidFile — header matched no known magic.
  • C1ZFormatUnknown, err — underlying read error.

func (C1ZFormat) String added in v0.10.0 

func (f C1ZFormat) String() string

String returns a stable human-readable name for the format.

type C1ZOption added in v0.1.8 

type C1ZOption func(*c1zOptions)

func WithBulkLoad added in v0.12.7 

func WithBulkLoad(enabled bool) C1ZOption

WithBulkLoad enables deferred secondary-index creation for a freshly-created destination c1z. See WithC1FBulkLoad for the full contract: it is opt-in and never the default; indexes are deferred only on EMPTY tables (a populated table keeps its indexes and degrades to normal mode with a warning); it implies skip-cleanup; pair it with WithSkipVacuum(true); and provision temp space + cache_size for the deferred index rebuild at Close.

func WithDecoderOptions added in v0.3.50 

func WithDecoderOptions(opts ...DecoderOption) C1ZOption

func WithEncoderConcurrency added in v0.6.6 

func WithEncoderConcurrency(concurrency int) C1ZOption

WithEncoderConcurrency sets the number of created encoders. Default is 1, which disables async encoding/concurrency. 0 uses GOMAXPROCS.

func WithEngine added in v0.10.0 

func WithEngine(engine Engine) C1ZOption

WithEngine selects the storage engine for newly created .c1z files. Default is EngineSQLite (v1 format). EnginePebble enables the v3 engine.

Reading existing files dispatches on the file's magic byte and is independent of this option.

func WithPayloadEncoding added in v0.10.0 

func WithPayloadEncoding(enc PayloadEncoding) C1ZOption

WithPayloadEncoding selects the c1z v3 envelope payload encoding for newly created files written by the Pebble engine. Default is PayloadEncodingTarZstd. PayloadEncodingTar skips the outer zstd compression — useful when Pebble's L5/L6 SSTs are already zstd-compressed at the engine layer, or when the storage target (S3 with Content-Encoding negotiation, etc.) compresses in transit.

No-op for SQLite engines; the encoding selector applies only to the v3 envelope written by Pebble.

func WithPragma added in v0.1.22 

func WithPragma(name string, value string) C1ZOption

WithPragma sets a sqlite pragma for the c1z file.

func WithReadOnly added in v0.6.6 

func WithReadOnly(readOnly bool) C1ZOption

WithReadOnly opens the c1z file in read only mode. Modifying the c1z will result in an error on close.

func WithSkipCleanup added in v0.8.22 

func WithSkipCleanup(skip bool) C1ZOption

WithSkipCleanup skips cleanup of old syncs when set to true.

func WithSkipVacuum added in v0.12.1 

func WithSkipVacuum(skip bool) C1ZOption

WithSkipVacuum skips the VACUUM step at the end of Cleanup() when set to true. The old-sync delete and WAL truncate inside Cleanup still run.

Trade-off: skipping VACUUM leaves freed pages on the SQLite freelist instead of reclaiming them, so the c1z file on disk grows across syncs with no upper bound until a real VACUUM runs. Use when the file is consumed immediately and re-encoded (e.g. iterative compaction where the output is supplanted on the next iteration); avoid when the c1z is intended to sit at rest or be read repeatedly. No effect when WithSkipCleanup is also set, since Cleanup returns early in that case.

func WithSyncLimit added in v0.7.0 

func WithSyncLimit(limit int) C1ZOption

WithSyncLimit sets the number of syncs to keep during cleanup. If not set, defaults to 2 (or BATON_KEEP_SYNC_COUNT env var if set).

func WithTmpDir added in v0.1.8 

func WithTmpDir(tmpDir string) C1ZOption

WithTmpDir sets the temporary directory to extract the c1z file to. If not provided, os.TempDir() will be used.

func WithV2GrantsWriter added in v0.9.9 

func WithV2GrantsWriter(enabled bool) C1ZOption

WithV2GrantsWriter toggles the slim-blob writer path for grants. See WithC1FV2GrantsWriter for details.

type C1ZStore added in v0.8.29 

type C1ZStore = c1zstore.Store

C1ZStore is the internal contract used by the sync pipeline, compactor, and related infrastructure to read and write a .c1z file. The interface lives in pkg/dotc1z/c1zstore (as c1zstore.Store) so storage engines can implement it without importing this package; this alias preserves the historical dotc1z name.

Implementations:

  • *C1File — the original SQLite-backed implementation (pkg/dotc1z/c1file.go).
  • *pebbleStore — the Pebble v3 engine implementation opened via NewStore(WithEngine(EnginePebble)) (pkg/dotc1z/pebble_store.go).

Both engines are registered statically; no extra imports are needed to open either format.

func NewStore added in v0.10.0 

func NewStore(ctx context.Context, outputFilePath string, opts ...C1ZOption) (C1ZStore, error)

NewStore opens outputFilePath through the registered engine registry. It is the engine-neutral constructor for callers that may opt into non-default engines. NewC1ZFile remains the concrete SQLite constructor for legacy callers that need *C1File.

type CloneSyncOption added in v0.13.0 

type CloneSyncOption = c1zstore.CloneSyncOption

CloneSyncOption configures a FileOps.CloneSync call. See c1zstore.CloneSyncOption.

func WithCloneTmpDir added in v0.13.0 

func WithCloneTmpDir(dir string) CloneSyncOption

WithCloneTmpDir sets the temporary directory used while assembling the cloned c1z. Replaces WithC1FTmpDir at FileOps.CloneSync call sites.

type CloneSyncOptions added in v0.13.0 

type CloneSyncOptions = c1zstore.CloneSyncOptions

CloneSyncOptions carries the engine-neutral knobs for FileOps.CloneSync. See c1zstore.CloneSyncOptions.

type ConvertOption added in v0.12.2 

type ConvertOption func(*convertConfig)

ConvertOption configures ToPebble.

func WithConvertBatchSize added in v0.12.2 

func WithConvertBatchSize(n int) ConvertOption

WithConvertBatchSize sets the per-Put* batch size. Values <= 0 are ignored.

func WithConvertTmpDir added in v0.12.2 

func WithConvertTmpDir(dir string) ConvertOption

WithConvertTmpDir sets the temp directory used for the destination engine's working files. Defaults to the source store's temp dir.

type ConvertStageStats added in v0.12.2 

type ConvertStageStats struct {
	Rows     int64
	Duration time.Duration
}

ConvertStageStats records the row count and wall-clock for one copy stage.

type ConvertStats added in v0.12.2 

type ConvertStats struct {
	SourceSyncID  string
	DestSyncID    string
	ResourceTypes ConvertStageStats
	Resources     ConvertStageStats
	Entitlements  ConvertStageStats
	Grants        ConvertStageStats
	Assets        ConvertStageStats
	AssetBytes    int64
	Total         time.Duration
}

ConvertStats is the per-stage instrumentation returned by ToPebble so the caller can see exactly where time and volume land on a real conversion.

type DecoderOption added in v0.0.24 

type DecoderOption func(*decoderOptions) error

DecoderOption is an option for creating a decoder.

func WithContext 

func WithContext(ctx context.Context) DecoderOption

WithContext sets a context, when cancelled, will cause subequent calls to Read() to return ctx.Error().

func WithDecoderConcurrency added in v0.3.50 

func WithDecoderConcurrency(n int) DecoderOption

WithDecoderConcurrency sets the number of created decoders. Default is 1, which disables async decoding/concurrency. 0 uses GOMAXPROCS. -1 uses GOMAXPROCS or 4, whichever is lower.

func WithDecoderMaxDecodedSize 

func WithDecoderMaxDecodedSize(n uint64) DecoderOption

WithDecoderMaxDecodedSize sets the maximum size of the decoded stream. This can be used to cap the resulting decoded stream size. Maximum is 1 << 63 bytes. Default is 1GiB.

func WithDecoderMaxMemory 

func WithDecoderMaxMemory(n uint64) DecoderOption

WithDecoderMaxMemory sets the maximum window size for streaming operations. This can be used to control memory usage of potentially hostile content. Maximum is 1 << 63 bytes. Default is 128MiB.

type Engine added in v0.10.0 

type Engine = c1zstore.Engine

Engine identifies a storage engine implementation. The engine is chosen by callers via WithEngine(...) on write; on read, the engine is dictated by the file's magic byte and (for v3) the manifest's engine field. The type lives in pkg/dotc1z/c1zstore so engine packages can name it without importing dotc1z.

type EngineDriver added in v0.10.0 

type EngineDriver interface {
	Engine() Engine
	Format() C1ZFormat
	OpenStore(ctx context.Context, outputFilePath string, opts StoreOptions) (C1ZStore, error)
}

EngineDriver opens a .c1z file for a specific storage engine. The SQLite and Pebble drivers are both registered statically by this package; RegisterEngine exists for additional engines.

func EngineDriverFor added in v0.10.0 

func EngineDriverFor(engine Engine) (EngineDriver, bool)

EngineDriverFor returns the registered driver for engine.

type FileOps added in v0.8.29 

type FileOps = c1zstore.FileOps

FileOps is the file-level operations sub-store of C1ZStore. See c1zstore.FileOps for the full contract.

type GrantAnnotation added in v0.8.29 

type GrantAnnotation = c1zstore.GrantAnnotation

GrantAnnotation is a row yielded by GrantStore.ListWithAnnotations. See c1zstore.GrantAnnotation.

type GrantStore added in v0.8.29 

type GrantStore = c1zstore.GrantStore

GrantStore is the grant-specific slice of C1ZStore. See c1zstore.GrantStore for the full contract.

type PayloadEncoding added in v0.10.0 

type PayloadEncoding = c1zstore.PayloadEncoding

PayloadEncoding selects the v3 envelope payload framing. Only the Pebble engine consults this; SQLite engines ignore it. See c1zstore.PayloadEncoding.

type PendingExpansion added in v0.8.29 

type PendingExpansion = c1zstore.PendingExpansion

PendingExpansion is a lightweight row yielded by GrantStore.PendingExpansion. See c1zstore.PendingExpansion.

type RollbackOption added in v0.12.2 

type RollbackOption func(*rollbackConfig)

RollbackOption configures RollbackExpansion.

func WithPreserveSuspectGrants added in v0.12.2 

func WithPreserveSuspectGrants() RollbackOption

WithPreserveSuspectGrants makes rollback KEEP suspect connector-sourced grants (Sources present, no self-source, no GrantImmutable) instead of deleting them. Default OFF: a suspect grant is an expander artifact in the common case and is deleted, but because Grant.Sources is public connector data and replay runs an empty connector, a genuinely connector-set grant of that shape would be permanently dropped — this option trades a possibly-stale expander grant left in the output for never silently dropping real connector data.

func WithSyncTokenRewrite added in v0.12.6 

func WithSyncTokenRewrite(fn func(token string) (string, error)) RollbackOption

WithSyncTokenRewrite supplies how the sync's persisted token is rewritten once the grants are rolled back. The function receives the current token and returns the replacement; rollback writes the result verbatim. Without it, rollback clears the token to empty, which makes a resumed syncer seed a fresh run. A caller that wants to preserve the token's other state passes a rewrite that re-marks the sync for expansion instead. Ignored on a dry run.

type RollbackResult added in v0.12.2 

type RollbackResult struct {
	SyncID string
	// GrantsDeleted is the number of purely expander-derived grants
	// removed: their Sources carried only foreign entitlement keys, never
	// a self-source for their own entitlement.
	GrantsDeleted int
	// SourcesCleared is the number of surviving direct grants whose
	// Sources were reset to empty, restoring the pre-expansion shape.
	SourcesCleared int
	// SuspectConnectorSourced counts grants that had Sources but no
	// self-source AND lacked the GrantImmutable annotation that every
	// expander-created grant carries — a shape that looks connector-set
	// rather than expander-set. By default these are deleted (and counted
	// in GrantsDeleted); with WithPreserveSuspectGrants they are kept
	// instead (counted in SuspectPreserved, not GrantsDeleted). Either way
	// the count is surfaced so a real connector-sourced grant is visible.
	// Zero in the common case.
	SuspectConnectorSourced int
	// SuspectPreserved counts suspect grants kept rather than deleted
	// because WithPreserveSuspectGrants was set. Always 0 in the default
	// (delete) mode; a subset of SuspectConnectorSourced when set.
	SuspectPreserved int
	DryRun           bool
}

RollbackResult reports what a rollback changed, or — for a dry run — what it would change.

type SessionStore added in v0.5.0 

type SessionStore interface {
	sessions.SessionStore
}

type StoreOptions added in v0.10.0 

type StoreOptions struct {
	TmpDir             string
	Pragmas            []StorePragma
	DecoderOptions     []DecoderOption
	ReadOnly           bool
	EncoderConcurrency int
	SyncLimit          int
	SkipCleanup        bool
	V2GrantsWriter     bool
	Engine             Engine

	// PayloadEncoding selects the v3 envelope payload framing for
	// engines that produce a v3 envelope (currently Pebble). Zero
	// value means "engine default" (PayloadEncodingTarZstd for Pebble).
	PayloadEncoding PayloadEncoding
}

StoreOptions is the engine-neutral form of C1ZOption values passed to a registered engine driver.

type StorePragma added in v0.10.0 

type StorePragma struct {
	Name  string
	Value string
}

StorePragma is a SQLite pragma forwarded through StoreOptions. Non-SQLite engines may ignore it.

type SyncMeta added in v0.8.29 

type SyncMeta = c1zstore.SyncMeta

SyncMeta is the sync-run-metadata sub-store of C1ZStore. See c1zstore.SyncMeta for the full contract.

type SyncRun added in v0.8.29 

type SyncRun = c1zstore.SyncRun

SyncRun is the exported shape of a sync run. See c1zstore.SyncRun.

Source Files

View all Source files

Directories

Path Synopsis
Package c1zstore defines the engine-neutral storage contract for .c1z files: the Store interface and its sub-stores (GrantStore, SyncMeta, FileOps), the row shapes they exchange, and the SDK-wide sync retention policy.
 Package c1zstore defines the engine-neutral storage contract for .c1z files: the Store interface and its sub-stores (GrantStore, SyncMeta, FileOps), the row shapes they exchange, and the SDK-wide sync retention policy.
engine
equivalence
Package equivalence runs the same workload through two storage backends and asserts they produce equivalent results across every reader method.
 Package equivalence runs the same workload through two storage backends and asserts they produce equivalent results across every reader method.
pebble
Package pebble is the v3 storage engine for baton-sdk.
 Package pebble is the v3 storage engine for baton-sdk.
pebble/codec
Package codec implements the v3 storage engine's record codec layer.
 Package codec implements the v3 storage engine's record codec layer.
pebble/microtests
Package microtests holds the 5 risk-validation tests that were written to prove the material design choices in RFC 0004 (storage engine v4) actually hold under Pebble's real behavior.
 Package microtests holds the 5 risk-validation tests that were written to prove the material design choices in RFC 0004 (storage engine v4) actually hold under Pebble's real behavior.
format
v3
Package v3 implements the C1Z3 envelope format: 5-byte magic, a length-prefixed proto manifest, and a payload (typically a zstd-tar of a Pebble directory).
 Package v3 implements the C1Z3 envelope format: 5-byte magic, a length-prefixed proto manifest, and a payload (typically a zstd-tar of a Pebble directory).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.