sync

package
v0.25.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 21, 2026 License: Apache-2.0 Imports: 52 Imported by: 0

Documentation

Index

Constants

View Source 
const IngestInvariantGeneration = "1"

IngestInvariantGeneration identifies the verification contract persisted in sync metadata. Increment it whenever the meaning of a covered invariant changes such that an older successful pass is not equivalent to the current one.

View Source 
const StateTokenVersion = 1

If you make a breaking change to the state token, you must increment this version.

View Source 
const StateTokenVersionTypeScoped = 2

StateTokenVersionTypeScoped marks checkpoints whose action state carries type-scoped or spawned-cursor markers. Older SDKs cannot interpret those actions: their JSON parser silently drops the marker fields, and the resulting actions dead-end against store pagination, sealing the sync as complete while missing every pending cursor's data. Version 2 defeats that: an older SDK fails the version check, falls back to the V0 parser, gets an empty action state, and restarts collection from Init inside the same sync run — redone work instead of silent data loss.

Variables

View Source 
var ErrArtifactUnusable = dotc1z.ErrArtifactUnusable

ErrArtifactUnusable is the storage verdict — the local c1z may not reflect a clean commit of this run's progress — and the only signal that permits a runner to discard a partial sync artifact (RFC 0009). It reaches a runner through Close() alone; Sync() errors and connector errors never carry it. Defined in pkg/dotc1z (storage owns the verdict) and re-exported here so runners depend on pkg/sync alone. Test with errors.Is.

View Source 
var ErrIngestInvariantViolated = errors.New("ingest invariant violated")

ErrIngestInvariantViolated classifies DATA VERDICTS — the store's content violates an invariant — as distinct from the pass's own IO failures (listing errors, probe errors, cancellation). A verdict is deterministic on an immutable dataset: retrying it re-fails forever, so runners map it to their non-retryable failure class (pkg/tasks/c1api wraps it with ErrTaskNonRetryable), while IO failures stay retryable. Test with errors.Is.

View Source 
var ErrNoSyncIDFound = fmt.Errorf("no syncID found after starting or resuming sync")
View Source 
var ErrSyncNotComplete = fmt.Errorf("sync exited without finishing")
View Source 
var ErrTooManyWarnings = fmt.Errorf("too many warnings, exiting sync")

Functions

func BuildCompactedToken added in v0.18.3 

func BuildCompactedToken(baseToken string, in CompactionTokenInput) (string, error)

BuildCompactedToken rewrites a compacted output's sync token with a compaction provenance section and folds each partial's timing stats into the top-level maps. baseToken is the base input's token ("" for rebuild outputs, which start empty); its resume state, skip flags, and timing stats are preserved as the starting point. Chained compactions accumulate: the original StatsSyncID, the uncapped partial count, and already-folded top-level timings carry forward; new partials are added on top.

func GetExpandableAnnotation added in v0.2.92 

func GetExpandableAnnotation(annos annotations.Annotations) (*v2.GrantExpandable, error)

func GetExternalResourceMatchAllAnnotation added in v0.2.84 

func GetExternalResourceMatchAllAnnotation(annos annotations.Annotations) (*v2.ExternalResourceMatchAll, error)

func GetExternalResourceMatchAnnotation added in v0.2.84 

func GetExternalResourceMatchAnnotation(annos annotations.Annotations) (*v2.ExternalResourceMatch, error)

func GetExternalResourceMatchIDAnnotation added in v0.2.90 

func GetExternalResourceMatchIDAnnotation(annos annotations.Annotations) (*v2.ExternalResourceMatchID, error)

func GraphFromStore added in v0.25.0 

func GraphFromStore(ctx context.Context, store c1zstore.Store, syncID string) (*expand.EntitlementGraph, error)

GraphFromStore loads the entitlement graph persisted in the c1z sidecar for syncID. Returns nil (no error) when the store lacks the capability, no graph was preserved, or the stored graph belongs to a different sync.

func GraphFromToken added in v0.25.0 

func GraphFromToken(stateStr string) (*expand.EntitlementGraph, error)

GraphFromToken parses a legacy sync token and returns its entitlement graph for compatibility tests. It returns nil if the token carried no graph.

Token graphs have no grant-generation binding and must not drive incremental reuse. Production readers must use GraphFromStore, which verifies that the sidecar graph describes the store's exact sealed grant generation.

func IsSyncPreservable added in v0.6.18 

func IsSyncPreservable(err error) bool

IsSyncPreservable returns true if the error returned by Sync() means that the sync artifact is useful. This either means that there was no error, or that the error is recoverable (we can resume the sync and possibly succeed next time). Timeouts (context.DeadlineExceeded or codes.DeadlineExceeded, e.g. an AWS Lambda hard timeout) are preservable because the sync can resume from the checkpoint.

FROZEN (RFC 0009): superseded by ShouldDiscardSyncArtifact (preserve-by-default). This behavior must not change — older runners branch on it, so widening it in place ships a silent retention change. Prose freeze rather than `Deprecated:` because unmigrated callers are intentional during rollout and must not fail staticcheck on an SDK bump.

func NeedsExpansion added in v0.12.2 

func NeedsExpansion(stateStr string) (bool, error)

func NewExpanderStore added in v0.25.0 

func NewExpanderStore(store c1zstore.Store) expand.ExpanderStore

NewExpanderStore adapts a c1zstore.Store into an expand.ExpanderStore, bridging engine differences (Pebble exposes StoreExpandedGrants on its Grants() sub-store, SQLite at top level). Use this instead of type-asserting the store, which is unsafe for Pebble.

func NormalizeWorkerCount added in v0.9.1 

func NormalizeWorkerCount(count int) int

NormalizeWorkerCount maps raw worker-count inputs (CLI / config sentinels) to the syncer's internal worker count: -1 selects min(max(GOMAXPROCS, 1), 4); any other value uses max(count, 0).

func PrepareExpansionReplayToken added in v0.12.6 

func PrepareExpansionReplayToken(stateStr string) (string, error)

PrepareExpansionReplayToken rewrites a finished sync's state token so the sync can be re-run through grant expansion, preserving the token's other recorded state rather than discarding it. It marks the sync as needing expansion and, when the action stack is empty, pushes an InitOp so the resumed syncer drives its work from the top. A finished sync's token has an empty action stack, so without the InitOp a resume would find nothing to do and exit before expanding; clearing the whole token would also drop the skip flags and exclusion-group bookkeeping the token carries.

func RunIngestInvariants added in v0.20.2 

func RunIngestInvariants(ctx context.Context, store connectorstore.Reader, policy IngestInvariantsPolicy) error

RunIngestInvariants evaluates the post-collection ingestion invariants over store, per the verdict table (ingestInvariants) and policy. Callers run it at a quiesced point — after every ingestion path has finished writing and before the sync is sealed — so a violating store is never published as complete. Idempotent: a resumed run re-evaluates with the same verdicts.

The syncer is one caller (runIngestionInvariants); the pass is a store-level function so store-producing pipelines without a syncer (the compactor's expand pass) can enforce the same contract.

func RunIngestInvariantsWithVerification added in v0.25.0 

func RunIngestInvariantsWithVerification(
	ctx context.Context,
	store connectorstore.Reader,
	policy IngestInvariantsPolicy,
) (*c1zstore.IngestInvariantVerification, error)

RunIngestInvariantsWithVerification evaluates the invariant pass and returns the verification metadata a store-producing caller must persist after the sync is sealed. It does not write the marker itself: publishing proof before EndSync would allow an unfinished artifact to claim verification.

func ShouldDiscardSyncArtifact added in v0.24.4 

func ShouldDiscardSyncArtifact(err error) bool

ShouldDiscardSyncArtifact reports whether err carries the storage verdict. Everything else — cancellation, timeouts, all connector-side failures — preserves the artifact (RFC 0009 invariants I1/I2). Pass the join of the Sync() and Close() errors; today the verdict comes from Close() alone.

Types

type Action 

type Action struct {
	ID                   string   `json:"id,omitempty"`
	Op                   ActionOp `json:"operation,omitempty"`
	PageToken            string   `json:"page_token,omitempty"`
	ResourceTypeID       string   `json:"resource_type_id,omitempty"`
	ResourceID           string   `json:"resource_id,omitempty"`
	ParentResourceTypeID string   `json:"parent_resource_type_id,omitempty"`
	ParentResourceID     string   `json:"parent_resource_id,omitempty"`
	// Spawned marks a sibling cursor enqueued by EnqueuePageTokens.
	// Progress accounting counts only the origin action for per-resource
	// phases. The marker is checkpointed so resume preserves that rule.
	Spawned bool `json:"spawned,omitempty"`
	// TypeScoped distinguishes whole-type grant/entitlement cursors from
	// per-resource actions. Do not infer this from an empty ResourceID:
	// malformed connector resources with empty ids can exist in old stores
	// and must retain the pre-type-scoped per-resource behavior.
	TypeScoped bool `json:"type_scoped,omitempty"`
	// TypeScopedPlanned records that a root entitlement/grant action has
	// already scheduled whole-type collection. Legacy checkpoints omit it,
	// causing an upgraded syncer to plan type-scoped work once on resume.
	TypeScopedPlanned bool `json:"type_scoped_planned,omitempty"`
}

Action stores the current operation, page token, and optional fields for which resource is being worked with.

type ActionOp 

type ActionOp uint8

ActionOp represents a sync operation. 

const (
	UnknownOp ActionOp = iota
	InitOp
	SyncResourceTypesOp
	SyncResourcesOp
	SyncEntitlementsOp
	ListResourcesForEntitlementsOp
	SyncGrantsOp
	SyncExternalResourcesOp
	SyncAssetsOp
	SyncGrantExpansionOp
	SyncTargetedResourceOp
	SyncStaticEntitlementsOp
)

Do not change the order of these constants, and only append new ones at the end. Otherwise resuming a sync started by an older version of baton-sdk will cause very strange behavior.

func (ActionOp) MarshalJSON 

func (s ActionOp) MarshalJSON() ([]byte, error)

MarshalJSON marshals the ActionOp into a json string.

func (ActionOp) String 

func (s ActionOp) String() string

String() returns the string representation for an ActionOp. This is used for marshalling the op.

func (*ActionOp) UnmarshalJSON 

func (s *ActionOp) UnmarshalJSON(data []byte) error

UnmarshalJSON unmarshals the input byte slice and updates this action op.

type CompactionRecordCounts added in v0.18.3 

type CompactionRecordCounts struct {
	Output   int64 `json:"output"`
	Added    int64 `json:"added,omitempty"`
	Replaced int64 `json:"replaced,omitempty"`
	Carried  int64 `json:"carried,omitempty"`
}

CompactionRecordCounts describes one record type's provenance in a compacted output. Output is the record count in the compacted artifact. Added counts records admitted with no incumbent, Replaced counts records that overrode a strictly-older incumbent, and Carried counts incumbents that survived untouched. Added/Replaced/Carried are only populated by the fold strategy — rebuild merges lose per-source attribution in their run-file paths, so rebuild outputs carry Output only.

type CompactionTokenInput added in v0.18.3 

type CompactionTokenInput struct {
	Mode           string
	BaseSyncID     string
	PartialSyncIDs []string
	// PartialTokens are the partials' marshalled sync tokens. Their timing
	// stats are folded into the compacted token's top-level maps. Entries
	// may be empty or unparseable (e.g. converted sqlite inputs carry no
	// token); those contribute no stats.
	PartialTokens []string
	RecordCounts  map[string]CompactionRecordCounts
}

CompactionTokenInput carries one compaction run's provenance into BuildCompactedToken.

type CompactionTokenStats added in v0.18.3 

type CompactionTokenStats struct {
	// Mode is the compaction strategy that produced this artifact
	// (fold / overlay / kway).
	Mode string `json:"mode"`
	// StatsSyncID is the original collection sync at the root of a fold
	// chain. Carried unchanged across chained folds. Top-level timings are
	// the approximate sum of that sync plus merged partials — not solely
	// this sync's execution.
	StatsSyncID string `json:"stats_sync_id,omitempty"`
	// BaseSyncID is the immediate base input of this compaction run (the id
	// the artifact carried before the rename).
	BaseSyncID string `json:"base_sync_id,omitempty"`
	// PartialSyncIDs lists merged partial sync ids, capped at
	// maxCompactionPartialIDs. PartialCount is the uncapped total and
	// accumulates across chained folds.
	PartialSyncIDs []string `json:"partial_sync_ids,omitempty"`
	PartialCount   int64    `json:"partial_count"`
	// RecordCounts is keyed by record type (resource_types, resources,
	// entitlements, grants).
	RecordCounts map[string]*CompactionRecordCounts `json:"record_counts,omitempty"`
}

CompactionTokenStats is the compaction provenance section of a compacted sync token. Timing stats for merged partials are folded into the token's top-level step_durations_ms / connector_call_stats / session_store_stats (approximate combined view); this section carries identity / record-count provenance only.

func CompactionStatsFromToken added in v0.18.3 

func CompactionStatsFromToken(token string) (*CompactionTokenStats, error)

CompactionStatsFromToken returns the compaction provenance section of a marshalled sync token, or nil when the token is empty or carries none.

type ConnectorCallStat added in v0.18.3 

type ConnectorCallStat struct {
	Count   int64 `json:"count"`
	TotalMs int64 `json:"total_ms"`
	MaxMs   int64 `json:"max_ms"`
}

ConnectorCallStat contains cumulative latency statistics for one connector method.

type EntitlementGraphStore added in v0.25.0 

type EntitlementGraphStore interface {
	PutEntitlementGraphBlob(ctx context.Context, data []byte) error
	GetEntitlementGraphBlob(ctx context.Context) ([]byte, error)
	DeleteEntitlementGraphBlob(ctx context.Context) error
}

EntitlementGraphStore is the optional store capability backing graph persistence in the c1z (Pebble implements it; SQLite does not). The blob format is owned by pkg/sync/expand.

type IngestInvariantsPolicy added in v0.20.2 

type IngestInvariantsPolicy struct {
	// ActiveSyncID scopes every store read to the sync under judgment.
	ActiveSyncID string
	// SyncType gates the full-keyspace invariants: store-derived
	// referential verdicts are only evaluable over a COMPLETE keyspace,
	// so I3/I4/I7/I8/I9 run on full syncs only. I5 runs on every sync
	// type (a conflict requires both rows present — valid evidence on a
	// partial store).
	SyncType connectorstore.SyncType
	// FailFast promotes every invariant verdict to a hard,
	// plainly-attributed failure: tolerated warns fail, and I4 (skipped
	// entirely in default mode) runs. Tests and equivalence harnesses
	// set it.
	FailFast bool
	// CompactionMerge marks a pass over a PRE-SEALED artifact this
	// process did not collect — the compactor's keep-newer merge
	// (whose key union manufactures shapes no single input contained:
	// dangling references, stranded InsertResourceGrants rows,
	// exclusion-group conflicts) and rollback-expansion's replay
	// (whose inputs include such merged artifacts). Verdicts that
	// would blame the connector are attributed to the merge and hard
	// arms soften to aggregated warnings: the hard arms exist to stop
	// a NEW collection from sealing bad data, not to re-adjudicate an
	// artifact that already sealed. A normal connector sync must never
	// set this.
	CompactionMerge bool
	// contains filtered or unexported fields
}

IngestInvariantsPolicy parameterizes one run of the ingestion invariant pass. The exported fields are the caller contract (the syncer today; a store-producing pipeline like the compactor's expand pass can run the same pass without a syncer). The unexported fields carry syncer-private evidence (the in-memory I4 schedule set) and the test-only halt hook; external callers leave them zero and the affected invariants degrade per their gates.

type IngestQualityCheckpoint added in v0.20.8 

type IngestQualityCheckpoint struct {
	SourceCacheReplayBlocked      bool   `json:"source_cache_replay_blocked,omitempty"`
	EntitlementsDropped           uint64 `json:"entitlements_dropped,omitempty"`
	GrantsDropped                 uint64 `json:"grants_dropped,omitempty"`
	GrantResourcesDropped         uint64 `json:"grant_resources_dropped,omitempty"`
	ExpansionResourceTypesDropped uint64 `json:"expansion_resource_types_dropped,omitempty"`
	ExpansionsDropped             uint64 `json:"expansions_dropped,omitempty"`
	InvalidResourceTypesObserved  uint64 `json:"invalid_resource_types_observed,omitempty"`
	InvalidResourcesObserved      uint64 `json:"invalid_resources_observed,omitempty"`
	InvalidEntitlementsObserved   uint64 `json:"invalid_entitlements_observed,omitempty"`
	ReasonFlags                   uint64 `json:"reason_flags,omitempty"`
}

IngestQualityCheckpoint is the checkpointed connector-ingestion quality summary. A nil value means legacy/unknown provenance, not a clean sync.

type Progress added in v0.0.25 

type Progress struct {
	Action               string
	ResourceTypeID       string
	ResourceID           string
	ParentResourceTypeID string
	ParentResourceID     string
	Count                uint32
}

func NewProgress added in v0.0.25 

func NewProgress(a *Action, c uint32) *Progress

type SessionStoreStat added in v0.18.3 

type SessionStoreStat struct {
	Count    int64 `json:"count"`
	Errors   int64 `json:"errors,omitempty"`
	Timeouts int64 `json:"timeouts,omitempty"`
	TotalMs  int64 `json:"total_ms"`
	MaxMs    int64 `json:"max_ms"`
}

SessionStoreStat contains cumulative latency and outcome counters for one session-store operation. Timeouts is the deadline-exceeded subset of Errors; MaxMs pinned at a fixed value with Timeouts ≈ Count is the signature of a backend whose every request times out.

type State 

type State interface {
	PushAction(ctx context.Context, action Action)
	FinishAction(ctx context.Context, action *Action)
	NextPage(ctx context.Context, actionID string, pageToken string) error
	EntitlementGraph(ctx context.Context) *expand.EntitlementGraph
	PeekEntitlementGraph() *expand.EntitlementGraph
	ClearEntitlementGraph(ctx context.Context)
	ClearEntitlementGraphTransientState(ctx context.Context)
	Current() *Action
	GetAction(id string) *Action
	PeekMatchingActions(ctx context.Context, op ActionOp) []*Action
	Marshal() (string, error)
	Unmarshal(input string) error
	NeedsExpansion() bool
	SetNeedsExpansion()
	HasExternalResourcesGrants() bool
	SetHasExternalResourcesGrants()
	ShouldFetchRelatedResources() bool
	SetShouldFetchRelatedResources()
	ShouldSkipEntitlementsAndGrants() bool
	SetShouldSkipEntitlementsAndGrants()
	ShouldSkipGrants() bool
	SetShouldSkipGrants()
	GetCompletedActionsCount() uint64
	AddStepDuration(bucket string, duration time.Duration)
	StepDurations() map[string]int64
	RecordConnectorCall(method string, duration time.Duration)
	MergeConnectorCallStat(method string, add ConnectorCallStat)
	ConnectorCallStats() map[string]ConnectorCallStat
	RecordSessionOp(op string, duration time.Duration, opErr error, timedOut bool)
	MergeSessionStat(op string, add SessionStoreStat)
	SessionStoreStats() map[string]SessionStoreStat
	SetIngestQuality(quality *IngestQualityCheckpoint)
	IngestQuality() *IngestQualityCheckpoint
}

type SyncOpt 

type SyncOpt func(s *syncer)

func WithC1ZPath added in v0.1.0 

func WithC1ZPath(path string) SyncOpt

WithC1ZPath sets the path to the c1z file. Either this or WithConnectorStore must be provided to create a new syncer.

func WithCompactionMergedStore added in v0.20.2 

func WithCompactionMergedStore() SyncOpt

WithCompactionMergedStore marks the store under this sync as a pre-sealed artifact this process did not collect — the compactor's keep-newer merge, or rollback-expansion's replay over an existing c1z (whose inputs include such merges). The ingestion invariants (ingest_invariants.go) then attribute merge-manufactured shapes — dangling references, stranded InsertResourceGrants rows, exclusion-group conflicts unioned from different input generations — to the merge instead of the connector, and soften the corresponding hard arms to aggregated warnings (fail-fast still promotes). The hard arms exist to stop a NEW collection from sealing bad data; passes over already-sealed artifacts observe and attribute instead. A normal connector sync must never set this.

func WithConnectorStore added in v0.1.0 

func WithConnectorStore(store c1zstore.Store) SyncOpt

WithConnectorStore sets the connector store to use. This is the preferred option. Either this or WithC1ZPath must be provided to create a new syncer.

func WithDontExpandGrants added in v0.3.48 

func WithDontExpandGrants() SyncOpt

WithDontExpandGrants sets whether to skip expanding grants. This is used for speeding up service mode connectors and reducing their c1z upload size. C1 will process the uploaded c1z and expand grants itself.

func WithEntitlementGraphInCheckpoints added in v0.23.0 

func WithEntitlementGraphInCheckpoints(enabled bool) SyncOpt

WithEntitlementGraphInCheckpoints serializes the entitlement graph into every checkpoint token. Off by default: the graph is a projection of data already in the store, and encoding it costs O(graph) memory several times over per checkpoint, which OOM-kills workers on large tenants.

Enable it to keep expansion progress across restarts. That matters only for a tenant whose expansion cannot finish within one worker or activity lifetime — without it, such a sync re-runs the load and expansion phases on every resume and can fail to converge. Note the two failure modes trade off directly: the tenants large enough to need cross-restart progress are the ones whose graph is expensive enough to encode that checkpointing may OOM.

func WithExternalResourceC1ZPath added in v0.2.84 

func WithExternalResourceC1ZPath(path string) SyncOpt

func WithExternalResourceEntitlementIdFilter added in v0.2.84 

func WithExternalResourceEntitlementIdFilter(entitlementId string) SyncOpt

func WithExternalResourceTraits added in v0.20.7 

func WithExternalResourceTraits(traits ...v2.ResourceType_Trait) SyncOpt

WithExternalResourceTraits sets the resource type traits the External Identity Matcher should sync from the external resource source and consider when matching grants (e.g. ExternalResourceMatch / ExternalResourceMatchAll annotations). When not called, the matcher falls back to TRAIT_USER/TRAIT_GROUP — the pre-CE-975 default — so existing connectors keep working unchanged. Passing any traits replaces that default entirely: an Azure connector matching service-principal role assignments against TRAIT_APP resources synced by baton-microsoft-entra while also keeping default user/group matching would pass TRAIT_USER, TRAIT_GROUP, TRAIT_APP.

func WithFailFastInvariants added in v0.20.2 

func WithFailFastInvariants() SyncOpt

WithFailFastInvariants promotes every ingestion-invariant verdict (see ingest_invariants.go) to a hard, plainly-attributed sync failure: tolerated warns fail, and I4 (skipped in default mode) runs. Tests and equivalence harnesses enable it; production default follows the per-invariant policy in the verdict table (ingestInvariants) — aggregated warnings with attribution for dangling references, hard failure for I5 and I3's InsertResourceGrants arm.

func WithMetricsHandler added in v0.12.5 

func WithMetricsHandler(h metrics.Handler) SyncOpt

WithMetricsHandler attaches a metrics.Handler that the syncer forwards to progresslog.NewProgressCounts so the grant-expansion OTel instruments (baton.sync.expand.actions_remaining / actions_burned / decompressed_bytes / decompressed_bytes_delta) actually reach the configured exporter instead of the default no-op handler.

Callers should pre-tag the handler with the dimensions they want to slice by (e.g. tenant_id, connector_id) via Handler.WithTags before passing it in — baton-sdk has no view of those identifiers.

func WithOnlyExpandGrants added in v0.3.8 

func WithOnlyExpandGrants() SyncOpt

WithOnlyExpandGrants sets whether to skip syncing resources and only expand grants.

func WithOptionalPreviousSyncC1ZPath added in v0.15.0 

func WithOptionalPreviousSyncC1ZPath(path string) SyncOpt

WithOptionalPreviousSyncC1ZPath is WithPreviousSyncC1ZPath with best-effort semantics: if the file is missing, corrupt, or written by an incompatible SDK, NewSyncer logs and proceeds WITHOUT replay instead of failing. Intended for cache-style replay sources the caller maintains automatically (the service-mode previous-sync spare) — a bad cache file must never fail a sync. Callers that name a specific file deliberately should use WithPreviousSyncC1ZPath, which surfaces open failures.

func WithPreserveEntitlementGraph added in v0.25.0 

func WithPreserveEntitlementGraph() SyncOpt

WithPreserveEntitlementGraph preserves the entitlement graph for later incremental expansion. Pebble stores it in the c1z sidecar; stores without that capability retain it in the final sync token as a legacy fallback.

func WithPreviousSyncC1ZPath added in v0.15.0 

func WithPreviousSyncC1ZPath(path string) SyncOpt

WithPreviousSyncC1ZPath points ETag-replay at a separate c1z holding the previous sync, instead of reading a previous sync from inside the live store.

This is required for the single-sync v3 (Pebble) engine: a Pebble c1z holds exactly one sync by contract, so there is no in-file "previous sync" to replay from (StartNewSync replaces the prior sync). Supplying the prior run's c1z here lets the syncer recover unchanged resources' ETags and carry their grants forward across runs. When unset, replay falls back to reading a previous sync from the live store (the SQLite multi-sync behavior), so existing callers are unaffected.

The file is opened read-only and engine-agnostically (the magic byte selects SQLite or Pebble), so the previous-sync c1z may use either engine.

func WithProgressHandler added in v0.0.25 

func WithProgressHandler(f func(s *Progress)) SyncOpt

WithProgressHandler sets a `progressHandler` for `NewSyncer` Options. The progress handler is called for sync action, such as listing resources, entitlements, grants, etc. If running in parallel mode, this function must be thread-safe.

func WithRunDuration 

func WithRunDuration(d time.Duration) SyncOpt

WithRunDuration sets a `time.Duration` for `NewSyncer` Options. `d` represents a duration. The elapsed time between two instants as an int64 nanosecond count.

func WithSessionStore added in v0.5.0 

func WithSessionStore(sessionStore sessions.SetSessionStore) SyncOpt

func WithSkipEntitlementsAndGrants added in v0.3.40 

func WithSkipEntitlementsAndGrants(skip bool) SyncOpt

WithSkipEntitlementsAndGrants sets whether to skip syncing entitlements and grants for resources. If true, only resources will be synced.

func WithSkipFullSync added in v0.2.15 

func WithSkipFullSync() SyncOpt

WithSkipFullSync skips syncing entirely.

func WithSkipGrants added in v0.5.1 

func WithSkipGrants(skip bool) SyncOpt

WithSkipGrants sets whether to skip syncing grants for resources. Entitlements will still be synced.

func WithStorageEngine added in v0.13.5 

func WithStorageEngine(engine c1zstore.Engine) SyncOpt

WithStorageEngine selects the dotc1z storage engine when opening the c1z file via WithC1ZPath. Empty uses the baton-sdk default.

func WithSyncID added in v0.3.8 

func WithSyncID(syncID string) SyncOpt

func WithSyncIdentity added in v0.12.5 

func WithSyncIdentity(id uotel.SyncIdentity) SyncOpt

WithSyncIdentity stamps connector identity onto sync and dotc1z spans (and the c1z size metric) so a single connector's work is filterable in APM. Attribute keys match the pprof.Do labels set by the platform sync activity, so spans and CPU profiles line up. Sync injects this into the run context via uotel.WithSyncIdentity, which is how it reaches dotc1z spans too.

func WithSyncResourceTypes added in v0.5.0 

func WithSyncResourceTypes(resourceTypeIDs []string) SyncOpt

WithSyncResourceTypes sets the resource types to sync. If empty (the default), all resource types will be synced.

func WithTargetedSyncResources added in v0.6.6 

func WithTargetedSyncResources(resources []*v2.Resource) SyncOpt

func WithTmpDir added in v0.1.8 

func WithTmpDir(path string) SyncOpt

func WithTransitionHandler 

func WithTransitionHandler(f func(s Action)) SyncOpt

WithTransitionHandler sets a `transitionHandler` for `NewSyncer` Options.

func WithWorkerCount added in v0.8.0 

func WithWorkerCount(count int) SyncOpt

WithWorkerCount sets the number of workers to use. If <=1, 1 worker is used (default). If > 1, parallel sync is used. If -1, the number of workers is set to the number of CPU cores or 4, whichever is lower. If < -1, 1 worker is used. (Nothing should do this, but there's no way to return an error in this option.)

type Syncer 

type Syncer interface {
	Sync(context.Context) error
	Close(context.Context) error
}

func NewSyncer 

func NewSyncer(ctx context.Context, c types.ConnectorClient, opts ...SyncOpt) (Syncer, error)

NewSyncer returns a new syncer object.

Source Files

View all Source files

Directories

Path Synopsis
scc
Package scc provides an iterative FW–BW SCC condensation for directed graphs, adapted for Baton’s entitlement graph.
 Package scc provides an iterative FW–BW SCC condensation for directed graphs, adapted for Baton’s entitlement graph.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.