Documentation
¶
Overview ¶
Package sync implements Fossil's multi-round sync and clone protocol.
The client side drives convergence: Sync opens a session against a Transport, exchanges xfer cards in rounds until both sides agree on the set of blobs, then returns a SyncResult. Clone creates a new repo file and populates it from the remote.
The server side is stateless per round: HandleSync processes one request and produces one response. Use XferHandler to embed sync in an http.ServeMux alongside operational routes like /healthz.
Observability is injected via the Observer interface — pass nil for zero-cost no-ops, or supply an implementation (e.g. OTelObserver from leaf/telemetry) for traces, metrics, and structured logs.
Index ¶
- Constants
- func HandleSync(ctx context.Context, r *repo.Repo, req *xfer.Message) (*xfer.Message, error)
- func HandleSyncWithOpts(ctx context.Context, r *repo.Repo, req *xfer.Message, opts HandleOpts) (*xfer.Message, error)
- func ServeHTTP(ctx context.Context, addr string, r *repo.Repo, h HandleFunc) error
- func XferHandler(r *repo.Repo, h HandleFunc) http.HandlerFunc
- type BuggifyChecker
- type CkinLockFail
- type CkinLockReq
- type CloneOpts
- type CloneResult
- type HTTPTransport
- type HandleEnd
- type HandleFunc
- type HandleOpts
- type HandleStart
- type MockTransport
- type Observer
- type PhantomStallError
- type RoundStats
- type SessionEnd
- type SessionStart
- type SyncOpts
- type SyncResult
- type SyncedTable
- type TableSyncEnd
- type TableSyncStart
- type Transport
Constants ¶
const ( // DefaultMaxSend is the default byte budget per round for file payloads. DefaultMaxSend = 250000 // MaxRounds caps the number of sync rounds before giving up. MaxRounds = 100 // MaxGimmeBase is the minimum gimme cap per round. MaxGimmeBase = 200 )
const DefaultCkinLockTimeout = 60 * time.Second
DefaultCkinLockTimeout is the default duration after which a ci-lock expires.
const DefaultCloneBatchBytes = 16_000_000
DefaultCloneBatchBytes bounds the wire bytes a single clone round emits.
A clone batch is bounded by output size, never by artifact count. A count cannot adapt to artifact size: 200 tiny control artifacts and 200 large blobs produce wildly different messages, and neither corresponds to what the transport can carry. §8.2 leaves the bound entirely to the server ("clients MUST NOT infer count or size from range"), so the only real constraint is that the response stays a size a server can hold and that a repository drains in rounds proportional to its bytes, not its artifacts.
emitCloneBatch now retransmits stored content per §7.2 (issue #98) rather than the fully expanded artifacts it used to send, so bytesSent tracks the same quantity canonical's 5,000,000 max-download bounds: repository bytes, not the 80x-220x-larger expanded form. This constant was chosen before that fix, against the expanded-bytes regime documented in issue #88 where a smaller value pushed fossil- and sqlite-sized corpora past MaxRounds -- that constraint is gone now that wire bytes and repository bytes are the same quantity. Retuning this value toward canonical's constant is issue #109's territory (server pagination/budget accounting), not this fix's; it is left at its prior value here to avoid colliding with that work in flight. 16,000,000 stays a safe, if generous, upper bound either way: it only widens how much stored content one round may carry.
This bounds an ordinary round at the budget plus the artifact that crosses it, and emitCloneBatch caps that sum at xfer.MaxDecompressedBytes so the client can always decode a round (issue #109). The one exception is a single artifact larger than that bound: it is sent alone, in a round of its own, to make progress -- so a repository holding one 2 GB artifact still emits a 2 GB round, but such an artifact is unclonable regardless. Measured worst case is ~25 MB across the three repositories above, since the largest single artifacts are 9.4 MB (libfossil) and 17.0 MB (sqlite). Against the count bound's unbounded 44.9 MB that is a large reduction, and concurrent clones multiply it.
Variables ¶
This section is empty.
Functions ¶
func HandleSync ¶
HandleSync processes an incoming xfer request and produces a response. Stateless per-round — the client drives convergence.
func HandleSyncWithOpts ¶
func HandleSyncWithOpts(ctx context.Context, r *repo.Repo, req *xfer.Message, opts HandleOpts) (*xfer.Message, error)
HandleSyncWithOpts processes an incoming xfer request with optional fault injection. Used by DST harness; production callers use HandleSync.
func ServeHTTP ¶
ServeHTTP starts an HTTP server that accepts Fossil xfer requests. Blocks until ctx is cancelled. Stock fossil clone/sync can connect.
func XferHandler ¶
func XferHandler(r *repo.Repo, h HandleFunc) http.HandlerFunc
XferHandler returns an http.HandlerFunc that decodes Fossil xfer requests, dispatches to the HandleFunc, and encodes the response. Use this to compose a custom mux when you need additional routes (e.g. /healthz) alongside xfer.
Types ¶
type BuggifyChecker ¶
BuggifyChecker is an optional fault injection interface. Pass nil in production — implementations should be nil-safe.
type CkinLockFail ¶
CkinLockFail reports that another client holds the check-in lock.
type CkinLockReq ¶
CkinLockReq requests a server-side check-in lock.
type CloneOpts ¶
type CloneOpts struct {
User string // Credentials for clone auth (also default admin user)
Password string
Version int // Protocol version (default 3)
Env *simio.Env // nil defaults to RealEnv
Observer Observer // nil defaults to no-op
Buggify BuggifyChecker // fault injection for DST (nil = no faults)
}
CloneOpts configures a clone operation.
type CloneResult ¶
type CloneResult struct {
Rounds int
BlobsRecvd int
ArtifactsLinked int // Manifests crosslinked into event table
ProjectCode string
ServerCode string
Messages []string // Informational messages from server
}
CloneResult reports what happened during a clone.
func Clone ¶
func Clone(ctx context.Context, path string, t Transport, opts CloneOpts) (r *repo.Repo, result *CloneResult, err error)
Clone performs a full repository clone from a remote Fossil server. It creates a new repository at path, runs the clone protocol until convergence, and returns the opened repo and a result summary. On error, the partially-created repo file is removed.
type HTTPTransport ¶
type HTTPTransport struct {
URL string // repo root, e.g. "http://localhost:8080"
}
HTTPTransport speaks Fossil's HTTP /xfer protocol. Fossil routes to /xfer based on Content-Type: application/x-fossil, NOT the URL path. URL should be the repo root (e.g. "http://localhost:8080").
type HandleFunc ¶
HandleFunc is the server-side sync handler signature. Transport listeners call this with decoded requests and write back the response.
type HandleOpts ¶
type HandleOpts struct {
Buggify BuggifyChecker // nil in production.
Observer Observer // nil defaults to no-op.
ContentCache *content.Cache // nil = no caching.
}
HandleOpts configures optional behavior for HandleSync.
type HandleStart ¶
type HandleStart struct {
Operation string // "sync" or "clone"
ProjectCode string
RemoteAddr string
}
HandleStart describes the beginning of a server-side sync request.
type MockTransport ¶
MockTransport replays canned responses for testing.
type Observer ¶
type Observer interface {
// Client-side session lifecycle.
Started(ctx context.Context, info SessionStart) context.Context
RoundStarted(ctx context.Context, round int) context.Context
RoundCompleted(ctx context.Context, round int, stats RoundStats)
Completed(ctx context.Context, info SessionEnd, err error)
// Per-error recording — called on individual protocol errors.
Error(ctx context.Context, err error)
// Server-side request lifecycle.
HandleStarted(ctx context.Context, info HandleStart) context.Context
HandleCompleted(ctx context.Context, info HandleEnd)
// Table sync lifecycle.
TableSyncStarted(ctx context.Context, info TableSyncStart)
TableSyncCompleted(ctx context.Context, info TableSyncEnd)
}
Observer receives lifecycle callbacks during sync and clone operations. A single Observer instance may be shared across multiple concurrent sessions. Pass nil for no-op default.
Performance: nopObserver implements all methods as empty functions. The only cost on the hot path is one indirect call per invocation (~2ns).
type PhantomStallError ¶
type PhantomStallError struct {
Count int // phantom UUIDs still outstanding when the clone stopped
Sample []string // up to phantomStallSampleSize lexicographically-smallest of those UUIDs, for diagnosis
}
PhantomStallError reports a clone that stopped making progress while phantom blobs — content referenced but never received — were still outstanding. Terminating a stalled clone is correct; reporting it as success is not, so this is always returned instead of nil in that case. Count is programmatically reachable so a caller can act on it without parsing the message.
func (*PhantomStallError) Error ¶
func (e *PhantomStallError) Error() string
type RoundStats ¶
type RoundStats struct {
FilesSent int
FilesReceived int
GimmesSent int
IgotsSent int
BytesSent int64
BytesReceived int64
}
RoundStats reports per-round activity for observer callbacks. Value type — no allocations on the nop path.
type SessionEnd ¶
type SessionEnd struct {
Operation string
Rounds int
FilesSent, FilesRecvd int
UVFilesSent, UVFilesRecvd int
UVGimmesSent int
BytesSent, BytesRecvd int64
ProjectCode string
Errors []string
}
SessionEnd describes the result of a sync or clone operation.
type SessionStart ¶
type SessionStart struct {
Operation string // "sync" or "clone"
Push, Pull bool
UV bool
ProjectCode string
PeerID string // identifies the leaf agent instance
}
SessionStart describes the beginning of a sync or clone operation.
type SyncOpts ¶
type SyncOpts struct {
Push, Pull bool // enable push/pull directions (at least one must be true)
ProjectCode string // repo's project-code — sent to identify the repository
ServerCode string // server-code from a previous session (cookie-like, speeds up sync)
User string // login user — empty means unauthenticated "nobody" sync
Password string // login password
PeerID string // identifies this leaf agent instance (for observability)
MaxSend int // byte budget per round for file payloads (0 defaults to DefaultMaxSend)
UV bool // enable unversioned file sync (wiki, forum, attachments)
XTableSync bool // enable extension table sync (peer_registry, etc.) — only between EdgeSync peers
Private bool // enable private artifact sync
Env *simio.Env // nil defaults to RealEnv
Buggify BuggifyChecker // nil in production — used by DST for fault injection
Observer Observer // nil defaults to no-op
CkinLock *CkinLockReq // nil = no lock requested
ContentCache *content.Cache // nil = no caching (every Expand walks the full delta chain)
}
SyncOpts configures a sync session.
type SyncResult ¶
type SyncResult struct {
Rounds, FilesSent, FilesRecvd int
UVFilesSent, UVFilesRecvd int
UVGimmesSent int
BytesSent, BytesRecvd int64
ArtifactsLinked int
Errors []string
CkinLockFail *CkinLockFail // nil = no conflict
}
SyncResult reports what happened during a sync.
type SyncedTable ¶
SyncedTable caches a table definition along with its metadata.
type TableSyncEnd ¶
TableSyncEnd describes the result of a table sync operation.
type TableSyncStart ¶
TableSyncStart describes the beginning of a table sync operation.
type Transport ¶
type Transport interface {
Exchange(ctx context.Context, request *xfer.Message) (*xfer.Message, error)
}
Transport sends an xfer request and returns the response. Implementations handle encoding (the request Message is already decoded), network I/O, and decoding. The xfer.Message uses zlib-compressed payloads internally — see xfer.Encode and xfer.Decode for wire format details.
Built-in implementations: HTTPTransport (Fossil HTTP /xfer protocol), MockTransport (canned responses for testing). The leaf agent adds NATSTransport for NATS-based peer-to-peer sync.