sync

package
v0.4.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 26, 2026 License: MIT Imports: 29 Imported by: 0

Documentation

Overview

Package sync implements Fossil's multi-round sync and clone protocol.

The client side drives convergence: Sync opens a session against a Transport, exchanges xfer cards in rounds until both sides agree on the set of blobs, then returns a SyncResult. Clone creates a new repo file and populates it from the remote.

The server side is stateless per round: HandleSync processes one request and produces one response. Use XferHandler to embed sync in an http.ServeMux alongside operational routes like /healthz.

Observability is injected via the Observer interface — pass nil for zero-cost no-ops, or supply an implementation (e.g. OTelObserver from leaf/telemetry) for traces, metrics, and structured logs.

Index

Constants

View Source
const (
	// DefaultMaxSend is the default byte budget per round for file payloads.
	DefaultMaxSend = 250000
	// MaxRounds caps the number of sync rounds before giving up.
	MaxRounds = 100
	// MaxGimmeBase is the minimum gimme cap per round.
	MaxGimmeBase = 200
)
View Source
const DefaultCkinLockTimeout = 60 * time.Second

DefaultCkinLockTimeout is the default duration after which a ci-lock expires.

View Source
const DefaultCloneBatchSize = 200

DefaultCloneBatchSize is the number of blobs sent per clone round.

Variables

View Source
var ErrDeltaSourceMissing = errors.New("delta source not found")

ErrDeltaSourceMissing is returned by storeReceivedFile when the delta source blob is not present in the repository. Clone uses this to create phantoms.

Functions

func HandleSync

func HandleSync(ctx context.Context, r *repo.Repo, req *xfer.Message) (*xfer.Message, error)

HandleSync processes an incoming xfer request and produces a response. Stateless per-round — the client drives convergence.

func HandleSyncWithOpts

func HandleSyncWithOpts(ctx context.Context, r *repo.Repo, req *xfer.Message, opts HandleOpts) (*xfer.Message, error)

HandleSyncWithOpts processes an incoming xfer request with optional fault injection. Used by DST harness; production callers use HandleSync.

func ServeHTTP

func ServeHTTP(ctx context.Context, addr string, r *repo.Repo, h HandleFunc) error

ServeHTTP starts an HTTP server that accepts Fossil xfer requests. Blocks until ctx is cancelled. Stock fossil clone/sync can connect.

func XferHandler

func XferHandler(r *repo.Repo, h HandleFunc) http.HandlerFunc

XferHandler returns an http.HandlerFunc that decodes Fossil xfer requests, dispatches to the HandleFunc, and encodes the response. Use this to compose a custom mux when you need additional routes (e.g. /healthz) alongside xfer.

Types

type BuggifyChecker

type BuggifyChecker interface {
	Check(site string, probability float64) bool
}

BuggifyChecker is an optional fault injection interface. Pass nil in production — implementations should be nil-safe.

type CkinLockFail

type CkinLockFail struct {
	HeldBy string
	Since  time.Time
}

CkinLockFail reports that another client holds the check-in lock.

type CkinLockReq

type CkinLockReq struct {
	ParentUUID string
	ClientID   string
}

CkinLockReq requests a server-side check-in lock.

type CloneOpts

type CloneOpts struct {
	User     string // Credentials for clone auth (also default admin user)
	Password string
	Version  int            // Protocol version (default 3)
	Env      *simio.Env     // nil defaults to RealEnv
	Observer Observer       // nil defaults to no-op
	Buggify  BuggifyChecker // fault injection for DST (nil = no faults)
}

CloneOpts configures a clone operation.

type CloneResult

type CloneResult struct {
	Rounds          int
	BlobsRecvd      int
	ArtifactsLinked int // Manifests crosslinked into event table
	ProjectCode     string
	ServerCode      string
	Messages        []string // Informational messages from server
}

CloneResult reports what happened during a clone.

func Clone

func Clone(ctx context.Context, path string, t Transport, opts CloneOpts) (r *repo.Repo, result *CloneResult, err error)

Clone performs a full repository clone from a remote Fossil server. It creates a new repository at path, runs the clone protocol until convergence, and returns the opened repo and a result summary. On error, the partially-created repo file is removed.

type HTTPTransport

type HTTPTransport struct {
	URL string // repo root, e.g. "http://localhost:8080"
}

HTTPTransport speaks Fossil's HTTP /xfer protocol. Fossil routes to /xfer based on Content-Type: application/x-fossil, NOT the URL path. URL should be the repo root (e.g. "http://localhost:8080").

func (*HTTPTransport) Exchange

func (t *HTTPTransport) Exchange(ctx context.Context, req *xfer.Message) (*xfer.Message, error)

type HandleEnd

type HandleEnd struct {
	CardsProcessed int
	FilesSent      int
	FilesReceived  int
	Err            error
}

HandleEnd describes the result of a server-side sync request.

type HandleFunc

type HandleFunc func(ctx context.Context, r *repo.Repo, req *xfer.Message) (*xfer.Message, error)

HandleFunc is the server-side sync handler signature. Transport listeners call this with decoded requests and write back the response.

type HandleOpts

type HandleOpts struct {
	Buggify      BuggifyChecker // nil in production.
	Observer     Observer       // nil defaults to no-op.
	ContentCache *content.Cache // nil = no caching.
}

HandleOpts configures optional behavior for HandleSync.

type HandleStart

type HandleStart struct {
	Operation   string // "sync" or "clone"
	ProjectCode string
	RemoteAddr  string
}

HandleStart describes the beginning of a server-side sync request.

type MockTransport

type MockTransport struct {
	Handler func(req *xfer.Message) *xfer.Message
}

MockTransport replays canned responses for testing.

func (*MockTransport) Exchange

func (t *MockTransport) Exchange(ctx context.Context, req *xfer.Message) (*xfer.Message, error)

type Observer

type Observer interface {
	// Client-side session lifecycle.
	Started(ctx context.Context, info SessionStart) context.Context
	RoundStarted(ctx context.Context, round int) context.Context
	RoundCompleted(ctx context.Context, round int, stats RoundStats)
	Completed(ctx context.Context, info SessionEnd, err error)

	// Per-error recording — called on individual protocol errors.
	Error(ctx context.Context, err error)

	// Server-side request lifecycle.
	HandleStarted(ctx context.Context, info HandleStart) context.Context
	HandleCompleted(ctx context.Context, info HandleEnd)

	// Table sync lifecycle.
	TableSyncStarted(ctx context.Context, info TableSyncStart)
	TableSyncCompleted(ctx context.Context, info TableSyncEnd)
}

Observer receives lifecycle callbacks during sync and clone operations. A single Observer instance may be shared across multiple concurrent sessions. Pass nil for no-op default.

Performance: nopObserver implements all methods as empty functions. The only cost on the hot path is one indirect call per invocation (~2ns).

type RoundStats

type RoundStats struct {
	FilesSent     int
	FilesReceived int
	GimmesSent    int
	IgotsSent     int
	BytesSent     int64
	BytesReceived int64
}

RoundStats reports per-round activity for observer callbacks. Value type — no allocations on the nop path.

type SessionEnd

type SessionEnd struct {
	Operation                 string
	Rounds                    int
	FilesSent, FilesRecvd     int
	UVFilesSent, UVFilesRecvd int
	UVGimmesSent              int
	BytesSent, BytesRecvd     int64
	ProjectCode               string
	Errors                    []string
}

SessionEnd describes the result of a sync or clone operation.

type SessionStart

type SessionStart struct {
	Operation   string // "sync" or "clone"
	Push, Pull  bool
	UV          bool
	ProjectCode string
	PeerID      string // identifies the leaf agent instance
}

SessionStart describes the beginning of a sync or clone operation.

type SyncOpts

type SyncOpts struct {
	Push, Pull   bool           // enable push/pull directions (at least one must be true)
	ProjectCode  string         // repo's project-code — sent to identify the repository
	ServerCode   string         // server-code from a previous session (cookie-like, speeds up sync)
	User         string         // login user — empty means unauthenticated "nobody" sync
	Password     string         // login password
	PeerID       string         // identifies this leaf agent instance (for observability)
	MaxSend      int            // byte budget per round for file payloads (0 defaults to DefaultMaxSend)
	UV           bool           // enable unversioned file sync (wiki, forum, attachments)
	XTableSync   bool           // enable extension table sync (peer_registry, etc.) — only between EdgeSync peers
	Private      bool           // enable private artifact sync
	Env          *simio.Env     // nil defaults to RealEnv
	Buggify      BuggifyChecker // nil in production — used by DST for fault injection
	Observer     Observer       // nil defaults to no-op
	CkinLock     *CkinLockReq   // nil = no lock requested
	ContentCache *content.Cache // nil = no caching (every Expand walks the full delta chain)
}

SyncOpts configures a sync session.

type SyncResult

type SyncResult struct {
	Rounds, FilesSent, FilesRecvd int
	UVFilesSent, UVFilesRecvd     int
	UVGimmesSent                  int
	BytesSent, BytesRecvd         int64
	ArtifactsLinked               int
	Errors                        []string
	CkinLockFail                  *CkinLockFail // nil = no conflict
}

SyncResult reports what happened during a sync.

func Sync

func Sync(ctx context.Context, r *repo.Repo, t Transport, opts SyncOpts) (result *SyncResult, err error)

Sync runs the client sync loop against the given transport. It returns once the protocol has converged or a fatal error occurs.

type SyncedTable

type SyncedTable struct {
	Info repo.TableInfo
	Def  repo.TableDef
}

SyncedTable caches a table definition along with its metadata.

type TableSyncEnd

type TableSyncEnd struct {
	Table    string
	Sent     int
	Received int
}

TableSyncEnd describes the result of a table sync operation.

type TableSyncStart

type TableSyncStart struct {
	Table     string
	LocalRows int
}

TableSyncStart describes the beginning of a table sync operation.

type Transport

type Transport interface {
	Exchange(ctx context.Context, request *xfer.Message) (*xfer.Message, error)
}

Transport sends an xfer request and returns the response. Implementations handle encoding (the request Message is already decoded), network I/O, and decoding. The xfer.Message uses zlib-compressed payloads internally — see xfer.Encode and xfer.Decode for wire format details.

Built-in implementations: HTTPTransport (Fossil HTTP /xfer protocol), MockTransport (canned responses for testing). The leaf agent adds NATSTransport for NATS-based peer-to-peer sync.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL