crdt

crdt

中文

A bounded Go CRDT library for convergent state, delta replication, recovery, and explicit protocol boundaries.

crdt provides deterministic CRDT primitives and framed binary codecs for replicas that must converge despite duplicate, reordered, or delayed delivery. It is a library, not a complete collaboration service: the host owns identity, authorization, storage, transport, membership, retention, and product invariants.

Start in three minutes

Requires Go 1.21 or later.

go get github.com/DarkInno/crdt@latest

package main

import (
	"fmt"
	"log"

	"github.com/DarkInno/crdt/counter"
)

func main() {
	left, err := counter.NewGCounter("left")
	if err != nil {
		log.Fatal(err)
	}
	right, err := counter.NewGCounter("right")
	if err != nil {
		log.Fatal(err)
	}
	if _, err := left.Increment(3); err != nil {
		log.Fatal(err)
	}
	if err := right.Merge(left); err != nil {
		log.Fatal(err)
	}
	value, err := right.Value()
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(value) // 3
}

For a checkout:

git clone https://github.com/DarkInno/crdt.git
cd crdt
make test

go test ./... at the repository root intentionally tests the dependency-free core only. make test traverses the core and every opt-in module.

What is included

• G-Counter, PN-Counter, G-Set, add-wins OR-Set, and causal MV-Register.
• Bounded canonical state/delta frames, deterministic snapshots, recovery plans, and persisted HLC state for reusable replica identities.
• A local, bounded multi-type undo/redo command stack plus a content-addressed snapshot version DAG for browser history and branches; both remain outside replication frames and are host-persisted metadata.
• RGA collaborative text with stable run-v2 frames by default, plus explicitly negotiated packed-v3 frames for dense HLC runs; stable bounded rich-text, observed-remove tree, and nested document-tree protocols; plus list and XML-fragment layers.
• Delta batching, Merkle anti-entropy, exact-acknowledgement tombstone-GC coordination, explicit local-only cleanup for disposable state, and manifest-bound replica/inbox recovery helpers.
• Opt-in modules for a bounded live WebSocket provider, a separate bbolt-backed durable relay with cursor replay, legacy optional state-vector catch-up, and opt-in HLC/Merkle no-state-vector anti-entropy, Redis/PostgreSQL/MySQL/SQL Server/SQLite durable-log implementations, a bounded WebRTC DataChannel bridge, and local bbolt/file checkpoint Store references.
• Optional, manifest-negotiated compression-aware outer frame v2 with explicit v1 conversion; it does not change CRDT TypeIDs or semantics.
• RGA diagnostic obfuscation that replaces text content while retaining an isolated debug timeline structure.

All implemented frame pairs are stable and use the zero-value ProtocolPolicy. LWW-Set/Map, scalar RGA v1, list RGA, run-v2 RGA, packed-v3 RGA, rich-text v1, and observed-remove tree v1 still require an authenticated exact manifest: a frame type alone is never a negotiated protocol, authenticated peer, or permission to compact tombstones.

Choose a path

The documentation index separates getting-started, integration, protocol/design, and operational material. Detailed performance evidence and deployment runbooks live there instead of making this entry page a manual.

Persistence and recovery

State bytes alone are not a recoverable replica for HLC-backed CRDTs. Persist the state frame, HLC state, and application delivery frontier/outbox atomically before reusing a replica ID. The opt-in persistence.Store contract has bbolt and file references for one typed CRDT schema and one active process; both validate concrete state before saving and on every load.

(cd examples && go run ./persistent-replica)
# recovered=true cursor=41 outbox_bytes=24

It is not a clustered database, authenticated transport, or generic business transaction manager. The host still owns encryption at rest, backup/restore, remote authorization, tenant isolation, membership, and tombstone lifecycle.

The durable package intentionally persists a relay operation log and replay cursor. Clients must persist their concrete CRDT checkpoint before advancing that cursor; read the local checkpoint and durable relay references together.

Modules and dependencies

The published root module, github.com/DarkInno/crdt, has no non-standard-library module dependencies. Durable storage, transports, and database backends are independently versioned opt-in modules, so a core-only consumer does not resolve their dependency graphs.

For example, an application choosing MySQL installs only the core, durable contract, and MySQL provider modules (plus its own selected driver):

go get github.com/DarkInno/crdt@latest
go get github.com/DarkInno/crdt/durable@latest
go get github.com/DarkInno/crdt/providers/mysql@latest

Package map

Verify and measure

Run focused checks while changing one package:

(cd persistence && go test .)
(cd examples && go test ./persistent-replica)
(cd persistence && go test -race .)
(cd persistence && go test -run='^$' -fuzz=FuzzUnmarshalCheckpoint -fuzztime=250000x -parallel=1 .)
(cd persistence && go test -run='^$' -fuzz=FuzzUnmarshalFileRecords -fuzztime=250000x -parallel=1 .)
(cd persistence && go test -run='^$' -bench='Benchmark((File)?Store(Save|Load|SaveParallel|Delete|LoadLegacyMigration)|(File)?ConfigFromLoader)$' -benchmem -benchtime=2s .)

Repository gates:

make test
make race
make vet
make coverage
make verify

make verify also runs bounded fuzzing, static analysis, linting, integration, and extreme scenarios. make benchmark is a controlled development measure, not a production capacity promise—repeat focused benchmarks on the target disk, CPU, Go version, network, and workload before selecting limits.

For host wiring of layered configuration, structured error codes, and bounded durable-relay telemetry, see production readiness.

Boundaries that matter

• CRC-32C, SHA-256, and a frame type detect format damage; they do not authenticate a peer. Bind exact manifests and protocol policies during an authenticated handshake.
• A greatest observed tag is not proof of contiguous delivery or permission to retire tombstones. Use the relevant frontier, inbox, and membership contracts.
• Both checkpoint backends require one active process and are not HA storage. bbolt has an exclusive file lock; the file reference has no inter-process lock and must never be shared by active pods.
• The library does not enforce business invariants. Validate identity, tenant, value permissions, rate limits, retention, and backup access in the host.

Contributing and releases

Contributions should include focused tests, preserve canonical encoding, bound untrusted input before allocation or mutation, and update the closest relevant documentation. Review CONTRIBUTING.md; keep beta changes on the reviewed beta-to-preprod-to-main release path and do not manually move published tags.

License

SPDX-License-Identifier: MIT. See LICENSE.