harmonylite

HarmonyLite

What is HarmonyLite?

HarmonyLite is a distributed SQLite replication system with leaderless architecture and eventual consistency. It enables robust multi-directional replication between nodes using NATS JetStream.

This project is a fork of Marmot, which appears to be no longer maintained.

Why HarmonyLite?

HarmonyLite continues and extends the vision of Marmot by providing:

• Leaderless, eventually consistent SQLite replication
• Easy horizontal scaling for read-heavy SQLite applications
• Minimal configuration with a modern, maintained codebase
• Enhanced testing and reliability
• Active development and community support

Quick Start

Download the latest release and extract:

tar vxzf harmonylite-v*.tar.gz

From the extracted directory, run the example cluster:

./examples/run-cluster.sh

Make changes to one database and watch them propagate:

# Insert data in the first database
sqlite3 /tmp/harmonylite-1.db
> PRAGMA trusted_schema = ON;
> INSERT INTO Books (title, author, publication_year) VALUES ('Project Hail Mary', 'Andy Weir', 2021);

# See it appear in the second database
sqlite3 /tmp/harmonylite-2.db
> SELECT * FROM Books;

What Makes HarmonyLite Different?

Unlike other SQLite replication solutions that require a leader-follower architecture, HarmonyLite:

• Has no primary node - any node can write to its local database
• Operates with eventual consistency - no global locking or blocking
• Requires no changes to your existing SQLite application code
• Runs as a sidecar to your existing processes

Features

• Multiple snapshot storage options:
  • NATS Blob Storage
  • WebDAV
  • SFTP
  • S3 Compatible (AWS S3, Minio, Blackblaze, SeaweedFS)
• Embedded NATS server
• Log compression for content-heavy applications
• Sleep timeout support for serverless environments
• Comprehensive E2E testing

Project Roadmap

Future plans for HarmonyLite include:

• Improved documentation and examples

Documentation

For detailed documentation, see the docs directory.

CLI Documentation

HarmonyLite is designed for simplicity with minimal configuration. Key command line options:

• config - Path to TOML configuration file
• node-id - Override node ID from config file
• cleanup - Clean up hooks and exit
• save-snapshot - Create and upload a snapshot
• cluster-addr - Binding address for cluster
• cluster-peers - Comma-separated list of NATS peers
• leaf-server - Leaf node connection list
• version - Display version information

See config.toml for detailed configuration options.

Version Information

HarmonyLite includes a comprehensive versioning system that provides detailed information about the build:

# Display version information
./harmonylite -version

The version information includes:

• Semantic version (based on Git tags)
• Git commit hash
• Git tag (if available)
• Build date and time
• Go version used for building
• Platform information

This helps with troubleshooting and ensures you're using the expected version in your deployment.

Development

Prerequisites

• Go 1.24 or later
• SQLite development libraries

Building from Source

make build

The Makefile automatically injects version information from Git during the build process.

Running Tests

Run all tests:

go test ./...

Run end-to-end tests only:

go run github.com/onsi/ginkgo/v2/ginkgo tests/e2e

macOS CGO Troubleshooting

On some macOS versions, you may encounter CGO compilation errors related to nullability type specifiers when building or testing. If you see errors like:

error: pointer is missing a nullability type specifier (_Nonnull, _Nullable, or _Null_unspecified)

Run tests with the following workaround:

CGO_CFLAGS="-O2 -g -Wno-nullability-completeness" go test ./...

License

MIT License - See LICENSE for details.

Acknowledgements

This project is a fork of Marmot by Zohaib Sibte Hassan. We are grateful for the solid foundation provided by the original project.