gomap

gomap (dev): HashDB + TreeDB

This repo is a development playground for two storage engines plus benchmarking tools:

• HashDB (HashDB/, package hashdb): mmap-backed hash index + slab value log.
• TreeDB (TreeDB/, package treedb): persistent B+Tree with an optional cached write-back layer.
• BTreeOnHashDB (HashDB/BTreeOnHashDB/): a B-Tree built on top of HashDB (benchmark/comparison).
• Unified Bench (cmd/unified_bench/): runs a consistent workload across engines.

Quickstart

• make test
• make build
• make unified-bench && ./bin/unified-bench

More docs:

• docs/README.md
• docs/GETTING_STARTED.md
• docs/TREEDB_CACHED_VS_BACKEND.md
• docs/TREEDB_CONCEPTS.md
• docs/TREEDB_RECOVERY.md
• CONTRIBUTING.md

Choosing An Engine

High-level guidance:

• HashDB: best for high-throughput random reads and perf experiments; use PutSync/DeleteSync/ApplyBatchSync for durable commits (non-*Sync writes are best-effort, and sharded HashDB uses a write-back cache with no WAL).
• HashDB: best for high-throughput random reads and perf experiments; use PutSync/DeleteSync/ApplyBatchSync for durable commits (non-*Sync writes are best-effort; sharded HashDB uses a write-back cache and can optionally enable a per-shard cache WAL via OpenWithOptions).
• TreeDB (cached, default): best for workloads dominated by many small random writes; use *Sync for durability.
• TreeDB (backend-only): best when you batch writes yourself or want the simplest engine path; scans can be faster.

Contracts (durability/locking/concurrency/iteration):

• docs/contracts/README.md

Benchmarking

Primary tool: cmd/unified_bench/ (side-by-side: HashDB, TreeDB, Badger, LevelDB).

• Run: make unified-bench && ./bin/unified-bench
• Sweep key counts (markdown output): ./bin/unified-bench -format markdown -keycounts 100000,1000000
• Update the README benchmark snapshot: make bench-readme

More details:

• cmd/unified_bench/README.md
• docs/BENCHMARK_SPEC.md

Generated by go run ./cmd/unified_bench -suite readme -format markdown.

Generated at: 2025-12-16T22:33:24Z Environment: darwin/arm64 | Go go1.25.5 | CPUs 8 | RAM 16 GiB | CPU Apple M3 | Model Mac15,13 Seed: 1

Key counts: 1…1,000,000 (valsize=128, batchsize=1000, range-queries=200, range-span=100)

Notes:

• Results depend on hardware and OS.
• TreeDBBackend (uncached) point ops are shown only at 10,000 keys (baseline) and excluded from larger sweeps.
• HashDB does not support ordered scans.

Graphs

Point Ops (writes + gets)

Key counts: 1, 10, 100, 1,000, 10,000, 100,000, 1,000,000

Sequential Write

Random Write

Random Read

TreeDBBackend baseline (point ops)

Key count: 10,000

Batch + Scans

Key counts: 1, 10, 100, 1,000, 10,000, 100,000, 1,000,000

Batch Write

Full Scan

Prefix Scan

Quick takeaways

• HashDB: great for high-throughput point reads/writes; no ordered scan API yet.
• TreeDB (cached): strong default for random-write-heavy workloads; scans include merge overhead.
• TreeDBBackend (uncached): best when you batch writes yourself; slow for per-key writes.
• Badger/LevelDB: useful baselines with different storage tradeoffs.

Testing

• go test ./...
• cd TreeDB && go test ./...
• cd cmd/unified_bench && go test ./...

Notes

• Exclusive open: TreeDB and HashDB take an exclusive lock on the DB directory (ErrLocked).
• On-disk formats and public APIs may evolve; see docs/API_STABILITY.md.