tiny

module
v0.3.17 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 3, 2026 License: MIT

README

TinyRaven

Open-source, self-hosted, drop-in alternative to Tinybird.

Built in Go on top of OSS ClickHouse. Same API. Your infrastructure.

Go

Status: pre-alpha — actively building Phase 1. No release yet. The commands and install methods below describe the target developer experience; track real progress in MILESTONE.md and the issues.


What is TinyRaven?

TinyRaven replicates Tinybird's full developer experience — HTTP ingestion, SQL pipes, REST endpoint publishing, a CLI, branching, and git workflows — but runs entirely on infrastructure you own. It speaks the exact same API as Tinybird, so existing client code works by changing one environment variable.

The model is ScyllaDB → Cassandra: identical API surface, leaner internals, fully open. Tinybird runs a private ClickHouse fork behind a Python/C++ stack; TinyRaven is a single Go binary in front of stock open-source ClickHouse.

  • Backend + CLI: one Go binary (tinyraven), net/http + chi
  • Database: OSS ClickHouse (Apache 2.0), target 26.3 LTS
  • Metadata + cache: Redis (AOF-persisted; system of record for metadata)
  • License: Apache 2.0 — free and feature-complete forever, no paywall, no gated "enterprise" features (ADR 0021)

Drop-in compatibility

Point your existing Tinybird project at TinyRaven by changing the host:

export TINYBIRD_HOST=https://tinyraven.example.com
export TINYBIRD_TOKEN=your_token
tinyraven deploy        # same .datasource / .pipe files, your backend
API parity
Endpoint Method Purpose
/v0/events POST JSON / NDJSON event ingestion
/v0/datasources POST Batch data import (NDJSON/CSV, append/replace)
/v0/pipes/{name}.{json,csv,ndjson,parquet} GET Parameterized query; per-pipe CACHE_TTL result cache
/v0/pipes/{name}/copy POST Run a copy pipe (also scheduled via COPY_SCHEDULE)
/v0/sql GET/POST Read-only ClickHouse SQL proxy
/v0/datasources, /v0/pipes (+ /{name}) GET Introspection — list/get datasources & pipes
/v0/tokens (+ /{name}) GET/POST/DELETE Token management (ADMIN)
/v0/jobs (+ /{id}) GET Async job listing / status
/v0/metrics, /health, /health/ready GET Prometheus metrics + liveness/readiness

.datasource and .pipe files use Tinybird's exact format, including {{Type(param, default)}} SQL templating. Native (non-Tinybird) endpoints live under /tr/v1/*; /v0 stays a frozen parity mirror.

Architecture

POST /v0/events batches through an in-process Gatherer into ClickHouse; GET /v0/pipes/{name}.json parses the {{Type(...)}} template, binds params as ClickHouse query parameters, and streams the result. ClickHouse holds event data, materialized views, and query execution; Redis holds the metadata registry (datasource/pipe definitions, tokens, deploy state) plus hot cache, AOF-persisted as a system of record. Branching = one ClickHouse database per git branch (tr_{branch}).

Full data flow, the deps table, and every locked decision live in PROMPT.md and the ADRs — the spec, not duplicated here.

Quickstart (planned)

brew install tinyraven        # installs the `tinyraven` binary
tinyraven local start                # ClickHouse + TinyRaven + Redis via Docker Compose

# ingest
curl -X POST "http://localhost:8000/v0/events?name=events" \
  -H "Authorization: Bearer $TR_TOKEN" \
  -d '{"user_id":"alice","event":"page_view"}'

# query a published pipe
curl "http://localhost:8000/v0/pipes/user_metrics.json?user_id=alice" \
  -H "Authorization: Bearer $TR_TOKEN"

Install (planned)

Platform Manager Command
macOS / Linux Homebrew brew tap ravencloak-org/tinyraven && brew install tinyraven
Debian / Ubuntu APT sudo apt-get install tinyraven (after adding the repo — see below)
RHEL / Fedora DNF/YUM sudo dnf install tinyraven (after adding the repo)
Windows Scoop scoop bucket add tinyraven https://github.com/ravencloak-org/scoop-bucket && scoop install tinyraven
Windows WinGet winget install Ravencloak.TinyRaven
Arch Linux AUR yay -S tinyraven-bin
Nix NUR nix profile install github:ravencloak-org/nur#tinyraven
Any Docker docker run -p 8000:8000 ghcr.io/ravencloak-org/tiny:latest serve
Any Binary Download from Releases and unpack tinyraven
Any Go go install github.com/ravencloak-org/tiny/cmd/tinyraven@latest

Package name is always tinyraven; the binary is always tinyraven. We never use tb (the Tinybird CLI) to avoid conflicts.

Full per-platform instructions — including the APT key/repo setup and how to verify the GPG signature of release checksums — live in docs/install.md.

Roadmap

Phase Deliverable Gate
1 — MVP tinyraven local start + events + pipes POST → GET round-trip works
2 — API tinyraven deploy + OpenAPI + metrics Full deploy + query cycle
3 — Workflows Branches + materialized views Zero-downtime migration
4 — Distribution Brew / APT / Heroku / AWS install One-click deploy everywhere
5 — Community Connectors + BI integrations 10k events/s benchmark

Full breakdown in MILESTONE.md. Architecture decisions live in PROMPT.md.

What TinyRaven is not

  • No built-in dashboard — API-first. Connect Metabase, Superset, or Grafana straight to ClickHouse.
  • No managed cloud — pure self-hosted FOSS.
  • No ClickHouse fork — stock OSS ClickHouse, as-is.

Contributing

Pre-alpha — the codebase is being bootstrapped. The best way to help right now is to browse the issues, grouped by phase milestone, and pick up a Phase 1 task.

License

Apache 2.0.


Deploy & Install

Distribution targets land in Phase 4. The one-liners and buttons below describe the intended experience; release artifacts publish on the first tagged release (git tag vX.Y.Z).

Install the tinyraven binary

Homebrew (macOS / Linux) — the recommended form is to tap first, then install the short name:

brew tap ravencloak-org/tinyraven   # adds the ravencloak-org/homebrew-tinyraven tap
brew install tinyraven              # installs the `tinyraven` binary

Why brew tap first? The bare brew install tinyraven only works for formulae in homebrew-core, which TinyRaven isn't in (yet). From our tap, the fully-qualified form is brew install ravencloak-org/tinyraven/tinyraven — running brew tap ravencloak-org/tinyraven once lets you use the short brew install tinyraven afterward. (Submitting to homebrew-core, which would enable the bare command for everyone, is a future option.)

APT (Debian / Ubuntu) — signed repo hosted on GitHub Pages:

curl -fsSL https://ravencloak-org.github.io/tiny/apt/KEY.gpg \
  | sudo gpg --dearmor -o /usr/share/keyrings/tinyraven.gpg
echo "deb [signed-by=/usr/share/keyrings/tinyraven.gpg] https://ravencloak-org.github.io/tiny/apt stable main" \
  | sudo tee /etc/apt/sources.list.d/tinyraven.list
sudo apt-get update && sudo apt-get install tinyraven

DNF / YUM (RHEL / Fedora) — signed repo:

sudo curl -fsSL https://ravencloak-org.github.io/tiny/rpm/tinyraven.repo \
  -o /etc/yum.repos.d/tinyraven.repo
sudo dnf install tinyraven

Windows — Scoop or WinGet:

scoop bucket add tinyraven https://github.com/ravencloak-org/scoop-bucket
scoop install tinyraven
# …or…
winget install Ravencloak.TinyRaven

Arch (AUR) · Nix · Docker · raw binary:

yay -S tinyraven-bin                                        # Arch User Repository
nix profile install github:ravencloak-org/nur#tinyraven    # Nix (NUR)
docker run -p 8000:8000 ghcr.io/ravencloak-org/tiny:latest serve

# Raw binary (any OS) — pick your platform from the Releases page:
curl -fsSL -o tinyraven.tar.gz \
  https://github.com/ravencloak-org/tiny/releases/latest/download/tinyraven_<ver>_linux_amd64.tar.gz
tar -xzf tinyraven.tar.gz && sudo install tr /usr/local/bin/tr

Binaries for Linux/macOS/Windows × amd64/arm64, plus .deb / .rpm packages and SHA256 checksums (GPG-signed), are built by GoReleaser on every tag. Full setup, signature verification, and per-platform notes: docs/install.md.

One-click cloud deploy

Deploy to Heroku Deploy on Railway Launch on AWS Deploy to DigitalOcean

Kubernetes (Helm)
helm install tinyraven ./charts/tinyraven \
  --set env.clickhouse.http=http://clickhouse:8123 \
  --set env.redis.addr=redis:6379
Per-platform guides
Platform Guide
Docker docs/deploy/docker.md
Kubernetes / Helm docs/deploy/kubernetes.md
Heroku docs/deploy/heroku.md
AWS (CloudFormation) docs/deploy/aws.md
Railway docs/deploy/railway.md
Dokploy + Cloudflare docs/deploy/dokploy.md

Coming from Tinybird? See docs/migrate-from-tinybird.md — install tinyraven, point TINYBIRD_HOST, tinyraven deploy.

Every target needs an external ClickHouse 26.3 and Redis — TinyRaven's tinyraven server is stateless.

Directories

Path Synopsis
cmd
tinyraven command
Command tr is the TinyRaven binary: both the HTTP server and the CLI.
Command tr is the TinyRaven binary: both the HTTP server and the CLI.
internal
api
Package api is the HTTP layer: chi router, request/response glue, and middleware.
Package api is the HTTP layer: chi router, request/response glue, and middleware.
apierr
Package apierr is the shared HTTP response envelope: the Tinybird-compatible error shape (ADR 0012) and JSON write helpers.
Package apierr is the shared HTTP response envelope: the Tinybird-compatible error shape (ADR 0012) and JSON write helpers.
auth
Package auth is the Redis-backed token store (ADR 0005).
Package auth is the Redis-backed token store (ADR 0005).
branch
Package branch resolves the current git branch to an isolated ClickHouse workspace database (tr_<branch>).
Package branch resolves the current git branch to an isolated ClickHouse workspace database (tr_<branch>).
clickhouse
Package clickhouse is TinyRaven's ClickHouse adapter.
Package clickhouse is TinyRaven's ClickHouse adapter.
config
Package config loads TinyRaven runtime configuration.
Package config loads TinyRaven runtime configuration.
datasource
Package datasource parses .datasource project files into model.Datasource values and persists them in the Redis-backed metadata registry (ADR 0001).
Package datasource parses .datasource project files into model.Datasource values and persists them in the Redis-backed metadata registry (ADR 0001).
deploy
Package deploy implements `tr deploy`: it parses every .datasource/.pipe file in a project directory, validates them all before touching ClickHouse (ADR 0027 — validate-all-then-apply), diffs each datasource against the live ClickHouse schema, applies safe additive migrations, creates materialized views (ADR 0010), and registers the datasource definitions in the metadata registry (ADR 0001).
Package deploy implements `tr deploy`: it parses every .datasource/.pipe file in a project directory, validates them all before touching ClickHouse (ADR 0027 — validate-all-then-apply), diffs each datasource against the live ClickHouse schema, applies safe additive migrations, creates materialized views (ADR 0010), and registers the datasource definitions in the metadata registry (ADR 0001).
gatherer
Package gatherer buffers incoming events in-process and flushes them to ClickHouse in batches (ADRs 0004, 0018).
Package gatherer buffers incoming events in-process and flushes them to ClickHouse in batches (ADRs 0004, 0018).
job
Package job is the Redis-backed model.JobStore (gap #8): persists completed (and, later, in-flight) async job records so GET /v0/jobs and GET /v0/jobs/{id} have something real to serve instead of a dead job_url.
Package job is the Redis-backed model.JobStore (gap #8): persists completed (and, later, in-flight) async job records so GET /v0/jobs and GET /v0/jobs/{id} have something real to serve instead of a dead job_url.
metrics
Package metrics owns the Prometheus registry and the collectors exposed at GET /v0/metrics (Phase 2).
Package metrics owns the Prometheus registry and the collectors exposed at GET /v0/metrics (Phase 2).
model
Package model is the shared contract for TinyRaven: the data types parsed from .datasource/.pipe files plus the interfaces that decouple the subsystems (gatherer, pipe executor, clickhouse, auth, api) from each other.
Package model is the shared contract for TinyRaven: the data types parsed from .datasource/.pipe files plus the interfaces that decouple the subsystems (gatherer, pipe executor, clickhouse, auth, api) from each other.
openapi
Package openapi emits an OpenAPI 3.0 spec for a TinyRaven deployment from the live pipe registry (ADR 0017).
Package openapi emits an OpenAPI 3.0 spec for a TinyRaven deployment from the live pipe registry (ADR 0017).
pipe
Package pipe parses .pipe project files into model.Pipe values, stores them in a hot-swappable in-memory registry (ADR 0020), and executes published endpoints against ClickHouse (ADRs 0003, 0009, 0012).
Package pipe parses .pipe project files into model.Pipe values, stores them in a hot-swappable in-memory registry (ADR 0020), and executes published endpoints against ClickHouse (ADRs 0003, 0009, 0012).
pipestats
Package pipestats implements model.StatsRecorder: it records per-query observability rows and flushes them to ClickHouse in batches via the shared CHInserter, mirroring the gatherer's max(N, interval) flush (ADR 0014).
Package pipestats implements model.StatsRecorder: it records per-query observability rows and flushes them to ClickHouse in batches via the shared CHInserter, mirroring the gatherer's max(N, interval) flush (ADR 0014).
ratelimit
Package ratelimit provides per-token request rate limiting for the API.
Package ratelimit provides per-token request rate limiting for the API.
scheduler
Package scheduler runs TYPE copy pipes on their COPY_SCHEDULE cron expression — the async-execution half of gap #8 that GET /v0/jobs (internal/job) left deferred.
Package scheduler runs TYPE copy pipes on their COPY_SCHEDULE cron expression — the async-execution half of gap #8 that GET /v0/jobs (internal/job) left deferred.
sqlproxy
Package sqlproxy serves GET/POST /v0/sql — a read-only ClickHouse SQL proxy.
Package sqlproxy serves GET/POST /v0/sql — a read-only ClickHouse SQL proxy.
scripts
loadtest command
Command loadtest is a stdlib-only load generator for the TinyRaven events API.
Command loadtest is a stdlib-only load generator for the TinyRaven events API.
querybench command
Command querybench is a stdlib-only query load generator for TinyRaven pipe endpoints.
Command querybench is a stdlib-only query load generator for TinyRaven pipe endpoints.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL