d3d-api

d3d-api

Go REST + WebSocket backend for d3dweb. Stores directed acyclic graphs (DAGs) in PostgreSQL and provides real-time collaborative editing via a WebSocket relay.

Tech stack

Prerequisites

• Go 1.23+
• Docker (for local PostgreSQL)
• gin for live-reload during development (optional: go install github.com/codegangsta/gin@latest)

Local development setup

1. Start PostgreSQL

docker run -d --name pg -p 5432:5432 -e POSTGRES_PASSWORD=postgres postgres:16-alpine
# or if the container already exists:
docker start pg

The schema is managed by embedded goose migrations and is applied automatically on startup (app/db/postgres/migrations/). No manual setup is required.

2. Configure the API

Copy samus_dev.toml as your local config (it is gitignored):

[samus]
    bind_addr     = ":3001"
    signing_key   = "dev-signing-key-change-in-prod"
    auth_provider = "localauth"

[postgres]
    dsn = "postgres://postgres:postgres@localhost:5432/samus?sslmode=disable"

Alternatively, spell out the connection as discrete fields under [postgresql] (a DSN is assembled for you, with the password properly escaped):

[postgresql]
    address  = "localhost:5432"
    user     = "samus"
    password = "dev-password-change-in-prod"
    database = "samus"

A non-empty dsn always wins when both forms are present.

3. Create the first user

localauth stores bcrypt hashed users in the users table:

go run . createUser admin changeme --config samus_dev.toml

4. Run the API

go run . --config samus_dev.toml

# with live-reload
gin --all run samus.go -- --config samus_dev.toml

Without --config, the app reads ./samus.toml. The API listens on the bind_addr from your config (e.g. http://localhost:3001).

API reference

All endpoints except /auth/login and /shares/exchange require a Bearer <jwt> token in the Authorization header (or ?token= / ?api-key= query params).

Auth

DAGs

Real-time collaboration

The WebSocket relay forwards two message types:

• diagram:updated — broadcast to all peers when a save occurs (echo-prevented via clientId)
• presence — relayed to other peers in the room (cursor position, node selection, display name)

Diagram history

Share links

Share tokens are signed with the same signing_key as regular auth tokens but carry iss: "d3d-share". The role claim is enforced server-side:

• view — POST /dag/:dag/update returns 403; WS is allowed (presence only)
• edit — full read/write access

Revoked JTIs are stored in the share_denylist table and checked on WS upgrade and token exchange.

Anonymous display names (e.g. "Teal Fox") are generated on share creation and returned by /shares/exchange so view-only users appear in the presence HUD with a readable identity.

Nodes / Edges / Menus

Auth providers

Set auth_provider in your config to one of (defaults to ldap if unset):

Running tests

go test ./...

The app/collab/ package has unit tests for the hub room registry and write pump.

Database tables

The schema lives in app/db/postgres/migrations/ and is applied automatically on startup by embedded goose migrations. To add a new migration, drop a NNNN_name.sql file in that directory.

Migrating from RethinkDB

cmd/migrate-rb2pg copies data from an existing RethinkDB database into Postgres. It is idempotent (rows already present are skipped), so it is safe to run more than once.

go run ./cmd/migrate-rb2pg \
  -rb-address localhost:28015 \
  -rb-database samus \
  -pg-dsn "postgres://postgres:postgres@localhost:5432/samus?sslmode=disable"

Behavior notes:

• Preserves UUIDs and timestamps exactly; RethinkDB zero-time values become NULL.
• diagram / snapshot_json (stored as JSON strings in RethinkDB) are validated and normalized into jsonb.
• Tables missing in RethinkDB are skipped; rows whose parent DAG is gone are reported as orphans and skipped.
• Per-table counts are logged (inserted / already present / orphaned / failed); a non-zero exit code indicates real failures.

For the full step-by-step production cutover (freeze, preflight, migrate, verify, rollback), see docs/cutover-runbook.md.

Deployment

The image runs the API binary, which reads its config from ./samus.toml (or the path given by --config). Provide a samus.toml with a [postgres] dsn (or a discrete [postgresql] block) pointing at your Postgres instance; samus.tmpl is an env-substitution template for rendering that config from environment variables (e.g. POSTGRES_DSN). CI is required to pass before deploy.

CI/CD

GitHub Actions drives quality gates and deploys:

Repository secrets required for deploy: FLY_API_TOKEN (from flyctl auth token or the Fly dashboard). GHCR auth uses the automatic GITHUB_TOKEN; no extra secret needed.

Branch protection on main should require the CI workflow to pass before merging.

Pre-commit hooks

A .pre-commit-config.yaml runs local quality gates before every commit:

• gitleaks — secrets scanning
• golangci-lint — linting (.golangci.yml)
• gofmt, go mod tidy drift check
• generic file hygiene (trailing-whitespace, end-of-file-fixer, check-yaml, check-added-large-files)

Install once:

make precommit-install   # pre-commit install

Run manually at any time:

make precommit-run       # pre-commit run --all-files

The gitleaks allowlist (public test fixtures in app/auth/ldap/ldap_test.go, docs samples) lives in .gitleaks.toml.