sqi-server

sqi-server

sqi-server is the control plane for a sqi render farm. A single binary runs the job scheduler, REST API, WebSocket gateway, embedded NATS JetStream broker, and embedded SQLite state store. Workers connect to it over NATS; clients interact with it over HTTP and WebSocket.

What it does

Build

Prerequisites: Go 1.23 or newer (version is pinned in go.mod).

# Build just the server binary into ./bin/
make build-server

# Or build both server and worker
make build

# Or build directly with go
go build -o bin/sqi-server ./cmd/sqi-server

Cross-compiled binaries for all target platforms (Linux, macOS, Windows × amd64/arm64) can be produced with:

make build-all

Run

# Start with all defaults (listens on 0.0.0.0:8080, SQLite at ./sqi.db)
./bin/sqi-server serve

# Start with a config file
./bin/sqi-server serve --config config/sqi-server.yaml

# Override specific values via flags or environment variables
SQI_LOG_FORMAT=text ./bin/sqi-server serve --http-addr 127.0.0.1:8090

# Print the effective merged configuration and exit (useful for debugging)
./bin/sqi-server config print

# Show build metadata
./bin/sqi-server version

The first time the server starts it creates the SQLite database and runs all pending migrations automatically. No manual schema setup is required.

See config/sqi-server.example.yaml for a fully commented configuration file, and docs/configuration.md for a complete reference of every option.

Smoke test

The following sequence verifies the server is healthy, accepts job submissions, and returns expected responses. It requires curl and jq.

1. Start the server

./bin/sqi-server serve &
SERVER_PID=$!
sleep 1

2. Liveness and readiness

curl -sf http://localhost:8080/healthz   # expects: {"status":"ok"}
curl -sf http://localhost:8080/readyz    # expects: {"status":"ok","checks":{"nats":"ok","sqlite":"ok"}}

3. Submit a minimal OpenJD job

JOB_ID=$(curl -sf -X POST http://localhost:8080/api/v1/jobs \
  -H 'Content-Type: application/yaml' \
  --data-binary '
specificationVersion: "jobtemplate-2023-09"
name: smoke-test
steps:
  - name: hello
    script:
      actions:
        onRun:
          command: echo
          args: ["hello from sqi"]
' | jq -r '.id')

echo "Submitted job: $JOB_ID"

4. Query the job

curl -sf http://localhost:8080/api/v1/jobs/$JOB_ID | jq '{id, name, status}'

5. List tasks

curl -sf http://localhost:8080/api/v1/jobs/$JOB_ID/tasks | jq '[.[] | {id, status}]'

6. Check Prometheus metrics

curl -sf http://localhost:8080/metrics | grep sqi_

7. Verify the OpenAPI spec is served

curl -sf http://localhost:8080/api/v1/openapi.yaml | head -5

8. Stop the server

kill $SERVER_PID

Database migrations

Schema migrations are embedded in the binary and applied automatically at startup. To manage migrations manually:

# Show current migration status
./bin/sqi-server migrate status

# Apply all pending migrations
./bin/sqi-server migrate up

# Roll back the most recent migration
./bin/sqi-server migrate down

Migration SQL files live in internal/store/migrations/.

Package layout

cmd/sqi-server/
└── main.go            Entry point — wires the cobra command tree, exits.

internal/
├── bus/               Typed NATS JetStream client wrapper and subject definitions.
├── config/            Config struct, layered loader, and startup validator.
├── discovery/         mDNS responder (grandcat/zeroconf).
├── health/            /healthz and /readyz handlers.
├── log/               slog-based structured logger and request-scoped middleware.
├── metrics/           Prometheus metric definitions and the /metrics handler.
├── middleware/         HTTP middleware (recovery, CORS, request ID, gzip, logger).
├── openjd/            OpenJD v2025-09 parser, validator, parameter-space expansion.
├── scheduler/         Assignment loop, worker registry, task state machine.
├── server/            HTTP server wiring — router, middleware stack, boot sequence.
├── store/             Store interface, SQLite implementation, migration runner.
├── ui/                Embedded web/dist assets, SPA fallback handler.
├── version/           Build metadata (version, commit, build date, Go version).
├── worker/            Worker wire protocol handlers and log ingestion.
└── ws/                WebSocket upgrade, subscription hub, NATS fanout.

Further reading

• docs/architecture.md — Component layout and job lifecycle data flow.
• docs/configuration.md — Complete configuration reference.
• docs/api.md — REST API reference with worked examples.
• docs/development.md — Local setup and contribution guide.