server

Secretli server

The API behind Secretli. It stores secrets that were encrypted before they arrived, hands them back to whoever proves they hold the link, and forgets them when they are opened or expire.

The server cannot read what it stores. Keys are derived and used only by the clients, the web app and the command-line client, which follow the format specified in secretli/format. The link's secret travels in the URL fragment, which never reaches a server. What arrives here is ciphertext, an encrypted metadata envelope, and three tokens, of which only SHA-256 hashes are kept. CI checks that the format library, the code that could derive a key, is never among this server's dependencies.

What it does

• Uploads: a secret arrives as a multipart upload session. Its parts are streamed into S3-compatible storage and assembled when the session completes, so no request ever carries the whole secret. Part n starts at (n−1) × 32 MiB and is exactly 32 MiB long, but for the last, which holds the rest; any other part is refused, so a session never stores more than it declared. A session that sends no new part for 30 minutes is abandoned, its reservation and its public id freed; the absolute limit stays 24 hours. Each client address may declare up to UPLOAD_BUDGET_BYTES_PER_HOUR for new uploads per hour, counted in memory only; past it, a start gets 429 with Retry-After. The secret's lifetime counts from the completed upload. Its times are kept to the minute, with one minute added so that it never lives shorter than chosen, and none of them tells how long the upload took.
• Retrieval: the link's metadata token unlocks the encrypted metadata. Its blob token opens a 15-minute retrieval session that reads the bundle by byte range. Opening a one-time secret closes it: nobody can open it again, and only the download that opened it can still read it, until that session ends. It closes only once storage shows it holds the object, so a one-time secret opened while storage is down stays live. A wrong blob token, a wrong password as a rule, gets 403 with details.attempts_left; the tenth deletes the secret as its owner could, so a password can't be guessed at for the secret's whole lifetime.
• Owner status: a link can tell whether a recipient has opened a reusable secret. Never when. Once a secret is deleted, expires or, being one-time, has been opened, nothing about it is told: its link gets the same 404 as one that never existed.
• Deletion: the owner link's deletion token removes a secret at once; its object leaves storage within a minute.
• Reserved links: a public id stays taken until the expiry of the secret that took it, also after the secret was opened or deleted, so nobody holding the link can put other content under it (see SECURITY.md).
• Short-code hand-off: a relay through which two devices pass a link after a password-authenticated key exchange. The server only sees public key-exchange shares and ciphertext.
• Cleanup: a worker runs every minute. It forgets secrets past their expiry and opened one-time secrets whose download ended, frees public ids past their expiry, abandons uploads that ran out of time, and is the only code that deletes from storage: every object is on file from before it is written until after it is deleted, so storage never holds one the database has forgotten.
• Storage outages: while object storage can't be reached or answers with server errors, every call that needs it (starting, uploading and completing an upload, opening a one-time secret, reading a range) gets 503 with Retry-After, which the clients wait out and repeat a few times; the cleanup removes in a later cycle whatever it could not remove. A one-time secret whose download an outage interrupts can still be lost once its session ends.
• Logs: a request is logged by its route, not its path, so the logs don't say which secret was asked for.

Endpoints

The secret, upload and transfer endpoints are rate limited per client address; health, version and metrics are not.

Running it

The server needs PostgreSQL and an S3-compatible object store. Migrations run at startup.

docker compose -f docker/docker-compose.yml --profile app up -d

starts the server on http://localhost:8080 with PostgreSQL and SeaweedFS. Without --profile app you get only the two backing services, for running the server from source:

docker compose -f docker/docker-compose.yml up -d
cp .env.example .env
make dev

Configuration

Environment variables; see .env.example.

Behind a reverse proxy, set TRUSTED_PROXIES to the proxy's address so rate limits key on the real client; otherwise forwarded headers are ignored.

Rate limits apply per client address, and an end-to-end suite sends far more requests from one address than a visitor would. Test environments set RATE_LIMIT_MULTIPLIER (for example to 100) instead of waiting out the limits. It is read once at startup and never from a request, anything but a whole number of at least 1 stops the server from starting, and a server with raised limits logs a warning. Production leaves it unset.

Storage lifecycle

The cleanup deletes every object itself, so the bucket needs no lifecycle rules to stay clean. In production it carries two anyway, as a safety net for what a database restore or a bug could leave behind. They are in deploy/storage-lifecycle.json, both on blobs/, where every secret's object lives:

• objects expire after 10 days. A secret's object lives at most about 8: an upload of up to 24 hours, then the longest lifetime of 7 days;
• unfinished multipart uploads are aborted after 2 days, since an upload session lasts at most 24 hours.

Storage counts from when an object was made and acts once a night, so it removes an object later than that, never earlier.

The rules live in the bucket, not in the code, so they are applied by hand once, and again whenever the file changes:

aws s3api put-bucket-lifecycle-configuration --bucket "$S3_BUCKET" --endpoint-url "$S3_ENDPOINT" \
  --lifecycle-configuration file://deploy/storage-lifecycle.json
aws s3api get-bucket-lifecycle-configuration --bucket "$S3_BUCKET" --endpoint-url "$S3_ENDPOINT"

TestStorageLifecycleOutlastsEverySecret keeps the rules in step with the server: it fails when a longer lifetime or upload session would let them take an object the server still needs.

Metrics

GET /metrics serves Prometheus metrics: Go runtime and process metrics, and these.

The gauges from secretli_secrets_live on are read from the database at every scrape, so every replica reports the same values: across replicas, take one of them with max, not sum. For example, all live secrets are sum(max by (kind) (secretli_secrets_live)). They are totals of the current state: no secret is told apart, and nothing is stored for them. Uploads and downloads under way, one-time secrets whose download is still running and open short-code transfers are not counted on purpose: each lasts only while one person acts, and a count of them, scraped every minute into a monitoring store, would record when someone uploaded, downloaded or handed over a secret.

Suggested alerts:

Development

make test        # everything, including the Postgres and S3 integration tests (needs Docker)
make test-short  # unit tests only
make api-test    # the API, over HTTP, against a running server; see below
make lint
make vuln

The API test (apitest/) checks a running server through its HTTP API alone, with no client and no format library: uploads in one and several parts, the upload rules, metadata, retrieval and byte ranges, one-time and reusable secrets, that a gone secret answers like one that never was and its link cannot be reused, deletion, times kept to the minute, and the short-code relay. The server never decrypts anything, so random bytes stand in for ciphertext. It sends more requests than the rate limits allow from one address, so the server under test needs raised limits:

make build && RATE_LIMIT_MULTIPLIER=100 ./bin/secretli   # with DATABASE_URL and S3_* set
SECRETLI_SERVER=http://localhost:8080 make api-test

One test waits about six minutes for a secret to expire and runs only with SECRETLI_SLOW_TESTS=1 set.

CI runs the API test against a fresh server at every change. It also runs the whole of Secretli with the change, from secretli/e2e: the web app and both clients against this server behind a gateway like production's. Only the latest command-line client is supported, so it runs with that one alone. Nothing is published unless both pass.

Database queries are generated with sqlc from internal/adapter/postgres/queries; run sqlc generate after changing them or the migrations.

Images

Every change on main that passes CI is published to the GitHub Container Registry:

ghcr.io/secretli/server:main
ghcr.io/secretli/server:sha-<commit>
ghcr.io/secretli/server:<YYYYMMDD-HHmmss>-<commit>

The images are built for linux/amd64 and linux/arm64 on distroless, carry an SBOM and SLSA provenance, and are signed with cosign, keyless:

cosign verify ghcr.io/secretli/server:main \
  --certificate-identity-regexp 'https://github.com/secretli/server/' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

Security

See SECURITY.md for reporting vulnerabilities.

License

MIT