helpers

helpers

Shared Go helpers in one module: github.com/open-rails/helpers. Requires Go 1.26.6 or newer. MigrateKit remains a separate module.

Import only the packages an application needs. The API core does not import Gin or River. Go builds the imported packages and their dependencies; all packages share this module's version and Go/toolchain requirements.

See the shared inbox guide for notification producers, existing-source projections and the PostgreSQL lifecycle.

API helpers

The core provides typed errors, stable type/code fields, optional request IDs, bounded/sanitized metadata, JSON writers and generic list/message/deletion responses. The Gin adapter delegates to those writers and adds pagination binding and locale middleware. It does not add a router or authentication system. Existing response bytes are retained, including omitted codes/discriminators and preserved request IDs.

Authentication and user-directory contracts

The auth and userinfo packages, with their conformance test packages, now belong to AuthKit v1.24.0 in the same AuthKit Go module:

The contracts use only the standard library, and their conformance packages add only their corresponding contract package. Importing them does not import, build or initialize AuthKit's service implementation. Custom host implementations remain supported.

These packages are removed from helpers in v1.11.0; there are no forwarding aliases. Update hosts and consuming libraries together because their Identity, Scope, Credential, User and related interfaces now have different Go package identities. Existing helpers tags remain available for applications that have not migrated. See AuthKit's migration instructions.

SMTP

smtp.New(smtp.Config{Host, Port, Username, Password, From}) returns a Sender, so the email provider is configuration: SendGrid is smtp.sendgrid.net:587 with username apikey and an API key with mail.send as the password; ZeptoMail, SES or a self-hosted server are another host. Port 465 is TLS from the first byte; any other port (0 is 587) upgrades with STARTTLS when offered, and must before credentials go to a host other than loopback. Authentication is PLAIN, else LOGIN. From is one RFC 5322 mailbox ("Shop <noreply@shop.example>"); a Message may carry its own. Send(ctx, Message{To, Subject, Text, HTML}) opens one connection per message (safe for concurrent use), bounded by ctx and Timeout (30s), and writes a multipart/alternative message with Date, Message-ID and encoded headers; a refusal wraps the server's *textproto.Error. CheckHealth connects, negotiates TLS and authenticates without sending. No error carries the password. Applications name the settings EMAIL_SMTP_HOST, EMAIL_SMTP_PORT, EMAIL_SMTP_USERNAME, EMAIL_SMTP_PASSWORD and EMAIL_SMTP_FROM.

smtptest.Start(t, smtptest.Options{...}) runs an SMTP server on 127.0.0.1 that captures and decodes what it receives (Wait, Messages), optionally requiring credentials, offering STARTTLS or speaking implicit TLS with a certificate ClientTLS trusts, refusing chosen recipients, and simulating an outage (Outage).

River composition

river.ApplyMigrations(ctx, pool, schema) creates the selected schema and applies River's migrations under a database/schema advisory lock. An empty schema means public. Schema creation happens inside the lock, and the lock uses a dedicated connection so a host pool with MaxConns=1 cannot stall itself. The host retains its pool; call this explicit initializer before starting workers or accepting requests that enqueue jobs.

river.New combines library contributions, validates registration and binds their producers to the same ordinary River client and actual host pool. Pass the same schema in riverqueue.Config.Schema; empty also means public. Construction does not migrate or start workers. The host starts/stops that client and retains ownership of the pool. These helpers add neither a scheduler nor billing logic.

When importing both the helpers and upstream River, use a descriptive alias such as riverhelpers for github.com/open-rails/helpers/river.

Dependency supervision

deps.New() supervises external dependencies. Each one is probed forever: every 10s (±10% jitter) while up, with full-jitter exponential backoff (500ms to 30s) while down, and it recovers after two consecutive successes. Optional dependencies go down on one failed probe, required ones after three (DownAfter, ProbeTimeout and ProbeInterval, the up-period, and DownInterval, the least period while failing, for probes that cost something, tune one dependency). Dependency.Report(err) marks it down on the first data-path connectivity error; a probe already in flight cannot undo that. OnUp/OnDown hooks run in order on their own goroutine with the supervisor's context; while they lag, pending transitions coalesce to one down plus the latest state, so a hung hook never stalls probing. Adding a dependency under an existing name replaces it; OnClose releases a probe's resources when it is replaced or the supervisor stops. Supervisor.Retry waits for a required dependency at startup instead of exiting. AddPostgres (built on PostgresProbe) checks Postgres over one dedicated connection, closed on replace and stop, so a saturated application pool does not read as an outage; PostgresUnavailable classifies connection-level errors.

deps.NewSwitch(dep, primary, newFallback) serves the primary while the dependency is up and a local fallback otherwise. deps.Call/deps.Do retry a call once on the fallback when the primary is unreachable. Fallback state is replaced, never merged, on recovery. Use it only for state whose per-process scope is acceptable (caches, rate-limit windows); state other replicas must see belongs in Postgres.

deps.NewRedis(RedisConfig) returns a redis.UniversalClient without dialing: master_name with sentinel_addrs gives a Sentinel failover client (the Sentinel password defaults to password), otherwise one of addrs gives a plain client and several a cluster client. AddRedis registers it as optional; its probe writes a short-lived key, so a primary that refuses writes (NOREPLICAS, READONLY) is down too.

HTTP: Gate() is the application listener's handler and can bind before the application is built. It serves /livez (always 200, never checks dependencies) and /readyz (200 once Open(app) is called, 503 after Drain()), and returns 503 for everything else until Open. OpsHandler() adds /statusz (JSON per dependency) and /metrics (app_ready, app_dependency_up{dependency,class}, app_dependency_transitions_total, Counters, GaugeFunc gauges and CounterFunc counters read at scrape time; names are validated and unique, label values escaped) for an internal port.

Cache

cache.New(cache.Options{...}, load) returns a *cache.Cache[K, V] in front of load(ctx, keys) (map[K]V, error); a key absent from the map is not found and is not cached. It holds mechanics only: the caller sets Name, RefreshAfter and MaxStale (required, 0 < RefreshAfter < MaxStale); LoadTimeout (10s), FailureBackoff (5s), MaxEntries (10,000) and Clock (real time) default.

• Get(ctx, keys) loads only missing keys, once across concurrent callers and overlapping batches. Loads run detached from the caller's cancellation, bounded by LoadTimeout; each caller still returns at its own deadline, and the load is still cached.
• After RefreshAfter the cached value is served while one background reload runs. A failed reload keeps the last good value until MaxStale after the last good load and retries at most once per FailureBackoff. A failed load of a key without a value returns that same error, without loading, for FailureBackoff.
• Refresh(ctx, keys) reloads now, joining an in-flight load, and waits; a failure keeps and returns the cached value with the error, and a not-found key is dropped.
• Invalidate(keys...)/InvalidateAll() win over in-flight loads and refreshes and clear recorded failures. Close() stops the background expiry goroutine of a cache being discarded; the cache still works after it.

Reload failures are logged with slog (cache=<Name>). cache.Snapshot() returns per-Name counts for the process (hits, misses, stale serves, loads, load failures); export them with deps.CounterFunc:

sup.CounterFunc("app_cache_events_total", "Cache events by cache name.", func() []deps.Sample {
	var out []deps.Sample
	for _, s := range cache.Snapshot() {
		for event, n := range map[string]uint64{"hit": s.Hits, "miss": s.Misses, "stale": s.StaleServes, "load": s.Loads, "load_failure": s.LoadFailures} {
			out = append(out, deps.Sample{Labels: []deps.Label{{"cache", s.Name}, {"event", event}}, Value: float64(n)})
		}
	}
	return out
})

The package wraps otter v2 at a main-branch pseudo-version with the refresh-versus-invalidate and waiter-context fixes missing from v2.3.0; move to v2.4.0 once it is tagged.

Migration

Replace the previous module imports with the corresponding packages in this module:

No forwarding packages are provided at the old import paths. Libraries and hosts that exchange River contribution types must use the shared package paths; update those libraries before updating their host composers. MigrateKit imports do not change.

The API helpers preserve APIKit's response contract. Replacing older doujins-org/ginapi writers can change envelope semantics and requires application-level compatibility checks. See the error envelope compatibility notes and frozen HTTP/parser fixtures for the recorded differences.

Injected-code scan

A reusable workflow scans a repository's tracked tree for the repo-injection worm (invisible padding, decode-and-eval, blockchain dead-drop C2, tampered build configs). Pin it by commit SHA:

jobs:
  injection-scan:
    uses: open-rails/helpers/.github/workflows/injection-scan.yml@<helpers commit SHA>

Locally: scripts/scan-injected-code.sh --root <repo>. Rules, thresholds and exclusions are in docs/injection-scan.md.

Validation

Use one entry point locally and in CI:

RIVER_TEST_DATABASE_URL='postgres://.../helpers_test?sslmode=disable' DEPS_TEST_REDIS_ADDR=127.0.0.1:6379 DEPS_TEST_SENTINEL_ADDR=127.0.0.1:26379 bash scripts/check.sh

It checks formatting, the single-module/package boundaries, vet/race tests, actual PostgreSQL River initialization/composition, Redis outage/recovery through a TCP cut, the frozen frontend-parser fixtures and reachable vulnerabilities. The PostgreSQL test login must be able to create temporary test databases; tests remove only their own databases/schemas. For a quick API-only check, run go test ./api/....

api/compat contains frozen wire/parser evidence, not reverse dependencies on applications. Parser origins and test adaptations are documented in SPA parser provenance.

License

All packages are covered by the root MIT license, which retains the copyrights of Shinobi Online LLC and Paul Fidika.