servicekit

module
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 22, 2026 License: MIT

README

servicekit

A Go SDK for building production services that speak REST and/or gRPC, backed by Postgres, with first-class observability (slog + OpenTelemetry traces + Prometheus metrics), reliable background workers, and a transactional outbox.

It distills patterns from several reference services into a single, importable module (github.com/assanoff/servicekit) plus a runnable example/ application that doubles as the scaffolding template for the servicekit CLI.

Layout

Package Purpose
errs Error type with stable codes, REST/gRPC mapping, validation, secret sanitization
logger slog-based logger: trace_id injection, level fan-out (stdout + Sentry), access log
otel Tracing bootstrap, span helpers, GetTraceID, route-excluding sampler
sqldb sqlx/pgx helpers: named queries, WithinTran, bulk insert/upsert/update, retries
dim Slim dependency-injection: lazy Provider[T], Once/OnceWithName, NewResource with init/cleanup logging
closer Graceful-shutdown registry: process-global default (Add/CloseSync) + instances (New/NewWithWait), LIFO
config 12-factor config via go-flags: one struct → CLI flags + env vars + --help, dotenv for local, subcommand dispatch
web/{rest,router,mid} Typed HTTP handlers over stdlib ServeMux + routegroup, middleware
grpcserver gRPC server as a worker.Runnable: recovery/trace/logging/metrics interceptors, errs→status mapping, health, reflection, transport tuning knobs
worker Unified worker abstraction: Runnable, Group, Loop, Pool, Processor[T], Backoff
queue Durable work queue (Schedule/Claim/MarkDone/MarkFailed): Postgres (FOR UPDATE SKIP LOCKED) + in-memory, plugs into worker.Processor as a Source/Sink
poller Typed cache of a periodically-refreshed value (Poller[T]), a worker.Runnable
broker Transport-agnostic Publisher/Consumer abstraction; CloudEvents v1.0 envelope; broker/rabbitmq implementation (confirm-mode publisher, worker.Runnable consumer)
outbox Transactional outbox: Event + Store + WithinTran (atomic domain-write + event), with Relay/Sweeper/Cleaner workers built on worker.Processor/Loop
delegate In-process synchronous event bus for decoupling domains without import cycles: Call (abort on first error) / Publish (run all, join errors). For atomic/durable events use outbox instead
i18n go-i18n wrapper: catalogs from fs.FS, Accept-Language matching + middleware, localizes *errs.Error by MessageID/Code with Args
auth Principal in context, credential extraction, pluggable Verifier, built-in JWT (HMAC/RSA/EC + JWKS via Keyfunc), Authenticate/Optional/RequireRole middleware
metrics Prometheus registry + HTTP middleware (gRPC collectors registered by grpcserver)
health Liveness/readiness handlers for Kubernetes probes
debugsrv Internal-only worker.Runnable serving net/http/pprof + optional metrics/health on a separate listener
migrate goose wrapper applying SQL migrations from an fs.FS via the Provider API (no goose global state)
httpmw Outbound HTTP-client middleware: RetryTransport retries 429/503 with worker.Backoff + RFC 7231 Retry-After
dbtest testcontainers Postgres for integration tests: start, migrate, connect, auto-teardown
apitest Stdlib-only HTTP test helpers: JSON requests, status/body assertions against an httptest.Server
safetick Panic recovery for worker ticks and consumer callbacks
pkg/httplog Vendored go-chi/httplog v3 (MIT), driven by our logger via Middleware(*logger.Logger, *Options)
cmd/servicekit Scaffolding CLI: servicekit new <module> (embedded starter, or gonew passthrough with --template)

(The CLI's per-model CRUD/REST/gRPC generators land in a later milestone.)

Installation

Your code always imports the same path — github.com/assanoff/servicekit/... — regardless of how the source is supplied, because Go resolves packages by import path. So both recipes below produce identical runtime behavior; they differ only in versioning and workflow. The key rule for the local recipe: keep the module path github.com/assanoff/servicekit and wire it in with replace or a workspace — do not rename the import paths (that forks the SDK).

Versioned, reproducible, go.sum-verified. This is what servicekit new generates.

go get github.com/assanoff/servicekit@v0.2.0   # or @latest

import (
    "github.com/assanoff/servicekit/web/router"
    "github.com/assanoff/servicekit/worker"
)

Update with go get github.com/assanoff/servicekit@latest. Pin a different version anytime by re-running go get …@vX.Y.Z.

Internal dependency (local source on disk)

Use this to develop your service and the SDK side by side, building against a local checkout. Two equivalent ways — pick one:

A. replace directive (per-module, committed to your app's go.mod):

# in your service module
go get github.com/assanoff/servicekit@v0.2.0          # keeps a sane version + go.sum entry
go mod edit -replace github.com/assanoff/servicekit=../servicekit
go mod tidy

// resulting go.mod
require github.com/assanoff/servicekit v0.2.0
replace github.com/assanoff/servicekit => ../servicekit

B. Go workspace (go.work, not committed — local to your machine):

cd /path/to/workspace        # parent dir holding both checkouts
go work init ./your-service ./servicekit

Either way the import paths stay github.com/assanoff/servicekit/... and the build uses the on-disk source. This is exactly how the in-repo example/ builds against the SDK (replace github.com/assanoff/servicekit => ../).

Caveats for the local recipe:

  • It builds against whatever is on disk — not version-pinned and not go.sum-verified for the replaced module; the path must exist everywhere you build (CI, teammates).
  • replace/go.work are not transitive and not published — they apply only to your own build. Fine for an application (the main module); for a library you publish, ship the external dependency and drop the replace before tagging.
  • Recommended flow: keep the external require pinned, add replace only while iterating on the SDK, then remove it.

i18n & auth

  • i18n.Translator loads JSON catalogs from an embed.FS, resolves the request language from Accept-Language, and localizes error responses: an app middleware translates any *errs.Error by its MessageID (or Code) with Args as template data. The example localizes widget errors to en/ru.
  • auth verifies bearer tokens via a pluggable Verifier (built-in JWT covers HMAC, RSA/EC PEM, and JWKS through a custom Keyfunc) and enforces RBAC with Authenticate + RequireRole. The example protects widget writes behind a widget:write role while keeping reads public.

Scaffolding

go run ./cmd/servicekit new github.com/you/svc          # embedded starter
go run ./cmd/servicekit new github.com/you/svc --template github.com/some/tmpl  # via gonew

Messaging & transactional outbox

broker + outbox give reliable, exactly-the-domain-write event publishing:

  • outbox.WithinTran writes a domain change and the events it emits in ONE transaction — an event is persisted iff its domain write commits.
  • outbox.Relay (a worker.Processor) drains pending events to a broker.Publisher with at-least-once delivery; Sweeper reclaims leases from crashed relays and Cleaner prunes terminal rows.
  • broker/rabbitmq publishes CloudEvents v1.0 with publisher confirms and consumes them via a supervised worker.Runnable.

The example wires it end-to-end: creating a widget emits widget.created through the outbox, the relay publishes it to RabbitMQ, and a consumer records it (see example/core/widget, widgetaudit, and the e2e test).

Reliable background processing

worker, queue, and sqldb compose into an at-least-once batch pipeline:

  • worker.Pool — bounded-concurrency fan-out of one-shot jobs.
  • worker.Processor[T] — a Source (claim) → Handler (process) → Sink (ack/retry) loop, run on a schedule via worker.Loop.
  • queue.PG — a Postgres queue that hands each ready task to exactly one consumer via FOR UPDATE SKIP LOCKED, so N replicas drain it safely; it satisfies worker.Source[Task]/Sink[Task] directly.

The example wires this end-to-end: POST /widgets/import enqueues a batch, and a supervised import worker bulk-inserts it (idempotently, via sqldb.BulkInsert + ON CONFLICT DO NOTHING). See example/core/widgetimport.

gRPC

grpcserver runs alongside (or instead of) the REST server — both are worker.Runnables supervised by worker.Group, toggled independently by config. It ships:

  • the same cross-cutting interceptors as REST (panic recovery, trace-id injection, structured access logs, Prometheus metrics) plus automatic *errs.Error → gRPC status mapping (error codes are aligned, so the mapping is a direct cast);
  • messages use the protobuf Opaque API via Protobuf Editions (edition = "2023", api_level = API_OPAQUE) — Google's official path to faster, lower-allocation, lazy-decoding generated code (replaces the third-party vtprotobuf; no extra dependency or codec);
  • transport tuning (MaxRecvMsgSize, SharedWriteBuffer, NumStreamWorkers, keepalive);
  • gRPC health service and optional reflection.

Code is generated with buf v2 (config in example/). gRPC v1.81 / protobuf v1.36.

make proto-tools   # install buf, protoc-gen-go, protoc-gen-go-grpc
make proto         # buf lint + generate (example/proto -> example/gen)

Quick start

make build      # build SDK + example
make test       # unit tests
cd example && go run ./cmd serve

See example/ for a full application wired with dim.

Directories

Path Synopsis
Package apitest provides helpers for end-to-end HTTP tests that drive a real application handler through an httptest.Server.
 Package apitest provides helpers for end-to-end HTTP tests that drive a real application handler through an httptest.Server.
Package auth provides authentication and authorization building blocks: a transport-neutral Principal carried in context, credential extraction from HTTP requests, a pluggable Verifier (token -> Principal), a built-in JWT verifier, and net/http middleware for authentication and role-based access control.
 Package auth provides authentication and authorization building blocks: a transport-neutral Principal carried in context, credential extraction from HTTP requests, a pluggable Verifier (token -> Principal), a built-in JWT verifier, and net/http middleware for authentication and role-based access control.
Package broker abstracts an event broker behind transport-agnostic Publisher and Consumer contracts.
 Package broker abstracts an event broker behind transport-agnostic Publisher and Consumer contracts.
rabbitmq
Package rabbitmq implements the broker abstraction over RabbitMQ using github.com/wagslane/go-rabbitmq, which adds automatic reconnection and publisher confirms on top of amqp091.
 Package rabbitmq implements the broker abstraction over RabbitMQ using github.com/wagslane/go-rabbitmq, which adds automatic reconnection and publisher confirms on top of amqp091.
Package closer manages graceful shutdown of resources.
 Package closer manages graceful shutdown of resources.
cmd
servicekit command
Command servicekit scaffolds and manages servicekit-based services.
 Command servicekit scaffolds and manages servicekit-based services.
Package config standardizes 12-factor configuration on jessevdk/go-flags: a single options struct whose fields carry `long`/`env`/`default`/`description` tags, so the same definition drives CLI flags, environment variables, and `--help`.
 Package config standardizes 12-factor configuration on jessevdk/go-flags: a single options struct whose fields carry `long`/`env`/`default`/`description` tags, so the same definition drives CLI flags, environment variables, and `--help`.
Package dbtest spins up disposable databases for integration tests using testcontainers.
 Package dbtest spins up disposable databases for integration tests using testcontainers.
Package debugsrv serves operational endpoints — net/http/pprof profiles plus optional metrics and health handlers — on a separate, internal-only listener.
 Package debugsrv serves operational endpoints — net/http/pprof profiles plus optional metrics and health handlers — on a separate, internal-only listener.
Package delegate is an in-process, synchronous event bus for decoupling domain packages that cannot import one another (avoiding import cycles).
 Package delegate is an in-process, synchronous event bus for decoupling domain packages that cannot import one another (avoiding import cycles).
Package dim is a slim, generics-based dependency-injection toolkit: lazy providers and managed resources built from plain functions, with structured logging of initialization and cleanup.
 Package dim is a slim, generics-based dependency-injection toolkit: lazy providers and managed resources built from plain functions, with structured logging of initialization and cleanup.
Package errs provides a single, transport-agnostic error type with stable codes, an HTTP (and later gRPC) status mapping, request validation helpers, and automatic redaction of secrets in user-facing messages.
 Package errs provides a single, transport-agnostic error type with stable codes, an HTTP (and later gRPC) status mapping, request validation helpers, and automatic redaction of secrets in user-facing messages.
Package grpcserver bootstraps a gRPC server with the same cross-cutting concerns as the REST stack — panic recovery, trace-id injection, structured access logging, Prometheus metrics, and errs->status mapping — and exposes it as a worker.Runnable so it can be supervised alongside the HTTP server and background workers.
 Package grpcserver bootstraps a gRPC server with the same cross-cutting concerns as the REST stack — panic recovery, trace-id injection, structured access logging, Prometheus metrics, and errs->status mapping — and exposes it as a worker.Runnable so it can be supervised alongside the HTTP server and background workers.
Package health provides liveness and readiness HTTP handlers suitable for Kubernetes probes.
 Package health provides liveness and readiness HTTP handlers suitable for Kubernetes probes.
Package httpmw provides outbound HTTP-client middleware: composable http.RoundTripper wrappers for resilient calls to upstream services.
 Package httpmw provides outbound HTTP-client middleware: composable http.RoundTripper wrappers for resilient calls to upstream services.
Package i18n is a thin wrapper over nicksnyder/go-i18n that loads message catalogs from an fs.FS (typically an embed.FS), resolves the request language from an Accept-Language header, and localizes *errs.Error messages by their MessageID/Code with Args as template data.
 Package i18n is a thin wrapper over nicksnyder/go-i18n that loads message catalogs from an fs.FS (typically an embed.FS), resolves the request language from an Accept-Language header, and localizes *errs.Error messages by their MessageID/Code with Args as template data.
Package logger provides a thin wrapper around log/slog that:
 Package logger provides a thin wrapper around log/slog that:
Package metrics exposes a Prometheus registry, an HTTP handler to scrape it, and a middleware that records request count and latency.
 Package metrics exposes a Prometheus registry, an HTTP handler to scrape it, and a middleware that records request count and latency.
Package migrate applies SQL schema migrations from an embedded filesystem.
 Package migrate applies SQL schema migrations from an embedded filesystem.
Package otel bootstraps OpenTelemetry tracing and provides helpers to inject a trace id into the context (and therefore into logs), open child spans, and propagate trace context across service boundaries.
 Package otel bootstraps OpenTelemetry tracing and provides helpers to inject a trace id into the context (and therefore into logs), open child spans, and propagate trace context across service boundaries.
Package outbox implements the transactional outbox pattern for reliably publishing domain events to a message broker.
 Package outbox implements the transactional outbox pattern for reliably publishing domain events to a message broker.
pkg
httplog
Package poller maintains a single "current value" that is refreshed by periodically calling a Getter.
 Package poller maintains a single "current value" that is refreshed by periodically calling a Getter.
Package queue is a durable work queue with at-least-once delivery, safe for concurrent consumers across processes.
 Package queue is a durable work queue with at-least-once delivery, safe for concurrent consumers across processes.
Package safetick provides panic recovery helpers for long-running background loops and message-consumer callbacks, so a single bad tick or message cannot crash the worker.
 Package safetick provides panic recovery helpers for long-running background loops and message-consumer callbacks, so a single bad tick or message cannot crash the worker.
Package sqldb provides Postgres helpers on top of jmoiron/sqlx and the pgx driver: connection setup, named query/exec wrappers with query logging, transactions, and bulk insert/upsert/update
 Package sqldb provides Postgres helpers on top of jmoiron/sqlx and the pgx driver: connection setup, named query/exec wrappers with query logging, transactions, and bulk insert/upsert/update
web
mid
Package mid provides standard net/http middleware for servicekit services: panic recovery, trace-context injection, access logging, request timeouts, and body size limits.
 Package mid provides standard net/http middleware for servicekit services: panic recovery, trace-context injection, access logging, request timeouts, and body size limits.
rest
Package rest is a tiny typed-handler layer over net/http.
 Package rest is a tiny typed-handler layer over net/http.
router
Package router wraps the standard library's net/http.ServeMux (Go 1.22+ method+path patterns) with go-pkgz/routegroup for nestable groups and per-group middleware, and bridges the typed rest.HandlerFunc into the mux.
 Package router wraps the standard library's net/http.ServeMux (Go 1.22+ method+path patterns) with go-pkgz/routegroup for nestable groups and per-group middleware, and bridges the typed rest.HandlerFunc into the mux.
Package worker unifies the background-execution patterns found across the reference services into one small vocabulary:
 Package worker unifies the background-execution patterns found across the reference services into one small vocabulary:

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.