servicekit

module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 22, 2026 License: MIT

README

servicekit

A Go SDK for building production services that speak REST and/or gRPC, backed by Postgres, with first-class observability (slog + OpenTelemetry traces + Prometheus metrics), reliable background workers, and a transactional outbox.

It distills patterns from several reference services into a single, importable module (github.com/assanoff/servicekit) plus a runnable example/ application that doubles as the scaffolding template for the servicekit CLI.

Layout

Package Purpose
errs Error type with stable codes, REST/gRPC mapping, validation, secret sanitization
logger slog-based logger: trace_id injection, level fan-out (stdout + Sentry), access log
otel Tracing bootstrap, span helpers, GetTraceID, route-excluding sampler
sqldb sqlx/pgx helpers: named queries, WithinTran, bulk insert/upsert/update, retries
dim Slim dependency-injection: lazy Provider[T], Once/OnceWithName, NewResource with init/cleanup logging
closer Graceful-shutdown registry: process-global default (Add/CloseSync) + instances (New/NewWithWait), LIFO
config 12-factor config via go-flags: one struct → CLI flags + env vars + --help, dotenv for local, subcommand dispatch
web/{rest,router,mid} Typed HTTP handlers over stdlib ServeMux + routegroup, middleware
grpcserver gRPC server as a worker.Runnable: recovery/trace/logging/metrics interceptors, errs→status mapping, health, reflection, transport tuning knobs
worker Unified worker abstraction: Runnable, Group, Loop, Pool, Processor[T], Backoff
queue Durable work queue (Schedule/Claim/MarkDone/MarkFailed): Postgres (FOR UPDATE SKIP LOCKED) + in-memory, plugs into worker.Processor as a Source/Sink
poller Typed cache of a periodically-refreshed value (Poller[T]), a worker.Runnable
broker Transport-agnostic Publisher/Consumer abstraction; CloudEvents v1.0 envelope; broker/rabbitmq implementation (confirm-mode publisher, worker.Runnable consumer)
outbox Transactional outbox: Event + Store + WithinTran (atomic domain-write + event), with Relay/Sweeper/Cleaner workers built on worker.Processor/Loop
i18n go-i18n wrapper: catalogs from fs.FS, Accept-Language matching + middleware, localizes *errs.Error by MessageID/Code with Args
auth Principal in context, credential extraction, pluggable Verifier, built-in JWT (HMAC/RSA/EC + JWKS via Keyfunc), Authenticate/Optional/RequireRole middleware
metrics Prometheus registry + HTTP middleware (gRPC collectors registered by grpcserver)
health Liveness/readiness handlers for Kubernetes probes
safetick Panic recovery for worker ticks and consumer callbacks
pkg/httplog Vendored go-chi/httplog v3 (MIT), driven by our logger via Middleware(*logger.Logger, *Options)
cmd/servicekit Scaffolding CLI: servicekit new <module> (embedded starter, or gonew passthrough with --template)

(Further packages — httpmw, migrate, dbtest, apitest — and the CLI's per-model CRUD generators land in later milestones.)

i18n & auth

  • i18n.Translator loads JSON catalogs from an embed.FS, resolves the request language from Accept-Language, and localizes error responses: an app middleware translates any *errs.Error by its MessageID (or Code) with Args as template data. The example localizes widget errors to en/ru.
  • auth verifies bearer tokens via a pluggable Verifier (built-in JWT covers HMAC, RSA/EC PEM, and JWKS through a custom Keyfunc) and enforces RBAC with Authenticate + RequireRole. The example protects widget writes behind a widget:write role while keeping reads public.

Scaffolding

go run ./cmd/servicekit new github.com/you/svc          # embedded starter
go run ./cmd/servicekit new github.com/you/svc --template github.com/some/tmpl  # via gonew

Messaging & transactional outbox

broker + outbox give reliable, exactly-the-domain-write event publishing:

  • outbox.WithinTran writes a domain change and the events it emits in ONE transaction — an event is persisted iff its domain write commits.
  • outbox.Relay (a worker.Processor) drains pending events to a broker.Publisher with at-least-once delivery; Sweeper reclaims leases from crashed relays and Cleaner prunes terminal rows.
  • broker/rabbitmq publishes CloudEvents v1.0 with publisher confirms and consumes them via a supervised worker.Runnable.

The example wires it end-to-end: creating a widget emits widget.created through the outbox, the relay publishes it to RabbitMQ, and a consumer records it (see example/core/widget, widgetaudit, and the e2e test).

Reliable background processing

worker, queue, and sqldb compose into an at-least-once batch pipeline:

  • worker.Pool — bounded-concurrency fan-out of one-shot jobs.
  • worker.Processor[T] — a Source (claim) → Handler (process) → Sink (ack/retry) loop, run on a schedule via worker.Loop.
  • queue.PG — a Postgres queue that hands each ready task to exactly one consumer via FOR UPDATE SKIP LOCKED, so N replicas drain it safely; it satisfies worker.Source[Task]/Sink[Task] directly.

The example wires this end-to-end: POST /widgets/import enqueues a batch, and a supervised import worker bulk-inserts it (idempotently, via sqldb.BulkInsert + ON CONFLICT DO NOTHING). See example/core/widgetimport.

gRPC

grpcserver runs alongside (or instead of) the REST server — both are worker.Runnables supervised by worker.Group, toggled independently by config. It ships:

  • the same cross-cutting interceptors as REST (panic recovery, trace-id injection, structured access logs, Prometheus metrics) plus automatic *errs.Error → gRPC status mapping (error codes are aligned, so the mapping is a direct cast);
  • messages use the protobuf Opaque API via Protobuf Editions (edition = "2023", api_level = API_OPAQUE) — Google's official path to faster, lower-allocation, lazy-decoding generated code (replaces the third-party vtprotobuf; no extra dependency or codec);
  • transport tuning (MaxRecvMsgSize, SharedWriteBuffer, NumStreamWorkers, keepalive);
  • gRPC health service and optional reflection.

Code is generated with buf v2 (config in example/). gRPC v1.81 / protobuf v1.36.

make proto-tools   # install buf, protoc-gen-go, protoc-gen-go-grpc
make proto         # buf lint + generate (example/proto -> example/gen)

Quick start

make build      # build SDK + example
make test       # unit tests
cd example && go run ./cmd serve

See example/ for a full application wired with dim.

Directories

Path Synopsis
Package auth provides authentication and authorization building blocks: a transport-neutral Principal carried in context, credential extraction from HTTP requests, a pluggable Verifier (token -> Principal), a built-in JWT verifier, and net/http middleware for authentication and role-based access control.
 Package auth provides authentication and authorization building blocks: a transport-neutral Principal carried in context, credential extraction from HTTP requests, a pluggable Verifier (token -> Principal), a built-in JWT verifier, and net/http middleware for authentication and role-based access control.
Package broker abstracts an event broker behind transport-agnostic Publisher and Consumer contracts.
 Package broker abstracts an event broker behind transport-agnostic Publisher and Consumer contracts.
rabbitmq
Package rabbitmq implements the broker abstraction over RabbitMQ using github.com/wagslane/go-rabbitmq, which adds automatic reconnection and publisher confirms on top of amqp091.
 Package rabbitmq implements the broker abstraction over RabbitMQ using github.com/wagslane/go-rabbitmq, which adds automatic reconnection and publisher confirms on top of amqp091.
Package closer manages graceful shutdown of resources.
 Package closer manages graceful shutdown of resources.
cmd
servicekit command
Command servicekit scaffolds and manages servicekit-based services.
 Command servicekit scaffolds and manages servicekit-based services.
Package config standardizes 12-factor configuration on jessevdk/go-flags: a single options struct whose fields carry `long`/`env`/`default`/`description` tags, so the same definition drives CLI flags, environment variables, and `--help`.
 Package config standardizes 12-factor configuration on jessevdk/go-flags: a single options struct whose fields carry `long`/`env`/`default`/`description` tags, so the same definition drives CLI flags, environment variables, and `--help`.
Package dim is a slim, generics-based dependency-injection toolkit: lazy providers and managed resources built from plain functions, with structured logging of initialization and cleanup.
 Package dim is a slim, generics-based dependency-injection toolkit: lazy providers and managed resources built from plain functions, with structured logging of initialization and cleanup.
Package errs provides a single, transport-agnostic error type with stable codes, an HTTP (and later gRPC) status mapping, request validation helpers, and automatic redaction of secrets in user-facing messages.
 Package errs provides a single, transport-agnostic error type with stable codes, an HTTP (and later gRPC) status mapping, request validation helpers, and automatic redaction of secrets in user-facing messages.
Package grpcserver bootstraps a gRPC server with the same cross-cutting concerns as the REST stack — panic recovery, trace-id injection, structured access logging, Prometheus metrics, and errs->status mapping — and exposes it as a worker.Runnable so it can be supervised alongside the HTTP server and background workers.
 Package grpcserver bootstraps a gRPC server with the same cross-cutting concerns as the REST stack — panic recovery, trace-id injection, structured access logging, Prometheus metrics, and errs->status mapping — and exposes it as a worker.Runnable so it can be supervised alongside the HTTP server and background workers.
Package health provides liveness and readiness HTTP handlers suitable for Kubernetes probes.
 Package health provides liveness and readiness HTTP handlers suitable for Kubernetes probes.
Package i18n is a thin wrapper over nicksnyder/go-i18n that loads message catalogs from an fs.FS (typically an embed.FS), resolves the request language from an Accept-Language header, and localizes *errs.Error messages by their MessageID/Code with Args as template data.
 Package i18n is a thin wrapper over nicksnyder/go-i18n that loads message catalogs from an fs.FS (typically an embed.FS), resolves the request language from an Accept-Language header, and localizes *errs.Error messages by their MessageID/Code with Args as template data.
Package logger provides a thin wrapper around log/slog that:
 Package logger provides a thin wrapper around log/slog that:
Package metrics exposes a Prometheus registry, an HTTP handler to scrape it, and a middleware that records request count and latency.
 Package metrics exposes a Prometheus registry, an HTTP handler to scrape it, and a middleware that records request count and latency.
Package otel bootstraps OpenTelemetry tracing and provides helpers to inject a trace id into the context (and therefore into logs), open child spans, and propagate trace context across service boundaries.
 Package otel bootstraps OpenTelemetry tracing and provides helpers to inject a trace id into the context (and therefore into logs), open child spans, and propagate trace context across service boundaries.
Package outbox implements the transactional outbox pattern for reliably publishing domain events to a message broker.
 Package outbox implements the transactional outbox pattern for reliably publishing domain events to a message broker.
pkg
httplog
Package poller maintains a single "current value" that is refreshed by periodically calling a Getter.
 Package poller maintains a single "current value" that is refreshed by periodically calling a Getter.
Package queue is a durable work queue with at-least-once delivery, safe for concurrent consumers across processes.
 Package queue is a durable work queue with at-least-once delivery, safe for concurrent consumers across processes.
Package safetick provides panic recovery helpers for long-running background loops and message-consumer callbacks, so a single bad tick or message cannot crash the worker.
 Package safetick provides panic recovery helpers for long-running background loops and message-consumer callbacks, so a single bad tick or message cannot crash the worker.
Package sqldb provides Postgres helpers on top of jmoiron/sqlx and the pgx driver: connection setup, named query/exec wrappers with query logging, transactions, and bulk insert/upsert/update.
 Package sqldb provides Postgres helpers on top of jmoiron/sqlx and the pgx driver: connection setup, named query/exec wrappers with query logging, transactions, and bulk insert/upsert/update.
web
mid
Package mid provides standard net/http middleware for servicekit services: panic recovery, trace-context injection, access logging, request timeouts, and body size limits.
 Package mid provides standard net/http middleware for servicekit services: panic recovery, trace-context injection, access logging, request timeouts, and body size limits.
rest
Package rest is a tiny typed-handler layer over net/http.
 Package rest is a tiny typed-handler layer over net/http.
router
Package router wraps the standard library's net/http.ServeMux (Go 1.22+ method+path patterns) with go-pkgz/routegroup for nestable groups and per-group middleware, and bridges the typed rest.HandlerFunc into the mux.
 Package router wraps the standard library's net/http.ServeMux (Go 1.22+ method+path patterns) with go-pkgz/routegroup for nestable groups and per-group middleware, and bridges the typed rest.HandlerFunc into the mux.
Package worker unifies the background-execution patterns found across the reference services into one small vocabulary:
 Package worker unifies the background-execution patterns found across the reference services into one small vocabulary:

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.