kit

package
v0.0.0-...-efeeb2b Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 24, 2026 License: Apache-2.0 Imports: 28 Imported by: 0

README

Kit Framework

Kit is a framework for building production grade scalable service applications that can run in Kubernetes.

The main goal of Kit is to provide a proven starting point that reduces the repetitive tasks required for a new project. It is designed to follow idiomatic code and Go best practices. Collectively, the project lays out everything logically to minimize guess work and enable engineers to quickly maintain a mental model for the project.

Starting point

A Kit service is a kit.Foundation, see sample/sampleapp for a complete service and make codegen to scaffold a new one.

func main() {
	if err := run(); err != nil {
		os.Exit(1)
	}
}

// run starts the service and blocks until it is stopped, the errors are logged before being returned.
func run() error {
	ctx := context.Background()

	// Initiate a logger with pre-configuration for production and telemetry.
	l, err := log.New()
	if err != nil {
		fmt.Fprintf(os.Stderr, "creating logger: %v\n", err)
		return err
	}
	defer l.Close()
	log.ReplaceGlobal(l)

	foundation, err := kit.NewFoundation("myservice", kit.WithLogger(l))
	if err != nil {
		l.Error(ctx, "creating foundation", log.Error(err))
		return err
	}

	// Serve blocks until the process receives an interrupt or terminate signal, the service is:
	// 1. instrumented: traces (OpenTelemetry), metrics (Prometheus) and logs with the trace ids
	// 2. probed: /healthz and /readyz endpoints, and profiled via /debug/pprof
	// 3. gracefully shut down
	if err := foundation.Serve(); err != nil {
		l.Error(ctx, "serving", log.Error(err))
		return err
	}
	return nil
}
Add a gRPC service and the gRPC gateway

The Foundation serves gRPC services and exposes them over HTTP with grpc-gateway. Define the protobuf API in the api folder, generate the code with make proto-gen, then register the service and its gateway handler:

foundation.RegisterService(func(s *grpc.Server) {
	pb.RegisterMyAppServiceServer(s, server)
})

// the handler is registered synchronously, the error is checked right after.
var gatewayErr error
foundation.RegisterServiceHandler(func(gw *runtime.ServeMux, conn *grpc.ClientConn) {
	gatewayErr = pb.RegisterMyAppServiceHandler(ctx, gw, conn)
})
if gatewayErr != nil {
	return gatewayErr
}

The requests are validated with their protovalidate rules, and the handlers return coded errors (see errors and application-error.md).

Configuration

The Foundation is configured with options, or environment variables (the options take precedence):

Option Environment variable Default
WithHTTPAddr FOUNDATION_HTTP_ADDRESS 0.0.0.0:8080
WithGRPCAddr FOUNDATION_GRPC_ADDRESS 0.0.0.0:8081
WithMetricsAddr FOUNDATION_METRICS_ADDRESS :9090
WithProbesAddr FOUNDATION_PROBES_ADDRESS :9091
WithLogger FOUNDATION_LOG_LEVEL (log) no-op logger
WithCORS, WithHTTPReadTimeout, WithHTTPWriteTimeout, WithGRPCServerOptions

The traces are exported to the OTLP collector set by OTEL_EXPORTER_OTLP_ENDPOINT (see telemetry).

Packages

Package Purpose
blob Object storage (MinIO / S3)
cache Key value cache (Redis)
config Configuration from YAML files (configmaps) and environment variables
errors Coded errors, generated from errors/errors.defs.yaml (make errgen)
grpc gRPC server and client: errors, validation, metrics, traces, retries
id Globally unique identifiers
namesgenerator Random names
log Structured logging with the trace ids
pagination Next page token pagination and list batching
password High-entropy random password generation
pubsub Publish and subscribe (GCP Pub/Sub, NATS, Kafka, MQTT)
retry Retries with backoff
sql PostgreSQL connection, migrations and test databases
telemetry OpenTelemetry tracer and meter providers, HTTP metrics
validator Struct validation
workerpool Bounded pool of goroutines

Tests

make test runs the unit tests, the tests needing a service (PostgreSQL, Redis, NATS, Pub/Sub emulator, Kafka, MQTT, MinIO) are skipped. make test-integration starts the services with docker compose and runs all the tests.

Documentation

Overview

Package kit provides the Foundation, the boilerplate of a production ready service: a gRPC server, an HTTP server with the grpc-gateway handlers and custom routes, the probes and metrics servers, the tracer and meter providers, and the graceful shutdown.

f, err := kit.NewFoundation("myservice", kit.WithLogger(l))
if err != nil {
	return err
}
f.RegisterService(func(s *grpc.Server) {
	pb.RegisterMyServiceServer(s, server)
})
f.RegisterServiceHandler(func(gw *runtime.ServeMux, conn *grpc.ClientConn) {
	gatewayErr = pb.RegisterMyServiceHandler(context.Background(), gw, conn)
})
return f.Serve()

The servers listen on 0.0.0.0:8080 (HTTP), 0.0.0.0:8081 (gRPC), :9090 (metrics) and :9091 (probes) by default, see the With* options and the FOUNDATION_* environment variables.

The sub packages are independent building blocks (errors, log, sql, pubsub, cache, blob...) usable without the Foundation.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Foundation

type Foundation struct {
	// contains filtered or unexported fields
}

Foundation provides a convenient way to build new services.

Foundation aims to provide a set of common boilerplate code for creating a production ready GRPC server and HTTP mux router (with grpc-gateway capabilities) and custom HTTP endpoints.

func NewFoundation

func NewFoundation(name string, optFns ...Option) (*Foundation, error)

NewFoundation creates a new foundation service. A list of configurable option can be passed as option and as env Variable eg:

// Setting up grpc server via option
kit.NewFoundation("myservice", kit.WithGRPCAddr("localhost:8089"))

// Setting up grpc server via Env
FOUNDATION_GRPC_ADDRESS=localhost:8089
kit.NewFoundation("myservice")

Order of priority for option is as follows:

1- Default configuration
2- Env variable
3- Options

func (*Foundation) RegisterHTTPHandler

func (f *Foundation) RegisterHTTPHandler(path string, fn http.HandlerFunc, methods ...string)

RegisterHTTPHandler registers a custom HTTP handler.

The path follows the http.ServeMux pattern syntax, path parameters can be read with r.PathValue and the handler is registered for each of the given methods (or for all methods if none given).

eg:

foundation.RegisterHTTPHandler("/users/{id}", func(w http.ResponseWriter, r *http.Request) {
	id := r.PathValue("id")
}, http.MethodGet)

func (*Foundation) RegisterLiveness

func (f *Foundation) RegisterLiveness(fn func() (string, error))

RegisterLiveness register a liveness function for /healthz

Many applications running for long periods of time eventually transition to broken states, and cannot recover except by being restarted. Kubernetes provides liveness probes to detect and remedy such situations.

func (*Foundation) RegisterReadiness

func (f *Foundation) RegisterReadiness(fn func() (string, error))

RegisterReadiness register a readiness function for /readyz

Sometimes, applications are temporarily unable to serve traffic. For example, an application might need to load a large amount of data or a large number of configuration files during startup. In such instances, we don’t want to kill the application, but we don’t want to send it requests either.

func (*Foundation) RegisterService

func (f *Foundation) RegisterService(fn RegisterServiceFunc)

RegisterService registers a grpc service handler.

func (*Foundation) RegisterServiceHandler

func (f *Foundation) RegisterServiceHandler(fn RegisterServiceHandlerFunc, muxOpts ...runtime.ServeMuxOption)

RegisterServiceHandler registers a grpc-gateway service handler.

func (*Foundation) Serve

func (f *Foundation) Serve() error

Serve configure and start serving request for the foundation service.

It blocks until the process receives an interrupt or terminate signal, or until one of the servers fails.

type Option

type Option func(*options)

Option defines a Foundation option.

func WithCORS

func WithCORS(opts cors.Options) Option

WithCORS enables CORS on the HTTP server with the given options.

func WithGRPCAddr

func WithGRPCAddr(addr string) Option

WithGRPCAddr defines the gRPC server host and port.

func WithGRPCServerOptions

func WithGRPCServerOptions(opts ...grpc.ServerOption) Option

WithGRPCServerOptions defines the gRPC server options.

func WithHTTPAddr

func WithHTTPAddr(addr string) Option

WithHTTPAddr defines the HTTP server host and port.

func WithHTTPReadTimeout

func WithHTTPReadTimeout(timeout time.Duration) Option

WithHTTPReadTimeout defines the read timeout of the HTTP server.

func WithHTTPWriteTimeout

func WithHTTPWriteTimeout(timeout time.Duration) Option

WithHTTPWriteTimeout defines the write timeout of the HTTP server.

func WithLogger

func WithLogger(logger *log.Logger) Option

WithLogger defines the logger of the foundation and of its servers, a nil logger is ignored.

func WithMetricsAddr

func WithMetricsAddr(addr string) Option

WithMetricsAddr defines the address of the HTTP server exposing the prometheus /metrics endpoint.

func WithProbesAddr

func WithProbesAddr(addr string) Option

WithProbesAddr defines the address of the HTTP server exposing the health probes and profiling endpoints.

type RegisterServiceFunc

type RegisterServiceFunc func(s *grpc.Server)

RegisterServiceFunc represents a function for registering a grpc service handler.

type RegisterServiceHandlerFunc

type RegisterServiceHandlerFunc func(gw *runtime.ServeMux, conn *grpc.ClientConn)

RegisterServiceHandlerFunc represents a function for registering a grpc gateway service handler.

Directories

Path Synopsis
Package blob provides interface that can be used to upload or fetch object from a Blob storage.
Package blob provides interface that can be used to upload or fetch object from a Blob storage.
minio
Package minio implements blob.Blob on top of MinIO, it works with any S3 compatible storage.
Package minio implements blob.Blob on top of MinIO, it works with any S3 compatible storage.
Package cache contains interfaces for caching data.
Package cache contains interfaces for caching data.
redis
Package redis implements cache.Cache on top of Redis.
Package redis implements cache.Cache on top of Redis.
Package config provides a simple logic that will read configuration from configmaps, io.Reader or envvar.
Package config provides a simple logic that will read configuration from configmaps, io.Reader or envvar.
Package errors provides simple error handling primitives.
Package errors provides simple error handling primitives.
Package grpc provides function for managing gRPC system calls (client and server).
Package grpc provides function for managing gRPC system calls (client and server).
Package id is a globally unique id generator suited for web scale.
Package id is a globally unique id generator suited for web scale.
Package log provides function for managing structured logs.
Package log provides function for managing structured logs.
Package namesgenerator generate random name.
Package namesgenerator generate random name.
Package pagination provides functions for managing next page token pagination as well as a batcher that will dynamically batch size a list query.
Package pagination provides functions for managing next page token pagination as well as a batcher that will dynamically batch size a list query.
v1
Package password provides a library for generating high-entropy random password strings via the crypto/rand package.
Package password provides a library for generating high-entropy random password strings via the crypto/rand package.
Package pubsub contains the interfaces to publish messages to topics and to subscribe to them, implemented by the backends:
Package pubsub contains the interfaces to publish messages to topics and to subscribe to them, implemented by the backends:
config
Package config creates the pubsub publishers and subscribers from a configuration, loaded with kit/config (YAML and environment variables).
Package config creates the pubsub publishers and subscribers from a configuration, loaded with kit/config (YAML and environment variables).
gcp
Package gcp implements the pubsub publisher and subscriber with Google Cloud Pub/Sub.
Package gcp implements the pubsub publisher and subscriber with Google Cloud Pub/Sub.
internal/tracing
Package tracing propagates the trace context through the pubsub messages and starts the publish and process spans of the backends, following the OpenTelemetry messaging semantic conventions (https://opentelemetry.io/docs/specs/semconv/messaging/messaging-spans/).
Package tracing propagates the trace context through the pubsub messages and starts the publish and process spans of the backends, following the OpenTelemetry messaging semantic conventions (https://opentelemetry.io/docs/specs/semconv/messaging/messaging-spans/).
internal/tracingtest
Package tracingtest provides helpers to test the trace propagation of the pubsub backends.
Package tracingtest provides helpers to test the trace propagation of the pubsub backends.
kafka
Package kafka implements the pubsub publisher and subscriber with Kafka (franz-go).
Package kafka implements the pubsub publisher and subscriber with Kafka (franz-go).
mqtt
Package mqtt implements the pubsub publisher and subscriber with MQTT v5 (paho).
Package mqtt implements the pubsub publisher and subscriber with MQTT v5 (paho).
nats
Package nats implements the pubsub publisher and subscriber with NATS JetStream.
Package nats implements the pubsub publisher and subscriber with NATS JetStream.
Package retry provides function for managing retry with backoff.
Package retry provides function for managing retry with backoff.
Package sql manages PostgreSQL connections instrumented with OpenTelemetry, the service migrations and the testing databases.
Package sql manages PostgreSQL connections instrumented with OpenTelemetry, the service migrations and the testing databases.
Package telemetry creates the OpenTelemetry providers of a service, set as the global providers.
Package telemetry creates the OpenTelemetry providers of a service, set as the global providers.
metric
Package metric registers and records prometheus metrics by name, the metrics are registered in the default prometheus registry.
Package metric registers and records prometheus metrics by name, the metrics are registered in the default prometheus registry.
Package validator implements value validations for structs and individual fields based on tags.
Package validator implements value validations for structs and individual fields based on tags.
Package workerpool concurrency limiting goroutine pool.
Package workerpool concurrency limiting goroutine pool.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL