worker

package
v0.12.14-0...-417a5dc Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 24, 2026 License: AGPL-3.0 Imports: 50 Imported by: 0

Documentation

Overview

Package worker implements the Restate workflow worker for Unkey's control plane.

The worker is the execution engine for long-running operations in Unkey's infrastructure, handling container builds, deployments, certificate management, and routing configuration through the Restate distributed workflow engine. It provides durable execution guarantees for operations that span multiple services and may take minutes to complete.

Architecture

The worker acts as a Restate service host, binding workflow services that handle container deployments, TLS certificate management, traffic routing, and versioning. It maintains connections to the primary database for persistent state, vault services for secrets and ACME certificates, S3-compatible storage for build artifacts, and ClickHouse for analytics.

Configuration

Configuration is loaded from a TOML file using config.Load:

cfg, err := config.Load[worker.Config]("/etc/unkey/unkey.toml")

The worker validates settings through struct tags and Config.Validate.

Usage

The worker is started with Run, which blocks until the context is cancelled or a fatal error occurs:

cfg, err := config.Load[worker.Config]("unkey.toml")
if err != nil {
    log.Fatal(err)
}
cfg.Clock = clock.New()

if err := worker.Run(ctx, cfg); err != nil {
    log.Fatal(err)
}

Startup Sequence

Run performs initialization in a specific order: configuration validation, vault services creation, database connection, build storage initialization, ACME provider setup, Restate server binding, admin registration with retry, wildcard certificate bootstrapping, health endpoint startup, and optional Prometheus metrics exposure.

Graceful Shutdown

When the context passed to Run is cancelled, the worker performs graceful shutdown by stopping the health server, closing database connections, and allowing in-flight Restate workflows to complete. The shutdown sequence is managed through a shutdown handler that reverses the startup order.

ACME Certificate Management

When ACME is enabled in configuration, the worker automatically manages TLS certificates using Let's Encrypt. It supports HTTP-01 challenges for regular domains and DNS-01 challenges (via Route53) for wildcard certificates. On startup with a configured default domain, Run calls [bootstrapWildcardDomain] to ensure the platform's wildcard certificate can be automatically renewed.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Run

func Run(ctx context.Context, cfg Config) error

Run starts the Restate worker service with the provided configuration.

This function initializes all required services and starts the Restate server for workflow execution. It performs these major initialization steps:

  1. Validates configuration and initializes structured logging
  2. Creates vault services for secrets and ACME certificates
  3. Initializes database and build storage
  4. Creates build service (docker/depot backend)
  5. Initializes ACME caches and providers (HTTP-01, DNS-01)
  6. Starts Restate server with workflow service bindings
  7. Registers with Restate admin API for service discovery
  8. Starts health check endpoint
  9. Optionally starts Prometheus metrics server

The worker handles graceful shutdown when context is cancelled, properly closing all services and database connections.

Returns an error if configuration validation fails, service initialization fails, or during server startup. Context cancellation results in clean shutdown with nil error.

Types

type AcmeConfig

type AcmeConfig struct {
	// Enabled determines whether ACME certificate management is active.
	// When true, certificates are automatically obtained and renewed.
	Enabled bool `toml:"enabled"`

	// EmailDomain is the domain used for ACME account emails.
	// Used for Let's Encrypt account registration and recovery.
	// Example: "unkey.com" creates "admin@unkey.com" for ACME account.
	EmailDomain string `toml:"email_domain" config:"default=unkey.com"`

	// Route53 configures DNS-01 challenges through AWS Route53 API.
	// Enables wildcard certificates for domains hosted on Route53.
	Route53 Route53Config `toml:"route53"`
}

AcmeConfig holds configuration for ACME TLS certificate management.

This configuration enables automatic certificate issuance and renewal through ACME protocol with support for multiple DNS providers.

type BuildPlatform

type BuildPlatform struct {
	// Platform is the original build platform string.
	// Example: "linux/amd64".
	Platform string

	// Architecture is the CPU architecture component.
	// Example: "amd64", "arm64".
	Architecture string
}

BuildPlatform represents parsed container build platform specification.

Contains the validated platform string separated into OS and architecture components for build backend integration.

type ClickHouseConfig

type ClickHouseConfig struct {
	// URL is the ClickHouse database connection string.
	// Used for analytics and operational metrics storage.
	URL string `toml:"url"`

	// AdminURL is the connection string for the ClickHouse admin user.
	// Used by ClickhouseUserService to create/configure workspace users.
	// Optional - if not set, ClickhouseUserService will not be enabled.
	AdminURL string `toml:"admin_url"`
}

ClickHouseConfig holds ClickHouse connection configuration.

type Config

type Config struct {
	// InstanceID is the unique identifier for this worker instance.
	// Used for logging, tracing, and cluster coordination.
	InstanceID string `toml:"instance_id"`

	// Region is the geographic region where this worker instance is running.
	// Used for logging and tracing context.
	Region string `toml:"region"`

	// Observability configures tracing, logging, and metrics. See [config.Observability].
	Observability config.Observability `toml:"observability"`

	// DefaultDomain is the fallback domain for system operations.
	// Used for sentinel deployment and automatic certificate bootstrapping.
	DefaultDomain string `toml:"default_domain" config:"default=unkey.app"`

	// DashboardURL is the base URL of the dashboard, used for constructing links
	// in GitHub PR comments and deployment status log URLs.
	DashboardURL string `toml:"dashboard_url" config:"default=https://app.unkey.com"`

	// BuildPlatformStr defines the target architecture for container builds.
	// Format: "linux/amd64", "linux/arm64". Only "linux" OS supported.
	BuildPlatformStr string `toml:"build_platform" config:"default=linux/amd64"`

	// SentinelImage is the container image used for new sentinel deployments.
	// Overrides default sentinel image with custom build or registry.
	SentinelImage string `toml:"sentinel_image" config:"default=ghcr.io/unkeyed/unkey:local"`

	// CnameDomain is the base domain for custom domain CNAME targets.
	// Each custom domain gets a unique subdomain like "{random}.{CnameDomain}".
	CnameDomain string `toml:"cname_domain" config:"required,nonempty"`

	// Database configures MySQL connections. See [config.DatabaseConfig].
	Database config.DatabaseConfig `toml:"database"`

	// Vault configures the encryption/decryption service. See [config.VaultConfig].
	Vault config.VaultConfig `toml:"vault"`

	// Acme configures automatic TLS certificate management.
	// Enables Let's Encrypt integration for domain certificates.
	Acme AcmeConfig `toml:"acme"`

	// Restate configures workflow engine integration.
	// Enables asynchronous deployment and certificate renewal workflows.
	Restate RestateConfig `toml:"restate"`

	// Depot configures Depot.dev build service integration.
	Depot DepotConfig `toml:"depot"`

	// Registry configures container registry authentication.
	Registry RegistryConfig `toml:"registry"`

	// ClickHouse configures ClickHouse connections.
	ClickHouse ClickHouseConfig `toml:"clickhouse"`

	// GitHub configures GitHub App integration for webhook-triggered deployments.
	GitHub *GitHubConfig `toml:"github"`

	// Heartbeat configures Checkly heartbeat URLs for health monitoring.
	Heartbeat HeartbeatConfig `toml:"heartbeat"`

	// Slack configures Slack webhook URLs for notifications.
	Slack SlackConfig `toml:"slack"`

	// Clock provides time operations for testing and scheduling.
	// Use clock.New() for production deployments.
	Clock clock.Clock `toml:"-"`
}

Config holds the complete configuration for the Restate worker service. It is designed to be loaded from a TOML file using config.Load:

cfg, err := config.Load[worker.Config]("/etc/unkey/unkey.toml")

Environment variables are expanded in file values using ${VAR} syntax before parsing. Struct tag defaults are applied to any field left at its zero value after parsing, and validation runs automatically via Config.Validate.

Clock is runtime-only and cannot be set through a config file. It is tagged toml:"-" and must be set programmatically after loading.

func (Config) GetBuildPlatform

func (c Config) GetBuildPlatform() (BuildPlatform, error)

GetBuildPlatform returns the parsed build platform.

This method returns the parsed BuildPlatform from the configured BuildPlatformStr string.

Returns BuildPlatform with parsed platform and architecture components, or an error if the platform string is invalid.

func (Config) GetDepotConfig

func (c Config) GetDepotConfig() DepotConfig

GetDepotConfig returns the depot configuration.

This method returns the DepotConfig from the main Config struct. Should only be called after Validate() succeeds to ensure depot configuration is complete and valid.

Returns the DepotConfig containing API URL and project region.

func (Config) GetRegistryConfig

func (c Config) GetRegistryConfig() RegistryConfig

GetRegistryConfig returns the registry configuration.

This method returns the RegistryConfig from the main Config struct. Should only be called after Validate() succeeds to ensure all required fields are present.

Returns RegistryConfig with URL, username, and password for container registry access.

func (*Config) Validate

func (c *Config) Validate() error

Validate checks the configuration for required fields and logical consistency.

This method performs comprehensive validation of all configuration sections including build backend, ACME providers, database connections, and required credentials. It ensures that conditional configuration (like ACME providers) has all necessary dependencies.

Returns an error if required fields are missing, invalid, or inconsistent. Provides detailed error messages to help identify configuration issues.

type DepotConfig

type DepotConfig struct {
	// APIUrl is the Depot API endpoint URL for build operations.
	// Example: "https://api.depot.dev".
	APIUrl string `toml:"api_url"`

	// ProjectRegion is the geographic region for build storage.
	// Affects build performance and data residency.
	// Options: "us-east-1", "eu-central-1". Default: "us-east-1".
	ProjectRegion string `toml:"project_region" config:"default=us-east-1"`
}

DepotConfig holds configuration for Depot.dev build service integration.

This configuration enables cloud-native container builds through Depot's managed build infrastructure with optimized caching.

type GitHubConfig

type GitHubConfig struct {
	// AppID is the GitHub App ID for authentication.
	AppID int64 `toml:"app_id"`

	// PrivateKeyPEM is the GitHub App private key in PEM format.
	PrivateKeyPEM string `toml:"private_key_pem"`

	// AllowUnauthenticatedDeployments controls whether deployments can skip
	// GitHub authentication. Set to true only for local development.
	// Production should keep this false to require GitHub App authentication.
	AllowUnauthenticatedDeployments bool `toml:"allow_unauthenticated_deployments"`
}

GitHubConfig holds configuration for GitHub App integration.

type HeartbeatConfig

type HeartbeatConfig struct {
	// CertRenewalURL is the Checkly heartbeat URL for certificate renewal.
	// When set, a heartbeat is sent after successful certificate renewal runs.
	// Optional - if empty, no heartbeat is sent.
	CertRenewalURL string `toml:"cert_renewal_url"`

	// QuotaCheckURL is the Checkly heartbeat URL for quota checks.
	// When set, a heartbeat is sent after successful quota check runs.
	// Optional - if empty, no heartbeat is sent.
	QuotaCheckURL string `toml:"quota_check_url"`

	// KeyRefillURL is the Checkly heartbeat URL for key refill runs.
	// When set, a heartbeat is sent after successful key refill runs.
	// Optional - if empty, no heartbeat is sent.
	KeyRefillURL string `toml:"key_refill_url"`

	// KeyLastUsedSyncURL is the Checkly heartbeat URL for key last-used sync runs.
	// When set, a heartbeat is sent after successful sync runs.
	// Optional - if empty, no heartbeat is sent.
	KeyLastUsedSyncURL string `toml:"key_last_used_sync_url"`
}

HeartbeatConfig holds Checkly heartbeat URLs for health monitoring.

type RegistryConfig

type RegistryConfig struct {
	// URL is the container registry endpoint URL.
	// Example: "registry.depot.dev" or "https://registry.example.com".
	URL string `toml:"url"`

	// Username is the registry authentication username.
	// Common values: "x-token" for token-based auth, or actual username.
	Username string `toml:"username"`

	// Password is the registry password or authentication token.
	// Should be stored securely and rotated regularly.
	Password string `toml:"password"`
}

RegistryConfig holds container registry authentication configuration.

This configuration provides credentials for accessing container registries used by build backends for pushing and pulling images.

type RestateConfig

type RestateConfig struct {
	// AdminURL is the Restate admin endpoint URL for service registration.
	// Used by the worker to register its workflow services.
	// Example: "http://restate:9070".
	AdminURL string `toml:"admin_url" config:"default=http://restate:9070"`

	// APIKey is the optional authentication key for Restate admin API requests.
	// If set, this key will be sent with all requests to the Restate admin API.
	APIKey string `toml:"api_key"`

	// HttpPort is the port where the worker listens for Restate requests.
	// This is the internal Restate server port, not the health check port.
	HttpPort int `toml:"http_port" config:"default=9080,min=1,max=65535"`

	// RegisterAs is the service URL used for self-registration with Restate.
	// Allows Restate to discover and invoke this worker's services.
	// Example: "http://worker:9080".
	RegisterAs string `toml:"register_as"`
}

RestateConfig holds configuration for Restate workflow engine integration.

This configuration enables asynchronous workflow execution through the Restate distributed system for deployment and certificate operations.

type Route53Config

type Route53Config struct {
	// Enabled determines whether Route53 DNS-01 challenges are used.
	// When true, wildcard certificates can be automatically obtained.
	Enabled bool `toml:"enabled"`

	// AccessKeyID is the AWS access key ID for Route53 API access.
	AccessKeyID string `toml:"access_key_id"`

	// SecretAccessKey is the AWS secret access key for Route53 API access.
	SecretAccessKey string `toml:"secret_access_key"`

	// Region is the AWS region where Route53 hosted zones are located.
	// Example: "us-east-1", "us-west-2".
	Region string `toml:"region" config:"default=us-east-1"`

	// HostedZoneID overrides automatic zone discovery.
	// Required when domains have complex CNAME setups that confuse
	// automatic zone lookup (e.g., wildcard CNAMEs to load balancers).
	HostedZoneID string `toml:"hosted_zone_id"`
}

Route53Config holds AWS Route53 configuration for ACME DNS-01 challenges.

This configuration enables automatic DNS record creation for wildcard TLS certificates through AWS Route53 DNS API.

type SlackConfig

type SlackConfig struct {
	// QuotaCheckWebhookURL is the Slack webhook URL for quota exceeded notifications.
	// When set, Slack notifications are sent when workspaces exceed their quota.
	// Optional - if empty, no Slack notifications are sent.
	QuotaCheckWebhookURL string `toml:"quota_check_webhook_url"`
}

SlackConfig holds Slack webhook URLs for notifications.

Directories

Path Synopsis
Package certificate implements ACME certificate workflows for SSL/TLS provisioning.
Package certificate implements ACME certificate workflows for SSL/TLS provisioning.
Package clickhouseuser implements ClickHouse user provisioning workflows.
Package clickhouseuser implements ClickHouse user provisioning workflows.
Package customdomain implements domain ownership verification workflows.
Package customdomain implements domain ownership verification workflows.
Package deploy orchestrates the deployment lifecycle for user applications.
Package deploy orchestrates the deployment lifecycle for user applications.
Package deployment provides a Restate virtual object that serialises all mutations targeting a single deployment.
Package deployment provides a Restate virtual object that serialises all mutations targeting a single deployment.
Package github provides a GitHub App client for repository access.
Package github provides a GitHub App client for repository access.
Package keyrefill implements a Restate workflow for refilling API key usage limits.
Package keyrefill implements a Restate workflow for refilling API key usage limits.
Package openapi provides a Restate virtual object that scrapes OpenAPI specifications from running user deployments.
Package openapi provides a Restate virtual object that scrapes OpenAPI specifications from running user deployments.
Package quotacheck implements a Restate workflow for monitoring workspace quota usage.
Package quotacheck implements a Restate workflow for monitoring workspace quota usage.
Package routing manages frontline route assignment for deployments.
Package routing manages frontline route assignment for deployments.
Package versioning provides per-region version counters for state synchronization.
Package versioning provides per-region version counters for state synchronization.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL