amp-common

module
v0.0.0-...-622fb6d Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 2, 2026 License: MIT

README

amp-common

Coverage

This repository contains shared Go libraries and utilities used across Ampersand projects. It provides a collection of reusable packages for common functionality like actor models, object pooling, concurrent execution, environment variable parsing, telemetry, and more.

amp-common logo

Overview

amp-common is a Go library (not a standalone application) that provides shared utilities and packages. It uses Go 1.25.5 and is published as github.com/amp-labs/amp-common.

Prerequisites

  • Go 1.25.5+
  • SSH Key Setup: This project uses private GitHub repositories. You need to set up SSH authentication:

    1. Ensure you have an SSH key configured for GitHub access to private amp-labs repositories

    2. Configure git to use SSH for GitHub:

      git config --global url."git@github.com:".insteadOf "https://github.com/"

    3. Set the GOPRIVATE environment variable (add to your shell profile):

      export GOPRIVATE=github.com/amp-labs/*

Development

Testing
make test              # Run all tests
go test -v ./...       # Run tests with verbose output
go test -v -run TestName ./package-name  # Run a specific test
Linting and Formatting
make fix               # Run all formatters and linters with auto-fix
make fix/sort          # Same as fix but with sorted output
make lint              # Run linters without auto-fix
make format            # Alias for 'make fix'
make fix-markdown      # Fix markdown files

The linting stack includes:

  • wsl - Whitespace linter (allows cuddle declarations)
  • gci - Go import formatter
  • golangci-lint - Comprehensive Go linter (configured via .golangci.yml)

Core Packages

Concurrency & Actor Model

actor - Actor model implementation with message passing

  • Generic Actor[Request, Response] with concurrent message processing
  • Actors have mailboxes (channels) and process messages sequentially
  • Includes Prometheus metrics for monitoring
  • Methods: Send, SendCtx, Request, RequestCtx, Publish, PublishCtx
  • Graceful panic recovery

pool - Generic object pooling with lifecycle management

  • Thread-safe pool for any io.Closer objects
  • Dynamic growth, configurable idle cleanup
  • Prometheus metrics for monitoring
  • Uses channels and semaphores for concurrency control

simultaneously - Parallel execution utility

  • Do(maxConcurrent int, ...func(context.Context) error) - Run functions in parallel
  • Returns first error encountered, cancels remaining on error
  • Automatic panic recovery with stack traces
  • Semaphore-based concurrency limiting
Configuration & Environment

envutil - Type-safe environment variable parsing

  • Fluent API with Reader[T] type for chaining operations
  • Built-in support for: strings, ints, bools, durations, URLs, UUIDs, file paths, etc.
  • Options pattern: Default(), Required(), Validate(), etc.
  • Example: envutil.Int("PORT", envutil.Default(8080)).Value()

startup - Application initialization and environment configuration

  • Load environment variables from files specified in ENV_FILE
  • Semicolon-separated file paths support (e.g., /path/to/.env;/path/to/.env.local)
  • Configurable override behavior for existing environment variables
  • Functions: ConfigureEnvironment(), ConfigureEnvironmentFromFiles(), WithAllowOverride()
Observability

telemetry - OpenTelemetry tracing integration

  • Initialize(ctx, config) - Set up OTLP tracing
  • LoadConfigFromEnv() - Load config from environment variables
  • Auto-detects Kubernetes environments and uses cluster-local collector
  • Environment variables: OTEL_ENABLED, OTEL_SERVICE_NAME, OTEL_EXPORTER_OTLP_TRACES_ENDPOINT

logger - Structured logging utilities

  • Built on Go's slog package
  • Integrates with OpenTelemetry context
  • Optional OpenTelemetry integration via go.opentelemetry.io/contrib/bridges/otelslog
    • Enable with Options{EnableOtel: true} when configuring logging
    • When enabled, logs are sent to both console and OpenTelemetry
    • Disabled by default (opt-in feature)
    • Allows logs to be correlated with traces and exported via OTLP
CLI & Commands

cli - CLI utilities for terminal interaction

  • Banner/divider generation with Unicode box drawing
  • BannerAutoWidth(), DividerAutoWidth() - Auto-detect terminal size
  • Prompt utilities for user input
  • Set AMP_NO_BANNER=true to suppress banners

cmd - Command execution wrapper

  • Fluent API for building exec.Cmd instances
  • Methods: SetDir(), SetStdin(), SetStdout(), SetStderr(), AppendEnv(), etc.
  • Returns exit code and error separately
Data Structures & Collections
  • maps - Generic map utilities with red-black tree implementation
  • set - Generic set implementation with red-black tree backing
  • tuple - Generic tuple types
  • collectable - Interface combining Hashable and Comparable for use in Map/Set data structures
  • sortable - Sortable interface with LessThan comparison for ordering
Resource Management
  • closer - Resource management utilities for io.Closer (Closer collector, CloseOnce, HandlePanic, CustomCloser)
  • using - Resource management pattern (try-with-resources/using statement)
  • should - Utilities for cleanup operations (e.g., should.Close())
  • shutdown - Graceful shutdown coordination
Error Handling & Control Flow
  • retry - Flexible retry mechanism with exponential backoff, jitter, and retry budgets
  • errors - Error utilities with collection support
  • try - Result type for error handling (Try[T] with Value and Error)
  • validate - Validation interfaces (HasValidate, HasValidateWithContext) with panic recovery and Prometheus metrics
Data Processing & Transformation
  • jsonpath - JSONPath bracket notation utilities for field mapping
  • xform - Type transformations and conversions
  • hashing - Hashing utilities
  • sanitize - String sanitization
  • compare - Comparison utilities
  • zero - Zero value utilities for generic types (Value[T](), IsZero[T](value))
Async & Concurrency Utilities
  • future - Future/Promise implementation for async programming (Go, GoContext, Await, Map, Combine)
  • bgworker - Background worker management
  • lazy - Lazy initialization with thread-safety
Optional & Pointer Utilities
  • optional - Type-safe Optional/Maybe type (Some[T], None[T], Map, FlatMap)
  • pointer - Pointer utilities (To[T], Value[T])
Misc Utilities
  • utils - Misc utilities (channels, context, JSON, sleep, dedup)
  • channels - Channel utilities (CloseChannelIgnorePanic)
  • contexts - Context utilities (EnsureContext, IsContextAlive, WithValue[K,V], GetValue[K,V])
  • envtypes - Common environment variable types (HostPort, Path)
  • emoji - Emoji constants for terminal output and UI (Rocket, Fire, ThumbsUp, Warning, etc.)
  • stage - Environment detection (local, test, dev, staging, prod)
  • script - Script execution utilities
  • build - Build information utilities
  • http/transport - HTTP transport configuration with DNS caching
  • assert - Assertion utilities for testing
  • debug - Debugging utilities (for local development only, not for production use)

Dependency Management

This is a Go module (github.com/amp-labs/amp-common). When changes are pushed to main, Cloud Build automatically:

  1. Creates a PR in the server repository to update the amp-common dependency
  2. Closes old auto-update PRs
  3. Auto-merges the new PR

Linter Configuration

The .golangci.yml enables most linters but disables:

  • gochecknoinits - Allows init() functions
  • exhaustruct - Zero-valued fields are acceptable
  • testpackage - Not doing black-box testing
  • wrapcheck - Too noisy
  • funlen, cyclop, gocognit - Function complexity checks disabled
  • gochecknoglobals - Global variables allowed for legitimate use cases

Special rules:

  • Variable naming accepts both "Id" and "ID" (via revive)
  • Short variable names allowed within 15 lines (via varnamelen)

Testing Philosophy

  • Tests use github.com/stretchr/testify for assertions
  • Package debug is for local debugging only and should not be imported in production code

Prometheus Metrics

Many packages expose Prometheus metrics:

  • Actor: message counts, processing time, panics, queue depth
  • Pool: object counts, creation/close events, errors
  • Metrics use subsystem labels for multi-tenancy

Troubleshooting

SSH Authentication Issues

If you encounter errors like Permission denied (publickey) or module download failures:

  1. Verify your SSH key is added to GitHub and has access to amp-labs repositories

  2. Test GitHub SSH connection:

    ssh -T git@github.com

    You should see: Hi username! You've successfully authenticated...

  3. Verify Go environment:

    go env GOPRIVATE  # Should show: github.com/amp-labs/*

  4. Test module access:

    go list -m github.com/amp-labs/amp-common
Common Issues

Problem: Module download fails with authentication errors

  • Solution: Ensure git config --global url."git@github.com:".insteadOf "https://github.com/" is set

Problem: IDE shows "module not found" errors

  • Solution: Restart your IDE after setting environment variables, ensure SSH key is loaded

Problem: Tests fail to import private dependencies

  • Solution: Verify GOPRIVATE is set in your environment and SSH authentication is working

Directories

Path Synopsis
Package actor provides an implementation of the actor model for concurrent message processing.
 Package actor provides an implementation of the actor model for concurrent message processing.
Package assert provides type assertion utilities with error handling.
 Package assert provides type assertion utilities with error handling.
Package bgworker provides background worker management with graceful lifecycle control.
 Package bgworker provides background worker management with graceful lifecycle control.
Package build provides utilities for parsing build information that is embedded at compile time.
 Package build provides utilities for parsing build information that is embedded at compile time.
Package channels provides utilities for working with Go channels, including channel creation with flexible sizing and safe channel closing.
 Package channels provides utilities for working with Go channels, including channel creation with flexible sizing and safe channel closing.
Package cli provides terminal interaction utilities including banners, dividers, and user prompts.
 Package cli provides terminal interaction utilities including banners, dividers, and user prompts.
Package closer provides utilities for managing io.Closer resources.
 Package closer provides utilities for managing io.Closer resources.
Package cmd provides a fluent API for building and executing shell commands.
 Package cmd provides a fluent API for building and executing shell commands.
Package collectable provides interfaces for hashable and comparable types used in Map and Set data structures.
 Package collectable provides interfaces for hashable and comparable types used in Map and Set data structures.
Package compare provides utilities for comparing values.
 Package compare provides utilities for comparing values.
Package contexts provides context utilities including lifecycle management and type-safe value storage.
 Package contexts provides context utilities including lifecycle management and type-safe value storage.
Package debug provides debugging utilities for local development only (not for production use).
 Package debug provides debugging utilities for local development only (not for production use).
Package emoji provides emoji constants for terminal output and UI.
 Package emoji provides emoji constants for terminal output and UI.
Package empty provides utilities for creating empty values of various types.
 Package empty provides utilities for creating empty values of various types.
Package envtypes provides common types used for parsing environment variables.
 Package envtypes provides common types used for parsing environment variables.
Package envutil provides type-safe environment variable parsing with a fluent API.
 Package envutil provides type-safe environment variable parsing with a fluent API.
Package errors provides error utilities with collection support for managing multiple errors.
 Package errors provides error utilities with collection support for managing multiple errors.
Package future provides a Future/Promise implementation for asynchronous programming in Go.
 Package future provides a Future/Promise implementation for asynchronous programming in Go.
Package hashing provides cryptographic hash utilities and hashable types for use with Map and Set collections.
 Package hashing provides cryptographic hash utilities and hashable types for use with Map and Set collections.
http
httplogger
Package httplogger provides utilities for logging HTTP requests and responses with structured logging.
 Package httplogger provides utilities for logging HTTP requests and responses with structured logging.
printable
Package printable provides utilities for converting HTTP request/response bodies into human-readable or loggable formats.
 Package printable provides utilities for converting HTTP request/response bodies into human-readable or loggable formats.
redact
Package redact provides utilities for redacting sensitive information from HTTP headers and URL query parameters.
 Package redact provides utilities for redacting sensitive information from HTTP headers and URL query parameters.
transport
Package transport provides HTTP transport configuration with DNS caching and connection pooling.
 Package transport provides HTTP transport configuration with DNS caching and connection pooling.
Package jsonpath provides utilities for working with JSONPath bracket notation.
 Package jsonpath provides utilities for working with JSONPath bracket notation.
Package lazy provides utilities for lazy initialization of values with thread-safe guarantees.
 Package lazy provides utilities for lazy initialization of values with thread-safe guarantees.
Package logger provides structured logging utilities built on Go's slog package with OpenTelemetry integration.
 Package logger provides structured logging utilities built on Go's slog package with OpenTelemetry integration.
Package maps provides generic map utilities, including a case-insensitive map implementation.
 Package maps provides generic map utilities, including a case-insensitive map implementation.
Package optional provides a type-safe Optional type for representing values that may or may not be present.
 Package optional provides a type-safe Optional type for representing values that may or may not be present.
Package pointer provides utilities for working with pointers in Go.
 Package pointer provides utilities for working with pointers in Go.
Package pool provides generic object pooling with lifecycle management and Prometheus metrics.
 Package pool provides generic object pooling with lifecycle management and Prometheus metrics.
Package retry provides a flexible and configurable retry mechanism for operations that may fail transiently.
 Package retry provides a flexible and configurable retry mechanism for operations that may fail transiently.
Package sanitize contains functions to sanitize filenames.
 Package sanitize contains functions to sanitize filenames.
Package script provides utilities for running scripts with standardized logging, signal handling, and exit code management.
 Package script provides utilities for running scripts with standardized logging, signal handling, and exit code management.
Package set provides generic set implementations with support for ordered and thread-safe variants.
 Package set provides generic set implementations with support for ordered and thread-safe variants.
Package should provides utilities for cleanup operations that should succeed but may fail in practice.
 Package should provides utilities for cleanup operations that should succeed but may fail in practice.
Package shutdown provides utilities for graceful shutdown coordination.
 Package shutdown provides utilities for graceful shutdown coordination.
Package simultaneously provides utilities for running functions concurrently with controlled parallelism.
 Package simultaneously provides utilities for running functions concurrently with controlled parallelism.
Package sortable provides wrapper types for primitive types that implement the Sortable interface, enabling their use as keys in sorted data structures.
 Package sortable provides wrapper types for primitive types that implement the Sortable interface, enabling their use as keys in sorted data structures.
Package spans provides utilities for creating OpenTelemetry spans with a fluent API.
 Package spans provides utilities for creating OpenTelemetry spans with a fluent API.
Package stage provides utilities for detecting and working with deployment environments.
 Package stage provides utilities for detecting and working with deployment environments.
Package startup provides utilities for application initialization and environment configuration.
 Package startup provides utilities for application initialization and environment configuration.
Package statemachine provides a declarative state machine framework for orchestrating complex workflows.
 Package statemachine provides a declarative state machine framework for orchestrating complex workflows.
actions
Package actions provides composable actions for state machine workflows, including sampling, elicitation, validation, and error handling primitives.
 Package actions provides composable actions for state machine workflows, including sampling, elicitation, validation, and error handling primitives.
helpers
Package helpers provides utility functions for state machine operations.
 Package helpers provides utility functions for state machine operations.
testing
Package testing provides testing utilities for state machine workflows.
 Package testing provides testing utilities for state machine workflows.
validator
Package validator provides validation and auto-fixing for state machine configurations.
 Package validator provides validation and auto-fixing for state machine configurations.
visualizer
Package visualizer generates visual diagrams from state machine configurations.
 Package visualizer generates visual diagrams from state machine configurations.
Package telemetry provides OpenTelemetry tracing integration with automatic service discovery and configuration.
 Package telemetry provides OpenTelemetry tracing integration with automatic service discovery and configuration.
Package tests provides utilities for managing test context with unique identifiers and test metadata.
 Package tests provides utilities for managing test context with unique identifiers and test metadata.
Package try provides a Result type for error handling in Go.
 Package try provides a Result type for error handling in Go.
Package tuple provides generic tuple types for grouping multiple values together.
 Package tuple provides generic tuple types for grouping multiple values together.
Package using provides a resource management pattern similar to C#'s "using" statement or Java's try-with-resources.
 Package using provides a resource management pattern similar to C#'s "using" statement or Java's try-with-resources.
Package utils provides miscellaneous utility functions for channels, context, JSON, sleep, dedup, and more.
 Package utils provides miscellaneous utility functions for channels, context, JSON, sleep, dedup, and more.
Package validate provides a unified validation framework for types that implement validation interfaces.
 Package validate provides a unified validation framework for types that implement validation interfaces.
Package xform provides type-safe transformation functions for converting and validating data.
 Package xform provides type-safe transformation functions for converting and validating data.
Package zero provides utilities for working with zero values of generic types.
 Package zero provides utilities for working with zero values of generic types.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.