app/

directory
v1.2.9 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 25, 2025 License: Apache-2.0

README

CloudZero Agent - Application Core

Overview

The app/ directory contains the complete application implementation for the CloudZero Agent for Kubernetes, organized using hexagonal architecture principles. This directory implements both the Application Core (domain logic) and the adapters (infrastructure) that enable cost allocation and monitoring for Kubernetes environments.

Architecture

The CloudZero Agent follows hexagonal (ports and adapters) architecture:

graph TD
    A["Primary Adapters"]
    B["Application Core"]
    C["Secondary Adapters"]
    A --> B
    B --> C

    subgraph "Primary Adapters (Input)"
        A1["HTTP Handlers"]
        A2["CLI Functions"]
        A3["Webhook Admission Controller"]
    end

    subgraph "Application Core"
        B1["Domain Services"]
        B2["Types & Interfaces"]
        B3["Business Logic"]
    end

    subgraph "Secondary Adapters (Output)"
        C1["Storage Systems"]
        C2["External APIs"]
        C3["File Systems"]
    end

    A1 --> B1
    A2 --> B1
    A3 --> B1
    B1 --> C1
    B1 --> C2
    B1 --> C3

Directory Structure

Core Components
  • types/ - Application Core type definitions, interfaces, and error constants
  • domain/ - Business logic and domain services implementing the core cost allocation pipeline
  • storage/ - Secondary adapters for persistent data storage (disk, SQLite)
Primary Adapters (Input Ports)
  • handlers/ - HTTP request handlers for Prometheus remote_write and API endpoints
  • functions/ - CLI applications for agent operations and management
  • http/ - HTTP infrastructure and middleware components
Supporting Infrastructure
  • config/ - Configuration management and validation systems
  • logging/ - Structured logging and instrumentation
  • utils/ - Utility packages for common operations
  • inspector/ - Agent diagnostics and monitoring tools

Data Flow

sequenceDiagram
    participant P as Prometheus
    participant H as HTTP Handler
    participant D as Domain Service
    participant S as Storage
    participant CZ as CloudZero API

    P->>H: remote_write request
    H->>D: PutMetrics(metrics)
    D->>D: Classify metrics
    D->>S: Store cost metrics
    D->>S: Store observability metrics
    S->>CZ: Upload via shipper

The CloudZero Agent processes metrics through this flow:

  1. Prometheus sends remote_write request to HTTP Handler
  2. Handler forwards metrics to Domain Service
  3. Domain Service classifies metrics (cost vs observability)
  4. Domain Service stores metrics in Storage
  5. Storage uploads data to CloudZero API via shipper

Key Services

Metric Collection Pipeline
  1. Collection - Receive Prometheus remote_write requests
  2. Classification - Separate cost vs observability metrics
  3. Storage - Buffer metrics to disk with compression
  4. Shipping - Upload processed data to CloudZero platform
Webhook System
  1. Admission Control - Intercept Kubernetes resource creation/updates
  2. Metadata Extraction - Capture labels and annotations
  3. Storage - Persist resource metadata for cost attribution
  4. Transmission - Send metadata to CloudZero for analysis

Integration Points

Prometheus Integration
  • Remote write protocol (v1 & v2)
  • Snappy compression support
  • Metric filtering and routing
Kubernetes Integration
  • Admission webhook for resource monitoring
  • Support for 15+ resource types
  • Label and annotation collection
CloudZero Platform Integration
  • Pre-signed S3 upload URLs
  • JSON and Parquet data formats
  • Cost allocation API endpoints

Development Guidelines

Adding New Components
  1. Identify Layer - Determine if component is a primary adapter, domain service, or secondary adapter
  2. Define Interfaces - Add necessary types to types/ package
  3. Implement Logic - Place business logic in appropriate domain services
  4. Create Tests - Ensure comprehensive test coverage
  5. Update Documentation - Maintain architectural documentation
Testing Strategy
  • Unit Tests - Test individual components in isolation
  • Integration Tests - Test adapter integration with external systems
  • Contract Tests - Validate interface contracts between layers
  • End-to-End Tests - Validate complete data flow

Configuration

The agent supports multiple configuration sources:

  • Environment variables
  • Configuration files (YAML/JSON)
  • Kubernetes ConfigMaps
  • CLI parameters

See config/ for detailed configuration options.

Monitoring

The agent provides comprehensive observability:

  • Prometheus metrics for collection statistics
  • Structured logging with zerolog
  • Health check endpoints
  • Storage usage monitoring

Security

  • TLS encryption for all external communication
  • Certificate management for webhook operations
  • Secret management for API credentials
  • Input validation and sanitization

Performance

  • Streaming JSON processing for large datasets
  • Brotli compression for efficient storage
  • Batch processing for optimal throughput
  • Memory-efficient metric handling

Limitations

  • Requires persistent storage for metric buffering
  • Webhook requires cluster admin permissions
  • Network connectivity required for CloudZero uploads
  • Memory usage scales with metric ingestion rate

Opportunities

  • Support for additional metric formats
  • Enhanced filtering capabilities
  • Real-time cost allocation alerts
  • Integration with additional monitoring systems

Directories

Path Synopsis
Package build contains build information for the application.
 Package build contains build information for the application.
Package compress provides functionality to compress a file into a tar.gz archive.
 Package compress provides functionality to compress a file into a tar.gz archive.
Package config contains code for all configs to share
 Package config contains code for all configs to share
gator
Package config implements the configuration for the aggregator..
 Package config implements the configuration for the aggregator..
validator
Package config contains configuration settings.
 Package config contains configuration settings.
webhook
Package config contains the configuration for the application.
 Package config contains the configuration for the application.
Package domain contains the Application Core business logic for the CloudZero Agent.
 Package domain contains the Application Core business logic for the CloudZero Agent.
certificate
Package certificate provides certificate generation and management functionality for the CloudZero Agent, including TLS certificate creation and Kubernetes webhook configuration management.
 Package certificate provides certificate generation and management functionality for the CloudZero Agent, including TLS certificate creation and Kubernetes webhook configuration management.
certificate/mocks
Package mocks is a generated GoMock package.
 Package mocks is a generated GoMock package.
diagnostic
Package diagnostic contains an interface to be implemented by diagnostics providers.
 Package diagnostic contains an interface to be implemented by diagnostics providers.
diagnostic/catalog
Package catalog contains the registry of diagnostics.
 Package catalog contains the registry of diagnostics.
diagnostic/cz
Package cz contains code for checking a CloudZero API token.
 Package cz contains code for checking a CloudZero API token.
diagnostic/k8s/namespace
Package namespace contains code for checking the Kubernetes configuration.
 Package namespace contains code for checking the Kubernetes configuration.
diagnostic/k8s/provider
Package provider contains code for checking the Kubernetes configuration.
 Package provider contains code for checking the Kubernetes configuration.
diagnostic/k8s/version
Package version contains code for checking the Kubernetes configuration.
 Package version contains code for checking the Kubernetes configuration.
diagnostic/kms
Package kms contains code for testing access the Kubernetes Management Service.
 Package kms contains code for testing access the Kubernetes Management Service.
diagnostic/prom/config
Package config contains a diagnostic provider for checking the Prometheus configuration.
 Package config contains a diagnostic provider for checking the Prometheus configuration.
diagnostic/prom/version
Package version contains a diagnostic provider for checking the Prometheus version.
 Package version contains a diagnostic provider for checking the Prometheus version.
diagnostic/runner
Package runner contains tools for running diagnostics.
 Package runner contains tools for running diagnostics.
diagnostic/stage
Package stage contains a diagnostic provider for checking the stage.
 Package stage contains a diagnostic provider for checking the stage.
diagnostic/webhook
Package webhook contains code for checking a CloudZero API token.
 Package webhook contains code for checking a CloudZero API token.
filter
Package filter provides high-performance metric filtering utilities for CloudZero Agent cost allocation.
 Package filter provides high-performance metric filtering utilities for CloudZero Agent cost allocation.
healthz
Package healthz provides a simple and extensible health check mechanism for HTTP services.
 Package healthz provides a simple and extensible health check mechanism for HTTP services.
housekeeper
Package housekeeper provides a mechanism for cleaning up stale data in a resource store.
 Package housekeeper provides a mechanism for cleaning up stale data in a resource store.
k8s
Package k8s gives a unified interface for k8s information to be retrieved.
 Package k8s gives a unified interface for k8s information to be retrieved.
k8s/mocks
Package mocks is a generated GoMock package.
 Package mocks is a generated GoMock package.
monitor
Package monitor provides secure dynamic secret management and certificate monitoring for CloudZero Agent operations.
 Package monitor provides secure dynamic secret management and certificate monitoring for CloudZero Agent operations.
pusher
Package pusher provides a mechanism for pushing metrics to a remote write endpoint.
 Package pusher provides a mechanism for pushing metrics to a remote write endpoint.
shipper
Package shipper provides metric upload and transmission services for the CloudZero Agent.
 Package shipper provides metric upload and transmission services for the CloudZero Agent.
transform
Package transform provides metric transformation capabilities for standardizing vendor-specific metrics into common formats for cost allocation.
 Package transform provides metric transformation capabilities for standardizing vendor-specific metrics into common formats for cost allocation.
transform/catalog
Package catalog provides a catalog-based metric transformer that routes metrics to registered specialized transformers.
 Package catalog provides a catalog-based metric transformer that routes metrics to registered specialized transformers.
transform/dcgm
Package dcgm provides NVIDIA DCGM metric transformation for cost allocation.
 Package dcgm provides NVIDIA DCGM metric transformation for cost allocation.
webhook
Package webhook provides Kubernetes admission webhook business logic for CloudZero Agent cost allocation.
 Package webhook provides Kubernetes admission webhook business logic for CloudZero Agent cost allocation.
webhook/backfiller
Package backfiller provides functionality to backfill Kubernetes Resource objects, and if enabled invokes the webhook domain logic
 Package backfiller provides functionality to backfill Kubernetes Resource objects, and if enabled invokes the webhook domain logic
webhook/handler
Package handler admission webhook handlers (hook.Handler) for various resource types.
 Package handler admission webhook handlers (hook.Handler) for various resource types.
webhook/helper
Package helper contains decode helper methods for transforming kubernetes metav1.Objects into K8sObjects
 Package helper contains decode helper methods for transforming kubernetes metav1.Objects into K8sObjects
webhook/hook
Package hook contains structures and interfaces for implementing admission webhooks handlers.
 Package hook contains structures and interfaces for implementing admission webhooks handlers.
functions
agent-inspector command
agent-validator command
agent-validator/config
Package config contains a CLI for managing configuration files.
 Package config contains a CLI for managing configuration files.
agent-validator/diagnose
Package diagnose contains a CLI for running diagnostics.
 Package diagnose contains a CLI for running diagnostics.
agent-validator/install
Package install contains a CLI for copying the executable to a destination.
 Package install contains a CLI for copying the executable to a destination.
certifik8s command
cluster-config command
cluster-config/loader
Package loader provides code to load all the different config types
 Package loader provides code to load all the different config types
collector command
helmless command
Package main implements a tool for comparing configured values against default values from a Helm chart.
 Package main implements a tool for comparing configured values against default values from a Helm chart.
helmless/overrides
Package overrides provides functionality for extracting configuration overrides by comparing configured values against default values from Helm charts.
 Package overrides provides functionality for extracting configuration overrides by comparing configured values against default values from Helm charts.
jsonbr2parquet command
scout command
shipper command
webhook command
Package handlers provides HTTP request handlers for CloudZero Agent Primary Adapter implementations.
 Package handlers provides HTTP request handlers for CloudZero Agent Primary Adapter implementations.
http
client
Package http provides HTTP client utilities for CloudZero Agent external API integrations.
 Package http provides HTTP client utilities for CloudZero Agent external API integrations.
middleware
Package middleware provides standard app middlware implementations
 Package middleware provides standard app middlware implementations
Package inspector provides a way to inspect HTTP responses from the CloudZero API to diagnose issues.
 Package inspector provides a way to inspect HTTP responses from the CloudZero API to diagnose issues.
Package logging provides structured logging infrastructure for CloudZero Agent operations.
 Package logging provides structured logging infrastructure for CloudZero Agent operations.
instr
Package instr provides instrumentation utilities.
 Package instr provides instrumentation utilities.
validator
Package logging contains utilities for logging.
 Package logging contains utilities for logging.
storage
core
Package core provides foundational database repository infrastructure for CloudZero Agent storage operations.
 Package core provides foundational database repository infrastructure for CloudZero Agent storage operations.
disk
Package disk implements the secondary adapter for persistent storage in hexagonal architecture.
 Package disk implements the secondary adapter for persistent storage in hexagonal architecture.
repo
Package repo provides concrete repository implementations for CloudZero Agent resource metadata storage.
 Package repo provides concrete repository implementations for CloudZero Agent resource metadata storage.
sqlite
Package sqlite provides SQLite database driver implementation for CloudZero Agent data persistence.
 Package sqlite provides SQLite database driver implementation for CloudZero Agent data persistence.
Package types defines event bus abstractions for inter-component communication in the CloudZero Agent.
 Package types defines event bus abstractions for inter-component communication in the CloudZero Agent.
clusterconfig
Package clusterconfig provides protobuf-based cluster configuration management for the CloudZero Agent.
 Package clusterconfig provides protobuf-based cluster configuration management for the CloudZero Agent.
mocks
Package mocks is a generated GoMock package.
 Package mocks is a generated GoMock package.
status
Package status provides thread-safe access to CloudZero Agent cluster health reporting.
 Package status provides thread-safe access to CloudZero Agent cluster health reporting.
Package utils provides utility functions and types for CloudZero Agent operational support.
 Package utils provides utility functions and types for CloudZero Agent operational support.
k8s
Package k8s contains helpers for working with the Kubernetes Client.
 Package k8s contains helpers for working with the Kubernetes Client.
lock
Package lock provides an interface for file-based locking.
 Package lock provides an interface for file-based locking.
parallel
Package parallel provides utilities for running tasks in parallel.
 Package parallel provides utilities for running tasks in parallel.
scout
Package scout provides cloud environment detection and metadata retrieval capabilities for cloud environments.
 Package scout provides cloud environment detection and metadata retrieval capabilities for cloud environments.
scout/auto
Package auto provides auto-detection capabilities for the CloudZero Scout.
 Package auto provides auto-detection capabilities for the CloudZero Scout.
scout/aws
Package aws provides AWS cloud environment detection and metadata retrieval capabilities using the EC2 instance metadata service (IMDS) v2 with fallback to v1.
 Package aws provides AWS cloud environment detection and metadata retrieval capabilities using the EC2 instance metadata service (IMDS) v2 with fallback to v1.
scout/azure
Package azure provides Azure cloud environment detection and metadata retrieval capabilities using the Azure Instance Metadata Service (IMDS).
 Package azure provides Azure cloud environment detection and metadata retrieval capabilities using the Azure Instance Metadata Service (IMDS).
scout/google
Package google provides functionality for detecting and gathering environment information from Google Cloud metadat services.
 Package google provides functionality for detecting and gathering environment information from Google Cloud metadat services.
scout/types
Package types defines core types and interfaces for cloud environment detection and metadata retrieval.
 Package types defines core types and interfaces for cloud environment detection and metadata retrieval.
scout/types/mocks
Package mocks is a generated GoMock package.
 Package mocks is a generated GoMock package.
telemetry
Package telemetry contains code for posting telemetry data to the CloudZero API.
 Package telemetry contains code for posting telemetry data to the CloudZero API.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.