Documentation
¶
Overview ¶
Package monitor provides secure dynamic secret management and certificate monitoring for CloudZero Agent operations. This package implements critical security infrastructure that enables zero-downtime credential rotation and certificate lifecycle management without requiring agent restarts or service disruption.
The monitor system operates as an Application Core component in the hexagonal architecture, providing secure credential management services to all agent components requiring authentication with external systems including the CloudZero platform, Kubernetes API, and Prometheus endpoints.
Key responsibilities:
- Dynamic API key rotation: Automatic detection and loading of updated CloudZero API keys
- Certificate lifecycle management: TLS certificate expiration monitoring and renewal
- Secret change detection: SHA256-based change detection to minimize unnecessary operations
- Security-first design: Credential redaction in logs and secure memory handling
- Production reliability: Panic recovery and graceful error handling for continuous operation
Architecture:
- secretsMonitor: Manages CloudZero API key rotation with configurable refresh intervals
- TLS monitors: Handle certificate expiration tracking and renewal notifications
- Change detection: Cryptographic hashing to identify actual secret changes
- Lifecycle management: Full start/stop/reset capabilities for operational flexibility
Security features:
- Credential redaction: Automatic masking of sensitive data in log outputs
- Minimal exposure: Secrets loaded only when changes are detected
- Memory protection: Secure handling and cleanup of sensitive credential data
- Audit logging: Comprehensive tracking of secret rotation events for compliance
The monitoring system enables production CloudZero deployments to maintain continuous operation while credentials are rotated according to organizational security policies, ensuring both security compliance and operational stability.
Integration points:
- CloudZero API client: Receives updated API keys for platform authentication
- TLS infrastructure: Monitors certificate expiration for webhook and metric endpoints
- Configuration services: Triggers configuration reload when secrets change
- Operational monitoring: Provides metrics and alerts for credential management health
Package monitor provides functionality to manage and reload TLS certificates dynamically. This package is designed to enhance the security and flexibility of TLS configurations by allowing dynamic reloading and rotation of TLS certificates. It supports various methods for triggering certificate reloads, such as file system changes, SIGHUP signals, and periodic intervals.
The main component of this package is the reconciler struct, which manages the TLS configuration and ensures that the latest certificates are always used. The reconciler can be customized using functional options to fit different use cases.
This package is valuable for applications that require robust and flexible TLS certificate management, ensuring that the TLS configuration stays up-to-date with the latest certificates without requiring application restarts.
Index ¶
- Variables
- func NewSecretMonitor(ctx context.Context, settings MonitoredAPIKey) types.Runnable
- func TLSConfig(opts ...Option) *tls.Config
- type MonitoredAPIKey
- type Option
- func WithCertificatesPaths(cert, key, ca string) Option
- func WithDurationReload(dur time.Duration) Option
- func WithOnReload(f func(*tls.Config)) Option
- func WithProvider(p TLSProvider) Option
- func WithReloadFunc(f func() bool) Option
- func WithRootsLimit(n uint) Option
- func WithSIGHUPReload(c chan os.Signal) Option
- func WithVerifyConnection() Option
- type TLSProvider
Constants ¶
This section is empty.
Variables ¶
var DefaultRefreshInterval = 1 * time.Minute
DefaultRefreshInterval defines the standard polling frequency for secret change detection. This interval balances security responsiveness with system resource utilization, providing timely credential rotation while avoiding excessive monitoring overhead.
The 1-minute interval enables:
- Rapid response to credential rotation (maximum 60-second delay)
- Reasonable resource consumption for continuous monitoring
- Compliance with typical organizational security policies
- Compatibility with automated secret management systems
Production deployments may adjust this interval based on:
- Security requirements and threat model
- Secret rotation frequency and urgency
- System resource constraints and monitoring overhead
- Integration with external credential management platforms
Functions ¶
func NewSecretMonitor ¶
func NewSecretMonitor(ctx context.Context, settings MonitoredAPIKey) types.Runnable
func TLSConfig ¶
TLSConfig creates a new tls.Config that automatically reconciles certificates after rotations. It accepts a variadic number of Option interfaces to customize the TLS configuration.
If no options are provided, it returns a default tls.Config.
Example:
tlsConfig := TLSConfig(
WithVerifyConnection(),
WithCertificatesPaths("/path/to/cert.pem", "/path/to/key.pem", "/path/to/ca.pem"),
WithRootsLimit(3),
WithReloadFunc(myReloadFunc),
)
server := &http.Server{
Addr: ":443",
TLSConfig: tlsConfig,
// Other server settings...
}
Types ¶
type MonitoredAPIKey ¶
type MonitoredAPIKey interface {
// GetAPIKey retrieves the currently loaded API key for CloudZero platform authentication.
// This method is called during monitoring cycles to obtain the current credential
// for change detection and validation purposes.
//
// Returns:
// - string: The current API key value (sensitive data requiring secure handling)
//
// Security considerations:
// - Returned value contains sensitive authentication data
// - Should not be logged or stored in plaintext
// - Memory should be cleared after use when possible
//
// Thread safety:
// - Must be safe for concurrent access during monitoring operations
// - Should provide consistent results during credential transition periods
GetAPIKey() string
// SetAPIKey triggers a refresh of the API key from the configured credential source.
// This method enables the monitor to request credential updates when changes are
// suspected or during periodic refresh cycles.
//
// Implementation behavior:
// - Should reload credentials from the authoritative source
// - May validate credential format and accessibility
// - Should update internal state atomically to prevent inconsistency
// - Should handle transient errors gracefully with appropriate retry logic
//
// Error conditions:
// - Credential source unavailable (network, filesystem, API errors)
// - Invalid credential format or structure
// - Authentication failures with credential management systems
// - Permission errors accessing credential storage
//
// Returns:
// - error: nil on successful refresh, error describing failure condition
SetAPIKey() error
}
MonitoredAPIKey defines the interface for components providing dynamically refreshable API keys. This interface enables the secrets monitor to work with various credential providers including file-based secrets, Kubernetes secrets, cloud provider key management services, and external credential rotation systems.
The interface supports both pull-based (GetAPIKey) and push-based (SetAPIKey) credential management patterns, enabling flexibility in how credentials are sourced and updated.
Implementation considerations:
- Thread safety: Methods may be called concurrently during monitoring operations
- Error handling: SetAPIKey errors indicate credential loading failures
- Security: GetAPIKey returns sensitive data that should be handled securely
- Performance: Methods should minimize latency as they're called during monitoring cycles
type Option ¶
type Option interface {
// contains filtered or unexported methods
}
Option represents a configuration option for the reconciler. It follows the functional options pattern, allowing for flexible and readable configuration.
func WithCertificatesPaths ¶
WithCertificatesPaths configures the reconciler to load TLS certificates from specified file paths. It sets up a file system provider that watches the provided certificate and key files, and optionally a CA bundle for root CAs.
Parameters: - cert: Path to the TLS certificate file. - key: Path to the TLS key file. - ca: Path to the CA bundle file. Can be empty if no CA rotation is needed.
Note: - The CA path can be empty if root CA rotation is not required. - If a CA path is provided, it should point to a valid CA bundle.
Example:
tlsConfig := TLSConfig(WithCertificatesPaths("/path/to/cert.pem", "/path/to/key.pem", "/path/to/ca.pem"))
func WithDurationReload ¶
WithDurationReload configures the reconciler to reload certificates at specified intervals. This is useful for ensuring that certificates are periodically refreshed.
Parameters: - dur: The duration between each reload attempt.
Example:
tlsConfig := TLSConfig(WithDurationReload(24*time.Hour))
func WithOnReload ¶
WithOnReload registers a callback function to be invoked after a certificate reload. This can be used for additional actions such as rotating session tickets or logging.
Parameters: - f: A function that takes a pointer to tls.Config and performs custom actions.
Note: - The callback is executed in its own goroutine to avoid blocking the reconciler.
Example:
onReload := func(cfg *tls.Config) {
log.Println("TLS certificates reloaded")
}
tlsConfig := TLSConfig(WithOnReload(onReload))
func WithProvider ¶
func WithProvider(p TLSProvider) Option
WithProvider sets the TLSProvider for the reconciler. The provider is responsible for retrieving the latest certificates upon receiving a reload signal.
This option configures the tls.Config to use the reconciler's methods for obtaining certificates, ensuring that the TLS configuration stays up-to-date with the latest certificates.
Example:
provider := NewMyTLSProvider() tlsConfig := TLSConfig(WithProvider(provider))
func WithReloadFunc ¶
WithReloadFunc registers a custom function to determine when a certificate reload is needed. This function is periodically or conditionally called to check if a reload should occur.
Parameters: - f: A function that returns a boolean indicating whether a reload is necessary.
Note: - Multiple goroutines may invoke this function concurrently. - Ensure that the function is thread-safe.
Example:
reloadFunc := func() bool {
// Custom logic to determine if reload is needed
return time.Now().Unix()%60 == 0
}
tlsConfig := TLSConfig(WithReloadFunc(reloadFunc))
func WithRootsLimit ¶
WithRootsLimit sets a limit on the number of old root CA certificates to retain in the pool. This is useful for maintaining backward compatibility, allowing verification of certificates issued before recent rotations.
Parameters: - n: The maximum number of root CA certificates to keep.
Note:
- This option works in tandem with WithVerifyConnection. If WithVerifyConnection is not used, this option has no effect.
Default: - If not set, the default limit is 2.
Example:
tlsConfig := TLSConfig(WithRootsLimit(3))
func WithSIGHUPReload ¶
WithSIGHUPReload configures the reconciler to reload certificates upon receiving a SIGHUP signal. This is useful for triggering certificate reloads without restarting the application.
Parameters: - c: A channel that receives os.Signal notifications.
Example:
signalChan := make(chan os.Signal, 1) signal.Notify(signalChan, syscall.SIGHUP) tlsConfig := TLSConfig(WithSIGHUPReload(signalChan))
func WithVerifyConnection ¶
func WithVerifyConnection() Option
WithVerifyConnection configures the TLS settings to verify the TLS connection's certificate. It sets tls.Config.VerifyConnection to use the reconciler's CA pool, which is managed based on certificate rotations. If no custom CA pool is provided, the system roots or platform verifier are used.
Additionally, this option sets tls.Config.InsecureSkipVerify to true to bypass the default Go TLS validation. This does not disable VerifyConnection but allows the reconciler to handle certificate verification.
Note: - The tls.Config.RootCAs and tls.Config.ClientCAs are ignored when this option is applied. - This option should be used in conjunction with WithRootsLimit for effective certificate pool management.
Example:
tlsConfig := TLSConfig(WithVerifyConnection())
type TLSProvider ¶
type TLSProvider interface {
// Certificates retrieves the most recently rotated client or server (leaf) certificate
// along with the root CA certificates.
// It returns an error if the retrieval fails.
//
// Note:
// - The method may return nil for root CAs if a predefined CA pool is set in tls.Config.
// - This allows for flexibility in managing CA pools, either dynamically or statically.
Certificates() (*tls.Certificate, []*x509.Certificate, error)
}
TLSProvider defines an interface for retrieving TLS certificates. Implementing this interface allows for dynamic reloading and rotation of TLS certificates, enhancing the security and flexibility of TLS configurations.
Source Files