cc-lib

cc-lib

Common ClusterCockpit golang packages providing reusable components for building HPC monitoring and metric collection applications.

Overview

cc-lib is a collection of Go packages developed for the ClusterCockpit project. These packages provide essential functionality for:

• Metric Collection: Receivers for various protocols (IPMI, Redfish, Prometheus, etc.)
• Data Processing: Message processing pipelines, resampling, and transformations
• Data Storage: Sinks for InfluxDB, NATS, Prometheus, and more
• Configuration: Flexible configuration management with validation
• Utilities: Caching, logging, topology detection, and helper functions

The library is designed to be modular, allowing you to use individual packages as needed in your own projects.

Packages

Core Messaging & Processing

Metric Collection

Data Storage & Output

Configuration & Logging

Utilities

Installation

go get github.com/ClusterCockpit/cc-lib/v2

Requirements:

• Go 1.24.0 or higher

Quick Start

Using the LRU Cache

import "github.com/ClusterCockpit/cc-lib/v2/lrucache"

cache := lrucache.New(1000) // maxmemory in arbitrary units

value := cache.Get("key", func() (interface{}, time.Duration, int) {
    // Compute expensive value
    result := fetchFromDatabase()
    return result, 10 * time.Minute, len(result)
})

Expanding Hostlists

import "github.com/ClusterCockpit/cc-lib/v2/hostlist"

hosts, err := hostlist.Expand("node[1-10],gpu[1-4]")
// Returns: [gpu1, gpu2, gpu3, gpu4, node1, node2, ..., node10]

Creating Messages

import "github.com/ClusterCockpit/cc-lib/v2/ccMessage"

msg, err := ccMessage.NewMessage(
    "temperature",
    map[string]string{"hostname": "node01", "type": "node"},
    map[string]string{"unit": "degC"},
    map[string]interface{}{"value": 45.2},
    time.Now(),
)

Using Configuration Management

import "github.com/ClusterCockpit/cc-lib/v2/ccConfig"

config := ccConfig.New()
config.AddFile("config.json")

// Access configuration
value := config.Get("key")

// Watch for changes
config.Watch(func() {
    log.Println("Configuration changed")
})

Documentation

• API Documentation: pkg.go.dev/github.com/ClusterCockpit/cc-lib/v2
• Package READMEs: Each package has its own README with detailed documentation and examples

Package Documentation

• ccConfig - Configuration management
• ccLogger - Logging utilities
• ccMessage - Message types and protocols
• ccTopology - System topology detection
• ccUnits - Unit conversion
• hostlist - Hostlist expansion
• lrucache - LRU cache with TTL
• messageProcessor - Message processing
• receivers - Metric receivers
• runtime - Runtime environment setup
• schema - JSON schema validation
• sinks - Metric sinks
• util - Utility functions

Testing

Run all tests:

go test ./...

Run tests with coverage:

go test -cover ./...

Run tests for a specific package:

go test -v ./lrucache

Contributing

Contributions are welcome! Please feel free to submit issues and pull requests.

Development

1. Clone the repository
2. Make your changes
3. Run tests: go test ./...
4. Submit a pull request

Projects Using cc-lib

• cc-metric-collector - Metric collection daemon
• cc-metric-store - Metric storage backend
• ClusterCockpit - Web interface and monitoring system

License

This project is licensed under the MIT License - see the LICENSE file for details.

Copyright (C) NHR@FAU, University Erlangen-Nuremberg.

Acknowledgments

Developed by the National High Performance Computing (NHR) center at FAU.

Additional contributors:

• Holger Obermaier (NHR@KIT)