schema

package
v2.13.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 27, 2026 License: MIT Imports: 15 Imported by: 5

README

Schema Package

The schema package provides core data structures and types for the ClusterCockpit system, including HPC job metadata, cluster configurations, performance metrics, user authentication, and validation utilities.

Overview

This package defines the fundamental schemas used throughout ClusterCockpit for representing:

  • Job Data: Complete metadata and metrics for HPC jobs
  • Cluster Configuration: Hardware topology and metric collection settings
  • Performance Metrics: Time series data with statistical aggregations
  • User Management: Authentication and authorization
  • Validation: JSON schema validation for data integrity

Key Components

Job Structures
Job

The central data structure containing all information about an HPC job:

  • Identification (cluster, job ID, user, project)
  • Resources (nodes, cores, accelerators, memory)
  • Timing (submission, start, duration)
  • State (current job state, monitoring status)
  • Metrics (performance statistics, time series data)
  • Metadata (tags, energy footprint)
JobMetric

Performance metric data with time series from individual hardware components and aggregated statistics.

JobState

Enumeration of job execution states matching common HPC schedulers (SLURM, PBS).

Cluster Configuration
Cluster

Complete HPC cluster definition with subclusters and metric collection settings.

SubCluster

Homogeneous partition within a cluster sharing identical hardware configuration.

Topology

Hardware topology mapping showing relationships between nodes, sockets, cores, hardware threads, and accelerators.

Metrics and Statistics
MetricScope

Hierarchical levels for metric measurement:

node > socket > memoryDomain > core > hwthread
(accelerator is a special scope at hwthread level)
Series

Time series of metric measurements from a single source (node, core, etc.) with min/avg/max statistics.

Float

Custom float64 type with special NaN handling:

  • NaN values serialize as JSON null
  • JSON null deserializes to NaN
  • Avoids pointer overhead for nullable metrics
  • Compatible with both JSON and GraphQL
User Management
User

User account with authentication and authorization information.

Role

Authorization hierarchy:

Anonymous < Api < User < Manager < Support < Admin
AuthSource

Authentication backends: LocalPassword, LDAP, Token, OIDC

Validation
Validate(kind Kind, r io.Reader) error

Validates JSON data against embedded JSON schemas:

  • Meta: Job metadata structure
  • Data: Job metric data
  • ClusterCfg: Cluster configuration

Usage Examples

Validating Cluster Configuration
import (
    "bytes"
    "github.com/ClusterCockpit/cc-lib/v2/schema"
)

// Validate cluster.json against schema
err := schema.Validate(schema.ClusterCfg, bytes.NewReader(clusterJSON))
if err != nil {
    log.Fatal("Invalid cluster configuration:", err)
}
Working with Metrics
// Create job metric data
jobMetric := &schema.JobMetric{
    Unit:     schema.Unit{Base: "FLOP/s", Prefix: "G"},
    Timestep: 60,
    Series: []schema.Series{
        {
            Hostname: "node001",
            Data:     []schema.Float{1.5, 2.0, 1.8, schema.NaN}, // NaN for missing data
            Statistics: schema.MetricStatistics{
                Min: 1.5,
                Avg: 1.77,
                Max: 2.0,
            },
        },
    },
}

// Add aggregated statistics
jobMetric.AddStatisticsSeries()
User Role Checking
user := &schema.User{
    Username: "alice",
    Roles:    []string{"user", "manager"},
}

// Check if user has manager role
if user.HasRole(schema.RoleManager) {
    // Grant project-level access
}

// Check for admin or support
if user.HasAnyRole([]schema.Role{schema.RoleAdmin, schema.RoleSupport}) {
    // Grant elevated privileges
}
Topology Navigation
// Get sockets used by a job's hardware threads
hwthreads := []int{0, 1, 2, 3, 20, 21, 22, 23}
sockets, exclusive := topology.GetSocketsFromHWThreads(hwthreads)

if exclusive {
    fmt.Printf("Job has exclusive access to sockets: %v\n", sockets)
} else {
    fmt.Printf("Job shares sockets: %v\n", sockets)
}

Database Models

The package includes database models for persistent storage:

  • NodeDB: Static node configuration
  • NodeStateDB: Time-stamped node state snapshots
  • Job: Contains both API fields and raw database fields (Raw*)

Raw database fields (RawResources, RawMetaData, etc.) store JSON blobs that are decoded into typed fields when loaded from the database.

JSON Schema Files

Embedded JSON schemas in schemas/ directory:

  • cluster.schema.json: Cluster configuration validation
  • job-meta.schema.json: Job metadata validation
  • job-data.schema.json: Job metric data validation
  • job-metric-data.schema.json: Individual metric validation
  • job-metric-statistics.schema.json: Metric statistics validation
  • unit.schema.json: Unit of measurement validation

Performance Considerations

Float Type

The custom Float type avoids pointer overhead for nullable metrics. In large-scale metric storage with millions of data points, this provides significant memory savings compared to *float64.

Series Marshaling

The Series.MarshalJSON() method provides optimized JSON serialization with fewer allocations, important for REST API performance when returning large metric datasets.

  • ccLogger: Logging utilities used for validation warnings
  • util: Helper functions for statistics (median calculation)
  • jsonschema/v5: JSON schema validation implementation

Testing

Run tests:

go test ./schema/... -v

Check test coverage:

go test ./schema/... -cover

View godoc:

go doc -all schema

Documentation

Overview

Package schema provides core data structures and types for the ClusterCockpit system.

This package defines the fundamental schemas used throughout ClusterCockpit for representing HPC job metadata, cluster configurations, performance metrics, user authentication, and validation utilities.

Key components:

Job Data Structures:

  • Job: Complete metadata for HPC jobs including resources, state, and statistics
  • JobMetric: Performance metrics data with time series and statistics
  • JobState: Enumeration of possible job states (running, completed, failed, etc.)

Cluster Configuration:

  • Cluster: HPC cluster definition with subclusters and metric configuration
  • SubCluster: Partition of a cluster with specific hardware topology
  • Topology: Hardware topology mapping (nodes, sockets, cores, accelerators)

Metrics and Statistics:

  • MetricScope: Hierarchical metric scopes (node, socket, core, hwthread, accelerator)
  • Series: Time series data for metrics with statistics
  • JobStatistics: Statistical aggregations (min, avg, max) for job metrics

User Management:

  • User: User account with roles, projects, and authentication information
  • Role: Authorization levels (admin, support, manager, user, api, anonymous)
  • AuthSource: Authentication source types (local, LDAP, token, OIDC)

Validation:

  • Validate: JSON schema validation for job metadata, job data, and cluster configs
  • Kind: Enumeration of schema types for validation

Special Types:

  • Float: Custom float64 wrapper that handles NaN as JSON null for efficient metric storage
  • Node: Node state information including scheduler and monitoring states

The types in this package are designed to be serialized to/from JSON and are used across REST APIs, GraphQL interfaces, and internal data processing pipelines.

Index

Constants

View Source 
const (
	MonitoringStatusDisabled            int32 = 0
	MonitoringStatusRunningOrArchiving  int32 = 1
	MonitoringStatusArchivingFailed     int32 = 2
	MonitoringStatusArchivingSuccessful int32 = 3
)

Variables

This section is empty.

Functions

func ConvertFloatToFloat64 

func ConvertFloatToFloat64(s []Float) []float64

ConvertFloatToFloat64 converts a slice of Float values to a slice of float64 values. NaN values in the Float slice will remain as NaN in the float64 slice.

func GetRoleString 

func GetRoleString(roleInt Role) string

GetRoleString returns the lowercase string representation of a Role enum value.

func GetValidRoles 

func GetValidRoles(user *User) ([]string, error)

GetValidRoles returns the list of assignable role names. Only admins may call this; returns an error if the user does not have the Admin role.

func GetValidRolesMap 

func GetValidRolesMap(user *User) (map[string]Role, error)

GetValidRolesMap returns a map of role names to Role enum values. Requires any authenticated (non-anonymous) user; returns an error for anonymous users.

func IsMetricGroupKey added in v2.13.0 

func IsMetricGroupKey(key string) bool

IsMetricGroupKey reports whether key denotes an array-valued metric group.

func IsValidRole 

func IsValidRole(role string) bool

IsValidRole reports whether the given string corresponds to a known role name.

func RegisterMetricGroup added in v2.13.0 

func RegisterMetricGroup(key string)

RegisterMetricGroup registers an additional top-level key as an array-valued metric group so that it is (de)serialized into JobData.Groups.

func Validate 

func Validate(k Kind, r io.Reader) error

Validate validates JSON data against an embedded JSON schema.

The kind parameter determines which schema is used:

  • Meta: Validates job metadata structure
  • Data: Validates job performance metric data
  • ClusterCfg: Validates cluster configuration

The reader should contain JSON-encoded data to validate. Returns nil if validation succeeds, or an error describing validation failures.

Example: 

err := schema.Validate(schema.ClusterCfg, bytes.NewReader(clusterJSON))

Types

type Accelerator 

type Accelerator struct {
	ID    string `json:"id"`    // Unique identifier for the accelerator (e.g., "0", "1", "GPU-0")
	Type  string `json:"type"`  // Type of accelerator (e.g., "Nvidia GPU", "AMD GPU")
	Model string `json:"model"` // Specific model name (e.g., "A100", "MI100")
}

Accelerator represents a hardware accelerator (e.g., GPU, FPGA) attached to a compute node. Each accelerator has a unique identifier and type/model information.

type AuthSource 

type AuthSource int

AuthSource identifies the authentication backend that validated a user. 

const (
	AuthViaLocalPassword AuthSource = iota // Local database password authentication
	AuthViaLDAP                            // LDAP directory authentication
	AuthViaToken                           // JWT or API token authentication
	AuthViaOIDC                            // OpenID Connect authentication
	AuthViaAll                             // Accepts any auth source (special case)
)

type AuthType 

type AuthType int

AuthType distinguishes between different authentication contexts. 

const (
	AuthToken   AuthType = iota // API token-based authentication
	AuthSession                 // Session cookie-based authentication
)

type Cluster 

type Cluster struct {
	Name         string          `json:"name"`         // Unique cluster name (e.g., "fritz", "alex")
	MetricConfig []*MetricConfig `json:"metricConfig"` // Cluster-wide metric configurations
	SubClusters  []*SubCluster   `json:"subClusters"`  // Homogeneous partitions within the cluster
}

Cluster represents a complete HPC cluster configuration. A cluster consists of one or more subclusters and defines metric collection/evaluation settings.

type ClusterSupport 

type ClusterSupport struct {
	Cluster     string   `json:"cluster"`     // Cluster name
	SubClusters []string `json:"subclusters"` // List of subcluster names supporting this metric
}

ClusterSupport indicates which subclusters within a cluster support a particular metric. Used to track metric availability across heterogeneous clusters.

type Float 

type Float float64

Float is a custom float64 type with special handling for NaN values in JSON and GraphQL serialization.

Standard Go encoding/json treats NaN as an error, but in metric data it's common to have missing or invalid measurements that should be represented as null in JSON. This type allows NaN values to be serialized as JSON null and vice versa, while avoiding the memory overhead of using *float64 pointers for every nullable metric value.

Key behaviors:

  • NaN values marshal to JSON null
  • JSON null unmarshals to NaN
  • Regular float values marshal/unmarshal normally
  • GraphQL marshaling follows the same null handling

This is particularly important for time series metric data where missing data points are common and need efficient representation. 

var (
	NaN Float = Float(math.NaN())
)

func ConvertToFloat 

func ConvertToFloat(input float64) Float

ConvertToFloat converts a regular float64 to a Float, treating -1.0 as a sentinel for NaN. This is useful when reading from systems that use -1.0 to indicate missing data.

func GetFloat64ToFloat 

func GetFloat64ToFloat(s []float64) []Float

GetFloat64ToFloat converts a slice of float64 values to a slice of Float values. This is the inverse operation of ConvertFloatToFloat64.

func (Float) Double 

func (f Float) Double() float64

func (Float) IsNaN 

func (f Float) IsNaN() bool

func (Float) MarshalGQL 

func (f Float) MarshalGQL(w io.Writer)

MarshalGQL implements the graphql.Marshaler interface. NaN will be serialized to `null`.

func (Float) MarshalJSON 

func (f Float) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface for Float. NaN values are serialized as JSON null. Regular float values are formatted with 2 decimal places using fixed-point notation.

func (*Float) UnmarshalGQL 

func (f *Float) UnmarshalGQL(v any) error

UnmarshalGQL implements the graphql.Unmarshaler interface.

func (*Float) UnmarshalJSON 

func (f *Float) UnmarshalJSON(input []byte) error

UnmarshalJSON implements the json.Unmarshaler interface for Float. JSON null is deserialized as NaN. Numeric strings are parsed as float64 values. Returns an error if the input is neither null nor a valid number.

type FloatArray 

type FloatArray []Float

FloatArray is an alias for []Float that can be marshaled to JSON more efficiently. This type exists to provide optimized JSON marshaling for arrays of Float values.

func (FloatArray) MarshalJSON added in v2.2.0 

func (fa FloatArray) MarshalJSON() ([]byte, error)

MarshalJSON implements json.Marshaler for FloatArray. It efficiently marshals a slice of Float values, converting NaN to null.

type GlobalMetricListItem 

type GlobalMetricListItem struct {
	Name         string           `json:"name"`                // Metric name
	Unit         Unit             `json:"unit"`                // Unit of measurement
	Scope        MetricScope      `json:"scope"`               // Metric scope level
	Footprint    string           `json:"footprint,omitempty"` // Footprint category
	Tooltip      string           `json:"tooltip,omitempty"`   // Tooltip text shown in frontend if defined
	Availability []ClusterSupport `json:"availability"`        // Where this metric is available
}

GlobalMetricListItem represents a metric in the global metric catalog. Tracks which clusters and subclusters support this metric across the entire system.

type Job 

type Job struct {
	Cluster            string             `json:"cluster" db:"cluster" example:"fritz"`
	SubCluster         string             `json:"subCluster" db:"subcluster" example:"main"`
	Partition          string             `json:"partition,omitempty" db:"cluster_partition" example:"main"`
	Project            string             `json:"project" db:"project" example:"abcd200"`
	User               string             `json:"user" db:"hpc_user" example:"abcd100h"`
	Shared             string             `json:"shared" db:"shared" enums:"none,single_user,multi_user"`
	State              JobState           `` /* 172-byte string literal not displayed */
	Tags               []*Tag             `json:"tags,omitempty"`
	RawEnergyFootprint []byte             `json:"-" db:"energy_footprint"`
	RawFootprint       []byte             `json:"-" db:"footprint"`
	RawMetaData        []byte             `json:"-" db:"meta_data"`
	RawResources       []byte             `json:"-" db:"resources"`
	Resources          []*Resource        `json:"resources"`
	EnergyFootprint    map[string]float64 `json:"energyFootprint"`
	Footprint          map[string]float64 `json:"footprint"`
	MetaData           map[string]string  `json:"metaData"`
	ConcurrentJobs     JobLinkResultList  `json:"concurrentJobs"`
	Energy             float64            `json:"energy" db:"energy"`
	ArrayJobID         int64              `json:"arrayJobId,omitempty" db:"array_job_id" example:"123000"`
	Walltime           int64              `json:"walltime,omitempty" db:"walltime" example:"86400" minimum:"1"`
	RequestedMemory    int64              `json:"requestedMemory,omitempty" db:"requested_memory" example:"128000" minimum:"1"` // in MB
	JobID              int64              `json:"jobId" db:"job_id" example:"123000"`
	Duration           int32              `json:"duration" db:"duration" example:"43200" minimum:"1"`
	SMT                int32              `json:"smt,omitempty" db:"smt" example:"4"`
	MonitoringStatus   int32              `json:"monitoringStatus,omitempty" db:"monitoring_status" example:"1" minimum:"0" maximum:"3"`
	NumAcc             int32              `json:"numAcc,omitempty" db:"num_acc" example:"2" minimum:"1"`
	NumHWThreads       int32              `json:"numHwthreads,omitempty" db:"num_hwthreads" example:"20" minimum:"1"`
	NumNodes           int32              `json:"numNodes" db:"num_nodes" example:"2" minimum:"1"`
	Statistics         JobStatisticsSet   `json:"statistics"`
	ID                 *int64             `json:"id,omitempty" db:"id"`
	SubmitTime         int64              `json:"submitTime,omitempty" db:"submit_time" example:"1649723812"`
	StartTime          int64              `json:"startTime" db:"start_time" example:"1649723812"`
}

Job represents complete metadata for an HPC job in ClusterCockpit.

This is the central data structure containing all information about a job including: - Identification: cluster, job ID, user, project - Resources: nodes, cores, accelerators, memory - Timing: submission, start time, duration - State: current job state and monitoring status - Metrics: performance statistics and time series data - Metadata: tags, energy footprint, custom metadata

The RawX fields are used for database serialization of complex nested structures that are stored as JSON blobs in the database and decoded into their respective typed fields (Resources, EnergyFootprint, Footprint, MetaData) when loaded.

Job model @Description Information of a HPC job.

func (Job) GoString 

func (j Job) GoString() string

type JobData 

type JobData struct {
	Metrics map[string]ScopedMetrics
	Groups  []MetricGroup
}

JobData holds all metric data of a HPC job.

Metrics maps a metric name to its scope-organized data, for example jobData.Metrics["cpu_load"]MetricScopeNode. Groups holds array-valued metric groups (filesystems, and later interconnects) whose members each carry their own scoped metrics; these cannot be expressed by the flat Metrics map because a plain map has nowhere to store per-instance identity.

Custom (Un)MarshalJSON keep the on-disk/on-wire layout identical to the JSON schema: flat metrics are top-level scope objects and each group is a top-level array (see job-data.schema.json).

func (*JobData) AddGroupInstance added in v2.13.0 

func (jd *JobData) AddGroupInstance(groupKey, name, typ string, metrics map[string]ScopedMetrics)

AddGroupInstance appends a named instance (with its own scoped metrics) to the group identified by groupKey, creating the group if necessary. This is the converter seam used to assemble grouped data from flat, selector-style query results without the caller knowing the JobData layout.

func (*JobData) AddNodeScope 

func (jd *JobData) AddNodeScope(metric string) bool

func (JobData) FlatMap added in v2.13.0 

func (jd JobData) FlatMap() map[string]ScopedMetrics

FlatMap returns the flat scoped-metrics map, providing the pre-struct access shape (metric -> scope -> *JobMetric) for consumers that do not need the grouped metrics.

func (JobData) MarshalJSON added in v2.13.0 

func (jd JobData) MarshalJSON() ([]byte, error)

MarshalJSON renders JobData in the job-data schema layout: every flat metric as a top-level scope object, and every group as a top-level array of {name, type, <metric>: <scopeObject>, ...} instances. Per-metric values reuse the existing *JobMetric/Series marshalling (NaN -> null preserved).

func (*JobData) RoundMetricStats 

func (jd *JobData) RoundMetricStats()

func (*JobData) Size 

func (jd *JobData) Size() int

func (*JobData) UnmarshalJSON added in v2.13.0 

func (jd *JobData) UnmarshalJSON(b []byte) error

UnmarshalJSON parses the job-data schema layout into JobData. Top-level keys are dispatched by group-key registration; as a robustness fallback an unregistered key whose value is a JSON array is also treated as a group, so a reader can ingest a new array group before it is registered. 

type JobLink struct {
	ID    int64 `json:"id"`    // Internal database ID
	JobID int64 `json:"jobId"` // The job's external job ID
}

JobLink represents a lightweight reference to a job, typically used for linking related jobs. Used to track concurrent jobs or job relationships without including full job metadata.

type JobLinkResultList 

type JobLinkResultList struct {
	Items []*JobLink `json:"items"` // List of job links
	Count int        `json:"count"` // Total count of available items
}

JobLinkResultList holds a paginated list of job links with a total count. Typically used for API responses that return lists of related jobs.

type JobMetric 

type JobMetric struct {
	StatisticsSeries *StatsSeries `json:"statisticsSeries,omitempty"` // Aggregated statistics over time
	Unit             Unit         `json:"unit"`                       // Unit of measurement
	Series           []Series     `json:"series"`                     // Individual time series data
	Timestep         int          `json:"timestep"`                   // Sampling interval in seconds
}

JobMetric contains time series data and statistics for a single metric.

The Series field holds time series data from individual nodes/hardware components, while StatisticsSeries provides aggregated statistics across all series over time.

func (*JobMetric) AddPercentiles 

func (jm *JobMetric) AddPercentiles(ps []int) bool

func (*JobMetric) AddStatisticsSeries 

func (jm *JobMetric) AddStatisticsSeries()

type JobState 

type JobState string

JobState represents the execution state of an HPC job. Valid states match common HPC scheduler states (SLURM, PBS, etc.). 

const (
	JobStateBootFail    JobState = "boot_fail"
	JobStateCancelled   JobState = "cancelled"
	JobStateCompleted   JobState = "completed"
	JobStateDeadline    JobState = "deadline"
	JobStateFailed      JobState = "failed"
	JobStateNodeFail    JobState = "node_fail"
	JobStateOutOfMemory JobState = "out_of_memory"
	JobStatePending     JobState = "pending"
	JobStatePreempted   JobState = "preempted"
	JobStateRunning     JobState = "running"
	JobStateSuspended   JobState = "suspended"
	JobStateTimeout     JobState = "timeout"
)

func (JobState) MarshalGQL 

func (e JobState) MarshalGQL(w io.Writer)

func (*JobState) UnmarshalGQL 

func (e *JobState) UnmarshalGQL(v any) error

func (JobState) Valid 

func (e JobState) Valid() bool

type JobStatistics 

type JobStatistics struct {
	Unit Unit    `json:"unit"`
	Avg  float64 `json:"avg" example:"2500" minimum:"0"` // Job metric average
	Min  float64 `json:"min" example:"2000" minimum:"0"` // Job metric minimum
	Max  float64 `json:"max" example:"3000" minimum:"0"` // Job metric maximum
}

JobStatistics model @Description Specification for job metric statistics.

type JobStatisticsSet added in v2.13.0 

type JobStatisticsSet struct {
	Metrics map[string]JobStatistics
	Groups  []StatsGroup
}

JobStatisticsSet is the value of Job.Statistics. Metrics holds the flat per-metric aggregates; Groups holds array-valued groups (filesystems, and later interconnects) whose members each carry their own statistics. Custom (Un)MarshalJSON keep the job-meta "statistics" layout: flat metrics as objects plus each group as a nested array.

func (*JobStatisticsSet) AddGroupInstance added in v2.13.0 

func (s *JobStatisticsSet) AddGroupInstance(groupKey, name, typ string, metrics map[string]JobStatistics)

AddGroupInstance appends a named statistics instance to the group identified by groupKey, creating the group if necessary.

func (JobStatisticsSet) MarshalJSON added in v2.13.0 

func (s JobStatisticsSet) MarshalJSON() ([]byte, error)

MarshalJSON renders the job-meta "statistics" layout: each flat metric as {unit,avg,min,max}, and each group as an array of {name, type, <metric>: {unit,avg,min,max}, ...} instances.

func (*JobStatisticsSet) UnmarshalJSON added in v2.13.0 

func (s *JobStatisticsSet) UnmarshalJSON(b []byte) error

UnmarshalJSON parses the job-meta "statistics" layout into a JobStatisticsSet. Array-valued group keys (registered, or fallback: value starts with '[') are decoded into Groups; all other keys into flat Metrics.

type Kind 

type Kind int

Kind identifies which JSON schema to use for validation. Each kind corresponds to a different embedded schema file. 

const (
	Meta       Kind = iota + 1 // Job metadata schema (job-meta.schema.json)
	Data                       // Job metric data schema (job-data.schema.json)
	ClusterCfg                 // Cluster configuration schema (cluster.schema.json)
)

type Metric 

type Metric struct {
	Name    string  `json:"name"`    // Metric name (e.g., "cpu_load", "mem_used")
	Unit    Unit    `json:"unit"`    // Unit of measurement
	Peak    float64 `json:"peak"`    // Peak/maximum expected value (best performance)
	Normal  float64 `json:"normal"`  // Normal/typical value (good performance)
	Caution float64 `json:"caution"` // Caution threshold (concerning but not critical)
	Alert   float64 `json:"alert"`   // Alert threshold (requires attention)
}

Metric defines thresholds for a performance metric used in job classification and alerts. Thresholds help categorize job performance: peak (excellent), normal (good), caution (concerning), alert (problem).

type MetricConfig 

type MetricConfig struct {
	Metric                            // Embedded metric thresholds
	Energy        string              `json:"energy"`                // Energy measurement method
	Scope         MetricScope         `json:"scope"`                 // Metric scope (node, socket, core, etc.)
	Aggregation   string              `json:"aggregation"`           // Aggregation function (avg, sum, min, max)
	Footprint     string              `json:"footprint,omitempty"`   // Footprint category
	Tooltip       string              `json:"tooltip,omitempty"`     // Tooltip text shown in frontend if defined
	SubClusters   []*SubClusterConfig `json:"subClusters,omitempty"` // Subcluster-specific overrides
	Timestep      int                 `json:"timestep"`              // Measurement interval in seconds
	Restrict      bool                `json:"restrict"`              // Restrict visibility to non user roles
	LowerIsBetter bool                `json:"lowerIsBetter"`         // Whether lower values are better
}

MetricConfig defines the configuration for a performance metric at the cluster level. Specifies how the metric is collected, aggregated, and evaluated across the cluster.

type MetricGroup added in v2.13.0 

type MetricGroup struct {
	Key       string
	Instances []MetricGroupInstance
}

MetricGroup is an array-valued group of named instances, identified by its top-level JSON key (e.g. "filesystems"). Instance order is preserved because the schema array order is meaningful for rendering.

type MetricGroupInstance added in v2.13.0 

type MetricGroupInstance struct {
	Name    string                   `json:"name"`
	Type    string                   `json:"type"`
	Metrics map[string]ScopedMetrics `json:"-"` // flattened onto the instance object by MarshalJSON
}

MetricGroupInstance is one named element of an array-valued metric group, such as a single filesystem (and, in the future, a single network interconnect). Alongside its identity (Name/Type) it carries its own set of scoped metrics keyed by metric name (e.g. "read_bw", "write_bw").

In the job-archive JSON, per-instance metrics are node-scoped; the scope map is kept general so finer scopes remain representable without type changes.

type MetricScope 

type MetricScope string

MetricScope defines the hierarchical level at which a metric is measured.

Scopes form a hierarchy from coarse-grained (node) to fine-grained (hwthread/accelerator): 

node > socket > memoryDomain > core > hwthread
accelerator is a special scope at the same level as hwthread

The scopePrecedence map defines numeric ordering for scope comparisons, which is used when aggregating metrics across different scopes. 

const (
	MetricScopeInvalid MetricScope = "invalid_scope"

	MetricScopeNode         MetricScope = "node"
	MetricScopeSocket       MetricScope = "socket"
	MetricScopeMemoryDomain MetricScope = "memoryDomain"
	MetricScopeCore         MetricScope = "core"
	MetricScopeHWThread     MetricScope = "hwthread"

	MetricScopeAccelerator MetricScope = "accelerator"
)

func (*MetricScope) LT 

func (e *MetricScope) LT(other MetricScope) bool

func (*MetricScope) LTE 

func (e *MetricScope) LTE(other MetricScope) bool

func (MetricScope) MarshalGQL 

func (e MetricScope) MarshalGQL(w io.Writer)

func (*MetricScope) Max 

func (e *MetricScope) Max(other MetricScope) MetricScope

func (*MetricScope) UnmarshalGQL 

func (e *MetricScope) UnmarshalGQL(v any) error

func (MetricScope) Valid 

func (e MetricScope) Valid() bool

type MetricStatistics 

type MetricStatistics struct {
	Avg float64 `json:"avg"` // Average/mean value
	Min float64 `json:"min"` // Minimum value
	Max float64 `json:"max"` // Maximum value
}

MetricStatistics holds statistical summary values for metric data. Provides the common statistical aggregations used throughout ClusterCockpit.

type MetricValue 

type MetricValue struct {
	Unit  Unit    `json:"unit"`  // Unit of measurement (e.g., FLOP/s, GB/s)
	Value float64 `json:"value"` // Numeric value of the measurement
}

MetricValue represents a single metric measurement with its associated unit. Used for hardware performance characteristics like FLOP rates and memory bandwidth.

type MonitoringState 

type MonitoringState string

MonitoringState indicates the health monitoring status of a node. Reflects whether metric collection is working correctly. 

const (
	MonitoringStateFull    MonitoringState = "full"    // All metrics being collected successfully
	MonitoringStatePartial MonitoringState = "partial" // Some metrics missing
	MonitoringStateFailed  MonitoringState = "failed"  // Metric collection failing
)

type Node 

type Node struct {
	Hostname        string              `json:"hostname"`        // Node hostname
	Cluster         string              `json:"cluster"`         // Cluster name
	SubCluster      string              `json:"subCluster"`      // Subcluster name
	MetaData        map[string]string   `json:"metaData"`        // Additional metadata
	HealthData      map[string][]string `json:"healthData"`      // Details about available metrics per node
	NodeState       SchedulerState      `json:"nodeState"`       // Scheduler/resource manager state
	HealthState     MonitoringState     `json:"healthState"`     // Monitoring system health
	CpusAllocated   int                 `json:"cpusAllocated"`   // Number of allocated CPUs
	MemoryAllocated int                 `json:"memoryAllocated"` // Allocated memory in MB
	GpusAllocated   int                 `json:"gpusAllocated"`   // Number of allocated GPUs
	JobsRunning     int                 `json:"jobsRunning"`     // Number of jobs running on this node
}

Node represents the current state and resource utilization of a compute node.

Combines scheduler state with monitoring health and current resource allocation. Used for displaying node status in dashboards and tracking node utilization.

type NodeDB 

type NodeDB struct {
	ID          int64  `json:"id" db:"id"`                                // Database ID
	Hostname    string `json:"hostname" db:"hostname" example:"fritz"`    // Node hostname
	Cluster     string `json:"cluster" db:"cluster" example:"fritz"`      // Cluster name
	SubCluster  string `json:"subCluster" db:"subcluster" example:"main"` // Subcluster name
	RawMetaData []byte `json:"-" db:"meta_data"`                          // Metadata as JSON blob
}

NodeDB is the database model for the node table. Stores static node configuration and metadata.

type NodePayload 

type NodePayload struct {
	Hostname        string   `json:"hostname"`        // Node hostname
	States          []string `json:"states"`          // State strings (flexible format)
	CpusAllocated   int      `json:"cpusAllocated"`   // Number of allocated CPUs
	MemoryAllocated int64    `json:"memoryAllocated"` // Allocated memory in MB
	GpusAllocated   int      `json:"gpusAllocated"`   // Number of allocated GPUs
	JobsRunning     int      `json:"jobsRunning"`     // Number of running jobs
}

NodePayload is the request body format for the node state REST API. Used when updateing node states from external monitoring or scheduler systems.

type NodeStateDB 

type NodeStateDB struct {
	ID              int64           `json:"id" db:"id"`                                                                                                         // Database ID
	TimeStamp       int64           `json:"timeStamp" db:"time_stamp" example:"1649723812"`                                                                     // Unix timestamp
	NodeState       SchedulerState  `json:"nodeState" db:"node_state" example:"completed" enums:"completed,failed,cancelled,stopped,timeout,out_of_memory"`     // Scheduler state
	HealthState     MonitoringState `json:"healthState" db:"health_state" example:"completed" enums:"completed,failed,cancelled,stopped,timeout,out_of_memory"` // Monitoring health
	HealthMetrics   string          `json:"healthMetrics" db:"health_metrics"`                                                                                  // JSON: {"missing":[...],"degraded":[...]}
	CpusAllocated   int             `json:"cpusAllocated" db:"cpus_allocated"`                                                                                  // Allocated CPUs
	MemoryAllocated int64           `json:"memoryAllocated" db:"memory_allocated"`                                                                              // Allocated memory (MB)
	GpusAllocated   int             `json:"gpusAllocated" db:"gpus_allocated"`                                                                                  // Allocated GPUs
	JobsRunning     int             `json:"jobsRunning" db:"jobs_running" example:"12"`                                                                         // Running jobs
	NodeID          int64           `json:"_" db:"node_id"`                                                                                                     // Foreign key to NodeDB
}

NodeStateDB is the database model for the node_state table. Stores time-stamped snapshots of node state and resource allocation.

type Resource 

type Resource struct {
	Hostname      string   `json:"hostname"`                // Node hostname
	Configuration string   `json:"configuration,omitempty"` // Optional configuration identifier
	HWThreads     []int    `json:"hwthreads,omitempty"`     // Allocated hardware thread IDs
	Accelerators  []string `json:"accelerators,omitempty"`  // Allocated accelerator IDs (e.g., GPU IDs)
}

Resource represents the hardware resources assigned to a job on a single compute node.

A job typically uses multiple Resource entries, one for each allocated node. HWThreads lists the specific hardware thread IDs allocated, allowing for precise CPU pinning analysis. Accelerators lists assigned GPU/accelerator IDs.

Resource model @Description A resource used by a job

type Role 

type Role int

Role defines the authorization level for a user in ClusterCockpit. Roles form a hierarchy with increasing privileges: Anonymous < Api < User < Manager < Support < Admin. 

const (
	RoleAnonymous Role = iota // Unauthenticated or guest access
	RoleAPI                   // API access (programmatic/service accounts)
	RoleUser                  // Regular user (can view own jobs)
	RoleManager               // Project manager (can view project jobs)
	RoleSupport               // Support staff (can view all jobs, limited admin)
	RoleAdmin                 // Full administrator access
	RoleError                 // Invalid/error role
)

type SchedulerState 

type SchedulerState string

SchedulerState represents the current state of a node in the HPC job scheduler. States typically reflect SLURM/PBS node states. 

const (
	NodeStateAllocated SchedulerState = "allocated" // Node is fully allocated to jobs
	NodeStateReserved  SchedulerState = "reserved"  // Node is reserved but not yet allocated
	NodeStateIdle      SchedulerState = "idle"      // Node is available for jobs
	NodeStateMixed     SchedulerState = "mixed"     // Node is partially allocated
	NodeStateDown      SchedulerState = "down"      // Node is down/offline
	NodeStateUnknown   SchedulerState = "unknown"   // Node state unknown
)

type ScopedJobStats 

type ScopedJobStats struct {
	Metrics map[string]ScopedMetricStats
	Groups  []ScopedStatsGroup
}

ScopedJobStats stores pre-computed statistics without the full time series data, reducing memory footprint when only aggregated values are needed. It mirrors JobData: flat metrics in Metrics, array-valued groups in Groups.

func (ScopedJobStats) MarshalJSON added in v2.13.0 

func (sjs ScopedJobStats) MarshalJSON() ([]byte, error)

MarshalJSON renders ScopedJobStats in the same top-level layout as JobData: flat metrics as scope objects, groups as top-level arrays of {name, type, <metric>: <scopeObject>} instances.

func (*ScopedJobStats) RoundScopedMetricStats added in v2.2.1 

func (sjs *ScopedJobStats) RoundScopedMetricStats()

func (*ScopedJobStats) UnmarshalJSON added in v2.13.0 

func (sjs *ScopedJobStats) UnmarshalJSON(b []byte) error

UnmarshalJSON parses the top-level layout into ScopedJobStats, dispatching array-valued group keys into Groups (see JobData.UnmarshalJSON).

type ScopedMetricStats added in v2.13.0 

type ScopedMetricStats map[MetricScope][]*ScopedStats

ScopedMetricStats maps a scope to its per-source statistical summaries.

type ScopedMetrics added in v2.13.0 

type ScopedMetrics map[MetricScope]*JobMetric

ScopedMetrics maps a hierarchical scope (node/socket/core/...) to its metric data. It is the value stored for a single metric name.

type ScopedStats 

type ScopedStats struct {
	Hostname string            `json:"hostname"`     // Source hostname
	ID       *string           `json:"id,omitempty"` // Optional scope ID
	Data     *MetricStatistics `json:"data"`         // Statistical summary
}

ScopedStats contains statistical summaries for a specific scope (e.g., one node, one socket). Used when full time series data isn't needed, only the aggregated statistics.

type ScopedStatsGroup added in v2.13.0 

type ScopedStatsGroup struct {
	Key       string
	Instances []ScopedStatsGroupInstance
}

ScopedStatsGroup is an array-valued group of ScopedStatsGroupInstance.

type ScopedStatsGroupInstance added in v2.13.0 

type ScopedStatsGroupInstance struct {
	Name    string                       `json:"name"`
	Type    string                       `json:"type"`
	Metrics map[string]ScopedMetricStats `json:"-"`
}

ScopedStatsGroupInstance is the ScopedJobStats analogue of MetricGroupInstance: one named instance carrying scope-organized statistics.

type Series 

type Series struct {
	ID         *string          `json:"id,omitempty"` // Optional ID (e.g., core ID, GPU ID)
	Hostname   string           `json:"hostname"`     // Source hostname
	Data       []Float          `json:"data"`         // Time series measurements
	Statistics MetricStatistics `json:"statistics"`   // Statistical summary (min/avg/max)
}

Series represents a single time series of metric measurements.

Each series corresponds to one source (e.g., one node, one core) identified by Hostname and optional ID. The Data field contains the time-ordered measurements, and Statistics provides min/avg/max summaries.

func (*Series) MarshalJSON 

func (s *Series) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface for Series. It manually builds the JSON output to reduce allocations compared to encoding/json. NaN data points are serialized as null. Only used via REST-API, not via GraphQL.

type StatsGroup added in v2.13.0 

type StatsGroup struct {
	Key       string
	Instances []StatsGroupInstance
}

StatsGroup is an array-valued statistics group identified by its top-level JSON key (e.g. "filesystems"). Instance order is preserved.

type StatsGroupInstance added in v2.13.0 

type StatsGroupInstance struct {
	Name    string                   `json:"name"`
	Type    string                   `json:"type"`
	Metrics map[string]JobStatistics `json:"-"` // flattened onto the instance object by MarshalJSON
}

StatsGroupInstance is one named element of an array-valued statistics group (a single filesystem, later a single interconnect). Its per-instance metrics are flat JobStatistics (the job-meta schema does not scope filesystem stats).

type StatsSeries 

type StatsSeries struct {
	Percentiles map[int][]Float `json:"percentiles,omitempty"` // Percentile values over time (e.g., 10th, 50th, 90th)
	Mean        []Float         `json:"mean"`                  // Mean values over time
	Median      []Float         `json:"median"`                // Median values over time
	Min         []Float         `json:"min"`                   // Minimum values over time
	Max         []Float         `json:"max"`                   // Maximum values over time
}

StatsSeries contains aggregated statistics across multiple time series over time.

Instead of storing individual series, this provides statistical summaries at each time step. For example, at time t, Mean[t] is the average value across all series at that time. Percentiles provides specified percentile values at each time step.

type SubCluster 

type SubCluster struct {
	Name            string          `json:"name"`                      // Name of the subcluster (e.g., "main", "gpu", "bigmem")
	Nodes           string          `json:"nodes"`                     // Node list in condensed format (e.g., "node[001-100]")
	ProcessorType   string          `json:"processorType"`             // CPU model (e.g., "Intel Xeon Gold 6148")
	Topology        Topology        `json:"topology"`                  // Hardware topology of nodes in this subcluster
	FlopRateScalar  MetricValue     `json:"flopRateScalar"`            // Theoretical scalar FLOP rate per node
	FlopRateSimd    MetricValue     `json:"flopRateSimd"`              // Theoretical SIMD FLOP rate per node
	MemoryBandwidth MetricValue     `json:"memoryBandwidth"`           // Theoretical memory bandwidth per node
	MetricConfig    []*MetricConfig `json:"metricConfig,omitempty"`    // Subcluster-specific metric configurations
	Footprint       []string        `json:"footprint,omitempty"`       // Default footprint metrics for jobs
	EnergyFootprint []string        `json:"energyFootprint,omitempty"` // Energy-related footprint metrics
	SocketsPerNode  int             `json:"socketsPerNode"`            // Number of CPU sockets per node
	CoresPerSocket  int             `json:"coresPerSocket"`            // Number of cores per CPU socket
	ThreadsPerCore  int             `json:"threadsPerCore"`            // Number of hardware threads per core (SMT level)
}

SubCluster represents a homogeneous partition of a cluster with identical hardware. A cluster may contain multiple subclusters with different processor types or configurations.

type SubClusterConfig 

type SubClusterConfig struct {
	Metric               // Embedded metric thresholds
	Footprint     string `json:"footprint,omitempty"` // Footprint category for this metric
	Energy        string `json:"energy"`              // Energy measurement configuration
	LowerIsBetter bool   `json:"lowerIsBetter"`       // Whether lower values indicate better performance
	Restrict      bool   `json:"restrict"`            // Restrict visibility to non user roles
	Remove        bool   `json:"remove"`              // Whether to exclude this metric for this subcluster
}

SubClusterConfig extends Metric with subcluster-specific metric configuration. Allows overriding metric settings for specific subclusters within a cluster.

type Tag 

type Tag struct {
	Type  string `json:"type" db:"tag_type" example:"Debug"`
	Name  string `json:"name" db:"tag_name" example:"Testjob"`
	Scope string `json:"scope" db:"tag_scope" example:"global"`
	ID    int64  `json:"id" db:"id"`
}

Tag model @Description Defines a tag using name and type.

type Topology 

type Topology struct {
	Node         []int          `json:"node"`                   // All hardware thread IDs on this node
	Socket       [][]int        `json:"socket"`                 // Hardware threads grouped by socket
	MemoryDomain [][]int        `json:"memoryDomain"`           // Hardware threads grouped by NUMA domain
	Die          [][]*int       `json:"die,omitempty"`          // Hardware threads grouped by die (optional)
	Core         [][]int        `json:"core"`                   // Hardware threads grouped by core
	Accelerators []*Accelerator `json:"accelerators,omitempty"` // Attached accelerators (GPUs, etc.)
	// contains filtered or unexported fields
}

Topology defines the hardware topology of a compute node, mapping the hierarchical relationships between hardware threads, cores, sockets, memory domains, and accelerators.

The topology is represented as nested arrays where indices represent hardware IDs:

  • Node: Flat list of all hardware thread IDs on the node
  • Socket: Hardware threads grouped by physical CPU socket
  • Core: Hardware threads grouped by physical core
  • MemoryDomain: Hardware threads grouped by NUMA domain
  • Die: Optional grouping by CPU die within sockets
  • Accelerators: List of attached hardware accelerators

func (*Topology) GetAcceleratorID 

func (topo *Topology) GetAcceleratorID(id int) (string, error)

GetAcceleratorID converts an integer accelerator index to its string ID. Returns an error if the index is out of range.

func (*Topology) GetAcceleratorIDs 

func (topo *Topology) GetAcceleratorIDs() []string

GetAcceleratorIDs returns a list of all accelerator IDs as strings. Capacity is pre-allocated to improve efficiency.

func (*Topology) GetAcceleratorIDsAsInt 

func (topo *Topology) GetAcceleratorIDsAsInt() ([]int, error)

GetAcceleratorIDsAsInt attempts to convert all accelerator IDs to integers. Returns an error if any accelerator ID is not a valid integer. This method assumes accelerator IDs are numeric strings. Capacity is pre-allocated to improve efficiency.

func (*Topology) GetCoresFromHWThreads 

func (topo *Topology) GetCoresFromHWThreads(
	hwthreads []int,
) (cores []int, exclusive bool)

GetCoresFromHWThreads returns core IDs that contain any of the given hardware threads. The exclusive return value is true if all hardware threads in the returned cores are present in the input list (i.e., the job has exclusive access to those cores).

func (*Topology) GetMemoryDomainsBySocket added in v2.6.0 

func (topo *Topology) GetMemoryDomainsBySocket(domainIDs []int) (map[int][]int, error)

GetMemoryDomainsBySocket groups the given memory domain IDs by their parent socket. Returns a map from socket ID to the list of memory domain IDs belonging to that socket.

func (*Topology) GetMemoryDomainsFromHWThreads 

func (topo *Topology) GetMemoryDomainsFromHWThreads(
	hwthreads []int,
) (memDoms []int, exclusive bool)

GetMemoryDomainsFromHWThreads returns memory domain IDs that contain any of the given hardware threads. The exclusive return value is true if all hardware threads in the returned memory domains are present in the input list (i.e., the job has exclusive access to those memory domains).

func (*Topology) GetSocketsFromCores 

func (topo *Topology) GetSocketsFromCores(
	cores []int,
) (sockets []int, exclusive bool)

GetSocketsFromCores returns socket IDs that contain any of the given cores. The exclusive return value is true if all hardware threads in the returned sockets belong to cores in the input list (i.e., the job has exclusive access to those sockets).

func (*Topology) GetSocketsFromHWThreads 

func (topo *Topology) GetSocketsFromHWThreads(
	hwthreads []int,
) (sockets []int, exclusive bool)

GetSocketsFromHWThreads returns socket IDs that contain any of the given hardware threads. The exclusive return value is true if all hardware threads in the returned sockets are present in the input list (i.e., the job has exclusive access to those sockets).

func (*Topology) InitTopologyMaps added in v2.6.0 

func (topo *Topology) InitTopologyMaps()

InitTopologyMaps initializes the topology mapping caches

type Unit 

type Unit struct {
	Base   string `json:"base"`             // Base unit (e.g., "B/s", "F/s", "W")
	Prefix string `json:"prefix,omitempty"` // SI prefix (e.g., "G", "M", "K", "T")
}

Unit represents a unit of measurement with optional SI prefix.

Examples:

  • {Base: "B/s", Prefix: "G"} = GB/s (gigabytes per second)
  • {Base: "F/s", Prefix: "T"} = TF/s (teraflops per second)
  • {Base: "", Prefix: ""} = dimensionless (e.g., CPU load)

type User 

type User struct {
	Username   string     `json:"username"`   // Unique username
	Password   string     `json:"-"`          // Password hash (never serialized to JSON)
	Name       string     `json:"name"`       // Full display name
	Email      string     `json:"email"`      // Email address
	Roles      []string   `json:"roles"`      // Assigned role names
	Projects   []string   `json:"projects"`   // Authorized project/account names
	AuthType   AuthType   `json:"authType"`   // How the user authenticated
	AuthSource AuthSource `json:"authSource"` // Which system authenticated the user
}

User represents a ClusterCockpit user account with authentication and authorization information.

Users are authenticated via various sources (local, LDAP, OIDC) and assigned roles that determine access levels. Projects lists the HPC projects/accounts the user has access to.

func (*User) GetAuthLevel 

func (u *User) GetAuthLevel() Role

GetAuthLevel returns the user's highest-privilege role. Returns RoleError if the user has no recognized roles.

func (*User) HasAllRoles 

func (u *User) HasAllRoles(queryroles []Role) bool

HasAllRoles reports whether the user has every one of the given roles.

func (*User) HasAnyRole 

func (u *User) HasAnyRole(queryroles []Role) bool

HasAnyRole reports whether the user has at least one of the given roles.

func (*User) HasNotRoles 

func (u *User) HasNotRoles(queryroles []Role) bool

HasNotRoles reports whether the user has none of the given roles.

func (*User) HasProject 

func (u *User) HasProject(project string) bool

HasProject reports whether the user is authorized for the given project name.

func (*User) HasRole 

func (u *User) HasRole(role Role) bool

HasRole reports whether the user has the specified role.

func (*User) HasValidRole 

func (u *User) HasValidRole(role string) (hasRole bool, isValid bool)

HasValidRole checks whether the user has the specified role and whether the role string is valid. Returns hasRole=true if the user has the role, and isValid=true if the role name is recognized.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.