Documentation
¶
Overview ¶
Package tagger provides automatic job tagging functionality for cc-backend. It supports detecting applications and classifying jobs based on configurable rules. Tags are automatically applied when jobs start or stop, or can be applied retroactively to existing jobs using RunTaggers.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Init ¶
func Init()
Init initializes the job tagger system and registers it with the job repository. This function is safe to call multiple times; initialization only occurs once. It should be called during application startup.
func RunTaggerByName ¶
RunTaggerByName starts a tagger by name asynchronously on all jobs. Returns an error if the name is unknown or the tagger is already running.
func RunTaggers ¶
func RunTaggers() error
RunTaggers applies all configured taggers to all existing jobs in the repository. This is useful for retroactively applying tags to jobs that were created before the tagger system was initialized or when new tagging rules are added. It fetches all jobs and runs both start and stop taggers on each one.
Types ¶
type AppTagger ¶
type AppTagger struct {
// contains filtered or unexported fields
}
AppTagger detects applications by matching patterns in job scripts. It loads application patterns from an external configuration directory and can dynamically reload configuration when files change. When a job script matches a pattern, the corresponding application tag is automatically applied.
func (*AppTagger) EventCallback ¶
func (t *AppTagger) EventCallback()
EventCallback is triggered when the configuration directory changes. It reloads all application pattern files from the watched directory.
func (*AppTagger) EventMatch ¶
EventMatch checks if a filesystem event should trigger configuration reload. It returns true if the event path contains "apps".
func (*AppTagger) Match ¶
Match attempts to detect the application used by a job by analyzing its job script. It fetches the job metadata, extracts the job script, and matches it against all configured application patterns using regular expressions. If a match is found, the corresponding application tag is added to the job. Multiple application tags can be applied if patterns for different apps match.
func (*AppTagger) Register ¶
Register initializes the AppTagger by loading application patterns from external folder. It sets up a file watch on ./var/tagger/apps if it exists, allowing for dynamic configuration updates without restarting the application. Returns an error if the configuration path does not exist or cannot be read.
type JobClassTagger ¶
type JobClassTagger struct {
// contains filtered or unexported fields
}
JobClassTagger classifies jobs based on configurable rules that evaluate job metrics and properties. Rules are loaded from an external configuration directory and can be dynamically reloaded when files change. When a job matches a rule, it is tagged with the corresponding classification and an optional hint message.
func (*JobClassTagger) EventCallback ¶
func (t *JobClassTagger) EventCallback()
EventCallback is triggered when the configuration directory changes. It reloads parameters and all rule files from the watched directory. FIXME: Only process the file that caused the event
func (*JobClassTagger) EventMatch ¶
func (t *JobClassTagger) EventMatch(s string) bool
EventMatch checks if a filesystem event should trigger configuration reload. It returns true if the event path contains "jobclasses".
func (*JobClassTagger) Match ¶
func (t *JobClassTagger) Match(job *schema.Job)
Match evaluates all classification rules against a job and applies matching tags. It retrieves job statistics and metric configurations, then tests each rule's requirements and main expression. For each matching rule, it:
- Applies the classification tag to the job
- Generates and stores a hint message based on the rule's template
The function constructs an evaluation environment containing:
- Job properties (duration, cores, nodes, state, etc.)
- Metric statistics (min, max, avg) and their configured limits
- Shared parameters defined in parameters.json
- Computed variables from the rule definition
Rules are evaluated in arbitrary order. Multiple rules can match and apply their tags to the same job. Hint messages from all matching rules are collected and stored as a combined message in the job metadata.
func (*JobClassTagger) Register ¶
func (t *JobClassTagger) Register() error
Register initializes the JobClassTagger by loading parameters and classification rules from external folder. It sets up a file watch on ./var/tagger/jobclasses if it exists, allowing for dynamic configuration updates without restarting the application. Returns an error if the configuration path does not exist or cannot be read.
type JobRepository ¶
type JobRepository interface {
// HasTag checks if a job already has a specific tag
HasTag(jobID int64, tagType string, tagName string) bool
// AddTagOrCreateDirect adds a tag to a job or creates it if it doesn't exist
AddTagOrCreateDirect(jobID int64, tagType string, tagName string) (tagID int64, err error)
// UpdateMetadata updates job metadata with a key-value pair
UpdateMetadata(job *schema.Job, key, val string) (err error)
}
JobRepository defines the interface for job database operations needed by the tagger. This interface allows for easier testing and decoupling from the concrete repository implementation.
type JobTagger ¶
type JobTagger struct {
// contains filtered or unexported fields
}
JobTagger coordinates multiple taggers that run at different job lifecycle events. It maintains separate lists of taggers that run when jobs start and when they stop.
func (*JobTagger) JobStartCallback ¶
JobStartCallback is called when a job starts. It runs all registered start taggers (e.g., application detection) on the job.
func (*JobTagger) JobStopCallback ¶
JobStopCallback is called when a job completes. It runs all registered stop taggers (e.g., job classification) on the job.
type RuleFormat ¶
type RuleFormat struct {
// Name is a human-readable description of the rule
Name string `json:"name"`
// Tag is the classification tag to apply if the rule matches
Tag string `json:"tag"`
// Parameters are shared values referenced in the rule (e.g., thresholds)
Parameters []string `json:"parameters"`
// Metrics are the job metrics required for this rule (e.g., "cpu_load", "mem_used")
Metrics []string `json:"metrics"`
// Requirements are boolean expressions that must be true for the rule to apply
Requirements []string `json:"requirements"`
// Variables are computed values used in the rule expression
Variables []Variable `json:"variables"`
// Rule is the boolean expression that determines if the job matches
Rule string `json:"rule"`
// Hint is a template string that generates a message when the rule matches
Hint string `json:"hint"`
}
RuleFormat defines the JSON structure for job classification rules. Each rule specifies requirements, metrics to analyze, variables to compute, and the final rule expression that determines if the job matches the classification.
type Tagger ¶
type Tagger interface {
// Register initializes the tagger and loads any required configuration.
// It should be called once before the tagger is used.
Register() error
// Match evaluates the tagger's rules against a job and applies appropriate tags.
// It is called for each job that needs to be evaluated.
Match(job *schema.Job)
}
Tagger is the interface that must be implemented by all tagging components. Taggers can be registered at job start or stop events to automatically apply tags.
type TaggerInfo ¶
type TaggerInfo struct {
Name string `json:"name"`
Type string `json:"type"`
Running bool `json:"running"`
}
TaggerInfo holds metadata about a tagger for JSON serialization.
func ListTaggers ¶
func ListTaggers() []TaggerInfo
ListTaggers returns information about all known taggers with their current running status.
type Variable ¶
type Variable struct {
// Name is the variable identifier used in rule expressions
Name string `json:"name"`
// Expr is the expression to evaluate (must return a numeric value)
Expr string `json:"expr"`
}
Variable defines a named expression that can be computed and reused in rules. Variables are evaluated before the main rule and their results are added to the environment.
Source Files