Help us shape the future of Go! Take the 2026 Go Developer Survey

structs

package
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 3, 2017 License: MIT Imports: 2 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AllocationResources

type AllocationResources struct {
	MemoryMB      int
	CPUMHz        int
	DiskMB        int
	MemoryPercent float64
	CPUPercent    float64
	DiskPercent   float64
}

AllocationResources represents the allocation resource utilization.

type ClusterAllocation

type ClusterAllocation struct {
	// NodeCount is the number of worker nodes in a ready and non-draining state across
	// the cluster.
	NodeCount int

	// ScalingMetric indicates the most-utilized allocation resource across the cluster.
	// The most-utilized resource is prioritized when making scaling decisions like
	// identifying the least-allocated worker node.
	ScalingMetric string

	// MaxAllowedUtilization represents the max allowed cluster utilization after
	// considering node fault-tolerance and task group scaling overhead.
	MaxAllowedUtilization int

	// ClusterTotalAllocationCapacity is the total allocation capacity across the cluster.
	TotalCapacity AllocationResources

	// ClusterUsedAllocationCapacity is the consumed allocation capacity across the cluster.
	UsedCapacity AllocationResources

	// TaskAllocation represents the total allocation requirements of a single instance
	// (count 1) of all running jobs across the cluster. This is used to practively
	// ensure the cluster has sufficient available capacity to scale each task by +1
	// if an increase in capacity is required.
	TaskAllocation AllocationResources

	// NodeList is a list of all worker nodes in a known good state.
	NodeList []string

	// NodeAllocations is a slice of node allocations.
	NodeAllocations []*NodeAllocation

	// ScalingDirection is the direction in/out of cluster scaling we require after
	// performning the proper evalutation.
	ScalingDirection string

	// LastScalingEvent represents the last time the daemon attempted a scaling action.
	LastScalingEvent time.Time
}

ClusterAllocation is the central object used to track cluster status and the data required to make scaling decisions.

type ClusterScaling

type ClusterScaling struct {
	// Enabled indicates whether cluster scaling actions are permitted.
	Enabled bool `mapstructure:"enabled"`

	// MaxSize in the maximum number of instances the nomad node worker count is
	// allowed to reach. This stops runaway increases in size due to misbehaviour
	// but should be set high enough to accomodate usual workload peaks.
	MaxSize int `mapstructure:"max_size"`

	// MinSize is the minimum number of instances that should be present within
	// the nomad node worker pool.
	MinSize int `mapstructure:"min_size"`

	// CoolDown is the number of seconds after a scaling activity completes before
	// another can begin.
	CoolDown float64 `mapstructure:"cool_down"`

	// NodeFaultTolerance is the number of Nomad worker nodes the cluster can
	// support losing, whilst still maintaining all existing workload.
	NodeFaultTolerance int `mapstructure:"node_fault_tolerance"`

	// AutoscalingGroup is the name of the ASG assigned to the Nomad worker nodes.
	AutoscalingGroup string `mapstructure:"autoscaling_group"`
}

ClusterScaling is the configuration struct for the Nomad worker node scaling activites.

type Config

type Config struct {
	// Consul is the location of the Consul instance or cluster endpoint to query
	// (may be an IP address or FQDN) with port.
	Consul string `mapstructure:"consul"`

	// Nomad is the location of the Nomad instance or cluster endpoint to query
	// (may be an IP address or FQDN) with port.
	Nomad string `mapstructure:"nomad"`

	// LogLevel is the level at which the application should log from.
	LogLevel string `mapstructure:"log_level"`

	// Enforce is the boolean falg which dicates whether or not scaling events are
	// actioned, or whether the application runs in report only mode.
	Enforce bool `mapstructure:"enforce"`

	// Region represents the AWS region the cluster resides in.
	Region string `mapstructure:"aws_region"`

	// ClusterScaling is the configuration struct that controls the basic Nomad
	// worker node scaling.
	ClusterScaling *ClusterScaling `mapstructure:"cluster_scaling"`

	// JobScaling is the configuration struct that controls the basic Nomad
	// job scaling.
	JobScaling *JobScaling `mapstructure:"job_scaling"`

	// Telemetry is the configuration struct that controls the telemetry settings.
	Telemetry *Telemetry `mapstructure:"telemetry"`

	// setKeys is the list of config keys that were overridden by the user.
	SetKeys map[string]struct{}

	// ConsulClient provides a client to interact with the Consul API.
	ConsulClient ConsulClient

	// NomadClient provides a client to interact with the Nomad API.
	NomadClient NomadClient
}

Config is the main configuration struct used to configure the replicator application.

func (*Config) Merge

func (c *Config) Merge(o *Config)

Merge takes the user override parameters and merges these into the default config parameters. User overrides will always take priority.

func (*Config) WasSet

func (c *Config) WasSet(key string) bool

WasSet determines if the given key was set by the user or uses the default values.

type ConsulClient

type ConsulClient interface {
	// GetJobScalingPolicies provides a list of Nomad jobs with a defined scaling
	// policy document at a specified Consuk Key/Value Store location. Supports
	// the use of an ACL token if required by the Consul cluster.
	GetJobScalingPolicies(*Config, NomadClient) ([]*JobScalingPolicy, error)
}

The ConsulClient interface is used to provide common method signatures for interacting with the Consul API.

type GroupScalingPolicy

type GroupScalingPolicy struct {
	// GroupName is the jobs Group name which this scaling policy represents.
	GroupName string `json:"name"`

	// TaskResources is a list
	Tasks TaskAllocation `json:"task_resources"`

	// ScalingMetric represents the most-utilized resource within the task group.
	ScalingMetric string

	// Scaling is a list of Scaling objects.
	Scaling *Scaling
}

GroupScalingPolicy represents the scaling policy of an individual group within a signle job.

type JobScaling

type JobScaling struct {
	// Enabled indicates whether job scaling actions are permitted.
	Enabled bool `mapstructure:"enabled"`

	// ConsulToken is the Consul ACL token used to access KeyValues from a
	// secure Consul installation.
	ConsulToken string `mapstructure:"consul_token"`

	// ConsulKeyLocation is the Consul key location where scaling policies are
	// defined.
	ConsulKeyLocation string `mapstructure:"consul_key_location"`
}

JobScaling is the configuration struct for the Nomad job scaling activities.

type JobScalingPolicies

type JobScalingPolicies []*JobScalingPolicy

JobScalingPolicies is a list of ScalingPolicy objects.

type JobScalingPolicy

type JobScalingPolicy struct {
	// JobName is the name of the Nomad job represented by the Consul Key/Value.
	JobName string

	// Enabled is a boolean falg which dictates whether scaling events for the job
	// should be enforced and is used for testing purposes.
	Enabled bool `json:"enabled"`

	// GroupScalingPolicies is a list of GroupScalingPolicy objects.
	GroupScalingPolicies []*GroupScalingPolicy `json:"groups"`
}

JobScalingPolicy is a struct which represents an individual job scaling policy document.

type NodeAllocation

type NodeAllocation struct {
	// NodeID is the unique ID of the worker node.
	NodeID string

	// NodeIP is the private IP of the worker node.
	NodeIP string

	// UsedCapacity represents the percentage of total cluster resources consumed by
	// the worker node.
	UsedCapacity AllocationResources
}

NodeAllocation describes the resource consumption of a specific worker node.

type NomadClient

type NomadClient interface {
	// ClusterAllocationCapacity determines the total cluster capacity and current
	// number of worker nodes.
	ClusterAllocationCapacity(*ClusterAllocation) error

	// ClusterAssignedAllocation determines the consumed capacity across the
	// cluster and tracks the resource consumption of each worker node.
	ClusterAssignedAllocation(*ClusterAllocation) error

	// DrainNode places a worker node in drain mode to stop future allocations and
	// migrate existing allocations to other worker nodes.
	DrainNode(string) error

	// EvaluateClusterCapacity determines if a cluster scaling action is required.
	EvaluateClusterCapacity(*ClusterAllocation, *Config) (bool, error)

	// EvaluateJobScaling compares the consumed resource percentages of a Job group
	// against its scaling policy to determine whether a scaling event is required.
	EvaluateJobScaling([]*JobScalingPolicy)

	// GetAllocationStats discovers the resources consumed by a particular Nomad
	// allocation.
	GetAllocationStats(*nomad.Allocation, *GroupScalingPolicy)

	// GetJobAllocations identifies all allocations for an active job.
	GetJobAllocations([]*nomad.AllocationListStub, *GroupScalingPolicy)

	// LeaderCheck determines if the node running replicator is the gossip pool
	// leader.
	LeaderCheck() bool

	// LeaseAllocatedNode determines the worker node consuming the least amount of
	// the cluster's mosted-utilized resource.
	LeastAllocatedNode(*ClusterAllocation) (string, string)

	// MostUtilizedResource calculates which resource is most-utilized across the
	// cluster. The worst-case allocation resource is prioritized when making
	// scaling decisions.
	MostUtilizedResource(*ClusterAllocation)

	// IsJobRunning checks to see whether the specified jobID has any currently
	// task groups on the cluster.
	IsJobRunning(string) bool

	// JobScale
	JobScale(*JobScalingPolicy)

	// TaskAllocationTotals calculates the allocations required by each running
	// job and what amount of resources required if we increased the count of
	// each job by one. This allows the cluster to proactively ensure it has
	// sufficient capacity for scaling events and deal with potential node failures.
	TaskAllocationTotals(*ClusterAllocation) error
}

NomadClient exposes all API methods needed to interact with the Nomad API, evaluate cluster capacity and allocations and make scaling decisions.

type Scaling

type Scaling struct {
	// Min in the minimum number of tasks the job should have running at any one
	// time.
	Min int `json:"min"`

	// Max in the maximum number of tasks the job should have running at any one
	// time.
	Max int `json:"max"`

	// ScaleDirection is populated by either out/in/none depending on the evalution
	// of a scaling event happening.
	ScaleDirection string

	// ScaleOut is the job scaling out policy which will contain the thresholds
	// which control scaling activies.
	ScaleOut *scaleout `json:"scaleout"`

	// ScaleIn is the job scaling in policy which will contain the thresholds
	// which control scaling activies.
	ScaleIn *scalein `json:"scalein"`
}

Scaling struct represents the scaling policy of a Nomad Job Group as well as details of any scaling activities which should take place during the current deamon run.

type TaskAllocation

type TaskAllocation struct {
	// TaskName is the name given to the task within the job specficiation.
	TaskName string

	// Resources tracks the resource requirements defined in the job spec and the
	// real-time utilization of those resources.
	Resources AllocationResources
}

TaskAllocation describes the resource requirements defined in the job specification.

type Telemetry

type Telemetry struct {
	// StatsdAddress specifies the address of a statsd server to forward metrics
	// to and should include the port.
	StatsdAddress string `mapstructure:"statsd_address"`
}

Telemetry is the struct that control the telemetry configuration. If a value is present then telemetry is enabled. Currently statsd is only supported for sending telemetry.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL