aws

package
v0.8.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 29, 2025 License: Apache-2.0 Imports: 21 Imported by: 0

Documentation

Overview

Package aws implements driver

Index

Constants

View Source
const HostReserved = "reserved"

HostReserved - custom status to set in the host for simplifying parallel ops in between the updates

Variables

This section is empty.

Functions

This section is empty.

Types

type Config

type Config struct {
	Region    string `json:"region"`     // AWS Region to connect to
	KeyID     string `json:"key_id"`     // AWS AMI Key ID
	SecretKey string `json:"secret_key"` // AWS AMI Secret Key

	// Optional
	AccountIDs   []string          `json:"account_ids"`   // AWS Trusted account IDs to filter vpc, subnet, sg, images, snapshots...
	InstanceTags map[string]string `json:"instance_tags"` // AWS Instance tags to use when this node provision them
	InstanceKey  string            `json:"instance_key"`  // AWS Instance Key Pair name to use while creating of the instance

	// Manage the AWS dedicated hosts to keep them busy and deallocate when not needed
	// Key of the map is name of the pool - will be used for identification of the pool
	DedicatedPool map[string]DedicatedPoolRecord `json:"dedicated_pool"`

	// Various options to not hardcode the important numbers
	SnapshotCreateWait util.Duration `json:"snapshot_create_wait"` // Maximum wait time for snapshot availability (create), default: 2h
	ImageCreateWait    util.Duration `json:"image_create_wait"`    // Maximum wait time for image availability (create/copy), default: 2h
}

Config - node driver configuration

func (*Config) Apply

func (c *Config) Apply(config []byte) error

Apply takes json and applies it to the config structure

func (*Config) Validate

func (c *Config) Validate() (err error)

Validate makes sure the config have the required defaults & that the required fields are set

type DedicatedPoolRecord

type DedicatedPoolRecord struct {
	Type  string   `json:"type"`  // Instance type handled by the dedicated hosts pool (example: "mac2.metal")
	Zones []string `json:"zones"` // Where to allocate the dedicated host (example: ["us-west-2a", "us-west-2c"])
	Max   uint     `json:"max"`   // Maximum dedicated hosts to allocate (they sometimes can handle more than 1 capacity slot)

	// AWS has a bug in it's API - when you getting the dedicated hosts availability it can say the
	// host is become available, but in fact it's not. Particularly we see that with mac machines
	// when they are returning after scrubbing. So this delay will add the requested delay between
	// previous state of the dedicated host and available state, so the allocations will not fail.
	PendingToAvailableDelay util.Duration `json:"pending_to_available_delay"`

	// Specifies when the dedicated host can be released after allocation. By default for mac type
	// it's set to [24h] but you can set to half a year or more to keep the host in your pool as
	// long as you need. If you want to set it to lower value for mac, then 24h - please consult
	// the AWS docs regarding that.
	//
	// [24h]: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-mac-instances.html#mac-instance-considerations
	ReleaseDelay util.Duration `json:"release_delay"`

	// Is a special optimization for the Mac dedicated hosts to send them in [scrubbing process] to
	// save money when we can't release the host due to Apple's license of [24 hours] min limit.
	//
	// Details:
	//
	// Apple forces AWS and any of their customers to keep the Mac dedicated hosts allocated for at
	// least [24 hours]. So after allocation you have no way to release the dedicated host even if
	// you don't need it. This makes the mac hosts very pricey for any kind of dynamic allocation.
	// In order to workaround this issue - Aquarium implements optimization to keep the Mac hosts
	// busy with [scrubbing process], which is triggered after the instance stop or termination and
	// puts Mac host in pending state for 1-2hr. That's the downside of optimization, because you
	// not be able to use the machine until it will become available again.
	//
	// That's why this ScrubbingDelay config exists - we need to give Mac host some time to give
	// the workload a chance to utilize the host. If it will not be utilized in this duration - the
	// manager will start the scrubbing process. When the host become old enough - the manager will
	// release it to clean up space for new fresh mac in the roster.
	//
	// * When this option is unset or 0 - no optimization is enabled.
	// * When it's set - then it's a duration to stay idle and then allocate and terminate empty
	// instance to trigger scrubbing.
	//
	// Current implementation is attached to state update, which could be API consuming, so this
	// duration should be >= 1 min, otherwise API requests will be too often.
	//
	// [24 hours]: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-mac-instances.html#mac-instance-considerations
	// [scrubbing process]: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/mac-instance-stop.html
	ScrubbingDelay util.Duration `json:"scrubbing_delay"`
}

DedicatedPoolRecord stores the configuration of AWS dedicated pool of particular type to manage aws ec2 allocate-hosts --availability-zone "us-west-2c" --auto-placement "on" --host-recovery "off" --host-maintenance "off" --quantity 1 --instance-type "mac2.metal"

type Driver

type Driver struct {
	// contains filtered or unexported fields
}

Driver implements provider.Driver interface

func (*Driver) Allocate

func (d *Driver) Allocate(def types.LabelDefinition, metadata map[string]any) (*types.ApplicationResource, error)

Allocate Instance with provided image

It selects the AMI and run instance Uses metadata to fill EC2 instance userdata

func (*Driver) AvailableCapacity

func (d *Driver) AvailableCapacity(_ types.Resources, def types.LabelDefinition) int64

AvailableCapacity allows Fish to ask the driver about it's capacity (free slots) of a specific definition

func (*Driver) Deallocate

func (d *Driver) Deallocate(res *types.ApplicationResource) error

Deallocate the resource

func (*Driver) GetTask

func (d *Driver) GetTask(name, options string) provider.DriverTask

GetTask returns task struct by name

func (*Driver) IsRemote

func (*Driver) IsRemote() bool

IsRemote needed to detect the out-of-node resources managed by this driver

func (*Driver) Name

func (d *Driver) Name() string

Name returns name of the driver

func (*Driver) Prepare

func (d *Driver) Prepare(config []byte) error

Prepare initializes the driver

func (*Driver) SetName

func (d *Driver) SetName(name string)

Name returns name of the gate

func (*Driver) Status

func (d *Driver) Status(res *types.ApplicationResource) (string, error)

Status shows status of the resource

func (*Driver) ValidateDefinition

func (d *Driver) ValidateDefinition(def types.LabelDefinition) error

ValidateDefinition checks LabelDefinition is ok

type Factory

type Factory struct{}

Factory implements provider.ProviderDriverFactory interface

func (*Factory) Name

func (*Factory) Name() string

Name shows name of the driver factory

func (*Factory) New

func (f *Factory) New() provider.Driver

New creates new provider driver

type Options

type Options struct {
	Image         string            `json:"image"`          // ID/Name of the image you want to use (name that contains * is usually a bad idea for reproducibility)
	InstanceType  string            `json:"instance_type"`  // Type of the instance from aws available list
	SecurityGroup string            `json:"security_group"` // ID/Name of the security group to use for the instance
	Tags          map[string]string `json:"tags"`           // Tags to add during instance creation
	EncryptKey    string            `json:"encrypt_key"`    // Use specific encryption key for the new disks
	Pool          string            `json:"pool"`           // Use machine from dedicated pool, otherwise will try to use one with auto-placement

	UserDataFormat string `json:"userdata_format"` // If not empty - will store the resource metadata to userdata in defined format
	UserDataPrefix string `json:"userdata_prefix"` // Optional if need to add custom prefix to the metadata key during formatting

	// TaskImage options
	TaskImageName       string `json:"task_image_name"`        // Create new image with defined name + "-DATE.TIME" suffix
	TaskImageEncryptKey string `json:"task_image_encrypt_key"` // KMS Key ID or Alias in format "alias/<name>" if need to re-encrypt the newly created AMI snapshots
}

Options for label definition

Example:

image: ami-abcdef123456
instance_type: c6a.4xlarge
security_group: sg-abcdef123456
tags:
  somekey: somevalue

func (*Options) Apply

func (o *Options) Apply(options util.UnparsedJSON) error

Apply takes json and applies it to the options structure

func (*Options) Validate

func (o *Options) Validate() error

Validate makes sure the options have the required defaults & that the required fields are set

type TaskImage

type TaskImage struct {
	*types.ApplicationTask     `json:"-"` // Info about the requested task
	*types.LabelDefinition     `json:"-"` // Info about the used label definition
	*types.ApplicationResource `json:"-"` // Info about the processed resource

	Full bool `json:"full"` // Make full (all disks including connected disks), or just the root OS disk image
	// contains filtered or unexported fields
}

TaskImage stores the task data

func (*TaskImage) Clone

func (t *TaskImage) Clone() provider.DriverTask

Clone makes a copy of the initial task to execute

func (*TaskImage) Execute

func (t *TaskImage) Execute() (result []byte, err error)

Execute - Image task could be executed during ALLOCATED & DEALLOCATE ApplicationStatus

func (*TaskImage) Name

func (*TaskImage) Name() string

Name returns name of the task

func (*TaskImage) SetInfo

SetInfo defines information of the environment

type TaskSnapshot

type TaskSnapshot struct {
	*types.ApplicationTask     `json:"-"` // Info about the requested task
	*types.LabelDefinition     `json:"-"` // Info about the used label definition
	*types.ApplicationResource `json:"-"` // Info about the processed resource

	Full bool `json:"full"` // Make full (all disks including OS image), or just the additional disks snapshot
	// contains filtered or unexported fields
}

TaskSnapshot stores the task data

func (*TaskSnapshot) Clone

func (t *TaskSnapshot) Clone() provider.DriverTask

Clone makes a copy of the initial task to execute

func (*TaskSnapshot) Execute

func (t *TaskSnapshot) Execute() (result []byte, err error)

Execute - Snapshot task could be executed during ALLOCATED & DEALLOCATE ApplicationStatus

func (*TaskSnapshot) Name

func (*TaskSnapshot) Name() string

Name returns name of the task

func (*TaskSnapshot) SetInfo

SetInfo defines information of the environment

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL