gce

package
v0.0.0-...-d35ba01 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 29, 2025 License: Apache-2.0 Imports: 33 Imported by: 0

Documentation

Overview

Package gce holds various helpers for testing the agents on GCE. This library is intended to work when run via Kokoro.

This library needs the following environment variables to be defined: PROJECT: What GCP project to use. ZONES: What GCP zones to run in as a comma-separated list, with optional integer weights attached to each zone, in the format: zone1=weight1,zone2=weight2. Any zone with no weight is given a default weight of 1.

The following variables are optional:

TEST_UNDECLARED_OUTPUTS_DIR: A path to a directory to write log files into. By default, a new temporary directory is created.

NETWORK_NAME: What GCP network name to use. KOKORO_BUILD_ID: supplied by Kokoro. KOKORO_BUILD_ARTIFACTS_SUBDIR: supplied by Kokoro. LOG_UPLOAD_URL_ROOT: A URL prefix (remember the trailing "/") where the test logs will be uploaded. If unset, this will point to ops-agents-public-buckets-test-logs, which should work for all tests triggered from GitHub.

USE_INTERNAL_IP: If set to "true", pass --no-address to gcloud when creating VMs. This will not create an external IP address for that VM (because those are expensive), and instead the VM will use cloud NAT to get to the external internet. ssh-ing to the VM is done via its internal IP address. Only useful on Kokoro.

SERVICE_EMAIL: If provided, which service account to use for spawned VMs. The default is the project's "Compute Engine default service account". TRANSFERS_BUCKET: A GCS bucket name to use to transfer files to testing VMs. The default is "stackdriver-test-143416-file-transfers". INSTANCE_SIZE: What size of VMs to make. Passed in to gcloud as --machine-type. If provided, this value overrides the selection made by the callers to this library.

Index

Constants

View Source 
const (
	// SuggestedTimeout is a recommended limit on how long a test should run before it is cancelled.
	// This cancellation does not happen automatically; each TestFoo() function must explicitly call
	// context.WithTimeout() to enable a timeout. It's a good idea to do this so that if a command
	// hangs, the test still will be cancelled eventually and all its VMs will be cleaned up.
	// This amount needs to be less than 4 hours, which is the limit on how long a Kokoro build can
	// take before it is forcibly killed.
	SuggestedTimeout = 2 * time.Hour

	// QueryMaxAttempts is the default number of retries when calling WaitForMetricSeries.
	// Retries are spaced by 10 seconds, so 40 retries denotes 6 minutes 40 seconds total.
	QueryMaxAttempts = 40 // 6 minutes 40 seconds total.

	// LogQueryMaxAttempts is the default number of retries when calling WaitForLog.
	// Retries are spaced by 30 seconds, so 15 retries denotes 7 minutes 30 seconds total.
	LogQueryMaxAttempts = 15 // 7 minutes 30 seconds total.

	DenyEgressTrafficTag = "test-ops-agent-deny-egress-traffic-tag"

	TraceQueryMaxAttempts = QueryMaxAttempts / traceQueryDerate
)

Variables

View Source 
var (
	ErrInvalidIteratorLength = errors.New("iterator length is less than the defined minimum length")
)

Functions

func ArbitraryImageSpec 

func ArbitraryImageSpec() string

ArbitraryImageSpec picks an arbitrary element from IMAGE_SPECS and returns it.

func AssertLogMissing 

func AssertLogMissing(ctx context.Context, logger *log.Logger, vm *VM, logNameRegex string, window time.Duration, query string) error

AssertLogMissing looks in the logging backend for a log matching the given query and returns success if no data is found. To consider possible transient errors while querying the backend we make queryMaxAttemptsMetricMissing query attempts.

func AssertMetricMissing 

func AssertMetricMissing(ctx context.Context, logger *log.Logger, vm *VM, metric string, isPrometheus bool, window time.Duration) error

AssertMetricMissing looks for data of a metric and returns success if no data is found. To consider possible transient errors while querying the backend we make queryMaxAttemptsMetricMissing query attempts.

func CleanupKeysOrDie 

func CleanupKeysOrDie()

CleanupKeysOrDie deletes ssh key files created in init(). It is intended to be called from inside TestMain() after tests have finished running.

func DeleteInstance 

func DeleteInstance(logger *log.Logger, vm *VM) error

DeleteInstance deletes the given VM instance synchronously. Does nothing if the VM was already deleted. Doesn't take a Context argument because even if the test has timed out or is cancelled, we still want to delete the VMs.

func DeleteManagedInstanceGroupVM 

func DeleteManagedInstanceGroupVM(logger *log.Logger, migVM *ManagedInstanceGroupVM) error

DeleteManagedInstanceGroupVM deletes the given Managed Instance Group VM instance synchronously. Does nothing if the Managed Instance Group VM was already deleted. Doesn't take a Context argument because even if the test has timed out or is cancelled, we still want to delete the Managed Instance Group.

func FetchMetadata 

func FetchMetadata(ctx context.Context, logger *log.Logger, vm *VM) (map[string]string, error)

FetchMetadata retrieves the instance metadata for the given VM.

func FirstImageSpec 

func FirstImageSpec() string

FirstImageSpec picks the first element from IMAGE_SPECS and returns it.

func InstallGrpcurlIfNeeded 

func InstallGrpcurlIfNeeded(ctx context.Context, logger *log.Logger, vm *VM) error

InstallGrpcurlIfNeeded installs grpcurl on instances that don't already have it installed.

func InstallGsutilIfNeeded 

func InstallGsutilIfNeeded(ctx context.Context, logger *log.Logger, vm *VM) error

InstallGsutilIfNeeded installs gsutil on instances that don't already have it installed. This is only currently the case for some old versions of SUSE.

func IsARM 

func IsARM(imageSpec string) bool

func IsCentOS 

func IsCentOS(imageSpec string) bool

func IsDLVMImage 

func IsDLVMImage(imageSpec string) bool

func IsDebianBased 

func IsDebianBased(imageSpec string) bool

func IsExhaustedRetriesMetricError 

func IsExhaustedRetriesMetricError(err error) bool

IsExhaustedRetriesMetricError returns true if the given error is an "exhausted retries" error returned from WaitForMetric.

func IsOpsAgentUAPPlugin 

func IsOpsAgentUAPPlugin() bool

func IsRHEL 

func IsRHEL(imageSpec string) bool

func IsSLESVM 

func IsSLESVM(vm *VM) bool

func IsSUSEImageSpec 

func IsSUSEImageSpec(imageSpec string) bool

func IsSUSEVM 

func IsSUSEVM(vm *VM) bool

func IsWindows 

func IsWindows(imageSpec string) bool

IsWindows returns whether the given image spec is a version of Windows (including Microsoft SQL Server).

func IsWindows2016 

func IsWindows2016(imageSpec string) bool

IsWindows2016 returns whether the given image is a Windows 2016 image.

func IsWindows2019 

func IsWindows2019(imageSpec string) bool

IsWindows2019 returns whether the given image is a Windows 2019 image.

func IsWindowsCore 

func IsWindowsCore(imageSpec string) bool

IsWindowsCore returns whether the given image spec is a version of Windows core.

func MapToCommaSeparatedList 

func MapToCommaSeparatedList(mapping map[string]string) string

MapToCommaSeparatedList converts a map of key-value pairs into a form that gcloud will accept, which is a comma separated list with "=" between each key-value pair. For example: "KEY1=VALUE1,KEY2=VALUE2"

func OSKind 

func OSKind(imageSpec string) string

OSKind returns "linux" or "windows" based on the given image spec.

func QueryLog 

func QueryLog(ctx context.Context, logger *log.Logger, vm *VM, logNameRegex string, window time.Duration, query string, maxAttempts int) (*cloudlogging.Entry, error)

QueryLog looks in the logging backend for a log matching the given query, over the trailing time interval specified by the given window. Returns the first log entry found, or an error if the log could not be found after some retries.

func RemoveExternalIP 

func RemoveExternalIP(ctx context.Context, logger *log.Logger, vm *VM) error

RemoveExternalIP deletes the external ip for an instance.

func RestartInstance 

func RestartInstance(ctx context.Context, logger *log.Logger, vm *VM) error

RestartInstance stops and starts the instance. It also waits for the instance to be started up post-restart.

func RetrieveContent 

func RetrieveContent(ctx context.Context, logger *log.Logger, vm *VM, remotePath string) (content string, err error)

RetrieveContent retrieves the file content from the the given file path from the remote VM

func RunForEachImage 

func RunForEachImage(t *testing.T, testBody func(t *testing.T, imageSpec string))

RunForEachImage runs a subtest for each image defined in IMAGE_SPECS.

func SetEnvironmentVariables 

func SetEnvironmentVariables(ctx context.Context, logger *log.Logger, vm *VM, envVariables map[string]string) error

SetEnvironmentVariables sets the environment variables in the envVariables map on the given vm in a os-dependent way. On Windows VMs, variables set this way are visible to all processes. On Linux VMs, variables set this way are visible to the Ops Agent services only.

func SetGcloudPath 

func SetGcloudPath(path string)

SetGcloudPath configures this library to use the passed-in gcloud binary instead of the default gcloud installed on the system.

func SetupGcloudConfigDir 

func SetupGcloudConfigDir(ctx context.Context, directory string) error

SetupGcloudConfigDir sets up a new gcloud configuration directory. This copies the contents of the context-specified configuration directory into the new directory. This only works on Linux.

func SetupLogger 

func SetupLogger(t *testing.T) *logging.DirectoryLogger

SetupLogger creates a new DirectoryLogger that will write to a directory based on t.Name() inside the directory TEST_UNDECLARED_OUTPUTS_DIR. If creating the logger fails, it will abort the test. At the end of the test, the logger will be cleaned up. TODO: Move this function along with logLocation() into the agents package, since nothing else in this file depends on DirectoryLogger anymore.

func StartInstance 

func StartInstance(ctx context.Context, logger *log.Logger, vm *VM) error

StartInstance boots a previously-stopped VM instance. Also waits for the instance to be started up.

func StopInstance 

func StopInstance(ctx context.Context, logger *log.Logger, vm *VM) error

StopInstance shuts down a VM instance.

func SyslogLocation 

func SyslogLocation(imageSpec string) string

SyslogLocation returns a filesystem path to the system log. This function assumes the image spec is some kind of Linux.

func UploadContent 

func UploadContent(ctx context.Context, logger *log.Logger, vm *VM, content io.Reader, remotePath string) (err error)

UploadContent takes an io.Reader and uploads its contents as a file to a given path on the given VM.

In order for this function to work, the currently active application default credentials (GOOGLE_APPLICATION_CREDENTIALS) need to be able to upload to fileTransferBucket, and also the role running on the remote VM needs to be given permission to read from that bucket. This was accomplished by adding the "Compute Engine default service account" for PROJECT as a "Storage Object Viewer" and "Storage Object Creator" on the bucket.

When making changes to this function, please run gce_testing_test.go (manually).

func WaitForLog 

func WaitForLog(ctx context.Context, logger *log.Logger, vm *VM, logNameRegex string, window time.Duration, query string) error

WaitForLog looks in the logging backend for a log matching the given query, over the trailing time interval specified by the given window. Returns an error if the log could not be found after LogQueryMaxAttempts retries.

func WaitForMetric 

func WaitForMetric(ctx context.Context, logger *log.Logger, vm *VM, metric string, window time.Duration, extraFilters []string, isPrometheus bool) (*monitoringpb.TimeSeries, error)

WaitForMetric looks for the given metrics in the backend and returns it if it exists. An error is returned otherwise. This function will retry "no data" errors a fixed number of times. This is useful because it takes time for monitoring data to become visible after it has been uploaded.

func WaitForMetricSeries 

func WaitForMetricSeries(ctx context.Context, logger *log.Logger, vm *VM, metric string, window time.Duration, extraFilters []string, isPrometheus bool, minimumRequiredSeries int) ([]*monitoringpb.TimeSeries, error)

WaitForMetricSeries looks for the given metrics in the backend and returns a slice if it exists. An error is returned otherwise. This function will retry "no data" errors a fixed number of times. This is useful because it takes time for monitoring data to become visible after it has been uploaded.

func WaitForTrace 

func WaitForTrace(ctx context.Context, logger *log.Logger, vm *VM, options WaitForTraceOptions) (*cloudtrace.Trace, error)

WaitForTrace looks for any trace from the given VM in the backend and returns it if it exists. An error is returned otherwise. This function will retry "no data" errors a fixed number of times. This is useful because it takes time for trace data to become visible after it has been uploaded.

Only the ProjectId and TraceId fields are populated. To get other fields, including spans, call traceClient.GetTrace with the TraceID returned from this function.

func WaitForTraceDeprecated 

func WaitForTraceDeprecated(ctx context.Context, logger *log.Logger, vm *VM, window time.Duration) (*cloudtrace.Trace, error)

Temporary overload for WaitForTrace. TODO: Switch all callers to WaitForTrace and delete this.

func WithGcloudConfigDir 

func WithGcloudConfigDir(ctx context.Context, directory string) context.Context

WithGcloudConfigDir returns a context that records the desired value of the gcloud configuration directory. Invoking RunGcloud with that context will set the configuration directory for the gcloud command to that value.

Types

type CommandOutput 

type CommandOutput struct {
	Stdout string
	Stderr string
}

CommandOutput holds the textual output from running a subprocess.

func AddTagToVm 

func AddTagToVm(ctx context.Context, logger *log.Logger, vm *VM, tags []string) (CommandOutput, error)

func DescribeVMDisk 

func DescribeVMDisk(ctx context.Context, logger *log.Logger, vm *VM) (CommandOutput, error)

DescribeVMDisk queries the VM disk information.

func RemoveTagFromVm 

func RemoveTagFromVm(ctx context.Context, logger *log.Logger, vm *VM, tags []string) (CommandOutput, error)

func RunGcloud 

func RunGcloud(ctx context.Context, logger *log.Logger, stdin string, args []string) (CommandOutput, error)

RunGcloud invokes a gcloud binary from runfiles and waits until it finishes. Returns the stdout and stderr and an error if the binary had a nonzero exit code. args is a slice containing the arguments to pass to gcloud.

Note: most calls to this function could be replaced by calls to the Compute API (https://cloud.google.com/compute/docs/reference/rest/v1). Various pros/cons of shelling out to gcloud vs using the Compute API are discussed here: http://go/sdi-gcloud-vs-api

func RunRemotely 

func RunRemotely(ctx context.Context, logger *log.Logger, vm *VM, command string) (_ CommandOutput, err error)

RunRemotely runs a command on the provided VM. The command should be a shell command if the VM is Linux, or powershell if the VM is Windows. Returns the stdout and stderr, plus an error if there was a problem.

'command' is what to run on the machine. Example: "cat /tmp/foo; echo hello" For extremely long commands, use RunScriptRemotely instead.

When making changes to this function, please test them by running gce_testing_test.go (manually).

func RunRemotelyStdin 

func RunRemotelyStdin(ctx context.Context, logger *log.Logger, vm *VM, stdin io.Reader, command string) (_ CommandOutput, err error)

RunRemotelyStdin is just like RunRemotely but it accepts an io.Reader for what data to pass in over standard input to the command.

func RunScriptRemotely 

func RunScriptRemotely(ctx context.Context, logger *log.Logger, vm *VM, scriptContents string, flags []string, env map[string]string) (CommandOutput, error)

RunScriptRemotely runs a script on the given VM. The script should be a shell script for a Linux VM and powershell for a Windows VM. env is a map containing environment variables to provide to the script as it runs. The environment variables and the flags will be wrapped in quotes. This function is necessary to handle long commands, particularly on Windows, since there is a length limit on the commands you can pass to RunRemotely: powershell will complain if its -EncodedCommand parameter is too long. It is highly recommended that any powershell script passed in here start with: $ErrorActionPreference = 'Stop' This will cause a broader class of errors to be reported as an error (nonzero exit code) by powershell.

type ManagedInstanceGroupVM 

type ManagedInstanceGroupVM struct {
	*VM
}

ManagedInstanceGroupVM represents an individual VM in a Managed Instace Group.

func CreateManagedInstanceGroupVM 

func CreateManagedInstanceGroupVM(origCtx context.Context, logger *log.Logger, options VMOptions) (*ManagedInstanceGroupVM, error)

CreateManagedInstanceGroupVM launches a new Managed Instance Group VM instance based on the given options. Also waits for the instance to be reachable over ssh. Returns a ManagedInstanceGroupVM object or an error (never both). The caller is responsible for deleting the ManagedInstanceGroupVM if (and only if) the returned error is nil.

func SetupManagedInstanceGroupVM 

func SetupManagedInstanceGroupVM(ctx context.Context, t *testing.T, logger *log.Logger, options VMOptions) *ManagedInstanceGroupVM

SetupManagedInstanceGroupVM creates an individual VM instance in a Managed Instance Group according to the given options. If the ManagedInstanceGroupVM creation fails, it will abort the test. At the end of the test, the ManagedInstanceGroupVM will be cleaned up.

func (ManagedInstanceGroupVM) AppHubWorkloadName 

func (migVM ManagedInstanceGroupVM) AppHubWorkloadName() string

func (ManagedInstanceGroupVM) InstanceTemplateName 

func (migVM ManagedInstanceGroupVM) InstanceTemplateName() string

func (ManagedInstanceGroupVM) ManagedInstanceGroupName 

func (migVM ManagedInstanceGroupVM) ManagedInstanceGroupName() string

type OS 

type OS struct {
	// The same as ID from /etc/os-release, or "windows".
	ID string
}

OS represents information on the Operating Systems.

type ThreadSafeWriter 

type ThreadSafeWriter struct {
	// contains filtered or unexported fields
}

func (*ThreadSafeWriter) Write 

func (writer *ThreadSafeWriter) Write(p []byte) (int, error)

type VM 

type VM struct {
	Name    string
	Project string
	Network string
	// The VMOptions.ImageSpec used to create the VM.
	ImageSpec   string
	OS          OS
	Zone        string
	MachineType string
	ID          int64
	// The IP address to ssh to. This is the external IP address, unless
	// USE_INTERNAL_IP is set to 'true'. See comment on extractIPAddress() for
	// rationale.
	IPAddress      string
	AlreadyDeleted bool
}

VM represents an individual virtual machine.

func CreateInstance 

func CreateInstance(origCtx context.Context, logger *log.Logger, options VMOptions) (*VM, error)

CreateInstance launches a new VM instance based on the given options. Also waits for the instance to be reachable over ssh. Returns a VM object or an error (never both). The caller is responsible for deleting the VM if (and only if) the returned error is nil.

func SetupVM 

func SetupVM(ctx context.Context, t *testing.T, logger *log.Logger, options VMOptions) *VM

SetupVM creates a new VM according to the given options. If VM creation fails, it will abort the test. At the end of the test, the VM will be cleaned up.

type VMOptions 

type VMOptions struct {
	// Required. Used to pass image/image family & image project in one string.
	//
	// Example Image Specs:
	// Image Family / Project: `<project>:<family>`
	// Specific Image / Project: `<project>=<image>“
	ImageSpec string
	// Optional. Set this to a duration like "3h" or "1d" to configure the VM to
	// be automatically deleted after the specified amount of time. This is
	// a recommended setting for short-lived VMs even if your code calls
	// DeleteInstance(), because this setting will take effect even if your code
	// crashes before calling DeleteInstance(), and besides DeleteInstance() can
	// fail. Calling DeleteInstance() is still recommended even if your code sets
	// a TimeToLive to free up VM resources as soon as possible.
	TimeToLive string
	// Optional. If missing, a random name will be generated.
	Name string
	// Optional. If missing, the environment variable PROJECT will be used.
	Project string
	// Optional. If missing, the environment variable ZONE will be used.
	Zone string
	// Optional.
	Metadata map[string]string
	// Optional.
	Labels map[string]string
	// Optional. If missing, the default is e2-standard-4.
	// Overridden by INSTANCE_SIZE if that environment variable is set.
	MachineType string
	// Optional. If missing, the default is 'global'.
	ImageFamilyScope string
	// Optional. If provided, these arguments are appended on to the end
	// of the "gcloud compute instances create" command.
	ExtraCreateArguments []string
}

VMOptions specifies settings when creating a VM via CreateInstance() or SetupVM().

type WaitForTraceOptions 

type WaitForTraceOptions struct {
	// Trailing time window to include in the query, measured from now.
	Window time.Duration

	// These are filter terms, for the filter field of:
	// https://cloud.google.com/trace/docs/reference/v2/rpc/google.devtools.cloudtrace.v1#listtracesrequest
	// WaitForTraces also adds a filter on the VM's instance ID automatically.
	Filters []string
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.