csp

package
v0.13.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 23, 2026 License: Apache-2.0 Imports: 16 Imported by: 0

Documentation

Overview

Package csp provides direct CSP (Cloud Service Provider) API call utilities. This bypasses CB-Spider for cases where direct SDK calls are more efficient or where CB-Spider does not provide the needed functionality.

Package csp provides direct CSP (Cloud Service Provider) API call utilities.

Index

Constants

View Source 
const BatchStatusCacheTTL = 10 * time.Second

BatchStatusCacheTTL is how long a batch status result is reused.

View Source 
const ManagerTagValue = "cb-tumblebug"

ManagerTagValue is the value TB writes to the sys.manager tag on CSP resources.

Variables

This section is empty.

Functions

func ApplyCredentialKeyMap added in v0.12.11 

func ApplyCredentialKeyMap(provider string, kvList []model.KeyValue) map[string]any

ApplyCredentialKeyMap transforms a credential key-value list using the CSP-specific key mapping. Keys not present in the map are passed through unchanged. Mapped keys with no incoming value are filled with "" so consumers of the secret (e.g. mc-terrarium's vault_kv_secret_v2 data sources) always find the full expected key set — same behavior as init/openbao/openbao-register-creds.py.

func BatchVMStatusCached added in v0.13.1 

func BatchVMStatusCached(ctx context.Context, provider, region string, fn BatchVMStatusFunc, instanceId string) (status string, found bool, err error)

BatchVMStatusCached returns the status of one instance through the shared per-region cache. found=false with err=nil means the CSP's batch response did not include the id (instance gone).

func BuildSecretPath 

func BuildSecretPath(ctx context.Context, provider string) string

BuildSecretPath builds the OpenBao secret path for a given CSP provider based on the credential holder from context. For "admin" holder: "secret/data/csp/{provider}" For other holders: "secret/data/users/{holder}/csp/{provider}"

func BuildSecretPathForHolder added in v0.12.11 

func BuildSecretPathForHolder(holder, provider string) string

BuildSecretPathForHolder builds the OpenBao secret path using holder and provider directly. Both holder and provider are lowercased to stay consistent with BuildSecretPath.

func CheckAvailability added in v0.12.7 

func CheckAvailability(ctx context.Context, q model.AvailabilityQuery) model.AvailabilityResult

CheckAvailability dispatches to the registered checker for q.Provider and caches the result for availabilityTTL. Concurrent misses for the same (provider, region, instanceType, disk) are deduplicated via singleflight.

If no checker is registered for the provider, the function returns a "no-checker" result with Available=true so that callers can proceed. All checker errors are also turned into non-fatal "Available=true" results with the error captured in Reason; this is intentional to avoid blocking provisioning on pre-check failures.

func CheckOpenBaoStatus added in v0.12.25 

func CheckOpenBaoStatus(ctx context.Context) model.OpenBaoStatusInfo

CheckOpenBaoStatus verifies that the OpenBao secret store is usable by CB-Tumblebug: endpoint reachable, initialized, unsealed, and the configured VAULT_TOKEN accepted. It stops at the first failed step, so Message always describes the most actionable problem.

func ClassifyProvisioningFailure added in v0.13.2 

func ClassifyProvisioningFailure(provider, region, attemptedZone, raw string) model.ProvisioningFailure

ClassifyProvisioningFailure converts a CSP error into a ProvisioningFailure.

provider selects the parser; attemptedZone is what CB-Tumblebug requested and is always recorded, because several CSPs never name the zone they rejected. The returned value is always usable: on an unrecognized message the class is FailureUnknown and RawMessage holds the (redacted, trimmed) original.

func CredKey added in v0.12.30 

func CredKey(parts ...string) string

CredKey returns a short, non-reversible id of the given credential values. Cache keys must include it so that a rotated credential yields a new entry.

func EnsurePlaceholderCredentialSecrets added in v0.12.25 

func EnsurePlaceholderCredentialSecrets(ctx context.Context)

EnsurePlaceholderCredentialSecrets writes an all-empty placeholder secret for every known CSP that has no secret yet under the default credential holder path. This keeps consumers that read all CSP secret paths (e.g. mc-terrarium's vault_kv_secret_v2 data sources during `tofu plan`) from hard-failing on CSPs whose credentials were never provided — they fail gracefully at auth time instead. Existing secrets are never touched (CAS-protected), so real credentials always win.

func ExplainCredentialError added in v0.12.30 

func ExplainCredentialError(provider string, err error) string

ExplainCredentialError turns a CSP error into a short, actionable message for the API caller. Unmatched errors are summarized rather than dropped.

func GetString 

func GetString(data map[string]any, key string) string

GetString safely extracts a string value from a map.

func InvalidateSecretCache added in v0.13.1 

func InvalidateSecretCache(path string)

InvalidateSecretCache drops a cached secret (call after writes).

func IsDefinitivePreCreationFailure added in v0.13.4 

func IsDefinitivePreCreationFailure(provider, errorMsg string) bool

IsDefinitivePreCreationFailure reports whether errorMsg represents a failure that occurs before any VM resource can possibly be created on the CSP (e.g. zone capacity exhausted, account quota exceeded, authentication failure, invalid parameter or image-spec mismatch). For such errors, it is impossible for an orphan VM to exist on the CSP, so orphan rescue (CSP API scanning) can be safely bypassed.

func IsManagedByTB added in v0.13.1 

func IsManagedByTB(name string, tags map[string]string) bool

IsManagedByTB reports whether a name/tag set identifies a resource created by TB. Tag keys are compared case-insensitively, with '.', '-' and '_' treated alike.

func IsTBUid added in v0.13.1 

func IsTBUid(name string) bool

IsTBUid reports whether a CSP resource name starts with a TB-generated uid (tb + 18 base32 chars).

func LoadClient added in v0.12.30 

func LoadClient(cache *sync.Map, account, credKey string) (any, bool)

LoadClient returns the client cached for this account and credential.

func LogCSP 

func LogCSP(provider, msg string)

LogCSP logs a CSP-related message with consistent prefix.

func NormalizeFailureMessage added in v0.13.2 

func NormalizeFailureMessage(raw string) string

NormalizeFailureMessage redacts secrets, cuts provider debris and caps length. Exported so callers that store a message without classifying it apply the same treatment.

func NormalizeTagKey added in v0.13.1 

func NormalizeTagKey(k string) string

NormalizeTagKey maps CSP-sanitized label keys (sys_uid, sys-uid) back to TB's sys.uid form.

func ReadOpenBaoSecret 

func ReadOpenBaoSecret(ctx context.Context, path string) (map[string]any, error)

ReadOpenBaoSecret reads a secret from OpenBao at the given path and returns the data map. It validates that VaultToken is set and the secret exists. A context is used for request-scoped cancellation and timeout.

func RedactErr added in v0.13.0 

func RedactErr(err error) string

RedactErr is RedactSecrets over an error, for use where an SDK error is wrapped into a message. It returns "" for a nil error so callers can format it directly.

func RedactSecrets added in v0.13.0 

func RedactSecrets(message string) string

RedactSecrets removes credential material from a message before it is logged or returned to a caller.

SDKs that sign with query parameters - Alibaba Cloud is the one CB-Tumblebug calls directly - put the whole signed URL in their error strings. Wrapping such an error with %w and logging it publishes the access key and signature; the same string is also copied into the errorMsg field of fetch results, so it reaches API clients too. Only the parameter values are removed: the host and the Action stay, which is what makes the error diagnosable.

func RegisterAvailabilityChecker added in v0.12.7 

func RegisterAvailabilityChecker(c AvailabilityChecker)

RegisterAvailabilityChecker registers a CSP-specific checker. It is safe to call from package init() functions.

func RegisterBatchTagHandler 

func RegisterBatchTagHandler(platform string, handler BatchTagHandler)

RegisterBatchTagHandler registers a batch tag upsert handler for a CSP. Called by CSP-specific packages during init().

func RegisterBatchVMControlHandlers added in v0.12.9 

func RegisterBatchVMControlHandlers(provider string, h BatchVMControlHandlers)

RegisterBatchVMControlHandlers registers bulk lifecycle control functions for a CSP. Each CSP package calls this from its init() function.

func RegisterBatchVMStatusHandler added in v0.12.9 

func RegisterBatchVMStatusHandler(provider string, fn BatchVMStatusFunc)

RegisterBatchVMStatusHandler registers a direct-SDK batch VM status function for a CSP. Each CSP package calls this from its init() function.

func RegisterFailureParser added in v0.13.2 

func RegisterFailureParser(p FailureParser)

RegisterFailureParser registers a CSP-specific parser. Safe to call from init().

func RegisterFindVMsByUIDHandler added in v0.13.4 

func RegisterFindVMsByUIDHandler(provider string, fn FindVMsByUIDFunc)

RegisterFindVMsByUIDHandler registers a direct-SDK targeted VM lookup function for a CSP.

func RegisterInventoryHandlers added in v0.13.1 

func RegisterInventoryHandlers(provider string, h InventoryHandlers)

RegisterInventoryHandlers registers direct-SDK inventory functions for a CSP (called from init()).

func RegisterRemediationTerminateHandler added in v0.13.1 

func RegisterRemediationTerminateHandler(provider string, fn BatchVMControlFunc)

RegisterRemediationTerminateHandler registers a direct terminate used ONLY by audit remediation (ghost/untracked VMs). Normal node control keeps going through CB-Spider for this CSP because Spider's terminate performs CSP-specific follow-up cleanup (floating IPs, NICs, firewall, ...).

func StoreBatchStatuses added in v0.13.1 

func StoreBatchStatuses(provider, region string, requested []string, statuses map[string]string)

StoreBatchStatuses seeds the cache from a batch fetch made elsewhere (e.g. the BatchSweeper).

func StoreClient added in v0.12.30 

func StoreClient(cache *sync.Map, account, credKey string, client any) any

StoreClient caches a client for this account and credential, and drops the entries this account had under a superseded credential so they do not accumulate.

func TryBatchUpsertTags 

func TryBatchUpsertTags(ctx context.Context, providerName, region, zone, cspResourceId, resourceType string, tags map[string]string) (bool, error)

TryBatchUpsertTags attempts to upsert multiple tags on a CSP resource in a single API call. resourceType is the CB-Tumblebug label type (e.g., "node", "vNet", "sshKey"). region is the CSP region, zone is the availability zone (used by GCP; can be empty for others). Returns (true, nil) if successfully handled by a direct CSP batch API. Returns (false, nil) if no batch handler exists for this CSP (caller should fall back to Spider). Returns (false, err) if a batch handler exists but failed (caller should fall back to Spider).

func ValidCredentialKeys added in v0.12.26 

func ValidCredentialKeys(provider string) []string

ValidCredentialKeys returns the sorted list of credential keys accepted for the given provider. The provider is platform-resolved first (e.g. "openstack-new01" -> "openstack") so derived CSPs share their base platform's key set. Returns nil when no key set is defined for the provider, in which case keys cannot be validated.

func ValidateCredentialKeys added in v0.12.26 

func ValidateCredentialKeys(provider string, keys []string) error

ValidateCredentialKeys checks that every provided credential key is a recognized key for the given provider. It validates the *format* (spelling) of keys, not their presence: any subset of the accepted keys is valid, so optional keys (e.g. S3AccessKey / S3SecretKey) may be omitted. Keys are case-sensitive. Unknown or empty keys are rejected with an error that lists the accepted keys so the caller can fix the request.

Validation is skipped (returns nil) for providers that have no defined key set, since the accepted format is unknown for them.

func WriteOpenBaoSecret added in v0.12.11 

func WriteOpenBaoSecret(ctx context.Context, path string, data map[string]any) error

WriteOpenBaoSecret writes key-value data to OpenBao at the given KV v2 path (upsert). ctx allows request-scoped cancellation and timeout, consistent with ReadOpenBaoSecret.

func WriteOpenBaoSecretIfAbsent added in v0.12.25 

func WriteOpenBaoSecretIfAbsent(ctx context.Context, path string, data map[string]any) (created bool, err error)

WriteOpenBaoSecretIfAbsent writes key-value data to OpenBao only when no version of the secret exists yet (KV v2 check-and-set with cas=0). This is atomic on the server side, so it can never overwrite a real credential written concurrently. Returns created=false with a nil error when the secret already exists.

Types

type AvailabilityChecker added in v0.12.7 

type AvailabilityChecker interface {
	// Provider returns the CSP identifier (must match model/csp constants).
	Provider() string
	// CheckInstance performs a pre-flight availability query for the given
	// instance type in the given region. Implementations should populate
	// Zones with per-zone disk-category availability when their CSP API
	// supports it. Implementations should return non-fatal results: a true
	// inability to determine availability should be returned as
	// (result with Available=true, Reason="...") with err=nil so that
	// provisioning is never blocked solely by a checker failure.
	CheckInstance(ctx context.Context, q model.AvailabilityQuery) (model.AvailabilityResult, error)
}

AvailabilityChecker is implemented by CSP-specific packages and registered via Register at init() time.

type BatchTagHandler 

type BatchTagHandler func(ctx context.Context, region, zone, cspResourceId, resourceType string, tags map[string]string) error

BatchTagHandler defines the function signature for CSP-specific batch tag upsert. Returns error if the operation fails. The handler should set all given tags on the CSP resource identified by cspResourceId in a single API call. resourceType is the CB-Tumblebug label type (e.g., "node", "vNet", "sshKey"). region is the CSP region (e.g., "us-east-1"), zone is the availability zone (e.g., "us-east-1a").

type BatchVMControlFunc added in v0.12.9 

type BatchVMControlFunc func(ctx context.Context, region string, instanceIds []string) (map[string]string, error)

BatchVMControlFunc sends a lifecycle control action to multiple instances in one SDK call. ctx must carry model.CtxKeyCredentialHolder for credential lookup. region is the CSP-native region identifier (e.g., "ap-northeast-2" for AWS). instanceIds are the CspResourceId values for each VM.

Returns a map of CspResourceId → transient TB status string (e.g., model.StatusSuspending). Missing keys mean the instance was not found or accepted; callers treat them as failed.

func GetBatchVMControlHandler added in v0.12.9 

func GetBatchVMControlHandler(provider, action string) (BatchVMControlFunc, bool)

GetBatchVMControlHandler returns the bulk control function for the given provider and action. action is case-insensitive: "suspend", "resume", "terminate", or "reboot".

func GetRemediationTerminateHandler added in v0.13.1 

func GetRemediationTerminateHandler(provider string) (BatchVMControlFunc, bool)

GetRemediationTerminateHandler returns the remediation terminate for a CSP, falling back to the bulk control Terminate when the CSP's direct terminate is a complete replacement for Spider's.

type BatchVMControlHandlers added in v0.12.9 

type BatchVMControlHandlers struct {
	Suspend   BatchVMControlFunc // e.g. AWS StopInstances
	Resume    BatchVMControlFunc // e.g. AWS StartInstances
	Terminate BatchVMControlFunc // e.g. AWS TerminateInstances
	Reboot    BatchVMControlFunc // e.g. Azure BeginRestart
}

BatchVMControlHandlers groups bulk lifecycle control functions for a CSP.

type BatchVMStatusFunc added in v0.12.9 

type BatchVMStatusFunc func(ctx context.Context, region string, instanceIds []string) (map[string]string, error)

BatchVMStatusFunc queries a CSP directly for the statuses of the given resource IDs. ctx must carry model.CtxKeyCredentialHolder for credential lookup. region is the CSP-specific region identifier (e.g., "ap-northeast-2" for AWS). instanceIds are the CspResourceId values for each VM — format varies per CSP: 

AWS / Alibaba: "i-0abc123def456"
Tencent:       "ins-xxxxxxxx"
Azure:         full ARM path "/subscriptions/{sub}/resourceGroups/{rg}/.../virtualMachines/{name}"
GCP:           instance name (zone-scoped; region used for zone-prefix filtering)

Returns a map of CspResourceId → TB status string (model.StatusRunning, etc.). Missing keys mean the instance was not found; treat as model.StatusUndefined.

func GetBatchVMStatusHandler added in v0.12.9 

func GetBatchVMStatusHandler(provider string) (BatchVMStatusFunc, bool)

GetBatchVMStatusHandler returns the registered BatchVMStatusFunc for the given provider, wrapped with uniform direct-SDK tracing. Existence checks that don't invoke are unaffected.

type DeleteResidualsFunc added in v0.13.1 

type DeleteResidualsFunc func(ctx context.Context, region, zone string, items []ResidualResource) map[string]error

DeleteResidualsFunc deletes residuals; the result is keyed by ResidualResource.Key() (nil error = deleted).

type DirectSpan added in v0.13.1 

type DirectSpan struct {
	// contains filtered or unexported fields
}

DirectSpan traces a direct-CSP-SDK operation started from a provider subpackage (pricing, image, ...) with the same start/end logs as the batch dispatch path. Usage: span := csp.BeginDirect(...); defer func() { span.End(retErr) }()

func BeginDirect added in v0.13.1 

func BeginDirect(provider, op, region string, count int) *DirectSpan

BeginDirect logs the start of a direct-CSP-SDK operation and returns a span to end it. op is a short verb ("pricing", "image-list", ...); region may be "global" for account-wide calls.

func (*DirectSpan) End added in v0.13.1 

func (s *DirectSpan) End(err error)

End logs completion with latency; a non-nil err switches OK -> Failed (Warn).

type FailureParser added in v0.13.2 

type FailureParser interface {
	// Provider returns the CSP identifier (must match model/csp constants).
	Provider() string
	// Parse extracts structured fields from a raw CSP error message. It
	// returns ok=false when the message is not one this provider recognizes,
	// letting the generic classifier handle it. Implementations must not
	// panic on arbitrary input.
	Parse(raw string) (model.ProvisioningFailure, bool)
}

FailureParser is implemented by CSP-specific packages and registered via RegisterFailureParser at init() time.

type FindVMsByUIDFunc added in v0.13.4 

type FindVMsByUIDFunc func(ctx context.Context, region string, uids []string) (map[string]string, error)

FindVMsByUIDFunc queries a CSP directly using targeted Name/Tag filters (no full region scan) to find instances matching any of the given UIDs. ctx must carry model.CtxKeyCredentialHolder for credential lookup. region is the CSP-specific region identifier. uids are the Node.Uid values to search for. Returns a map of UID -> CspResourceId. Missing keys mean the instance was not found on the CSP.

func GetFindVMsByUIDHandler added in v0.13.4 

func GetFindVMsByUIDHandler(provider string) (FindVMsByUIDFunc, bool)

GetFindVMsByUIDHandler returns the registered FindVMsByUIDFunc for the given provider.

type InventoryHandlers added in v0.13.1 

type InventoryHandlers struct {
	ListVMs         ListVMsFunc
	ListResiduals   ListResidualsFunc
	DeleteResiduals DeleteResidualsFunc
}

InventoryHandlers groups the direct-SDK "truth" functions for a CSP.

func GetInventoryHandlers added in v0.13.1 

func GetInventoryHandlers(provider string) (InventoryHandlers, bool)

GetInventoryHandlers returns the registered inventory functions for a provider.

type ListResidualsFunc added in v0.13.1 

type ListResidualsFunc func(ctx context.Context, region, zone string) ([]ResidualResource, error)

ListResidualsFunc lists TB-managed residual resources no longer attached to a VM.

type ListVMsFunc added in v0.13.1 

type ListVMsFunc func(ctx context.Context, region, zone string) ([]VMRecord, error)

ListVMsFunc lists every VM in a region (paginated internally). zone is only used by zone-scoped CSPs (KT).

type ResidualResource added in v0.13.1 

type ResidualResource struct {
	Type   string `json:"type"` // nic, publicIp, disk, eni, eip, volume, floatingIp
	Id     string `json:"id"`
	Name   string `json:"name,omitempty"`
	Zone   string `json:"zone,omitempty"`
	Detail string `json:"detail,omitempty"`
}

ResidualResource is a VM-adjacent resource left behind after a VM is gone (NIC, public IP, disk, ...).

func (ResidualResource) Key added in v0.13.1 

func (r ResidualResource) Key() string

Key identifies a residual across list/delete calls.

type VMRecord added in v0.13.1 

type VMRecord struct {
	CspResourceId string            `json:"cspResourceId"`
	Name          string            `json:"name"`
	Status        string            `json:"status"` // TB status string
	Zone          string            `json:"zone,omitempty"`
	PublicIP      string            `json:"publicIp,omitempty"`
	Tags          map[string]string `json:"tags,omitempty"`
}

VMRecord is a CSP-side VM as observed directly through the CSP SDK (never via CB-Spider metadata).

Source Files

View all Source files

Directories

Path Synopsis
Package alibaba provides direct-SDK helpers for Alibaba Cloud that complement the cb-spider based flow of cb-tumblebug.
 Package alibaba provides direct-SDK helpers for Alibaba Cloud that complement the cb-spider based flow of cb-tumblebug.
Package azure provides direct Azure API call utilities for cases where CB-Spider is too slow or does not provide adequate functionality.
 Package azure provides direct Azure API call utilities for cases where CB-Spider is too slow or does not provide adequate functionality.
Package ibm provides direct IBM Cloud VPC SDK calls (truth surface: inventory, terminate, residuals).
 Package ibm provides direct IBM Cloud VPC SDK calls (truth surface: inventory, terminate, residuals).
Package kt provides direct KT Cloud VPC (D1, OpenStack-based) SDK calls (truth surface).
 Package kt provides direct KT Cloud VPC (D1, OpenStack-based) SDK calls (truth surface).
Package ncp provides direct Naver Cloud Platform (VPC) SDK calls (truth surface).
 Package ncp provides direct Naver Cloud Platform (VPC) SDK calls (truth surface).
Package nhn provides direct NHN Cloud (OpenStack) SDK calls (truth surface).
 Package nhn provides direct NHN Cloud (OpenStack) SDK calls (truth surface).
Package openstackcommon holds helpers shared by OpenStack-based direct clients (NHN, KT).
 Package openstackcommon holds helpers shared by OpenStack-based direct clients (NHN, KT).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.