Documentation
¶
Overview ¶
Package purchase handles the purchase workflow including approvals and execution.
Index ¶
- Constants
- Variables
- func CheckStepCommitmentCap(plan *config.PurchasePlan, recs []config.RecommendationRecord) error
- func OrphanExecutionError(execution *config.PurchaseExecution) error
- func ParseReapAfterFromEnv() time.Duration
- func PrefetchLadderVisibility(ctx context.Context, store interface{ ... }, factory LadderSPDescriberFactory, ...) prefetchedVisibility
- func RedriveRefusalReason(exec *config.PurchaseExecution) string
- func SingleCloudAccountIDFromRecs(recs []config.RecommendationRecord) (*string, error)
- func ValidateApprovalToken(execution *config.PurchaseExecution, token string) error
- type AsyncMessage
- type ClaimedRowCheck
- type CommitmentVisibility
- type FinalizeResult
- type FireResult
- type LadderReconcileDeps
- type LadderReconcileResult
- type LadderReconcileStore
- type LadderSPDescriberFactory
- type Manager
- func (m *Manager) ApproveAndExecute(ctx context.Context, executionID, actor string, transitionedBy *string, ...) (*config.PurchaseExecution, string, error)
- func (m *Manager) ApproveExecution(ctx context.Context, executionID, token, actor string, check ClaimedRowCheck) (string, error)
- func (m *Manager) AttachResolvedRecommendations(ctx context.Context, exec *config.PurchaseExecution, ...) (bool, error)
- func (m *Manager) CancelExecution(ctx context.Context, executionID, token, actor string) error
- func (m *Manager) DirectExecute(ctx context.Context, executionID, actor string, transitionedBy *string, ...) (*config.PurchaseExecution, string, error)
- func (m *Manager) FinalizeInFlightRevocations(ctx context.Context) (*FinalizeResult, error)
- func (m *Manager) FireScheduledDelayedPurchases(ctx context.Context) (*FireResult, error)
- func (m *Manager) LadderSPDescriber(ctx context.Context, cloudAccountID, region string) (SPDescriber, error)
- func (m *Manager) ProcessMessage(ctx context.Context, body string) error
- func (m *Manager) ProcessScheduledPurchases(ctx context.Context) (*ProcessResult, error)
- func (m *Manager) ReapStuckExecutions(ctx context.Context, reapAfter time.Duration) (*ReapResult, error)
- func (m *Manager) RecoverStrandedApprovals(ctx context.Context) (int, error)
- func (m *Manager) RunPlannedPurchaseNow(ctx context.Context, executionID, actor string, transitionedBy *string, ...) (*config.PurchaseExecution, string, error)
- func (m *Manager) SendUpcomingPurchaseNotifications(ctx context.Context) (*NotificationResult, error)
- func (m *Manager) SetRecommendationSource(src planstep.Source)
- type ManagerConfig
- type MessageType
- type NotificationResult
- type ProcessResult
- type PurchaseDefaults
- type ReapResult
- type SPDescriber
- type STSClient
Constants ¶
const DefaultReapAfter = 10 * time.Minute
DefaultReapAfter is the default age at which a running execution is considered stuck (approved rows use approvedReapAfter). The issue (#678) calls for "configurable via env var, default 10m" — the threshold is intentionally conservative: the happy-path executor completes in <60s; the longest legitimately-slow paths (multi-account fan-out, provider rate-limit backoff) settle in <3min. 10 minutes leaves multiple multiples of headroom so the reaper never fights a real executor.
const LadderFiredStaleAfter = 6 * time.Hour
LadderFiredStaleAfter is how long a fired tranche may stay unresolved before the reconciler logs an alert for it. It never changes the tranche: an unresolved fired tranche keeps netting, so the ladder cannot over-buy.
Variables ¶
var ( ErrInvalidApprovalToken = errors.New("invalid approval token") ErrApprovalTokenExpired = errors.New("approval token has expired") )
ErrInvalidApprovalToken and ErrApprovalTokenExpired let the HTTP layer map a bad email-link token to a 4xx instead of a 500.
var ErrClaimedRowRefused = errors.New("purchase refused after claim: the claimed execution no longer passes the authorization checks")
ErrClaimedRowRefused marks a run refused by the caller's ClaimedRowCheck after the claim was won and before any provider call (issue #415). The row ends "failed" with no_submission stamped; the check's own error is wrapped alongside it so callers can still map it (for example to a 403).
var ErrFourEyesDenied = errors.New("approval declined")
ErrFourEyesDenied wraps every refusal by the 4-eyes approval policy, as opposed to a failure to evaluate it (config or store errors).
var ErrPlanStepNoRecommendations = errors.New("plan step has no recommendations attached, so it cannot buy anything (platform#631: plan steps cannot yet carry recommendations)")
ErrPlanStepNoRecommendations marks a plan-step execution that carries no recommendations. Nothing attaches recommendations to a plan step yet (platform#609), so such a step can only "succeed" by buying nothing, which would complete it and advance the ramp. It is refused instead.
var ErrStaleAzurePricing = errors.New("stale Azure recommendation pricing; refresh recommendations and submit a new purchase")
var ErrStepCommitmentCap = errors.New("plan step refused by commitment cap")
ErrStepCommitmentCap marks a plan step refused by the per-plan spend cap.
Functions ¶
func CheckStepCommitmentCap ¶
func CheckStepCommitmentCap(plan *config.PurchasePlan, recs []config.RecommendationRecord) error
CheckStepCommitmentCap refuses a plan step's recommendations unless the plan carries a cap and the step's TOTAL commitment (upfront + monthly * term * 12, see RecommendationRecord.TotalCommitment) fits under it. It fails closed:
- a nil plan, a plan without MaxStepCommitment (nil or <= 0), or a non-finite cap refuses everything;
- a recommendation with a zero, negative or non-finite total commitment is refused (Azure monthly commitments have upfront 0, so upfront alone would pass anything);
- no recommendations is not a step that can buy, so it is not this check's concern and returns nil (the no-recommendations refusal owns it).
It only ever refuses; nothing here buys.
func OrphanExecutionError ¶
func OrphanExecutionError(execution *config.PurchaseExecution) error
OrphanExecutionError returns a descriptive error when the execution has no resolvable CloudAccountID and the provider is explicitly non-AWS (issue #609). AWS executions with a nil CloudAccountID are passed through because the ambient-host-account fallback from PR #607/#604 handles them. Empty provider is treated as AWS (legacy rows pre-dating multi-cloud support).
An execution is NOT considered orphan if any individual recommendation carries its own non-nil CloudAccountID — multi-rec fan-out executions can have a nil execution-level field while individual recs still name the target account.
Exported so the HTTP layer (internal/api) can call the same guard without duplicating the predicate. Extracted from ApproveExecution to keep that function below the gocyclo threshold — same pattern as loadCancelableExecution.
func ParseReapAfterFromEnv ¶
ParseReapAfterFromEnv reads PURCHASE_APPROVED_REAP_AFTER from the environment and parses it via time.ParseDuration. Falls back to DefaultReapAfter on either an absent env var, a parse failure, or a non-positive duration (0s / negative) — each variant logs a WARN so ops can spot a typo without the reaper silently running at the default. Never panics: a misconfigured env var must not crash the Lambda's other scheduled tasks.
Non-positive values are explicitly rejected: time.ParseDuration accepts "0s" and "-5m" as valid, but feeding them into ListStuckExecutions would either match every row (0s) or invert the SELECT into "updated_at < NOW() + |d|" (negative) and reap fresh executions. The store has its own guard (defense-in-depth) but rejecting here keeps the misconfig visible in the WARN log rather than as a confusing store error.
func PrefetchLadderVisibility ¶
func PrefetchLadderVisibility(ctx context.Context, store interface { GetExecutionByID(ctx context.Context, id string) (*config.PurchaseExecution, error) }, factory LadderSPDescriberFactory, fired []config.LadderTrancheDB) prefetchedVisibility
PrefetchLadderVisibility resolves, outside any transaction, whether the commitments of each fired tranche's completed execution are visible. It makes no provider call when nothing is fired or no execution has completed. Any lookup or API error is recorded against the affected executions and logged; nothing is ever reported visible on an error.
func RedriveRefusalReason ¶
func RedriveRefusalReason(exec *config.PurchaseExecution) string
RedriveRefusalReason returns a short operator-facing reason why exec must not be re-driven, or "" when every recommendation on it carries a provider-side guarantee that a second attempt collapses onto the first.
This is the single source of truth for re-drive safety. Both the reaper's automatic in-place re-drive (via allRecsSafeToRedrive) and the user-facing Retry endpoint (Handler.checkRetryEligibilityGates in internal/api) gate on it, so a provider/service that is unsafe for one is unsafe for the other. Before issue #1668 only the reaper consulted it, and clicking Retry on a landed Azure savings-plans row bought a second savings plan, which cannot be canceled.
Safe providers / services (issue #639):
- AWS RDS, ElastiCache, MemoryDB, OpenSearch and Savings Plans: a deterministic reservation ID or ClientToken collapses a repeat onto the first attempt (#636/#638).
- Azure reservations (compute, relational-db, cache, nosql, memorydb, search, data-warehouse): DoIdempotentPurchaseTwoStep performs a tag-based lookup before purchasing (#729 / #721).
- GCP compute (CUDs): server-side RequestId + deterministic name from the token (#654).
NOT safe:
- AWS EC2 and Redshift (slugs ec2, compute, redshift, data-warehouse): the purchase APIs take no client token, so a lost response may have bought the commitment and a second attempt would buy another (MON-02). The provider sends the call once and reports "purchase outcome unknown".
- Azure savings-plans: the OrderAlias API uses time.Now().UnixNano() as the alias name; there is no server-side idempotency key and no tag-based lookup implemented yet. Re-driving would create a duplicate savings plan.
- Any provider this function does not recognize, rather than assuming a guard exists.
Empty provider ("") is treated as AWS (pre-multi-cloud legacy rows).
It answers exactly one question: could re-driving these recommendations buy something twice. An execution with no recommendations buys nothing, so it has no duplicate risk and gets no refusal here. Callers that need "there is nothing worth re-driving" must say so themselves, as allRecsSafeToRedrive does above.
The reason is rendered verbatim to the operator, so it explains the refusal in product terms rather than naming internals.
func SingleCloudAccountIDFromRecs ¶
func SingleCloudAccountIDFromRecs(recs []config.RecommendationRecord) (*string, error)
SingleCloudAccountIDFromRecs returns the cloud_account_id shared by every SELECTED recommendation, nil when no selected recommendation carries one (the ambient single-account deployment), and errAmbiguousAccountScope when the selected recs span more than one account or mix attributed and unattributed entries (#1902). Only selected recs count: they are the recs the money moves for (processPurchaseRecommendations buys selectedIndices). Exported so the API boundary (validateExecutePurchaseRecommendations) applies the identical rule before an execution row exists.
func ValidateApprovalToken ¶
func ValidateApprovalToken(execution *config.PurchaseExecution, token string) error
ValidateApprovalToken checks that the execution carries a non-empty token, that the supplied token matches the stored one using constant-time comparison (Finding #4 -- prevents timing attacks), and that the token has not expired (issue #397). Legacy rows with a nil ApprovalTokenExpiresAt pass the TTL check for backward compatibility. Extracted from ApproveExecution to keep that function under the gocyclo threshold. Exported so the HTTP layer's delayed-approval branch (which never calls ApproveExecution) validates the token the same way (issue #586).
execution.ApprovalToken is the SHA-256 hex digest stored at rest (issue #103), never the raw secret; config.ApprovalTokenMatches hashes the supplied token and compares digests in constant time.
Types ¶
type AsyncMessage ¶
type AsyncMessage struct {
Type MessageType `json:"type"`
ExecutionID string `json:"execution_id,omitempty"`
PlanID string `json:"plan_id,omitempty"`
Token string `json:"token,omitempty"`
ActorEmail string `json:"actor_email,omitempty"`
}
AsyncMessage represents an SQS message for async purchase processing.
ActorEmail is REQUIRED for approve/cancel messages: the SQS path must carry the verified session email of the operator who initiated the action so the consumer can enforce the same per-account contact_email gate as the HTTP path's authorizeApprovalAction. Messages without ActorEmail are rejected (no tokenless legacy fallback) — this is the CRITICAL hardening for replayed/forwarded payloads. Producers in this repo have always paired token+actor; legacy in-flight messages pre-dating this field are rejected by design and the operator can re-issue the action via the HTTP route.
type ClaimedRowCheck ¶
type ClaimedRowCheck func(ctx context.Context, claimed *config.PurchaseExecution) error
ClaimedRowCheck re-runs a caller's authorization checks (permission Constraints such as MaxPurchaseAmount) against the row the claim CAS returned, which is exactly the row executeAndFinalize buys. A caller's earlier check ran on a snapshot read before the CAS, so a repricing or a recommendation attach landing in between would otherwise be bought unchecked (issue #415). A nil check skips the re-check; only callers with no pre-CAS Constraints check (the SQS approve worker) pass nil.
type CommitmentVisibility ¶
type CommitmentVisibility interface {
CommitmentVisible(ctx context.Context, tranche config.LadderTrancheDB, executionID string) (bool, error)
}
CommitmentVisibility reports whether the commitment bought by a completed execution already shows up in the provider's layer listing. The production adapter lands with the executor; an error or false keeps the tranche fired.
type FinalizeResult ¶
type FinalizeResult struct {
// Found is the number of purchase_history rows with revocation_in_flight=true
// and revoked_at IS NULL that the sweep saw.
Found int
// Finalized is the number of rows successfully stamped by MarkPurchaseRevoked.
Finalized int
// Errored is the number of rows the MarkPurchaseRevoked call failed for.
Errored int
}
FinalizeResult summarizes one sweep of FinalizeInFlightRevocations. Returned for the scheduled-task handler to log and surface in CloudWatch.
type FireResult ¶
type FireResult struct {
// Found is the number of status=scheduled rows whose scheduled_execution_at
// is in the past (SELECT pre-CAS).
Found int `json:"found"`
// Fired is the number of rows successfully transitioned from "scheduled" to
// "approved" and then through executeAndFinalize. Fired <= Found.
Fired int `json:"fired"`
// RaceLost is the number of rows where the scheduled->approved CAS was
// rejected by a concurrent operation (e.g. the user clicked Revoke between
// the SELECT and the CAS). This is normal and not an error.
RaceLost int `json:"race_lost"`
// Errored is the number of rows where the fire attempt failed for a reason
// other than a CAS race (DB error, provider error). Worth alerting on.
Errored int `json:"errored"`
}
FireResult carries the aggregate outcome of a single FireScheduledDelayedPurchases sweep. Callers log these counts and return them to the Lambda scheduler so they appear in CloudWatch Logs.
type LadderReconcileDeps ¶
type LadderReconcileDeps struct {
Store ladderReconcileDB
SP LadderSPDescriberFactory
Now func() time.Time
}
LadderReconcileDeps is what one scheduled reconcile run needs.
type LadderReconcileResult ¶
type LadderReconcileResult struct {
Checked, Completed, Failed, Unresolved, Stale, LostRace int
}
LadderReconcileResult counts what one sweep did.
func ReconcileLadderTranches ¶
func ReconcileLadderTranches(ctx context.Context, store LadderReconcileStore, vis CommitmentVisibility, now time.Time) (LadderReconcileResult, error)
ReconcileLadderTranches moves fired tranches to completed or failed from the state of their executions. Each tranche is decided inside one transaction that holds the tranche row and the retry-chain tail row, so a concurrent Retry cannot link a successor between the decision and the write. Tranches it cannot prove stay fired and keep netting; those older than LadderFiredStaleAfter are logged as errors. Not scheduled anywhere yet.
func RunLadderReconcile ¶
func RunLadderReconcile(ctx context.Context, d LadderReconcileDeps) (LadderReconcileResult, error)
RunLadderReconcile is the entry point for the future scheduled reconcile task: prefetch visibility, then ReconcileLadderTranches against the map. Nothing schedules it yet. With no fired tranche it touches no provider API.
type LadderReconcileStore ¶
type LadderReconcileStore interface {
ListFiredLadderTranches(ctx context.Context, limit int) ([]config.LadderTrancheDB, error)
WithTx(ctx context.Context, fn func(tx pgx.Tx) error) error
LockLadderTrancheChainTx(ctx context.Context, tx pgx.Tx, trancheID string) ([]config.LadderChainNode, error)
TransitionLadderTrancheTx(ctx context.Context, tx pgx.Tx, id string, from, to ladder.TrancheStatus, reason string) (bool, error)
}
LadderReconcileStore is the slice of the Postgres store the reconciler uses.
type LadderSPDescriberFactory ¶
type LadderSPDescriberFactory func(ctx context.Context, cloudAccountID, region string) (SPDescriber, error)
LadderSPDescriberFactory returns a describer authenticated for one cloud account and region.
type Manager ¶
type Manager struct {
// contains filtered or unexported fields
}
Manager handles purchase workflow.
func NewManager ¶
func NewManager(cfg ManagerConfig) *Manager
NewManager creates a new purchase manager.
func (*Manager) ApproveAndExecute ¶
func (m *Manager) ApproveAndExecute(ctx context.Context, executionID, actor string, transitionedBy *string, check ClaimedRowCheck) (*config.PurchaseExecution, string, error)
ApproveAndExecute atomically flips a pending/notified execution to "approved" (stamping ApprovedBy) and then runs the purchase synchronously. Callers MUST have already authorized the actor — this method does no RBAC or token validation; it is shared between:
- ApproveExecution (token/SQS path: token validated by the caller)
- approvePurchaseViaSession (session path: RBAC validated by the caller)
- directExecutePurchase (execute_mode="direct": RBAC validated by the caller)
Concurrency: TransitionExecutionStatus uses an atomic UPDATE WHERE status IN ('pending','notified'). Two callers racing to approve the same row will see exactly one win — the loser receives a clear "cannot transition" error. Cross-execution concurrency is unaffected: each approval drives its own executeAndFinalize, which already fans out per-account in parallel via executeMultiAccount.
The returned row carries the approval's final status, also on an execution error; it is nil when the run never started. check runs against the claimed row before any provider call; see ClaimedRowCheck.
func (*Manager) ApproveExecution ¶
func (m *Manager) ApproveExecution(ctx context.Context, executionID, token, actor string, check ClaimedRowCheck) (string, error)
ApproveExecution is the token-authenticated approve entry point used by the legacy email-link flow and the SQS approve worker. After validating the approval token it hands off to ApproveAndExecute, which performs the atomic status transition, runs the AWS purchase synchronously, and mints the fresh revocation token returned here.
actor carries the email of the operator who triggered the approval (session-authed click on the HTTP path; verified actor_email on the SQS path) so it can be stamped onto ApprovedBy. Empty actor is recorded as NULL — the column is nullable TEXT and we don't want to claim "approved by nobody".
check is the caller's ClaimedRowCheck, run against the claimed row before any provider call (issue #415); the SQS approve worker passes nil.
Returns the raw revocation token minted by ApproveAndExecute on success (see that function's doc comment for why this must be a return value rather than something the caller re-fetches from the DB). Empty on failure, and best-effort empty if the mint itself failed after an otherwise-successful approve.
func (*Manager) AttachResolvedRecommendations ¶
func (m *Manager) AttachResolvedRecommendations(ctx context.Context, exec *config.PurchaseExecution, recs []config.RecommendationRecord) (bool, error)
AttachResolvedRecommendations persists the recommendations a plan step resolved onto its still-empty pending/notified execution, together with the suppressions that stop Opportunities (and the next step's resolution) from offering the same recs again. Both writes happen in one transaction: a crash leaves the row empty with no suppressions, or populated with them.
Returns false (and writes nothing) when another resolver attached first or the row left pending/notified; the caller re-reads the row instead of overwriting it. On success exec is updated in place.
func (*Manager) CancelExecution ¶
CancelExecution cancels a pending execution. actor carries the email of the session-authenticated user who clicked cancel; verified by the caller (HTTP path: authorizeApprovalAction; SQS path: verifyAsyncApprovalActor) before reaching here. Same empty-actor rationale as ApproveExecution.
Concurrency: CancelExecutionAtomic uses a conditional UPDATE WHERE status IN ('pending','notified') so a concurrent approve that wins the race causes zero rows to be affected and the caller receives a clean error with the current status rather than silently overwriting the approved row. This is the token/email-link cancel analog of the atomic guard TransitionExecutionStatus provides for ApproveAndExecute.
func (*Manager) DirectExecute ¶
func (m *Manager) DirectExecute(ctx context.Context, executionID, actor string, transitionedBy *string, check ClaimedRowCheck) (*config.PurchaseExecution, string, error)
DirectExecute is ApproveAndExecute for a session with the direct-execute permission: the same 4-eyes-gated, CAS-guarded funnel, but it stamps the executed_* audit fields once the CAS has won, so a refused or lost direct execute leaves no executed_at on a row that never ran.
func (*Manager) FinalizeInFlightRevocations ¶
func (m *Manager) FinalizeInFlightRevocations(ctx context.Context) (*FinalizeResult, error)
FinalizeInFlightRevocations sweeps purchase_history rows with revocation_in_flight=true and revoked_at IS NULL and retries MarkPurchaseRevoked for each. These rows represent cases where the Azure Return API call succeeded but the subsequent DB write failed — the finalize_revocations scheduled tick ensures the audit record is eventually consistent without requiring the user to retry (which would be rejected by Azure as "already returned").
The sweep uses a fixed timestamp of time.Now() at the START of the sweep so all rows finalized in one pass share the same revokedAt wall-clock value, which makes log correlation easier.
Per-row error handling: rows that fail MarkPurchaseRevoked after retries are logged and counted in FinalizeResult.Errored but do not abort the sweep — the sweep continues to the next row so a single stuck row does not block finalization of all in-flight rows.
func (*Manager) FireScheduledDelayedPurchases ¶
func (m *Manager) FireScheduledDelayedPurchases(ctx context.Context) (*FireResult, error)
FireScheduledDelayedPurchases runs a sweep that fires all purchase_executions with status="scheduled" and scheduled_execution_at <= NOW(). For each row:
- Atomically transition scheduled -> approved via TransitionExecutionStatus (CAS; skips the row if the revoke handler won the race and flipped it to canceled first).
- Stamp ApprovedBy = "scheduler" to preserve audit trail.
- Run executeAndFinalize to call the cloud SDK and flip the row to completed/failed.
Safety: idempotent across duplicate invocations (the CAS guard prevents double-firing). Per-row error isolation: a failure on row N never blocks rows N+1..K. Counts are aggregated in FireResult for the caller to log.
func (*Manager) LadderSPDescriber ¶
func (m *Manager) LadderSPDescriber(ctx context.Context, cloudAccountID, region string) (SPDescriber, error)
LadderSPDescriber is the production factory: the same per-account credential resolution the purchase itself used.
func (*Manager) ProcessMessage ¶
ProcessMessage handles SQS messages for async purchase processing. Supported message types:
- execute_purchase: Execute a scheduled purchase by execution_id
- approve: Approve a pending execution (requires execution_id and token)
- cancel: Cancel a pending execution (requires execution_id and token)
- send_notification: Send notification for upcoming purchases
func (*Manager) ProcessScheduledPurchases ¶
func (m *Manager) ProcessScheduledPurchases(ctx context.Context) (*ProcessResult, error)
ProcessScheduledPurchases checks for and executes scheduled purchases.
func (*Manager) ReapStuckExecutions ¶
func (m *Manager) ReapStuckExecutions(ctx context.Context, reapAfter time.Duration) (*ReapResult, error)
ReapStuckExecutions runs one sweep that finds purchase_executions stuck in running longer than reapAfter, or in approved longer than approvedReapAfter(reapAfter), and atomically transitions them to "failed" via the existing TransitionExecutionStatus CAS. Each successful transition also persists a canonical error message so the History UI (issue #621) can show why the row was reaped and the operator knows it's safe to retry.
Approved rows get the later threshold so the process sweep's RecoverStrandedApprovals can re-drive them first (issue #894); the reaper only fails the ones recovery never picked up, e.g. on deployments that do not schedule the process sweep.
Safety properties:
- Local-status-only: the reaper never touches provider commitments. If the real executor did get a commitment created on the provider before dying, the row is still flipped to failed locally — the operator's retry hits the idempotency path (#636/#638/#652) which surfaces a duplicate-reservation error and short-circuits cleanly.
- CAS-protected: if the real executor wakes up and finishes between the SELECT and the CAS, TransitionExecutionStatus returns an error ("cannot transition from completed/failed"); we log at INFO and move on. The real executor wins the race.
- Per-row error-isolation: a failure on row N never blocks rows N+1..K. Counts are aggregated in ReapResult for the caller to log.
func (*Manager) RecoverStrandedApprovals ¶
RecoverStrandedApprovals finds executions stuck in the "approved" status past staleApprovedThreshold and either re-drives them idempotently (executions where every rec is safe to re-drive and the row has a durable ExecutionID) or drives them into a terminal "failed" state (rows with unsafe recs or without a stable ExecutionID).
Idempotent re-drive path (issue #639): all AWS, Azure reservations, and GCP compute service clients derive or look up a deterministic idempotency key from DeriveIdempotencyToken(idempotencyLineageKey(exec), i). This is an IN-PLACE re-drive — the same row, so the lineage key is unchanged — which reproduces the same token, so the cloud provider dedupes the second call and no double-purchase occurs. (The retry/fan-out double-buy hole #1012 closes is about NEW rows minting a fresh ExecutionID; that does not apply here.) The row transitions directly to "completed" (or "failed"/"partially_completed" on a genuine error), bypassing the manual Retry step required by the old safe-fail path. See allRecsSafeToRedrive for which provider/service combinations are eligible.
Safe-fail path: Azure savings-plans recs are excluded because the OrderAlias API uses a timestamp-based alias name with no idempotency key. Executions without a stable ExecutionID (legacy rows) also fall through because idempotencyLineageKey(exec) falls back to "" for them and DeriveIdempotencyToken("", i) would produce the same token set for every such row. These fall through to the original behavior: the row is atomically transitioned to "failed" so it surfaces in History and can be Retry-ed by an operator after confirming the cloud-side state.
The transition in the safe-fail path is atomic: TransitionExecutionStatus only flips rows still in "approved", so if the original run finally completes between the stale SELECT and this UPDATE, the transition is a no-op and the genuine "completed" status is preserved.
func (*Manager) RunPlannedPurchaseNow ¶
func (m *Manager) RunPlannedPurchaseNow(ctx context.Context, executionID, actor string, transitionedBy *string, check ClaimedRowCheck) (*config.PurchaseExecution, string, error)
RunPlannedPurchaseNow lets an operator force a scheduled purchase (pending or paused) to execute immediately instead of waiting for its scheduled date, going through the exact same 4-eyes gate and synchronous executeAndFinalize funnel ApproveAndExecute uses (issue #218). The previous runPlannedPurchase implementation only flipped the row's status to "running" and returned success: no executor consumes "running" rows (the scheduler picks up pending/notified, the reaper only cleans up a stuck "running" row after 10 minutes), so nothing ever bought the purchase and the row later reads "failed" out from under the operator who was told it ran.
Sharing transitionApproveAndExecute's CAS with ApproveAndExecute and claimAndExecute is also what makes this idempotent: a second "Run now" click, or the scheduler racing the same pending row, loses the atomic UPDATE ... WHERE status IN (...) and gets a clean "cannot transition" error rather than executing the purchase a second time.
The returned row carries the run-now audit stamp and its final status, also on an execution error; it is nil when the run never started. check runs against the claimed row before any provider call; see ClaimedRowCheck.
func (*Manager) SendUpcomingPurchaseNotifications ¶
func (m *Manager) SendUpcomingPurchaseNotifications(ctx context.Context) (*NotificationResult, error)
SendUpcomingPurchaseNotifications sends notifications for upcoming automated purchases.
func (*Manager) SetRecommendationSource ¶
SetRecommendationSource wires the strict recommendation read the plan-step attach resolves through. Late-bound because the scheduler is built after the manager and imports this package.
type ManagerConfig ¶
type ManagerConfig struct {
AmbientAWSCreds aws.CredentialsProvider
EmailSender email.SenderInterface
STSClient STSClient
AssumeRoleSTS credentials.STSClient
CredentialStore credentials.CredentialStore
ProviderFactory provider.FactoryInterface
ConfigStore config.StoreInterface
OIDCSigner oidc.Signer
OIDCIssuerURL string
DefaultPaymentOption string
DefaultRampSchedule string
DashboardURL string
NotificationDaysBefore int
DefaultCoverage float64
DefaultTerm int
}
ManagerConfig holds configuration for the purchase manager.
type MessageType ¶
type MessageType string
MessageType defines the types of async messages that can be processed.
const ( // MessageTypeExecutePurchase triggers execution of a scheduled purchase. MessageTypeExecutePurchase MessageType = "execute_purchase" // MessageTypeApprove approves a pending execution. MessageTypeApprove MessageType = "approve" // MessageTypeCancel cancels a pending execution. MessageTypeCancel MessageType = "cancel" // MessageTypeSendNotification sends a notification for upcoming purchase. MessageTypeSendNotification MessageType = "send_notification" )
type NotificationResult ¶
type NotificationResult struct {
Notified int `json:"notified"`
}
NotificationResult holds the result of sending notifications.
type ProcessResult ¶
type ProcessResult struct {
Errors []string `json:"errors,omitempty"`
Processed int `json:"processed"`
Executed int `json:"executed"`
Failed int `json:"failed"`
Recovered int `json:"recovered,omitempty"`
}
ProcessResult holds the result of processing scheduled purchases.
type PurchaseDefaults ¶
PurchaseDefaults holds default purchase settings.
type ReapResult ¶
type ReapResult struct {
// Found is the number of rows the SELECT returned (i.e. stuck rows the
// sweep saw — pre-CAS).
Found int `json:"found"`
// Reaped is the number of rows that successfully transitioned to
// "failed" via the atomic CAS. Reaped <= Found; the gap is rows the
// real executor finished between the SELECT and the CAS (CAS race
// rejected the reap — correct behavior, not an error).
Reaped int `json:"reaped"`
// RaceLost is the number of rows where the CAS rejected the reap
// because the row's status changed between the SELECT and the
// transition (the real executor woke up and beat us). Logged at
// INFO; not surfaced as an error.
RaceLost int `json:"race_lost"`
// Errored is the number of rows where the persistence step failed
// (CAS itself errored or the canonical-error follow-up save failed).
// These are real ops issues worth surfacing.
Errored int `json:"errored"`
}
ReapResult summarizes one sweep of ReapStuckExecutions. Returned for the scheduled-task handler to log + surface in CloudWatch / metrics.
type SPDescriber ¶
type SPDescriber interface {
DescribeSavingsPlans(ctx context.Context, in *sdksp.DescribeSavingsPlansInput, optFns ...func(*sdksp.Options)) (*sdksp.DescribeSavingsPlansOutput, error)
}
SPDescriber is the one Savings Plans API call the visibility prefetch needs. *savingsplans.Client satisfies it.
Source Files
¶
- account_scope.go
- approvals.go
- attach_recommendations.go
- azure_pricing.go
- claim.go
- execution.go
- fan_out_shape.go
- finalize_revocations.go
- ladder_execution_recheck.go
- ladder_reconcile.go
- ladder_visibility.go
- manager.go
- messages.go
- notifications.go
- plan_step_attach.go
- plan_step_managed.go
- reaper.go
- recover_reaped.go
- scheduled_fire.go
- step_cap.go