Documentation
¶
Overview ¶
The operationutils package provides helpers that glue together Clusters Service and asynchronous operations initiated by RP frontend pods in response to client requests.
For background reading about Azure's asynchronous operation contract for Resource Providers, see the Resource Provider Contract.
The ARO-HCP RP uses type api.Operation to represent an asynchronous operation. These structs get converted to JSON format and stored in Cosmos DB as so-called "operation documents".
At the time of this writing the RP backend defers most of the actual work involved in an operation to Clusters Service. The controllers in this package merely update operation documents in Cosmos DB to reflect the status of the actual operation in Clusters Service.
Generally speaking, the lifecycle of an operation document is as follows:
A frontend pod creates the operation document in Cosmos DB before responding to the client requesting the operation.
On the backend, the new operation document is first noticed by a "dispatch controller" that is dedicated to the operation's particular request type and resource type, such as "create cluster" or "delete node pool". The dispatch controller makes the appropriate calls to dispatch the operation to Clusters Service.
Once the operation is dispatched to Clusters Service, an "operation controller" begins polling the Clusters Service resource associated with the operation for status changes, and updates the operation document in Cosmos DB accordingly.
Meanwhile, the frontend will have exposed a status endpoint for this operation for the client to poll. (The endpoint is returned to the client as a header in the initial response.) The response body format of this endpoint is defined by Azure, but the operation document in Cosmos DB has all the required details to build a compliant response.
Operation documents in Cosmos DB are transient by way of a time-to-live (TTL) value. Once this TTL period (currently 7 days) expires, the Cosmos DB service will automatically delete the operation document.
Index ¶
- Constants
- func ClusterServiceInProgressMessage(state arohcpv1alpha1.ClusterState) string
- func ClusterServiceOperationMessage(clusterStatus *arohcpv1alpha1.ClusterStatus, opError *coreapi.CloudErrorBody) string
- func CompareOperationState(lhs, rhs *OperationState) int
- func ConvertClusterStatus(ctx context.Context, clusterServiceClient ocm.ClusterServiceClientSpec, ...) (coreapi.ProvisioningState, *coreapi.CloudErrorBody, error)
- func ConvertExternalAuthStatus(operation *coreapi.Operation, ...) (coreapi.ProvisioningState, *coreapi.CloudErrorBody, error)
- func ConvertNodePoolStatus(operation *coreapi.Operation, nodePoolStatus *arohcpv1alpha1.NodePoolStatus) (coreapi.ProvisioningState, *coreapi.CloudErrorBody, error)
- func DeadlineExceededMessage(deadlineSentence, remainingChecks string) string
- func NeedToPatchOperation(oldOperation *coreapi.Operation, newOperationStatus coreapi.ProvisioningState, ...) bool
- func NodePoolServiceInProgressMessage(state NodePoolStateValue) string
- func NodePoolServiceOperationMessage(nodePoolStatus *arohcpv1alpha1.NodePoolStatus, opError *coreapi.CloudErrorBody) string
- func PatchOperation(ctx context.Context, clock utilsclock.PassiveClock, ...) error
- func PickWorstCloudErrorCode(states []*OperationState, provisioningState coreapi.ProvisioningState) string
- func PostAsyncNotification(ctx context.Context, notificationClient *http.Client, ...) error
- func SetDeleteOperationAsCompleted(ctx context.Context, clock utilsclock.PassiveClock, ...) error
- func UpdateOperationStatus(ctx context.Context, clock utilsclock.PassiveClock, ...) error
- type ExternalAuthStateValue
- type NodePoolStateValue
- type OperationState
- type PostAsyncNotificationFunc
Constants ¶
const (
InflightChecksFailedProvisionErrorCode = "OCM4001"
)
Variables ¶
This section is empty.
Functions ¶
func ClusterServiceInProgressMessage ¶
func ClusterServiceInProgressMessage(state arohcpv1alpha1.ClusterState) string
ClusterServiceInProgressMessage returns a short description of a non-terminal Cluster Service cluster state, or empty for terminal states.
func ClusterServiceOperationMessage ¶
func ClusterServiceOperationMessage(clusterStatus *arohcpv1alpha1.ClusterStatus, opError *coreapi.CloudErrorBody) string
ClusterServiceOperationMessage is the OperationState message for a converted ClusterStatus: opError.Message if set, otherwise the in-progress CS state.
func CompareOperationState ¶
func CompareOperationState(lhs, rhs *OperationState) int
func ConvertClusterStatus ¶
func ConvertClusterStatus(ctx context.Context, clusterServiceClient ocm.ClusterServiceClientSpec, operation *coreapi.Operation, clusterStatus *arohcpv1alpha1.ClusterStatus, clusterServiceID metadataapi.InternalID) (coreapi.ProvisioningState, *coreapi.CloudErrorBody, error)
ConvertClusterStatus attempts to translate a ClusterStatus object from Cluster Service into an ARM provisioning state and, if necessary, a structured OData error.
func ConvertExternalAuthStatus ¶
func ConvertExternalAuthStatus(operation *coreapi.Operation, externalAuthStatus *arohcpv1alpha1.ExternalAuthStatus) (coreapi.ProvisioningState, *coreapi.CloudErrorBody, error)
func ConvertNodePoolStatus ¶
func ConvertNodePoolStatus(operation *coreapi.Operation, nodePoolStatus *arohcpv1alpha1.NodePoolStatus) (coreapi.ProvisioningState, *coreapi.CloudErrorBody, error)
ConvertNodePoolStatus attempts to translate a NodePoolStatus object from Cluster Service into an ARM provisioning state and, if necessary, a structured OData error.
func DeadlineExceededMessage ¶
DeadlineExceededMessage returns deadlineSentence, appending remainingChecks when it is non-empty.
func NeedToPatchOperation ¶
func NeedToPatchOperation(oldOperation *coreapi.Operation, newOperationStatus coreapi.ProvisioningState, newOperationError *coreapi.CloudErrorBody) bool
func NodePoolServiceInProgressMessage ¶
func NodePoolServiceInProgressMessage(state NodePoolStateValue) string
NodePoolServiceInProgressMessage returns a short description of a non-terminal Cluster Service node pool state, or empty for terminal states.
func NodePoolServiceOperationMessage ¶
func NodePoolServiceOperationMessage(nodePoolStatus *arohcpv1alpha1.NodePoolStatus, opError *coreapi.CloudErrorBody) string
NodePoolServiceOperationMessage is the OperationState message for a converted NodePoolStatus: opError.Message if set, else the CS message, else the in-progress state.
func PatchOperation ¶
func PatchOperation(ctx context.Context, clock utilsclock.PassiveClock, resourcesDBClient corecosmosstorage.ResourcesDBClient, oldOperation *coreapi.Operation, newOperationStatus coreapi.ProvisioningState, newOperationError *coreapi.CloudErrorBody, postAsyncNotificationFn PostAsyncNotificationFunc) error
PatchOperation patches the status and error fields of an OperationDocument.
func PickWorstCloudErrorCode ¶
func PickWorstCloudErrorCode(states []*OperationState, provisioningState coreapi.ProvisioningState) string
PickWorstCloudErrorCode selects a code from states matching provisioningState. Invalid* codes rank worst, followed by other codes, then InternalServerError. Successful states have no error code. Otherwise, missing codes default to InternalServerError; equal priorities keep the first code.
func PostAsyncNotification ¶
func SetDeleteOperationAsCompleted ¶
func SetDeleteOperationAsCompleted(ctx context.Context, clock utilsclock.PassiveClock, resourcesDBClient corecosmosstorage.ResourcesDBClient, operation *coreapi.Operation, postAsyncNotificationFn PostAsyncNotificationFunc) error
SetDeleteOperationAsCompleted updates Cosmos DB to reflect a completed resource deletion.
func UpdateOperationStatus ¶
func UpdateOperationStatus(ctx context.Context, clock utilsclock.PassiveClock, resourcesDBClient corecosmosstorage.ResourcesDBClient, existingOperation *coreapi.Operation, newOperationStatus coreapi.ProvisioningState, newOperationError *coreapi.CloudErrorBody, postAsyncNotificationFn PostAsyncNotificationFunc) error
UpdateOperationStatus updates Cosmos DB to reflect an updated resource status. If the operation has an associated resource, both documents are updated atomically using a transactional batch to prevent a window where the operation shows a terminal status but the resource still reflects the previous provisioning state.
The resource update is skipped (but the operation is still updated) when:
- the operation has no ExternalID (no associated resource)
- the resource document was deleted (404 not found)
- a different operation now owns the resource (ActiveOperationID mismatch)
- the resource is already at the target non-terminal provisioning state
In all of these cases the operation document is still persisted and ARM is notified, so the operation reaches its terminal state and does not get stuck.
Types ¶
type ExternalAuthStateValue ¶
type ExternalAuthStateValue string
Copied from uhc-clusters-service, because the OCM SDK does not define this for some reason.
const ( ExternalAuthStateReady ExternalAuthStateValue = "ready" ExternalAuthStateUninstalling ExternalAuthStateValue = "uninstalling" ExternalAuthStateError ExternalAuthStateValue = "error" )
type NodePoolStateValue ¶
type NodePoolStateValue string
Copied from uhc-clusters-service, because the OCM SDK does not define this for some reason.
const ( NodePoolStateValidating NodePoolStateValue = "validating" NodePoolStatePending NodePoolStateValue = "pending" NodePoolStateInstalling NodePoolStateValue = "installing" NodePoolStateReady NodePoolStateValue = "ready" NodePoolStateUpdating NodePoolStateValue = "updating" NodePoolStateValidatingUpdate NodePoolStateValue = "validating_update" NodePoolStatePendingUpdate NodePoolStateValue = "pending_update" NodePoolStateUninstalling NodePoolStateValue = "uninstalling" NodePoolStateRecoverableError NodePoolStateValue = "recoverable_error" NodePoolStateError NodePoolStateValue = "error" )
type OperationState ¶
type OperationState struct {
// Source is a name that identifies the source of the operation state.
Source string `json:"source"`
ProvisioningState coreapi.ProvisioningState `json:"provisioningState"`
Message string `json:"message"`
// CloudErrorCode defaults to InternalServerError for non-successful states.
CloudErrorCode string `json:"cloudErrorCode"`
// Error is the customer-safe error for a failed operation
Error *coreapi.CloudErrorBody `json:"error,omitempty"`
}
func NewFailedOperationState ¶
func NewFailedOperationState(code, message string, operationError *coreapi.CloudErrorBody) *OperationState
NewFailedOperationState creates a failed operation state with an explicit code, diagnostic message and optional customer-safe error. An empty code defaults to InternalServerError.
func NewOperationState ¶
func NewOperationState(provisioningState coreapi.ProvisioningState, message string) *OperationState
NewOperationState creates a new operation state with the given provisioning state and message, without a source.
func PickWorstOperationState ¶
func PickWorstOperationState(states []*OperationState) (*OperationState, error)
PickWorstOperationState expects states pre-sorted and returns the worst state with merged messages.
Only sources that report an actual message are included in the merged message: a source tied for the worst provisioning state with no message of its own has nothing blocking to report (e.g. it is simply still in progress), so it is omitted rather than rendered as a confusing "<no_message>" placeholder that reads like an error.
Customer-safe errors from failed states are collected independently of their messages. No errors yields nil; one retains its message and details; multiple are wrapped in a combined error with the original errors in Details. The resulting error uses the worst code from all states with the selected provisioning state.
func (*OperationState) WithCloudErrorCode ¶
func (s *OperationState) WithCloudErrorCode(code string) *OperationState
WithCloudErrorCode sets the error code when a more specific classification is available.
func (*OperationState) WithSource ¶
func (s *OperationState) WithSource(source string) *OperationState
WithSource sets the source of the operation state.
type PostAsyncNotificationFunc ¶
func PostAsyncNotificationFn ¶
func PostAsyncNotificationFn(notificationClient *http.Client) PostAsyncNotificationFunc
Source Files
Directories