Documentation
¶
Overview ¶
Package workqueue contains an interface for a simple key workqueue abstraction.
Index ¶
- Constants
- Variables
- func GetRequeueDelay(err error) (time.Duration, bool)
- func NonRetriableError(err error, reason string) error
- func RegisterWorkqueueServiceServer(s grpc.ServiceRegistrar, srv WorkqueueServiceServer)
- func RequeueAfter(delay time.Duration) error
- type Client
- type InProgressKey
- type Interface
- type Key
- type NoRetryDetails
- type ObservedInProgressKey
- type Options
- type OwnedInProgressKey
- type ProcessRequest
- func (*ProcessRequest) Descriptor() ([]byte, []int)deprecated
- func (x *ProcessRequest) GetDelaySeconds() int64
- func (x *ProcessRequest) GetKey() string
- func (x *ProcessRequest) GetPriority() int64
- func (*ProcessRequest) ProtoMessage()
- func (x *ProcessRequest) ProtoReflect() protoreflect.Message
- func (x *ProcessRequest) Reset()
- func (x *ProcessRequest) String() string
- type ProcessResponse
- type QueuedKey
- type UnimplementedWorkqueueServiceServer
- type UnsafeWorkqueueServiceServer
- type WorkqueueServiceClient
- type WorkqueueServiceServer
Examples ¶
Constants ¶
const (
WorkqueueService_Process_FullMethodName = "/chainguard.workqueue.WorkqueueService/Process"
)
Variables ¶
var ( // BackoffPeriod is the unit of backoff used when requeueing keys. // This unit is combined with the number of attempts to determine the // wait period before a key should be reprocessed. BackoffPeriod = 30 * time.Second // MaximumBackoffPeriod is a cap on the period a key must wait before // being retried. MaximumBackoffPeriod = 10 * time.Minute )
Note that these are variables, so that they can be modified by tests and made flags in binary entrypoints.
var File_workqueue_proto protoreflect.FileDescriptor
var WorkqueueService_ServiceDesc = grpc.ServiceDesc{ ServiceName: "chainguard.workqueue.WorkqueueService", HandlerType: (*WorkqueueServiceServer)(nil), Methods: []grpc.MethodDesc{ { MethodName: "Process", Handler: _WorkqueueService_Process_Handler, }, }, Streams: []grpc.StreamDesc{}, Metadata: "workqueue.proto", }
WorkqueueService_ServiceDesc is the grpc.ServiceDesc for WorkqueueService service. It's only intended for direct use with grpc.RegisterService, and not to be introspected or modified (even as a copy)
Functions ¶
func GetRequeueDelay ¶ added in v0.6.161
GetRequeueDelay extracts the requeue delay from an error if it's a requeue error. Returns the delay and true if the error is a requeue error, or 0 and false otherwise.
func NonRetriableError ¶
NoRetryDetails marks the error as non-retriable with the given reason. If this error is returned to the dispatcher, it will not requeue the key.
func RegisterWorkqueueServiceServer ¶
func RegisterWorkqueueServiceServer(s grpc.ServiceRegistrar, srv WorkqueueServiceServer)
func RequeueAfter ¶ added in v0.6.161
RequeueAfter returns an error that indicates the work item should be requeued after the specified delay.
Example ¶
ExampleRequeueAfter demonstrates how to use RequeueAfter in a callback.
package main
import (
"context"
"fmt"
"time"
"github.com/chainguard-dev/terraform-infra-common/pkg/workqueue"
)
// ExampleWorker demonstrates how to implement a worker that can request
// custom requeue delays.
type ExampleWorker struct {
workqueue.UnimplementedWorkqueueServiceServer
}
func (w *ExampleWorker) Process(_ context.Context, req *workqueue.ProcessRequest) (*workqueue.ProcessResponse, error) {
// Example 1: Process successfully
if req.Key == "success" {
fmt.Println("Processing successful")
return &workqueue.ProcessResponse{}, nil
}
// Example 2: Requeue with a 5-minute delay
if req.Key == "rate-limited" {
fmt.Println("Rate limited, requeueing after 5 minutes")
return &workqueue.ProcessResponse{
RequeueAfterSeconds: 300, // 5 minutes
}, nil
}
// Example 3: Requeue with exponential backoff based on external state
if req.Key == "backoff" {
retryCount := getRetryCount(req.Key) // hypothetical function
delay := time.Duration(retryCount) * time.Minute
fmt.Printf("Requeueing with %v delay\n", delay)
return &workqueue.ProcessResponse{
RequeueAfterSeconds: int64(delay.Seconds()),
}, nil
}
// Example 4: Traditional error handling (uses default backoff)
if req.Key == "error" {
return nil, fmt.Errorf("processing failed")
}
// Example 5: Non-retriable error
if req.Key == "permanent-failure" {
return nil, workqueue.NonRetriableError(
fmt.Errorf("unrecoverable error"),
"Resource does not exist",
)
}
return &workqueue.ProcessResponse{}, nil
}
// ExampleRequeueAfter demonstrates how to use RequeueAfter in a callback.
func main() {
callback := func(_ context.Context, _ string, _ workqueue.Options) error {
// Do some work...
// Request requeue with a 30-second delay
return workqueue.RequeueAfter(30 * time.Second)
}
// This would be used in a dispatcher
err := callback(context.Background(), "example-key", workqueue.Options{})
delay, ok := workqueue.GetRequeueDelay(err)
fmt.Printf("Requeue requested: %v, delay: %v\n", ok, delay)
}
func getRetryCount(_ string) int {
// This is a placeholder for demonstration
return 1
}
Output: Requeue requested: true, delay: 30s
Types ¶
type Client ¶
type Client interface {
WorkqueueServiceClient
Close() error
}
func NewWorkqueueClient ¶
type InProgressKey ¶
type InProgressKey interface {
Key
// Requeue returns this key to the queue.
Requeue(context.Context) error
// RequeueWithOptions returns this key to the queue with custom options.
RequeueWithOptions(context.Context, Options) error
}
InProgressKey is a shared interface that all in-progress key types must implement.
type Interface ¶
type Interface interface {
// Queue adds an item to the workqueue.
Queue(ctx context.Context, key string, opts Options) error
// Enumerate returns:
// - a list of all of the in-progress keys, and
// - a list of the next "N" keys in the queue (according to its configured ordering), or
// - an error if the workqueue is unable to enumerate the keys.
Enumerate(ctx context.Context) ([]ObservedInProgressKey, []QueuedKey, error)
}
Interface is the interface that workqueue implementations must implement.
type Key ¶
type Key interface {
// Name is the name of the key.
Name() string
// Priority is the priority of the key.
Priority() int64
}
Key is a shared interface that all key types must implement.
type NoRetryDetails ¶
type NoRetryDetails struct {
Message string `protobuf:"bytes,1,opt,name=message,proto3" json:"message,omitempty"` // A message describing why the key should not be retried.
// contains filtered or unexported fields
}
NoRetryDetails is a marker message that indicates that the key should not be retried.
func GetNonRetriableDetails ¶
func GetNonRetriableDetails(err error) *NoRetryDetails
GetNonRetriableDetails extracts the NoRetryDetails from the error if it exists. If the error is nil or does not contain NoRetryDetails, it returns nil.
func (*NoRetryDetails) Descriptor
deprecated
func (*NoRetryDetails) Descriptor() ([]byte, []int)
Deprecated: Use NoRetryDetails.ProtoReflect.Descriptor instead.
func (*NoRetryDetails) GetMessage ¶
func (x *NoRetryDetails) GetMessage() string
func (*NoRetryDetails) ProtoMessage ¶
func (*NoRetryDetails) ProtoMessage()
func (*NoRetryDetails) ProtoReflect ¶
func (x *NoRetryDetails) ProtoReflect() protoreflect.Message
func (*NoRetryDetails) Reset ¶
func (x *NoRetryDetails) Reset()
func (*NoRetryDetails) String ¶
func (x *NoRetryDetails) String() string
type ObservedInProgressKey ¶
type ObservedInProgressKey interface {
InProgressKey
// IsOrphaned checks whether the key has been orphaned by it's owner.
IsOrphaned() bool
}
ObservedInProgressKey is a key that we have observed to be in progress, but that we are not the owner of.
type Options ¶
type Options struct {
// Priority is the priority of the key.
// Higher values are processed first.
Priority int64
// NotBefore is the earliest time that the key should be processed.
// When deduplicating, the oldest time is used.
NotBefore time.Time
// Delay is an optional duration to wait before processing the key.
// This is used when requeueing with a custom delay.
Delay time.Duration
}
Options is a set of options that can be passed when queuing a key.
type OwnedInProgressKey ¶
type OwnedInProgressKey interface {
InProgressKey
// Complete marks the key as successfully completed, and removes it from
// the in-progress key set.
Complete(context.Context) error
// Deadletter permanently removes this key from the queue, indicating it has
// failed after exceeding the maximum retry attempts.
Deadletter(context.Context) error
// GetAttempts returns the current attempt count for the key.
GetAttempts() int
// Context is the context of the process heartbeating the key.
Context() context.Context
}
OwnedInProgressKey is an in-progress key where we have initiated the work, and own until it completes either successfully (Complete), or unsuccessfully (Requeue or Fail).
type ProcessRequest ¶
type ProcessRequest struct {
// The key of the work item
Key string `protobuf:"bytes,1,opt,name=key,proto3" json:"key,omitempty"`
// The (optional) priority of the work item, where higher numbers are processed first.
Priority int64 `protobuf:"varint,2,opt,name=priority,proto3" json:"priority,omitempty"`
// The (optional) delay in second to wait before processing the work item.
DelaySeconds int64 `protobuf:"varint,3,opt,name=delay_seconds,json=delaySeconds,proto3" json:"delay_seconds,omitempty"`
// contains filtered or unexported fields
}
func (*ProcessRequest) Descriptor
deprecated
func (*ProcessRequest) Descriptor() ([]byte, []int)
Deprecated: Use ProcessRequest.ProtoReflect.Descriptor instead.
func (*ProcessRequest) GetDelaySeconds ¶
func (x *ProcessRequest) GetDelaySeconds() int64
func (*ProcessRequest) GetKey ¶
func (x *ProcessRequest) GetKey() string
func (*ProcessRequest) GetPriority ¶
func (x *ProcessRequest) GetPriority() int64
func (*ProcessRequest) ProtoMessage ¶
func (*ProcessRequest) ProtoMessage()
func (*ProcessRequest) ProtoReflect ¶
func (x *ProcessRequest) ProtoReflect() protoreflect.Message
func (*ProcessRequest) Reset ¶
func (x *ProcessRequest) Reset()
func (*ProcessRequest) String ¶
func (x *ProcessRequest) String() string
type ProcessResponse ¶
type ProcessResponse struct {
// Optional: If set, indicates the work item should be requeued after the specified delay.
// If not set or 0, the default behavior applies (success if no error, or exponential backoff on error).
// This field is only honored when the RPC returns successfully (no error).
RequeueAfterSeconds int64 `protobuf:"varint,1,opt,name=requeue_after_seconds,json=requeueAfterSeconds,proto3" json:"requeue_after_seconds,omitempty"`
// contains filtered or unexported fields
}
func (*ProcessResponse) Descriptor
deprecated
func (*ProcessResponse) Descriptor() ([]byte, []int)
Deprecated: Use ProcessResponse.ProtoReflect.Descriptor instead.
func (*ProcessResponse) GetRequeueAfterSeconds ¶ added in v0.6.161
func (x *ProcessResponse) GetRequeueAfterSeconds() int64
func (*ProcessResponse) ProtoMessage ¶
func (*ProcessResponse) ProtoMessage()
func (*ProcessResponse) ProtoReflect ¶
func (x *ProcessResponse) ProtoReflect() protoreflect.Message
func (*ProcessResponse) Reset ¶
func (x *ProcessResponse) Reset()
func (*ProcessResponse) String ¶
func (x *ProcessResponse) String() string
type QueuedKey ¶
type QueuedKey interface {
Key
// Start initiates processing of the key, returning an OwnedInProgressKey
// on success and an error on failure.
Start(context.Context) (OwnedInProgressKey, error)
}
QueuedKey is a key that is in the queue, waiting to be processed.
type UnimplementedWorkqueueServiceServer ¶
type UnimplementedWorkqueueServiceServer struct{}
UnimplementedWorkqueueServiceServer must be embedded to have forward compatible implementations.
NOTE: this should be embedded by value instead of pointer to avoid a nil pointer dereference when methods are called.
func (UnimplementedWorkqueueServiceServer) Process ¶
func (UnimplementedWorkqueueServiceServer) Process(context.Context, *ProcessRequest) (*ProcessResponse, error)
type UnsafeWorkqueueServiceServer ¶
type UnsafeWorkqueueServiceServer interface {
// contains filtered or unexported methods
}
UnsafeWorkqueueServiceServer may be embedded to opt out of forward compatibility for this service. Use of this interface is not recommended, as added methods to WorkqueueServiceServer will result in compilation errors.
type WorkqueueServiceClient ¶
type WorkqueueServiceClient interface {
Process(ctx context.Context, in *ProcessRequest, opts ...grpc.CallOption) (*ProcessResponse, error)
}
WorkqueueServiceClient is the client API for WorkqueueService service.
For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.
func NewWorkqueueServiceClient ¶
func NewWorkqueueServiceClient(cc grpc.ClientConnInterface) WorkqueueServiceClient
type WorkqueueServiceServer ¶
type WorkqueueServiceServer interface {
Process(context.Context, *ProcessRequest) (*ProcessResponse, error)
// contains filtered or unexported methods
}
WorkqueueServiceServer is the server API for WorkqueueService service. All implementations must embed UnimplementedWorkqueueServiceServer for forward compatibility.