Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ProcessParallel ¶
func ProcessParallel[T, R any](ctx context.Context, items []T, processFunc func(T) R, opts ...ParallelOptions) (results []R, lastProcessed T, err error)
ProcessParallel processes a slice of items concurrently and returns the results. It supports generic types for both input items (T) and results (R).
Parameters:
- ctx: Caller's context. The fan-out inherits its deadline/cancellation (e.g. the API-GW/Lambda time budget), so dispatch stops once the caller's budget expires. processFunc must honor this same deadline through the context it closes over, so an in-flight AWS call unwinds instead of pinning a worker.
- items: Slice of input items to process
- processFunc: Function that processes each item and returns a result. Make sure to handle panics in the function. Also, keep this function simple and fast.
- opts: Optional ParallelOptions to configure processing behavior
Returns:
- results: Slice of processed results (nil if CollectResults is false)
- lastProcessed: The last successfully processed item
- err: Error from context timeout/cancellation if it occurred
The function handles panics in individual goroutines gracefully, ensuring that a panic in one routine doesn't affect others. It also tracks the last successfully processed item even when not collecting results.
If you plan to change the function, run all benchmark tests: TestGoroutineCount, BenchmarkMemoryUsage, BenchmarkParallelProcessing against the original function to ensure the performance is not degraded
Types ¶
type ParallelOptions ¶
type ParallelOptions struct {
MaxRoutines uint // Maximum number of concurrent goroutines (default: 10)
Timeout time.Duration // Optional timeout for processing, this is checked at the start of each item processing and hence should have a buffer to account for the total time taken to process one item
CollectResults bool // Whether to collect and return results (default: true)
}
ParallelOptions configures the behavior of parallel processing operations. It allows customization of concurrency limits, timeout duration, and result collection.