parallel

package
v0.8.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 23, 2026 License: Apache-2.0 Imports: 6 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ProcessParallel

func ProcessParallel[T, R any](ctx context.Context, items []T, processFunc func(T) R, opts ...ParallelOptions) (results []R, lastProcessed T, err error)

ProcessParallel processes a slice of items concurrently and returns the results. It supports generic types for both input items (T) and results (R).

Parameters:

  • ctx: Caller's context. The fan-out inherits its deadline/cancellation (e.g. the API-GW/Lambda time budget), so dispatch stops once the caller's budget expires. processFunc must honor this same deadline through the context it closes over, so an in-flight AWS call unwinds instead of pinning a worker.
  • items: Slice of input items to process
  • processFunc: Function that processes each item and returns a result. Make sure to handle panics in the function. Also, keep this function simple and fast.
  • opts: Optional ParallelOptions to configure processing behavior

Returns:

  • results: Slice of processed results (nil if CollectResults is false)
  • lastProcessed: The last successfully processed item
  • err: Error from context timeout/cancellation if it occurred

The function handles panics in individual goroutines gracefully, ensuring that a panic in one routine doesn't affect others. It also tracks the last successfully processed item even when not collecting results.

If you plan to change the function, run all benchmark tests: TestGoroutineCount, BenchmarkMemoryUsage, BenchmarkParallelProcessing against the original function to ensure the performance is not degraded

Types

type ParallelOptions

type ParallelOptions struct {
	MaxRoutines    uint          // Maximum number of concurrent goroutines (default: 10)
	Timeout        time.Duration // Optional timeout for processing, this is checked at the start of each item processing and hence should have a buffer to account for the total time taken to process one item
	CollectResults bool          // Whether to collect and return results (default: true)
}

ParallelOptions configures the behavior of parallel processing operations. It allows customization of concurrency limits, timeout duration, and result collection.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL