Documentation
¶
Overview ¶
Package parallel provides generic parallel processing utilities for file and data processing workloads.
Index ¶
- func CalculateWorkers(numItems int, taskType TaskType) int
- func Collect[T any, R any](items []T, fn func(item T) (R, bool)) []R
- func Process[T any, R any](items []T, fn ProcessFunc[T, *R]) []*R
- func ProcessWithErrors[T any, R any](items []T, fn ProcessWithErrorFunc[T, R]) []*R
- type ProcessFunc
- type ProcessWithErrorFunc
- type TaskType
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CalculateWorkers ¶
CalculateWorkers determines the optimal number of workers based on the number of items to process and the type of workload.
func Collect ¶
Collect is a convenience function that processes items and collects results into a slice using a simple transform function.
func Process ¶
func Process[T any, R any](items []T, fn ProcessFunc[T, *R]) []*R
Process executes the given function on each item in parallel using a worker pool. It returns all non-nil results. The order of results is not guaranteed.
func ProcessWithErrors ¶
func ProcessWithErrors[T any, R any](items []T, fn ProcessWithErrorFunc[T, R]) []*R
ProcessWithErrors executes the given function on each item in parallel. Items that return errors are skipped. Returns all successful results.
Types ¶
type ProcessFunc ¶
ProcessFunc is a function that processes an item and returns a result. It should handle its own errors internally and return nil for failed items.
type ProcessWithErrorFunc ¶
ProcessWithError is similar to Process but allows the processing function to return an error. Items that error are skipped in the results.
Source Files