Documentation
¶
Overview ¶
Package parallel provides concurrent, context-aware sequence transforms for Go 1.23 iter.Seq values.
The parallel package is useful when each item in a sequence needs independent CPU or I/O work and you want to spread that work across a fixed number of workers. It keeps the same lazy iteration style as the rest of the library while letting callers control concurrency with standard context cancellation.
Example:
ctx := context.Background()
numbers := slices.Values([]int{1, 2, 3, 4})
for v := range parallel.ParallelMap(ctx, numbers, func(n int) int {
return n * 2
}, 2) {
fmt.Println(v)
}
Any important notes about behaviour such as laziness, context cancellation, memory usage. ParallelMap and WorkerPoolMap may yield results out of order when multiple workers are used, and they stop quickly when the context is cancelled or the consumer stops early. BatchMap preserves batch order and does not spin up worker goroutines by itself.
Index ¶
- func BatchMap[T, U any](ctx context.Context, seq iter.Seq[T], fn func([]T) []U, batchSize int) iter.Seq[U]
- func OrderedParallelMap[T, U any](ctx context.Context, seq iter.Seq[T], fn func(T) U, workers int) iter.Seq[U]
- func ParallelMap[T, U any](ctx context.Context, seq iter.Seq[T], fn func(T) U, workers int) iter.Seq[U]
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func BatchMap ¶
func BatchMap[T, U any](ctx context.Context, seq iter.Seq[T], fn func([]T) []U, batchSize int) iter.Seq[U]
BatchMap groups items into batches of size n and processes each batch.
Example ¶
package main
import (
"context"
"fmt"
"github.com/MostafaMagdSalama/vortex/parallel"
"slices"
)
func main() {
numbers := slices.Values([]int{1, 2, 3, 4})
for v := range parallel.BatchMap(context.Background(), numbers, func(batch []int) []int {
out := make([]int, len(batch))
for i, n := range batch {
out[i] = n * 10
}
return out
}, 2) {
fmt.Println(v)
}
}
Output: 10 20 30 40
func OrderedParallelMap ¶ added in v0.1.8
func OrderedParallelMap[T, U any]( ctx context.Context, seq iter.Seq[T], fn func(T) U, workers int, ) iter.Seq[U]
OrderedParallelMap applies fn to each element of seq concurrently using workers goroutines and yields results in the original input order.
Example ¶
package main
import (
"context"
"fmt"
"github.com/MostafaMagdSalama/vortex/parallel"
"slices"
)
func main() {
ctx := context.Background()
numbers := slices.Values([]int{1, 2, 3, 4, 5})
// results always come back in original order
for v := range parallel.OrderedParallelMap(ctx, numbers, func(n int) int {
return n * 2
}, 3) {
fmt.Println(v)
}
}
Output: 2 4 6 8 10
Example (Strings) ¶
package main
import (
"context"
"fmt"
"github.com/MostafaMagdSalama/vortex/parallel"
"slices"
"strings"
)
func main() {
ctx := context.Background()
words := slices.Values([]string{"hello", "world", "foo"})
for v := range parallel.OrderedParallelMap(ctx, words, strings.ToUpper, 2) {
fmt.Println(v)
}
}
Output: HELLO WORLD FOO
func ParallelMap ¶
func ParallelMap[T, U any](ctx context.Context, seq iter.Seq[T], fn func(T) U, workers int) iter.Seq[U]
ParallelMap processes each element concurrently with n workers.
Example ¶
package main
import (
"context"
"fmt"
"github.com/MostafaMagdSalama/vortex/parallel"
"slices"
)
func main() {
numbers := slices.Values([]int{1, 2, 3})
for v := range parallel.ParallelMap(context.Background(), numbers, func(n int) int {
return n * 2
}, 1) {
fmt.Println(v)
}
}
Output: 2 4 6
Types ¶
This section is empty.
Source Files