Documentation
¶
Index ¶
- Constants
- Variables
- func Batch[T any](items []T, batchSize int) [][]T
- func Contains[T comparable](list []T, item T) bool
- func ContainsAll[T comparable](list1, list2 []T) bool
- func ContainsAny[T comparable](A []T, B []T) bool
- func ContainsDeepEqual[T any](list []T, item T) bool
- func Convert[T any, Y any](from T, converter func(T) Y) Y
- func Copy[K comparable, V any](items map[K]V) map[K]V
- func Dedupe[T comparable](items []T) []T
- func DedupeByHash[T any](items []T, hashFn datastructures.HashFn[T]) []T
- func Each[T any](items []T, fn func(T))
- func EachMergeErrs[T any](items []T, fn func(T) error) error
- func EqualIgnoreOrder[T comparable](slices ...[]T) bool
- func Filter[T any](from []T, filter func(T) bool) []T
- func FilterMap[K comparable, V any](from map[K]V, filter func(k K, v V) bool) map[K]V
- func Find[T interface{}](from []T, filter func(T) bool) (item T, wasFound bool)
- func Generalize[T any](from []T) []any
- func GoEach[T any](items []T, fn func(T), opts ...Option)
- func GoEachMapWithErrs[K comparable, V any](items map[K]V, fn func(K, V) error, opts ...Option) (errs []error)
- func GoEachWithErrs[T any](items []T, fn func(T) error, opts ...Option) (errs []error)
- func GoMap[T any, Y any](items []T, converter func(T) Y, opts ...Option) []Y
- func GoMapToMany[T any, Y any](items []T, converter func(T) []Y, opts ...Option) (results []Y)
- func GoMapToManyWithErrs[T any, Y any](items []T, converter func(T) ([]Y, error), opts ...Option) (results []Y, errs []error)
- func GoMapToSlice[K comparable, V any, Z any](items map[K]V, converter func(K, V) Z, opts ...Option) []Z
- func GoMapToSliceWithErrs[K comparable, V any, Z any](items map[K]V, converter func(K, V) (Z, error), opts ...Option) (results []Z, errs []error)
- func GoMapWithErrs[T any, Y any](items []T, f func(T) (Y, error), opts ...Option) (results []Y, errs []error)
- func GoToMap[T any, K comparable, V any](items []T, f func(T) (K, V), opts ...Option) map[K]V
- func GoToMapWithErrs[T any, K comparable, V any](items []T, f func(T) (K, V, error), opts ...Option) (results map[K]V, errs []error)
- func HasNonNilError(errs []error) bool
- func Intersection[T comparable](a, b []T) []T
- func Join[T any](items []T, separator string) string
- func Map[T any, Y any](from []T, converter func(T) Y) []Y
- func MapMergeErrs[T any, Y any](from []T, converter func(T) (Y, error)) ([]Y, error)
- func MapToSlice[K comparable, V any, Z any](from map[K]V, converter func(k K, v V) Z) []Z
- func MapWithErrs[T any, Y any](from []T, converter func(T) (Y, error)) ([]Y, []error)
- func Max[T cmp.Ordered](values ...T) T
- func MergeErrors(errs ...error) error
- func Min[T cmp.Ordered](values ...T) T
- func Reduce[T any, Y any](from []T, to Y, reducer func(T, Y) Y) Y
- func Remove[T any](slice []T, i int) []T
- func RemoveNils[T any](from []T) []T
- func RunWithRetries[T any](fn func(t T) error, item T, numRetries int, backoffInterval time.Duration) error
- func Shuffle[T any](items []T) []T
- func ToMap[T any, K comparable, V any](from []T, converter func(T) (K, V)) map[K]V
- func WrapError(err error, message string) error
- func WrapErrorf(err error, message string, args ...any) error
- func WrapErrorsIntoFirst(errs ...error) error
- type Option
- type OptionConfig
Constants ¶
const ( AbsoluteMaxConcurrency = 100 DefaultMaxConcurrency = 20 )
Variables ¶
var ConcurrencyLimitOption = func(limit int) Option { return func(opt *OptionConfig) *OptionConfig { opt.ConcurrencyLimit = Max(limit, 1) return opt } }
ConcurrencyLimitOption limits the concurrency of a concurrent mapping function to protect against open file limits, connection limits, etc.
var DiscardResultIfErrOption = func() Option { return func(oc *OptionConfig) *OptionConfig { oc.DiscardResultsIfErr = true return oc } }
DiscardResultIfErrOption will make mapping functions not map results where errors are detected. For instance, if the provided fn() (result, error) function returns an error the result will not be added to the mapped result.
var RandomOrderOption = func() Option { return func(oc *OptionConfig) *OptionConfig { oc.RandomOrder = true return oc } }
RandomOrderOption will make the targeted function randomly order it's execution rather than iterating over elements in order.
var RateLimitOption = func(rate int, rateTimeframe time.Duration) Option { return func(oc *OptionConfig) *OptionConfig { rate = Max(rate, -rate) memoryBackend := ratelimit.NewMemoryRateLimiterBackend(rate, rateTimeframe, rate) oc.limiter = ratelimit.NewRateLimiter(ratelimit.FailClosed, memoryBackend) return oc } }
RateLimitOption limits maximum iterations that may be executed over a specified timeframe For instance, if rate is 5 and time.duration is 1 millisecond. The max throughput of the optioned function is 5 executions per millisecond.
var RetryOption = func(numRetries int, backoffInterval time.Duration) Option { return func(oc *OptionConfig) *OptionConfig { oc.RetryCount = numRetries oc.RetryBackoffInterval = backoffInterval return oc } }
RetryOption will make the targeted function attempt to retry any executed functions numRetries times. For instance, if a list has 5 items, and numRetries is 2, the provided mapping function could be executed a total of 15 times, 3 per item, due to 2 retries. With each retry, backoffInterval grows by backoffInterval. If backoffInterval is 1s, the first retry will be 1s after the first attempt, and the second will wait 2s before trying again, and so on.
Functions ¶
func Batch ¶
Batch splits a slice into smaller slices of a specified size. If the batchSize <= 0, it panics
func Contains ¶
func Contains[T comparable](list []T, item T) bool
Contains returns true/false on whether item is contained in list.
func ContainsAll ¶
func ContainsAll[T comparable](list1, list2 []T) bool
ContainsAll checks if all elements of list1 are present in list2. It does not require the elements to be contiguous.
func ContainsAny ¶
func ContainsAny[T comparable](A []T, B []T) bool
ContainsAny compares slices A and B and returns true if at least one element of A exists in B
func ContainsDeepEqual ¶
ContainsDeepEqual operates similar to Contains except performs comparisons by reflect.DeepEqual. This is capable of comparing types that do not implement 'comparable'
func Copy ¶
func Copy[K comparable, V any](items map[K]V) map[K]V
Copy deep copies a map to a new map of the same type. This does not mutate the original
func DedupeByHash ¶
func DedupeByHash[T any](items []T, hashFn datastructures.HashFn[T]) []T
DedupeByHash dedupes the items between 2 slices using a hashFn that should return equal values for two items that are considered equal
func EachMergeErrs ¶
EachMergeErrs runs the provided fn() over every item and aggregates all errors into a single error. If no error is found, nil is returned.
func EqualIgnoreOrder ¶
func EqualIgnoreOrder[T comparable](slices ...[]T) bool
EqualIgnoreOrder compares N slices for equal values ignoring the order of the items in the slices. Items must be comparable.
func FilterMap ¶
func FilterMap[K comparable, V any](from map[K]V, filter func(k K, v V) bool) map[K]V
FilterMap removes any items from map[K]V that _do not_ match the provided filter function.
func Find ¶
Find retrieves the first element in a slice matching the filter. Nil of type T is returned if nothing matches
func Generalize ¶
Generalize converts a slice[T] a slice[]any. Rarely useful except in particular cases.
func GoEach ¶
GoEach runs a function across each item of a slice concurrently. Options Support:
- ConcurrencyLimitOption
- RateLimitOption
- RandomOrderOption
func GoEachMapWithErrs ¶
func GoEachMapWithErrs[K comparable, V any](items map[K]V, fn func(K, V) error, opts ...Option) (errs []error)
GoEachMapWithErrs runs a function across each item of a map concurrently. Errors are aggregated and returned. Options Support:
- ConcurrencyLimitOption
- RateLimitOption
func GoEachWithErrs ¶
GoEachWithErrs runs a function across each item of a slice concurrently. Errors are aggregated and returned. Options Support:
- ConcurrencyLimitOption
- RateLimitOption
- RandomOrderOption
- RetryOption
func GoMap ¶
GoMap - Fast Map inside of a goroutine with safe aggregation This function is particularly useful if your converter function has a form of IO Wait (i.e, queries a DB, makes a HTTP call, or reads from disk) Options Support:
- ConcurrencyLimitOption
- RateLimitOption
- DiscardResultIfErrOption
- RandomOrderOption
func GoMapToMany ¶
GoMapToMany - Fast Map inside of a goroutine where each function run may return N elements with safe aggregation This function is particularly useful if your converter function has a form of IO Wait (i.e, queries a DB, makes a HTTP call, or reads from disk) Options Support:
- ConcurrencyLimitOption
- RateLimitOption
- DiscardResultIfErrOption
func GoMapToManyWithErrs ¶
func GoMapToManyWithErrs[T any, Y any](items []T, converter func(T) ([]Y, error), opts ...Option) (results []Y, errs []error)
GoMapToManyWithErrs - Fast Map inside a goroutine where each function run may return N elements or an error with safe aggregation This function is particularly useful if your converter function has a form of IO Wait (i.e, queries a DB, makes a HTTP call, or reads from disk) Options Support:
- ConcurrencyLimitOption
- RateLimitOption
- DiscardResultIfErrOption
- RandomOrderOption
- RetryOption
func GoMapToSlice ¶
func GoMapToSlice[K comparable, V any, Z any](items map[K]V, converter func(K, V) Z, opts ...Option) []Z
GoMapToSlice - Fast Map of a Map[T]Y to []Z -- runs in a goroutine with safe aggregation This function is particularly useful if your converter function has a form of IO Wait (i.e, queries a DB, makes a HTTP call, or reads from disk) Options Support:
- ConcurrencyLimitOption
- RateLimitOption
- DiscardResultIfErrOption
func GoMapToSliceWithErrs ¶
func GoMapToSliceWithErrs[K comparable, V any, Z any](items map[K]V, converter func(K, V) (Z, error), opts ...Option) (results []Z, errs []error)
GoMapToSliceWithErrs - Fast Map of a Map[K]V to []Z -- runs in a goroutine with safe aggregation of results and errors This function is particularly useful if your converter function has a form of IO Wait (i.e, queries a DB, makes a HTTP call, or reads from disk) Options Support:
- ConcurrencyLimitOption
- RateLimitOption
- DiscardResultIfErrOption
- RetryOption
func GoMapWithErrs ¶
func GoMapWithErrs[T any, Y any](items []T, f func(T) (Y, error), opts ...Option) (results []Y, errs []error)
GoMapWithErrs - Fast Map inside of a goroutine with safe aggregation of results & errors This function is particularly useful if your converter function has a form of IO Wait (i.e, queries a DB, makes a HTTP call, or reads from disk) Options Support:
- ConcurrencyLimitOption
- RateLimitOption
- DiscardResultIfErrOption
- RandomOrderOption
- RetryOption
func GoToMap ¶
func GoToMap[T any, K comparable, V any](items []T, f func(T) (K, V), opts ...Option) map[K]V
GoToMap converts a slice[] to a map[K]V via a provided converter function. Options Support:
- ConcurrencyLimitOption
- RateLimitOption
- DiscardResultIfErrOption
- RandomOrderOption
func GoToMapWithErrs ¶
func GoToMapWithErrs[T any, K comparable, V any](items []T, f func(T) (K, V, error), opts ...Option) (results map[K]V, errs []error)
GoToMapWithErrs - ToMap converts a slice[] to a map[K]V via a provided converter function. This function is particularly useful if your converter function has a form of IO Wait (i.e, queries a DB, makes a HTTP call, or reads from disk) Options Support:
- ConcurrencyLimitOption
- RateLimitOption
- DiscardResultIfErrOption
- RandomOrderOption
- RetryOption
func HasNonNilError ¶
func Intersection ¶
func Intersection[T comparable](a, b []T) []T
Intersection returns the intersection of two slices
func Join ¶
Join merges a slice[T] to a string separated by the provided separator. Items are rendered via fmt %v syntax.
func MapMergeErrs ¶
MapMergeErrs converts a slice of T to a slice of Y. Errors are merged, then returned.
func MapToSlice ¶
func MapToSlice[K comparable, V any, Z any](from map[K]V, converter func(k K, v V) Z) []Z
MapToSlice converts a map[K]V to a slice of type Z via provided converter function.
func MapWithErrs ¶
MapWithErrs converts a slice of T to a slice of Y. Errors are aggregated and returned.
func MergeErrors ¶
func Reduce ¶
Reduce converts []T to a single Y via a reducer function. `to` is provided to each iteration of reducer and will be returned at the end.
func Remove ¶
Remove deletes an element with index i from the slice and shifts all necessary items to preserver order
func RemoveNils ¶
func RemoveNils[T any](from []T) []T
RemoveNils removes any nil values from a slice of T -- this is safe for finding boxed and unboxed nils.
func RunWithRetries ¶
func RunWithRetries[T any](fn func(t T) error, item T, numRetries int, backoffInterval time.Duration) error
RunWithRetries will run fn numRetries+1 times, if an error is returned, the fn will be run again, until numRetries is exceeded or a non nil error is returned.
func Shuffle ¶
func Shuffle[T any](items []T) []T
Shuffle returns a COPY of the provided slice with shuffled elements
func ToMap ¶
func ToMap[T any, K comparable, V any](from []T, converter func(T) (K, V)) map[K]V
ToMap converts a slice[] to a map[K]V via a provided converter function.
func WrapError ¶
WrapError prepends a message to an error, and returns a new error while preserving its type.
func WrapErrorf ¶
WrapErrorf performs the same action as WrapError, but allows the user to format the message (similar to fmt.Printf).
func WrapErrorsIntoFirst ¶
WrapErrorsIntoFirst wraps all errors into the first preserving type.
Types ¶
type Option ¶
type Option func(option *OptionConfig) *OptionConfig
Source Files