Documentation
¶
Overview ¶
Package cursorediterator provides a series of specialized iterator builders that support construction of a tree of iterators, returning standard Go Seq2 iterators which wrap all the complexity.
This is useful for building paginated APIs, where the client can request a specific page of results, and the server can return a cursor to the next page, while automatically handling all the complexities of context cancellation, error handling, and cursor management.
Take a look at integration_test.go for a full example of how to use this package in practice.
Index ¶
- func CountingIterator[I any](source iter.Seq2[I, error], callback func(int)) iter.Seq2[I, error]
- func CursoredParallelIterators[I any](ctx context.Context, currentCursor Cursor, concurrency uint16, ...) iter.Seq2[ItemAndCursor[I], error]
- func CursoredProducerMapperIterator[C any, P any, I any](ctx context.Context, currentCursor Cursor, concurrency uint16, ...) iter.Seq2[ItemAndCursor[I], error]
- func CursoredWithIntegerHeader[I any](ctx context.Context, currentCursor Cursor, ...) iter.Seq2[ItemAndCursor[I], error]
- func DisableCursorsInContext(ctx context.Context) context.Context
- func Empty[I any](ctx context.Context, cursor Cursor) iter.Seq2[ItemAndCursor[I], error]
- func Spanned[I any](ctx context.Context, source iter.Seq2[I, error], tracer trace.Tracer, ...) iter.Seq2[I, error]
- func UncursoredEmpty[I any]() iter.Seq2[I, error]
- func YieldsError[I any](err error) iter.Seq2[I, error]
- type Chunk
- type ChunkOrHold
- type Cursor
- type CursorFromStringConverter
- type HoldForMappingComplete
- type ItemAndCursor
- type Next
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CountingIterator ¶
CountingIterator wraps a Seq2 iterator and counts the number of items yielded, invoking the callback with the final count once the iterator completes.
func CursoredParallelIterators ¶
func CursoredParallelIterators[I any]( ctx context.Context, currentCursor Cursor, concurrency uint16, orderedIterators ...Next[I], ) iter.Seq2[ItemAndCursor[I], error]
CursoredParallelIterators is a function that takes a context, a Cursor, a concurrency level, and a list of `next` iterators, which are executed in parallel but whose results are yielded in the order matching the order of the iterators.
The integer value found at the HEAD of the Cursor represents the starting iterator to execute in the list of `orderedIterators` iterators. If the Cursor is empty, all iterators in the list will be executed in parallel.
Yielded items are automatically decorated with the correct prefix for the Cursor, representing the index of the iterator in the list of `orderedIterators` iterators.
func CursoredProducerMapperIterator ¶
func CursoredProducerMapperIterator[C any, P any, I any]( ctx context.Context, currentCursor Cursor, concurrency uint16, cursorFromStringConverter CursorFromStringConverter[C], cursorToStringConverter func(C) (string, error), producer func(ctx context.Context, currentIndex C, remainingCursor Cursor) iter.Seq2[ChunkOrHold[P, C], error], mapper func(ctx context.Context, remainingCursor Cursor, chunk P) iter.Seq2[ItemAndCursor[I], error], ) iter.Seq2[ItemAndCursor[I], error]
CursoredProducerMapperIterator is an iterator that takes a chunk of items produced by a `producer` iterator and maps them to items of type `I` using a `mapper` function. The `producer` function runs in parallel with the `mapper`. The `mapper` function runs with the specified concurrency level for parallel processing. If the producer yields HoldForMappingComplete, further calls to the producer will wait until the mapper has completed its current chunk.
func CursoredWithIntegerHeader ¶
func CursoredWithIntegerHeader[I any]( ctx context.Context, currentCursor Cursor, header func(ctx context.Context, startIndex int) iter.Seq2[I, error], next Next[I], ) iter.Seq2[ItemAndCursor[I], error]
CursoredWithIntegerHeader is a function that takes a Cursor, a `header` iterator, a `next` iterator and executes the `header` iterator with a Cursor of an integer value.
The integer value found at the HEAD of the Cursor represents the starting index for the next iteration of the `header` iterator, if any. For example, if the `header` iterator last yields a value of 5, then the returned Cursor will have an entry of "6", indicating that the `header` iterator should start yielding values *from* index 6.
If the Cursor is empty, the `header` iterator will be called with an index of 0.
If the `header` iterator has completed all its values, then the HEAD of Cursor will be -1, and the `header` iterator will not be called again.
Unlike the other blocks, the `header` iterator does not have to return the next Cursor at all, as yielded items are automatically decorated with the next item's index.
func DisableCursorsInContext ¶ added in v1.46.1
DisableCursorsInContext returns a new context with cursors disabled. When cursors are disabled, all cursored iterator functions will return nil cursors instead of constructing them.
func Spanned ¶ added in v1.46.0
func Spanned[I any]( ctx context.Context, source iter.Seq2[I, error], tracer trace.Tracer, spanName string, spanAttrs ...attribute.KeyValue, ) iter.Seq2[I, error]
Spanned wraps an iterator sequence, recording any errors to the provided span, and ending the span when the iteration is complete.
func UncursoredEmpty ¶
UncursoredEmpty is a function that returns an empty iterator sequence without a cursor.
Types ¶
type Chunk ¶ added in v1.46.0
Chunk holds a chunk of items and the portion of the Cursor for the chunk, indicating where the current chunk should resume.
type ChunkOrHold ¶ added in v1.46.0
ChunkOrHold is an enum-like type that can hold one of three values: - Chunk: contains a chunk and the cursor position for the current chunk - HoldForMappingComplete: signals that the producer should wait for mapper completion
type Cursor ¶
type Cursor []string
cursor is a type alias for a slice of strings, representing a cursor. Each entry in the slice is dependent on the structure of the iterator and should be defined accordingly. The cursor is ordered back-to-front, with the most recent entry at the end of the slice.
func CursorCustomHeadValue ¶
func CursorCustomHeadValue[T any](c Cursor, converter CursorFromStringConverter[T]) (T, Cursor, error)
CursorCustomHeadValue extracts the last value from the cursor (HEAD) using a custom converter. It returns the converted value, the remaining cursor, and an error if the conversion fails. If the cursor is empty, it returns the zero value of type T and nil.
type CursorFromStringConverter ¶
CursorFromStringConverter is a function type that converts a string to a value of type T. It returns the converted value and an error if the conversion fails.
type HoldForMappingComplete ¶
HoldForMappingComplete signals that the producer should wait until the mapper has completed processing its current chunk before producing more.
type ItemAndCursor ¶
type ItemAndCursor[I any] struct { // Item is the item of type I. Item I // Cursor is the cursor for the item, to resume execution within the iterator. // For flat iterators, the HEAD value is typically the integer of the index of the // *next* item to yield. For nested iterators, the HEAD value is typically an index // into the current branch of the iterator tree. Cursor Cursor }
ItemAndCursor represents an item along with its cursor.
Source Files