Documentation
¶
Overview ¶
Package dataloader provides a generic batching and caching mechanism for loading data efficiently. It reduces the number of database queries by batching multiple individual requests together and caching results to avoid duplicate fetches.
The primary use case is in GraphQL resolvers where you might have N+1 query problems. Instead of making individual database calls for each item, the dataloader batches requests together and executes them in a single operation.
Basic usage example:
// Create a loader for User entities
userLoader := dataloader.NewStoreLoader(ctx, userStore.FindMany, func(u User) string { return u.ID })
// Use in resolvers - these calls will be batched together
user1, err := userLoader.FetchOne(ctx, "user-id-1")
user2, err := userLoader.FetchOne(ctx, "user-id-2")
user3, err := userLoader.FetchOne(ctx, "user-id-3")
// For database operations with a DB connection:
dbLoader := dataloader.NewStoreLoaderWithDB(ctx, db, dbStore.FindMany, func(u User) string { return u.ID })
For parameterized queries, use FetcherParam:
// Create a parameterized loader
type UserParams struct { Active bool }
paramLoader := dataloader.NewStoreLoaderParam(ctx,
func(ctx context.Context, param UserParams, ids []string) ([]User, error) {
return userStore.FindManyFiltered(ctx, ids, param.Active)
},
func(u User) string { return u.ID },
)
activeUser, err := paramLoader.FetchOneParam(ctx, "user-id", UserParams{Active: true})
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type AggFetchFunc ¶ added in v0.34.0
type AggFetchFunc[K comparable, V any] func(context.Context, []K) ([]V, error)
AggFetchFunc defines the signature for functions that fetch aggregated data. The function receives a context and slice of IDs to fetch, and should return all values associated with any of those IDs (potentially multiple values per ID).
type AggFetchParamFunc ¶ added in v0.34.0
type AggFetchParamFunc[K, P comparable, V any] func(context.Context, P, []K) ([]V, error)
AggFetchParamFunc defines the signature for functions that fetch aggregated data with parameters. The function receives a context, parameter value, and slice of IDs to fetch, and should return all values associated with any of those IDs for the given parameters.
type AggFetchResult ¶ added in v0.34.0
type AggFetchResult[K comparable, V any] struct { ID K // The unique identifier Values []V // All values associated with this ID }
AggFetchResult holds the aggregated results for a single ID. It contains the ID and all values associated with that ID.
type AggFetcher ¶ added in v0.34.0
type AggFetcher[K comparable, V any] struct { *Fetcher[K, AggFetchResult[K, V]] // Embedded Fetcher for aggregated results }
AggFetcher provides batched loading of aggregated data. Unlike the basic Fetcher which returns at most one value per ID, AggFetcher collects multiple values for each ID and returns them as a slice.
This is useful for one-to-many relationships where you need to fetch all related items for a set of parent IDs, such as fetching all alerts for multiple services.
Type parameters:
- K: The type of the unique identifier (key) for items being fetched
- V: The type of values being aggregated
func NewStoreLoaderAgg ¶ added in v0.34.0
func NewStoreLoaderAgg[V any, K comparable](ctx context.Context, fetchMany AggFetchFunc[K, V], idFunc IDFunc[K, V]) *AggFetcher[K, V]
NewStoreLoaderAgg creates a new AggFetcher for loading aggregated data from a store. This is useful for one-to-many relationships where you need to batch-load multiple related items for each parent ID.
The fetchMany function should return all values for the given IDs. Values can be returned in any order and multiple values can have the same ID (as determined by idFunc). The AggFetcher will automatically group them by ID.
Example usage:
// Fetch all alerts for multiple services
alertLoader := dataloader.NewStoreLoaderAgg(ctx,
func(ctx context.Context, serviceIDs []string) ([]Alert, error) {
return alertStore.FindByServiceIDs(ctx, serviceIDs)
},
func(alert Alert) string { return alert.ServiceID },
)
alerts, err := alertLoader.FetchAggregate(ctx, "service-123")
func (*AggFetcher[K, V]) FetchAggregate ¶ added in v0.34.0
func (af *AggFetcher[K, V]) FetchAggregate(ctx context.Context, id K) ([]V, error)
FetchAggregate retrieves all values associated with the given ID. Multiple calls within the same batch window will be batched together into a single call to the underlying fetch function.
Returns a slice of all values associated with the ID, or nil if no values are found. An error is returned if the fetch operation fails or the context is cancelled.
type AggFetcherParam ¶ added in v0.34.0
type AggFetcherParam[K, P comparable, V any] struct { *FetcherParam[K, P, AggFetchResult[K, V]] // Embedded FetcherParam for aggregated results }
AggFetcherParam provides batched loading of aggregated data with parameters. Unlike AggFetcher, this allows you to pass additional parameters that modify the fetch behavior. It maintains separate Fetcher instances for each unique parameter combination.
This is useful for parameterized one-to-many relationships, such as fetching all active alerts for multiple services, or all escalation policies for multiple teams with specific filters.
Type parameters:
- K: The type of the unique identifier (key) for items being fetched
- P: The type of additional parameters that can be passed to modify fetch behavior
- V: The type of values being aggregated
func NewStoreLoaderAggParam ¶ added in v0.34.0
func NewStoreLoaderAggParam[V any, K, P comparable](ctx context.Context, fetchMany AggFetchParamFunc[K, P, V], idFunc IDFunc[K, V]) *AggFetcherParam[K, P, V]
NewStoreLoaderAggParam creates a new AggFetcherParam for loading aggregated data from a store with parameters. This allows you to batch requests not just by ID, but also by parameter values, while still aggregating multiple results per ID.
The fetchMany function should handle the parameter and return all values for the given IDs that match the parameter criteria. Values can be returned in any order and multiple values can have the same ID (as determined by idFunc). The AggFetcherParam will automatically group them by ID.
Example usage:
type AlertParams struct { Status string }
alertLoader := dataloader.NewStoreLoaderAggParam(ctx,
func(ctx context.Context, params AlertParams, serviceIDs []string) ([]Alert, error) {
return alertStore.FindByServiceIDsAndStatus(ctx, serviceIDs, params.Status)
},
func(alert Alert) string { return alert.ServiceID },
)
openAlerts, err := alertLoader.FetchAggregateParam(ctx, "service-123", AlertParams{Status: "open"})
func (*AggFetcherParam[K, P, V]) FetchAggregateParam ¶ added in v0.34.0
func (af *AggFetcherParam[K, P, V]) FetchAggregateParam(ctx context.Context, id K, param P) ([]V, error)
FetchAggregateParam retrieves all values associated with the given ID using the specified parameters. Requests with the same parameter values will be batched together, while requests with different parameters will use separate batches.
The method lazily creates new Fetcher instances for each unique parameter combination. Returns a slice of all values associated with the ID for the given parameters, or nil if no values are found. An error is returned if the fetch operation fails or the context is cancelled.
type FetchFunc ¶ added in v0.34.0
type FetchFunc[K comparable, V any] func(context.Context, []K) ([]V, error)
type FetchParamFunc ¶ added in v0.34.0
type FetchParamFunc[K, P comparable, V any] func(context.Context, P, []K) ([]V, error)
FetchParamFunc defines the signature for functions that fetch data with parameters. The function receives a context, parameter value, and slice of IDs to fetch.
type Fetcher ¶ added in v0.34.0
type Fetcher[K comparable, V any] struct { // FetchFunc is called to retrieve data for a batch of IDs. // It should return values in any order - the Fetcher will map them back to requests // using the ID extracted via IDFunc. FetchFunc func(ctx context.Context, ids []K) ([]V, error) // IDFunc extracts the unique identifier from a value. This function is required // and is used to match returned values back to their corresponding requests. IDFunc func(V) K // MaxBatch sets the maximum number of IDs to include in a single batch. // When this limit is reached, a new batch is started immediately. MaxBatch int // Delay specifies how long to wait before executing a batch. This allows // multiple requests to accumulate into a single batch, improving efficiency. Delay time.Duration // contains filtered or unexported fields }
Fetcher provides batched loading of data with caching. It batches individual requests together to reduce the number of calls to the underlying data source, and caches results to avoid duplicate fetches within the same Fetcher instance.
Type parameters:
- K: The type of the unique identifier (key) for items being fetched
- V: The type of values being fetched
The Fetcher is safe for concurrent use. All methods can be called from multiple goroutines simultaneously.
func NewStoreLoader ¶ added in v0.30.0
func NewStoreLoader[V any, K comparable](ctx context.Context, fetchMany FetchFunc[K, V], idFunc IDFunc[K, V]) *Fetcher[K, V]
NewStoreLoader creates a new Fetcher for loading data from a store without parameters. It's a convenience function for the common case where you only need to batch by ID.
The fetchMany function should return values in any order, and the Fetcher will map them back to the correct requests using the ID field.
func NewStoreLoaderWithDB ¶ added in v0.32.0
func NewStoreLoaderWithDB[V any, K comparable]( ctx context.Context, db gadb.DBTX, fetchMany func(context.Context, gadb.DBTX, []K) ([]V, error), idFunc IDFunc[K, V], ) *Fetcher[K, V]
NewStoreLoaderWithDB creates a new Fetcher that automatically passes a database connection to the fetch function. This is a convenience wrapper around NewStoreLoader for database-based loading operations.
func (*Fetcher[K, V]) Close ¶ added in v0.34.0
func (f *Fetcher[K, V]) Close()
Close waits for all pending batches to complete. This should be called when the Fetcher is no longer needed to ensure proper cleanup and prevent goroutine leaks.
func (*Fetcher[K, V]) FetchOne ¶ added in v0.34.0
FetchOne retrieves a single value by its ID. Multiple calls to FetchOne within the same batch window (defined by Delay) will be batched together into a single call to FetchFunc.
The method returns a pointer to the value if found, or nil if not found. An error is returned if the fetch operation fails or the context is cancelled.
type FetcherParam ¶ added in v0.34.0
type FetcherParam[K, P comparable, V any] struct { FetchFunc FetchParamFunc[K, P, V] // Function called to fetch data with parameters IDFunc IDFunc[K, V] // Function to extract ID from values MaxBatch int // Maximum batch size for each parameter's fetcher Delay time.Duration // Delay before executing batches // contains filtered or unexported fields }
FetcherParam provides batched loading of data with parameters. Unlike the basic Fetcher, this allows you to pass additional parameters that modify the fetch behavior. It maintains separate Fetcher instances for each unique parameter combination.
Type parameters:
- K: The type of the unique identifier (key) for items being fetched
- P: The type of additional parameters that can be passed to modify fetch behavior
- V: The type of values being fetched
Example usage:
type UserParams struct { Active bool }
paramLoader := dataloader.NewStoreLoaderParam(
ctx,
func(ctx context.Context, param UserParams, ids []string) ([]User, error) {
return userStore.FindManyFiltered(ctx, ids, param.Active)
},
func(u User) string { return u.ID },
)
activeUser, err := paramLoader.FetchOneParam(ctx, "user-id", UserParams{Active: true})
func NewStoreLoaderParam ¶ added in v0.34.0
func NewStoreLoaderParam[K, P comparable, V any](ctx context.Context, fetchMany FetchParamFunc[K, P, V], idFunc IDFunc[K, V]) *FetcherParam[K, P, V]
NewStoreLoaderParam creates a new FetcherParam for loading data from a store with parameters. This allows you to batch requests not just by ID, but also by parameter values.
The fetchMany function should handle the parameter and return values in any order. The FetcherParam will create separate Fetcher instances for each unique parameter combination.
func (*FetcherParam[K, P, V]) Close ¶ added in v0.34.0
func (fp *FetcherParam[K, P, V]) Close()
func (*FetcherParam[K, P, V]) FetchOneParam ¶ added in v0.34.0
func (fp *FetcherParam[K, P, V]) FetchOneParam(ctx context.Context, id K, param P) (*V, error)
FetchOneParam retrieves a single value by its ID using the specified parameters. Requests with the same parameter values will be batched together, while requests with different parameters will use separate batches.
The method lazily creates new Fetcher instances for each unique parameter combination. This allows efficient batching while still supporting parameterized queries.
type IDFunc ¶ added in v0.34.0
type IDFunc[K comparable, V any] func(V) K
type Loader ¶ added in v0.30.0
type Loader[K comparable, V any] = Fetcher[K, V]
Source Files