batch

package

v0.2.0 Latest Latest Go to latest Published: Sep 6, 2026 License: MIT Imports: 6 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/DemografixGenderize/demografix-cli

Links

Open Source Insights

Documentation ¶

Overview ¶

Package batch groups names by country and chunks them to the API's 100-per-request cap, then runs the requests with bounded concurrency while preserving input order. It is shared by the predict commands and enrich.

Index ¶

func Chunk[T any](xs []T, size int) [][]T
func Names(items []Item) []string
type Item
- func Items(names []string, country string) []Item
type Outcome
- func RunRequests[T any](ctx context.Context, reqs []Request, total int, concurrency int, ...) ([]Outcome[T], api.Quota, error)
type Request
- func Plan(items []Item, size int) []Request

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

func Chunk ¶

func Chunk[T any](xs []T, size int) [][]T

Chunk splits xs into slices of at most size elements.

func Names ¶

func Names(items []Item) []string

Names returns the names of items in order.

Types ¶

type Item ¶

type Item struct {
	Index   int
	Name    string
	Country string // normalized; "" means no country (always "" for nationality)
}

Item is one name carrying its original input index, which is the key used to restore output order regardless of completion order.

func Items ¶

func Items(names []string, country string) []Item

Items builds positional items for a single fixed country (or "").

type Outcome ¶

type Outcome[T any] struct {
	Item  Item
	Value T
	Done  bool
	Err   error
}

Outcome is the result for one item, placed at outcomes[Item.Index]. Done is true only when the item's request succeeded; an item whose request was never scheduled (e.g. cancelled after a 429) has Done false and Err nil.

func RunRequests ¶

func RunRequests[T any](
	ctx context.Context,
	reqs []Request,
	total int,
	concurrency int,
	fn func(context.Context, Request) ([]T, api.Quota, error),
) ([]Outcome[T], api.Quota, error)

RunRequests runs fn for each request with at most concurrency in flight, placing each item's result at outcomes[Item.Index] so input order is preserved. A 429 cancels remaining scheduling; other errors do not abort the run. The returned Quota is the freshest view (smallest Remaining seen) and the returned error is the first error encountered, if any.

type Request ¶

type Request struct {
	Country string
	Items   []Item
}

Request is one HTTP call's worth of work: a single country and at most MaxBatch names.

func Plan ¶

func Plan(items []Item, size int) []Request

Plan groups items by country (sorted, for reproducibility) and chunks each group to size. Output order is restored later via Item.Index, so request order does not affect correctness.

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL