fetch

package

v0.12.4 Latest Latest Go to latest Published: Apr 22, 2026 License: MIT Imports: 7 Imported by: 0

Documentation ¶

Index ¶

func AllTables() []string
func CoreTables() []string
func URLForTable(name string) (string, bool)
type Config
type Fetcher
- func New(cfg Config) *Fetcher
- func (f *Fetcher) FetchTable(name, url string, force bool) error

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

func AllTables ¶

func AllTables() []string

AllTables returns the names of all available parquet tables.

func CoreTables ¶

func CoreTables() []string

CoreTables returns the names of the core parquet tables.

func URLForTable ¶

func URLForTable(name string) (string, bool)

URLForTable returns the download URL for a named table, and whether it exists.

Types ¶

type Config ¶

type Config struct {
	BaseURL  string
	DataDir  string
	Parallel int
}

Config holds configuration for a Fetcher.

type Fetcher ¶

type Fetcher struct {
	// contains filtered or unexported fields
}

Fetcher downloads parquet tables from OSF into a local data directory.

func New ¶

func New(cfg Config) *Fetcher

New creates a Fetcher with the given configuration.

func (*Fetcher) FetchTable ¶

func (f *Fetcher) FetchTable(name, url string, force bool) error

FetchTable downloads a single parquet table by name, using an atomic rename (.tmp -> final) to avoid partial writes. If force is false and the final file already exists, it is skipped.

Source Files ¶

View all Source files

fetcher.go

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL