Documentation
¶
Index ¶
- Constants
- func CheckCacheFile(filename string, maxAge time.Duration) ([]byte, error)
- func EnsureDankRoot() error
- func FetchSocrata[T any](cfg SocrataConfig, appToken string, maxCacheAge time.Duration) ([]T, error)
- func GetDankCacheDir() string
- func GetDankCachePathname(filename string) string
- func GetDankDir() string
- func GetDankRoot() string
- func MakeCacheFile(filename string) (*os.File, error)
- func SQLString(str string) string
- func SetDankRoot(root string)
- func WriteCSV[T CSVExportable](filename string, items []T) error
- func WriteJSON[T any](filename string, items []T) error
- type CSVExportable
- type SocrataConfig
Constants ¶
const ( DankDir = ".dank" // DankDir is the directory where dank-extract stores its data. CacheDir = "cache" // CacheDir is the directory under DankDir where dank-extract stores its cache files. )
Variables ¶
This section is empty.
Functions ¶
func CheckCacheFile ¶
CheckCacheFile checks DankDir/cache for a file. Returns its bytes and error, if any. If the file is not found, it returns an error. If the file is older than maxAge, it returns an error.
func EnsureDankRoot ¶
func EnsureDankRoot() error
EnsureDankRoot ensures the current DankRoot exists, creating it if needed. Returns an error, if any.
func FetchSocrata ¶
func FetchSocrata[T any](cfg SocrataConfig, appToken string, maxCacheAge time.Duration) ([]T, error)
FetchSocrata fetches data from a Socrata API endpoint with caching and pagination. It handles the common pattern of: check cache, paginate requests, unmarshal, cache.
func GetDankCacheDir ¶
func GetDankCacheDir() string
GetDankCacheDir returns the path to the CacheDir directory.
func GetDankCachePathname ¶
GetDankCachePathname returns the path to the given filename within the CacheDir.
func MakeCacheFile ¶
MakeCacheFile creates a cache file in the DankDir/cache directory and returns its handles. Returns nil with any error.
func SetDankRoot ¶
func SetDankRoot(root string)
SetDankRoot sets the root directory for dank-extract.
func WriteCSV ¶
func WriteCSV[T CSVExportable](filename string, items []T) error
WriteCSV writes any slice of CSVExportable items to a CSV file
Types ¶
type CSVExportable ¶
CSVExportable is an interface for types that can be exported to CSV
type SocrataConfig ¶
type SocrataConfig struct {
URL string // API endpoint URL
CacheFilename string // Filename for caching results
OrderBy string // Field to order by (required for pagination)
BatchSize int // Records per request (default 5000, set higher to disable pagination)
}
SocrataConfig holds configuration for a Socrata API endpoint
Source Files
Directories