Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
// TempRepoDir is a temporary location to schema files downloaded from GitHub.
TempRepoDir string
// CacheDir is the root path for all cached files.
CacheDir string
// CacheSfgaDir is the path SFGA database.
CacheSfgaDir string
// JobsNum is the number of concurrent jobs to run.
JobsNum int
// BatchSize is the number of records to insert in one transaction.
BatchSize int
// BadRow dets decision what to do if a row has more/less fields
// than it should.
BadRow gnfmt.BadRow
// WithZipOutput is a flag to return zipped SFGAarchive outpu.
WithZipOutput bool
// WithQuotes tells that coldp file has `"` to escape new lines and
// delimiters inside fields. If true, RFC-based CSV algorithm is used.
WithQuotes bool
}
Config is a configuration object for the Catalogue of Life Data Package Archive (CoLDP) data processing.
type Option ¶
type Option func(*Config)
Option is a function type that allows to standardize how options to the configuration are organized.
func OptCacheDir ¶
OptCacheDir sets the root path for all temporary files.
func OptCacheSfgaDir ¶
OptCacheSfgaDir sets the path to store resulting sqlite file with data imported from CoLDP file.
func OptJobsNum ¶
OptJobsNum sets the number of concurrent jobs to run.
func OptWithQuotes ¶ added in v0.5.0
OptWithQuotes tells reader that CoLDP file uses quotes in CSV to help data integrity when a field contains either new lines, or designated field delimiters.
func OptWithZipOutput ¶
OptWithZipOutput sets output as binary SQLite file.
Source Files
Click to show internal directories.
Click to hide internal directories.