Documentation
¶
Overview ¶
Package importer streams CSV and XLSX uploads into typed rows. A heading row maps declared columns onto a row struct through scalar codecs; each row is then checked by ordinary validation rules. Valid rows reach the application handler in bounded chunks while invalid rows are reported with their row number and issues. Input, row, cell, decompressed XML and shared-string sizes are all bounded; no complete file or result set is held in memory.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Column ¶
type Column[R any] struct { // contains filtered or unexported fields }
Column maps one file heading onto a field of row type R. Construct it with Field or NullableField. Headings match after trimming spaces, ignoring case.
func Field ¶
func Field[R, V any](heading string, codec foundryhttp.QueryCodec[V], assign func(*R, V)) Column[R]
Field declares a required heading and a non-empty cell parsed by codec. An empty cell is a required issue. assign stores the parsed value in the row.
func NullableField ¶
func NullableField[R, V any](heading string, codec foundryhttp.QueryCodec[V], assign func(*R, value.Nullable[V])) Column[R]
NullableField declares an optional heading. An empty cell, or a file without the heading, assigns value.Null; other cells are parsed by codec.
type Definition ¶
type Definition[R any] struct { // contains filtered or unexported fields }
Definition is an immutable, validated import declaration.
func Define ¶
func Define[R any](spec Spec[R]) Definition[R]
Define copies column declarations and initializes an import definition, retaining validation errors for Validate.
func (Definition[R]) Run ¶
Run reads the source, maps and validates rows, and calls handle with chunks of valid rows. Invalid rows are reported and skipped; blank rows are ignored. Chunks already handled are not undone when a later chunk, a limit or the input fails: run the handler inside a caller-owned transaction and inspect Report.Failed to make an import all-or-nothing. Codec, rule and handler callbacks run synchronously in file order; panic/Goexit are faults.
func (Definition[R]) Validate ¶
func (d Definition[R]) Validate() error
Validate returns the definition's initialization error, or an invalid fault for an undefined import.
type Failure ¶
Failure reports the issues of one rejected row. Paths name the column heading (for parsing issues) or the row's validation field.
type Handler ¶
Handler receives each chunk of valid rows in file order. The slice is owned by the handler. Returning an error stops the import.
type HeadingError ¶
type HeadingError struct {
// contains filtered or unexported fields
}
HeadingError reports a heading row that cannot be mapped: missing required headings or repeated declared headings. Nothing was passed to the handler.
func (*HeadingError) Error ¶
func (*HeadingError) Error() string
Error returns the heading mismatch message.
func (*HeadingError) Is ¶
func (*HeadingError) Is(target error) bool
Is reports whether the target is fault.Invalid.
func (*HeadingError) Issues ¶
func (e *HeadingError) Issues() []contract.Issue
Issues returns a copy of the heading issues.
type Limits ¶
type Limits struct {
// MaxRows bounds data rows after the heading; exceeding it fails the import.
MaxRows int
// MaxColumns bounds the columns of any row.
MaxColumns int
// MaxCellBytes bounds one cell; a longer cell is a row issue.
MaxCellBytes int
// MaxRowBytes bounds one CSV record, including quoted line breaks.
// Longer records fail the import: they cannot be parsed incrementally.
MaxRowBytes int
// MaxInputBytes bounds CSV bytes read or the XLSX archive size.
MaxInputBytes int64
// MaxXMLBytes bounds all decompressed XLSX parts that are read. No single
// XML token may exceed eight times MaxCellBytes plus 4 KiB, whatever this
// total allows. Raise it together with MaxRows for very wide sheets.
MaxXMLBytes int64
// MaxSharedStrings and MaxSharedStringBytes bound the XLSX string table,
// the only part that must be held in memory for random access.
// MaxFailures bounds retained row failures; later failures are counted.
MaxFailures int
// ChunkSize is the number of valid rows passed to each handler call.
ChunkSize int
Validation validation.Limits
}
Limits bound one import. DefaultLimits is the source of the defaults.
func DefaultLimits ¶
func DefaultLimits() Limits
DefaultLimits returns default import input, row, validation, and reporting budgets.
type Report ¶
type Report struct {
Rows int `json:"rows"`
Imported int `json:"imported"`
Failed int `json:"failed"`
Failures []Failure `json:"failures,omitempty"`
}
Report summarizes an import. Rows counts non-blank data rows read; Imported counts rows accepted by the handler; Failed counts rejected rows, of which at most Limits.MaxFailures are retained in Failures.
type Row ¶
Row is one valid parsed row and its 1-based row number: the worksheet row for XLSX, or the line on which the record starts for CSV.
type Source ¶
type Source struct {
// contains filtered or unexported fields
}
Source is one uploaded table. The caller owns the underlying reader and closes it after Run returns. Construct it with CSVSource or XLSXSource.
func CSVSource ¶
CSVSource reads comma-separated UTF-8 text (RFC 4180 quoting). A leading byte-order mark is ignored.
func XLSXSource ¶
XLSXSource reads the first worksheet of an Office Open XML workbook. The archive is random access, so the caller supplies its exact size.
type Spec ¶
type Spec[R any] struct { Columns []Column[R] Rules []validation.Rule[R] HeadingRow int TimeZone temporal.ZoneName Limits Limits }
Spec declares one import. Rules run on each completely parsed row. An empty TimeZone means UTC; it converts spreadsheet date-time cells, which have no zone, into instants. HeadingRow is 1-based; zero means the first row. Zero Limits means DefaultLimits.
Source Files