Documentation
¶
Overview ¶
Package csvimport is Atlas's in-process worker for the CSV-to-JSON connector task (ADR-0139), and the CSV parser the API's upload-validation endpoint shares with it (ADR-0084).
Like every package under connector/, it rides the standard service-task seam: a CSV connector task compiles to a job carrying compiler.CsvImportJobTypeIndex, and Handler picks that job up off the processor goroutine, after fsync, so parsing a file never allocates on the hot path or runs on the recovery path (I1/I4). Unlike its siblings it talks to no external system — the "connector" reads a variable and writes rows back — which is why it also serves as the parser behind the operator-facing CSV upload check.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Handler ¶
func Handler(store VarStore, lookup ProcessLookup) job.OutputHandler
Handler is the in-process worker for a CSV-import service task (compiler.CsvImportJobType). It converts an uploaded CSV into a JSON `rows` collection so a batch of records is ingested entirely within the process, the file having arrived through a user-task form.
It serves two authoring shapes on the one reserved job type:
- A first-class CSV-to-JSON connector task (ADR-0139): the source variable, delimiter, header handling, columns, and result variable are authored on the task and compiled into a connector detail, which the worker reads from the compiled process (like the mail/rest workers). This is preferred when present.
- The ADR-0087 variable convention: with no connector detail, the worker reads `csvText` and `columnConfig` up the task's scope chain and writes `rows` + `rowCount`, so already-deployed models keep running unchanged.
A missing/empty source, an absent or malformed layout, or an unparseable CSV is a worker error: the job fails and (retries exhausted) raises an incident, rather than silently producing an empty batch.
func ParseRows ¶
ParseRows parses CSV data against the predefined column layout into a list of row objects ({fieldName: value}). It is a pure, side-effect-free transform that runs in the API/side-effect phase, never on the processor hot path (ADR-0084, invariant 1/5).
Type coercion is deliberately lenient: a cell that will not coerce to its declared type is kept as its raw string, so dirty records flow through to be validated and corrected rather than being rejected at ingestion — that is the point of the feature. A *structural* mismatch (a configured header absent from the file, an unparseable CSV, an invalid layout) is an error.
Types ¶
type Column ¶
type Column struct {
Name string `json:"name"` // field name in the row object (required, unique)
Header string `json:"header,omitempty"` // CSV header to read; defaults to Name when the file has a header row
Index *int `json:"index,omitempty"` // 0-based source column, used when the file has no header row
Type string `json:"type,omitempty"` // "string" (default), "number", "integer", "boolean"
}
Column maps one CSV column into a field of the produced row object. The source column is located by Header (when the file has a header row) or by the 0-based Index (when it does not); the cell is coerced to Type. This is the "predefined column layout" a Quality Manager's upload is checked against (ADR-0084).
type Config ¶
type Config struct {
Columns []Column `json:"columns"`
Delimiter string `json:"delimiter,omitempty"` // single character; defaults to ","
HasHeader *bool `json:"hasHeader,omitempty"` // defaults to true (a header row is present)
}
Config is the predefined column layout an uploaded CSV is parsed against.
type ProcessLookup ¶
type ProcessLookup func(defKey uint64) *compiler.CompiledProcess
ProcessLookup resolves a process-definition key to its compiled process, so the worker can read a CSV connector task's authored layout from the model it belongs to (ADR-0139) — mirroring the mail/rest/DMN workers' ProcessLookup.
type VarStore ¶
type VarStore interface {
VariablesOfScope(scope uint64, fn func(v *model.VariableValue) error) error
GetElementInstance(key uint64) (*model.ElementInstanceValue, bool, error)
}
VarStore is the slice of the state store the CSV-import worker reads — a scope's variables and an element instance's parent scope. A narrow interface so the scope-chain walk (and its error paths) are testable with a fake (*state.Store satisfies it).
Source Files