Documentation
¶
Overview ¶
Package ndjson provides NDJSON (newline-delimited JSON) import and export for the pulse I/O pipeline.
Package ndjson provides NDJSON (newline-delimited JSON) import and export for the pulse I/O pipeline.
Index ¶
Constants ¶
const OverlayTrailerKey = "_overlays"
OverlayTrailerKey is the JSON object key the trailing sidecar line carries. Reserved namespace — pulse cohort schemas never produce a field with this name (the underscore prefix is reserved per the Crosstab long-form `_margin` precedent). Readers detect the trailer by checking whether a line's parsed object contains this key.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Reader ¶
type Reader struct {
// contains filtered or unexported fields
}
Reader reads NDJSON data from a byte source. Each line is one JSON object.
func NewReaderFromBytes ¶
NewReaderFromBytes creates an NDJSON reader from raw bytes.
func (*Reader) ReadHeader ¶
ReadHeader returns column names derived from the keys of the first JSON object.
func (*Reader) ReadOverlays ¶ added in v0.19.0
func (r *Reader) ReadOverlays() ([]*types.OverlayLayer, error)
ReadOverlays walks the NDJSON source and returns the overlay layers carried by the trailing `{"_overlays": [...]}` line, if present. Returns nil (not error) when no trailer is found — matches the Arrow / Excel / Parquet "absent means nil" contract.
The reader scans every line; if multiple trailing-shaped lines are present (a defensive case the writer never produces) the LAST one wins. Lines whose first parsed key is not the trailer key are skipped silently — host record lines remain consumable through ReadRows.
ReadOverlays is independent of ReadHeader / ReadRows — the underlying byte source (data or afero path) is re-read so calling ReadOverlays before, after, or instead of ReadRows is safe.
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
Writer writes NDJSON data to a buffer.
func NewWriterToBuffer ¶
func NewWriterToBuffer() *Writer
NewWriterToBuffer creates an NDJSON writer that writes to an internal buffer.
func (*Writer) Close ¶
Close flushes and writes to the target path if configured. When SetOverlays handed the writer a non-empty []*OverlayLayer slice the trailing `{"_overlays": [...]}` block is appended to the internal buffer BEFORE the buffer is flushed to fs — so both Bytes() and the on-disk artefact carry the trailer. Idempotent: a second Close() re-flushes the same buffer without appending the trailer twice.
func (*Writer) SetOverlays ¶ added in v0.19.0
func (w *Writer) SetOverlays(layers []*types.OverlayLayer)
SetOverlays records the Response.Overlays layers the export pipeline wants the writer to embed in the NDJSON file. Each layer lands inside a single trailing line `{"_overlays": [...]}` appended after the last host-record line per research/export-embedding-shape.md § 6. nil or empty layers leave the NDJSON output byte-identical to a pre-overlay export (no trailer line lands). Implements pio.OverlayAwareWriter.
Emission happens at Close() time so the host record stream stays untouched until the file is finalised. Must be called BEFORE Close.
func (*Writer) WriteHeader ¶
WriteHeader records the column names for use in subsequent WriteRow calls. NDJSON has no header line; the columns are used as keys in each JSON object.
Source Files