internal/

Package arrowio supplies streamio's Apache Arrow IPC (file format) capabilities: decoding into canonical records and encoding records back into Arrow IPC documents.

Package csvio owns streamio's CSV and TSV support: one package for both, since they differ only in field delimiter.

Package decompress turns a compressed source (gzip, zstd, bzip2, xz, LZ4, Snappy, or Brotli) into the sized, randomly-addressable options.Source every format's decoder needs.

Package formatio defines the per-format capabilities streamio composes: streaming a file's own bytes (RawSource), decoding into canonical records (RecordDecoder), and encoding records as another format's documents (RecordEncoder).

Package jsonio owns streamio's JSON output path: encoding canonical record.Records as JSON documents, and decoding NDJSON back into them.

Package options defines streamio's Config and the functional Option funcs that build it, along with the transform rules and format constants ProcessFile's callers configure a run with.

Package parquetio supplies streamio's Parquet capabilities: raw row-group passthrough, decoding into canonical records, and encoding records back into Parquet documents.

Package pool runs the decode/dispatch worker pool shared by every streamio route: each format only supplies a formatio.RawSource (RunRaw) or a decoder/encoder pair (RunRecords).

Package profile summarizes a stream of canonical records: how many there are, how often each attribute is non-empty, and which values occur most.

Package record defines streamio's canonical, format-neutral row: the intermediate every cross-format conversion goes through between a decoder and an encoder.

Package sniff identifies an input's compression and document format from its leading bytes, using the file extension only as a hint for inputs the content can't settle.