csvpputil

csvpputil

Utility package for converting CSV++ data to JSON and YAML formats.

Requirements

This package uses encoding/json/jsontext from Go's experimental JSON v2 implementation.

GOEXPERIMENT=jsonv2 go build ./...
GOEXPERIMENT=jsonv2 go test ./...

Features

• Streaming JSON output - Memory-efficient for large files
• YAML output - With preserved key order
• Full CSV++ field type support - SimpleField, ArrayField, StructuredField, ArrayStructuredField

API

Streaming Writers

For large datasets, use streaming writers to minimize memory usage.

JSONArrayWriter

w := csvpputil.NewJSONArrayWriter(os.Stdout, headers)

for _, record := range records {
    if err := w.Write(record); err != nil {
        return err
    }
}

if err := w.Close(); err != nil {
    return err
}

YAMLArrayWriter

w := csvpputil.NewYAMLArrayWriter(os.Stdout, headers,
    csvpputil.WithYAMLCapacity(1000), // optional: pre-allocate buffer
)

for _, record := range records {
    if err := w.Write(record); err != nil {
        return err
    }
}

if err := w.Close(); err != nil {
    return err
}

Options:

• WithYAMLCapacity(n) - Pre-allocates the internal buffer for n records, reducing memory allocations when the approximate record count is known.

Note: YAML output is buffered until Close() due to go-yaml library constraints.

Convenience Functions

For small to medium datasets, use these one-shot functions.

Marshal Functions

// CSV++ to JSON bytes
jsonBytes, err := csvpputil.MarshalJSON(headers, records)

// CSV++ to YAML bytes
yamlBytes, err := csvpputil.MarshalYAML(headers, records)

Write Functions

// Write JSON to io.Writer
err := csvpputil.WriteJSON(w, headers, records)

// Write YAML to io.Writer
err := csvpputil.WriteYAML(w, headers, records)

Example

package main

import (
    "fmt"
    "log"

    "github.com/osamingo/go-csvpp"
    "github.com/osamingo/go-csvpp/csvpputil"
)

func main() {
    headers := []*csvpp.ColumnHeader{
        {Name: "name", Kind: csvpp.SimpleField},
        {Name: "age", Kind: csvpp.SimpleField},
    }

    records := [][]*csvpp.Field{
        {{Value: "Alice"}, {Value: "30"}},
        {{Value: "Bob"}, {Value: "25"}},
    }

    // Convert to JSON
    data, err := csvpputil.MarshalJSON(headers, records)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(string(data))
    // Output: [{"name":"Alice","age":"30"},{"name":"Bob","age":"25"}]

    // Convert to YAML
    yamlData, err := csvpputil.MarshalYAML(headers, records)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Print(string(yamlData))
    // Output:
    // - name: Alice
    //   age: "30"
    // - name: Bob
    //   age: "25"
}

Field Type Mapping

License

See the LICENSE file in the repository root.