jsonl

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 5, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Overview

Package jsonl reads and writes graphs in newline-delimited JSON (NDJSON / JSON Lines) format.

Records have one of three shapes:

{"type": "node", "id": "alice"}
{"type": "edge", "src": "alice", "dst": "bob", "weight": 7}
{"type": "property", "id": "alice", "key": "age", "value": "30", "kind": "int64"}

The 'weight' field is optional and defaults to 0. Property records are produced and consumed by WriteWithProps / ReadWithProps.

Index

Examples

Constants

View Source
const DefaultMaxBytes int64 = 1 << 30 // 1 GiB

DefaultMaxBytes is the default ceiling, in bytes, on the amount of input the default read entry points will consume before failing with ErrInputTooLarge. It guards against memory exhaustion from untrusted files (a crafted multi-gigabyte line, for example). The capped variants (ReadIntoCappedCtx, ReadWithPropsCappedCtx) accept an explicit ceiling; a value of zero or less disables the cap.

Variables

View Source
var ErrInputTooLarge = errors.New("jsonl: input exceeds maximum size")

ErrInputTooLarge is returned by the read functions when the input stream exceeds the configured byte ceiling. The reader fails as soon as the limit is crossed, before the offending line is fully buffered, so allocation stays bounded.

View Source
var ErrUnknownType = errors.New("jsonl: unknown record type")

ErrUnknownType is returned by ReadInto, ReadIntoCtx, ReadWithProps, and ReadWithPropsCtx when a record's "type" field contains a value that is not one of the recognised literals ("node", "edge", "property").

Functions

func ReadInto

func ReadInto(r io.Reader, cfg adjlist.Config) (*adjlist.AdjList[string, int64], int, error)

ReadInto consumes a JSON Lines stream from r and builds an adjacency list. Node records pre-intern endpoints; edge records add the edge with optional weight.

func ReadIntoCappedCtx added in v0.2.0

func ReadIntoCappedCtx(ctx context.Context, r io.Reader, cfg adjlist.Config, maxBytes int64) (*adjlist.AdjList[string, int64], int, error)

ReadIntoCappedCtx is ReadIntoCtx with an explicit input-size ceiling. When maxBytes > 0 the reader fails with ErrInputTooLarge the moment consumption exceeds the limit, before the offending line is fully buffered; a value of zero or less disables the cap.

func ReadIntoCtx

func ReadIntoCtx(ctx context.Context, r io.Reader, cfg adjlist.Config) (*adjlist.AdjList[string, int64], int, error)

ReadIntoCtx is the context-aware variant of ReadInto. ctx.Err() is checked every 4096 rows. The input is capped at DefaultMaxBytes; use ReadIntoCappedCtx for an explicit ceiling.

On any error — a parse error, context cancellation, or the ErrInputTooLarge cap — the returned graph is nil; the import is all-or-nothing at the in-memory level, so a caller cannot accidentally commit a half-built graph. The typed error is returned unchanged; only the graph value is discarded.

func ReadWithProps

func ReadWithProps(r io.Reader, cfg adjlist.Config) (*lpg.Graph[string, int64], int, error)

ReadWithProps consumes a JSON Lines stream from r and builds a labelled property graph. It handles "node", "edge", and "property" record types. Property records must appear after the "node" record for the referenced ID.

func ReadWithPropsCappedCtx added in v0.2.0

func ReadWithPropsCappedCtx(ctx context.Context, r io.Reader, cfg adjlist.Config, maxBytes int64) (*lpg.Graph[string, int64], int, error)

ReadWithPropsCappedCtx is ReadWithPropsCtx with an explicit input-size ceiling. When maxBytes > 0 the reader fails with ErrInputTooLarge the moment consumption exceeds the limit, before the offending line is fully buffered; a value of zero or less disables the cap.

func ReadWithPropsCtx

func ReadWithPropsCtx(ctx context.Context, r io.Reader, cfg adjlist.Config) (*lpg.Graph[string, int64], int, error)

ReadWithPropsCtx is the context-aware variant of ReadWithProps. ctx.Err() is checked every 4096 rows. The input is capped at DefaultMaxBytes; use ReadWithPropsCappedCtx for an explicit ceiling.

On any error — a parse error, context cancellation, or the ErrInputTooLarge cap — the returned graph is nil; the import is all-or-nothing at the in-memory level, so a caller cannot accidentally commit a half-built graph. The typed error is returned unchanged; only the graph value is discarded.

func Write

func Write(w io.Writer, a *adjlist.AdjList[string, int64]) (int, error)

Write streams every node and edge of a to w as JSON Lines. Nodes come first, then edges, so that on-read every endpoint is known before its referencing edge.

Example

ExampleWrite shows a JSON Lines round-trip: marshal a directed, weighted graph to NDJSON (one node/edge record per line) with Write, then unmarshal it back with ReadInto and confirm the structure.

package main

import (
	"bytes"
	"fmt"

	"github.com/FlavioCFOliveira/GoGraph/graph/adjlist"
	"github.com/FlavioCFOliveira/GoGraph/graph/io/jsonl"
)

func main() {
	cfg := adjlist.Config{Directed: true}

	src := adjlist.New[string, int64](cfg)
	_ = src.AddEdge("a", "b", 7)
	_ = src.AddEdge("b", "c", 9)

	var buf bytes.Buffer
	if _, err := jsonl.Write(&buf, src); err != nil {
		panic(err)
	}

	dst, records, err := jsonl.ReadInto(&buf, cfg)
	if err != nil {
		panic(err)
	}

	fmt.Println("records read:", records)
	fmt.Println("order:", dst.Order())
	fmt.Println("size:", dst.Size())
	fmt.Println("b->c:", dst.HasEdge("b", "c"))
}
Output:
records read: 5
order: 3
size: 2
b->c: true

func WriteCtx

func WriteCtx(ctx context.Context, w io.Writer, a *adjlist.AdjList[string, int64]) (int, error)

WriteCtx is the context-aware variant of Write. ctx.Err() is checked every 4096 records; on cancellation flushes the buffer and returns (recordsWritten, wrapped ctx.Err()).

func WriteWithProps

func WriteWithProps(w io.Writer, g *lpg.Graph[string, int64]) (int, error)

WriteWithProps streams the full contents of an lpg.Graph to w as JSON Lines. Output order is: all node records, then all edge records, then all property records. This ordering ensures that ReadWithProps can reconstruct the graph in a single pass.

Example

ExampleWriteWithProps shows the labelled-property-graph round-trip: WriteWithProps emits a property record per typed property and ReadWithProps restores it, so a string property recovers its value.

package main

import (
	"bytes"
	"fmt"

	"github.com/FlavioCFOliveira/GoGraph/graph/adjlist"
	"github.com/FlavioCFOliveira/GoGraph/graph/io/jsonl"
	"github.com/FlavioCFOliveira/GoGraph/graph/lpg"
)

func main() {
	cfg := adjlist.Config{Directed: true}

	src := lpg.New[string, int64](cfg)
	_ = src.AddEdge("alice", "bob", 1)
	_ = src.SetNodeProperty("alice", "city", lpg.StringValue("Lisbon"))

	var buf bytes.Buffer
	if _, err := jsonl.WriteWithProps(&buf, src); err != nil {
		panic(err)
	}

	dst, _, err := jsonl.ReadWithProps(&buf, cfg)
	if err != nil {
		panic(err)
	}

	got, ok := dst.GetNodeProperty("alice", "city")
	city, _ := got.String()
	fmt.Println("alice has city:", ok)
	fmt.Println("alice.city:", city)
}
Output:
alice has city: true
alice.city: Lisbon

func WriteWithPropsCtx

func WriteWithPropsCtx(ctx context.Context, w io.Writer, g *lpg.Graph[string, int64]) (int, error)

WriteWithPropsCtx is the context-aware variant of WriteWithProps. ctx.Err() is checked every 4096 records.

Types

type Record

type Record struct {
	Type   string `json:"type"`
	ID     string `json:"id,omitempty"`     // node key — used by "node" and "property" records
	Src    string `json:"src,omitempty"`    // edge source
	Dst    string `json:"dst,omitempty"`    // edge destination
	Weight int64  `json:"weight,omitempty"` // edge weight (defaults to 0)
	Key    string `json:"key,omitempty"`    // property key — used by "property" records
	Value  string `json:"value,omitempty"`  // property value serialised as a string
	Kind   string `json:"kind,omitempty"`   // property kind: "string","int64","float64","bool","time","bytes"
}

Record is the wire shape of a JSON-Lines event.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL