Documentation
¶
Overview ¶
Package jsonl reads and writes graphs in newline-delimited JSON (NDJSON / JSON Lines) format.
Records have one of three shapes:
{"type": "node", "id": "alice"}
{"type": "edge", "src": "alice", "dst": "bob", "weight": 7}
{"type": "property", "id": "alice", "key": "age", "value": "30", "kind": "int64"}
The 'weight' field is optional and defaults to 0. Property records are produced and consumed by WriteWithProps / ReadWithProps.
Index ¶
- Constants
- Variables
- func ReadInto(r io.Reader, cfg adjlist.Config) (*adjlist.AdjList[string, int64], int, error)
- func ReadIntoCappedCtx(ctx context.Context, r io.Reader, cfg adjlist.Config, maxBytes int64) (*adjlist.AdjList[string, int64], int, error)
- func ReadIntoCtx(ctx context.Context, r io.Reader, cfg adjlist.Config) (*adjlist.AdjList[string, int64], int, error)
- func ReadWithProps(r io.Reader, cfg adjlist.Config) (*lpg.Graph[string, int64], int, error)
- func ReadWithPropsCappedCtx(ctx context.Context, r io.Reader, cfg adjlist.Config, maxBytes int64) (*lpg.Graph[string, int64], int, error)
- func ReadWithPropsCtx(ctx context.Context, r io.Reader, cfg adjlist.Config) (*lpg.Graph[string, int64], int, error)
- func Write(w io.Writer, a *adjlist.AdjList[string, int64]) (int, error)
- func WriteCtx(ctx context.Context, w io.Writer, a *adjlist.AdjList[string, int64]) (int, error)
- func WriteWithProps(w io.Writer, g *lpg.Graph[string, int64]) (int, error)
- func WriteWithPropsCtx(ctx context.Context, w io.Writer, g *lpg.Graph[string, int64]) (int, error)
- type Record
Examples ¶
Constants ¶
const DefaultMaxBytes int64 = 1 << 30 // 1 GiB
DefaultMaxBytes is the default ceiling, in bytes, on the amount of input the default read entry points will consume before failing with ErrInputTooLarge. It guards against memory exhaustion from untrusted files (a crafted multi-gigabyte line, for example). The capped variants (ReadIntoCappedCtx, ReadWithPropsCappedCtx) accept an explicit ceiling; a value of zero or less disables the cap.
Variables ¶
var ErrInputTooLarge = errors.New("jsonl: input exceeds maximum size")
ErrInputTooLarge is returned by the read functions when the input stream exceeds the configured byte ceiling. The reader fails as soon as the limit is crossed, before the offending line is fully buffered, so allocation stays bounded.
var ErrUnknownType = errors.New("jsonl: unknown record type")
ErrUnknownType is returned by ReadInto, ReadIntoCtx, ReadWithProps, and ReadWithPropsCtx when a record's "type" field contains a value that is not one of the recognised literals ("node", "edge", "property").
Functions ¶
func ReadInto ¶
ReadInto consumes a JSON Lines stream from r and builds an adjacency list. Node records pre-intern endpoints; edge records add the edge with optional weight.
func ReadIntoCappedCtx ¶ added in v0.2.0
func ReadIntoCappedCtx(ctx context.Context, r io.Reader, cfg adjlist.Config, maxBytes int64) (*adjlist.AdjList[string, int64], int, error)
ReadIntoCappedCtx is ReadIntoCtx with an explicit input-size ceiling. When maxBytes > 0 the reader fails with ErrInputTooLarge the moment consumption exceeds the limit, before the offending line is fully buffered; a value of zero or less disables the cap.
func ReadIntoCtx ¶
func ReadIntoCtx(ctx context.Context, r io.Reader, cfg adjlist.Config) (*adjlist.AdjList[string, int64], int, error)
ReadIntoCtx is the context-aware variant of ReadInto. ctx.Err() is checked every 4096 rows. The input is capped at DefaultMaxBytes; use ReadIntoCappedCtx for an explicit ceiling.
On any error — a parse error, context cancellation, or the ErrInputTooLarge cap — the returned graph is nil; the import is all-or-nothing at the in-memory level, so a caller cannot accidentally commit a half-built graph. The typed error is returned unchanged; only the graph value is discarded.
func ReadWithProps ¶
ReadWithProps consumes a JSON Lines stream from r and builds a labelled property graph. It handles "node", "edge", and "property" record types. Property records must appear after the "node" record for the referenced ID.
func ReadWithPropsCappedCtx ¶ added in v0.2.0
func ReadWithPropsCappedCtx(ctx context.Context, r io.Reader, cfg adjlist.Config, maxBytes int64) (*lpg.Graph[string, int64], int, error)
ReadWithPropsCappedCtx is ReadWithPropsCtx with an explicit input-size ceiling. When maxBytes > 0 the reader fails with ErrInputTooLarge the moment consumption exceeds the limit, before the offending line is fully buffered; a value of zero or less disables the cap.
func ReadWithPropsCtx ¶
func ReadWithPropsCtx(ctx context.Context, r io.Reader, cfg adjlist.Config) (*lpg.Graph[string, int64], int, error)
ReadWithPropsCtx is the context-aware variant of ReadWithProps. ctx.Err() is checked every 4096 rows. The input is capped at DefaultMaxBytes; use ReadWithPropsCappedCtx for an explicit ceiling.
On any error — a parse error, context cancellation, or the ErrInputTooLarge cap — the returned graph is nil; the import is all-or-nothing at the in-memory level, so a caller cannot accidentally commit a half-built graph. The typed error is returned unchanged; only the graph value is discarded.
func Write ¶
Write streams every node and edge of a to w as JSON Lines. Nodes come first, then edges, so that on-read every endpoint is known before its referencing edge.
Example ¶
ExampleWrite shows a JSON Lines round-trip: marshal a directed, weighted graph to NDJSON (one node/edge record per line) with Write, then unmarshal it back with ReadInto and confirm the structure.
package main
import (
"bytes"
"fmt"
"github.com/FlavioCFOliveira/GoGraph/graph/adjlist"
"github.com/FlavioCFOliveira/GoGraph/graph/io/jsonl"
)
func main() {
cfg := adjlist.Config{Directed: true}
src := adjlist.New[string, int64](cfg)
_ = src.AddEdge("a", "b", 7)
_ = src.AddEdge("b", "c", 9)
var buf bytes.Buffer
if _, err := jsonl.Write(&buf, src); err != nil {
panic(err)
}
dst, records, err := jsonl.ReadInto(&buf, cfg)
if err != nil {
panic(err)
}
fmt.Println("records read:", records)
fmt.Println("order:", dst.Order())
fmt.Println("size:", dst.Size())
fmt.Println("b->c:", dst.HasEdge("b", "c"))
}
Output: records read: 5 order: 3 size: 2 b->c: true
func WriteCtx ¶
WriteCtx is the context-aware variant of Write. ctx.Err() is checked every 4096 records; on cancellation flushes the buffer and returns (recordsWritten, wrapped ctx.Err()).
func WriteWithProps ¶
WriteWithProps streams the full contents of an lpg.Graph to w as JSON Lines. Output order is: all node records, then all edge records, then all property records. This ordering ensures that ReadWithProps can reconstruct the graph in a single pass.
Example ¶
ExampleWriteWithProps shows the labelled-property-graph round-trip: WriteWithProps emits a property record per typed property and ReadWithProps restores it, so a string property recovers its value.
package main
import (
"bytes"
"fmt"
"github.com/FlavioCFOliveira/GoGraph/graph/adjlist"
"github.com/FlavioCFOliveira/GoGraph/graph/io/jsonl"
"github.com/FlavioCFOliveira/GoGraph/graph/lpg"
)
func main() {
cfg := adjlist.Config{Directed: true}
src := lpg.New[string, int64](cfg)
_ = src.AddEdge("alice", "bob", 1)
_ = src.SetNodeProperty("alice", "city", lpg.StringValue("Lisbon"))
var buf bytes.Buffer
if _, err := jsonl.WriteWithProps(&buf, src); err != nil {
panic(err)
}
dst, _, err := jsonl.ReadWithProps(&buf, cfg)
if err != nil {
panic(err)
}
got, ok := dst.GetNodeProperty("alice", "city")
city, _ := got.String()
fmt.Println("alice has city:", ok)
fmt.Println("alice.city:", city)
}
Output: alice has city: true alice.city: Lisbon
Types ¶
type Record ¶
type Record struct {
Type string `json:"type"`
ID string `json:"id,omitempty"` // node key — used by "node" and "property" records
Src string `json:"src,omitempty"` // edge source
Dst string `json:"dst,omitempty"` // edge destination
Weight int64 `json:"weight,omitempty"` // edge weight (defaults to 0)
Key string `json:"key,omitempty"` // property key — used by "property" records
Value string `json:"value,omitempty"` // property value serialised as a string
Kind string `json:"kind,omitempty"` // property kind: "string","int64","float64","bool","time","bytes"
}
Record is the wire shape of a JSON-Lines event.