Documentation
¶
Index ¶
- Constants
- func DeriveTableName(filePath string) string
- func IsJSONLFile(filePath string) bool
- func NewJsonFSSource(sys fs.FS, filePath string) *arraysource.Model
- func NewJsonFileSource(filePath string) *arraysource.Model
- func NewJsonSource(jsonString string, tableName string, isJSONL bool) *arraysource.Model
- func NormalizeRows(rows []map[string]any) []map[string]any
- func NormalizeValue(v any) any
- func ParseJSONArrayReader(r io.Reader) ([]map[string]any, error)
- func ParseJSONFSFile(sys fs.FS, filePath string) ([]map[string]any, error)
- func ParseJSONFile(filePath string) ([]map[string]any, error)
- func ParseJSONLReader(r io.Reader) ([]map[string]any, error)
- func ParseJSONString(content string, isJSONL bool) ([]map[string]any, error)
Constants ¶
const MaxJSONRows = 100000
MaxJSONRows limits the number of data rows (objects) that can be loaded from a single JSON/JSONL file to prevent unbounded memory/CPU consumption.
Variables ¶
This section is empty.
Functions ¶
func DeriveTableName ¶ added in v0.37.0
DeriveTableName extracts the table name from the file path by taking the base filename and removing the extension. "data/users.json" → "users", "events.jsonl" → "events"
func IsJSONLFile ¶ added in v0.37.0
IsJSONLFile checks if the file extension indicates JSONL/NDJSON format.
func NewJsonFSSource ¶ added in v0.40.0
func NewJsonFSSource(sys fs.FS, filePath string) *arraysource.Model
NewJsonFSSource reads a JSON or JSONL file from an embedded filesystem (embed.FS / fs.FS) and returns an array-backed data source ready for querying.
func NewJsonFileSource ¶
func NewJsonFileSource(filePath string) *arraysource.Model
NewJsonFileSource reads a JSON or JSONL file and returns an array-backed data source ready for querying with the array driver.
The file must contain either:
- A JSON array of objects: [{"id":1,"name":"Alice"},...]
- JSONL/NDJSON (one object per line): {"id":1,"name":"Alice"}\n{"id":2,...}
JSONL mode is auto-detected from the file extension (.jsonl or .ndjson). For .json files, the content is parsed as a single JSON array.
JSON has native types, so no string-to-type inference is needed — values are already int, float64, bool, string, or nil. String values that match RFC3339 format are converted to time.Time so the array driver maps them to DATETIME columns. Nested objects and arrays are stored as JSON strings (queryable via SQLite JSON functions).
The table name is derived from the filename (without the extension). For example, "data/users.json" → table name "users". Override with the .Table() method on the returned model if needed.
database.Query().
Model(neat.NewJsonFileSource("data/users.json")).
Where("active = ?", true).
Get(&users)
Panics if the file cannot be opened, parsed, or contains an empty array — there is no header row like CSV to infer column names from, so an empty JSON array cannot produce a queryable table. These are programmer errors (wrong path, malformed file), not runtime conditions.
func NewJsonSource ¶
func NewJsonSource(jsonString string, tableName string, isJSONL bool) *arraysource.Model
NewJsonSource parses a JSON or JSONL string and returns an array-backed data source ready for querying with the array driver.
The content must be either:
- A JSON array of objects: [{"id":1,"name":"Alice"},...]
- JSONL/NDJSON (one object per line): {"id":1,"name":"Alice"}\n{"id":2,...}
Pass isJSONL=true for JSONL content, false for a JSON array. For JSONL, empty lines are skipped.
JSON has native types, so no string-to-type inference is needed — values are already int, float64, bool, string, or nil. String values that match RFC3339 format are converted to time.Time so the array driver maps them to DATETIME columns. Nested objects and arrays are stored as JSON strings (queryable via SQLite JSON functions).
The table name must be provided explicitly since there is no filename to derive it from. Override with the .Table() method on the returned model if needed.
database.Query().
Model(neat.NewJsonSource(jsonString, "users", false)).
Where("active = ?", true).
Get(&users)
Panics if the content cannot be parsed — this is a programmer error (malformed JSON), not a runtime condition. Panics if the content cannot be parsed or is an empty array — there is no header row like CSV to infer column names from, so an empty JSON array cannot produce a queryable table. This is a programmer error, not a runtime condition.
func NormalizeRows ¶ added in v0.37.0
NormalizeRows processes raw JSON rows to make them compatible with the array driver:
- String values matching RFC3339 are converted to time.Time
- Nested maps and arrays are stored as JSON strings
- Other values (int, float64, bool, nil, string) are kept as-is
func NormalizeValue ¶ added in v0.37.0
NormalizeValue converts a JSON-native value into a form the array driver can handle:
- float64 with integer value → int64 (JSON has no int type, but the array driver infers INTEGER columns from int64 values)
- map[string]any → JSON string (via json.Marshal)
- []any → JSON string (via json.Marshal)
- string matching RFC3339 → time.Time
- everything else → unchanged
func ParseJSONArrayReader ¶ added in v0.37.0
ParseJSONArrayReader parses a JSON array of objects from any io.Reader. Returns an error if there is trailing data after the closing bracket. Enforces MaxJSONRows limit incrementally during decoding.
func ParseJSONFSFile ¶ added in v0.40.0
ParseJSONFSFile reads a JSON or JSONL file from an fs.FS (or os.Open if sys is nil) and returns raw rows.
func ParseJSONFile ¶ added in v0.37.0
ParseJSONFile reads a JSON or JSONL file and returns raw rows.
func ParseJSONLReader ¶ added in v0.37.0
ParseJSONLReader parses JSONL (one JSON object per line) from any io.Reader. Empty lines are skipped. Enforces MaxJSONRows limit incrementally during decoding.
Types ¶
This section is empty.
Source Files