README
¶
go-simple-serializer
Description
go-simple-serializer (aka GSS) is a simple library for serializing/deserializing objects that aims to decrease the burden on developers to support multiple serialization formats in their applications.
Using cross compilers, this library can also be called by other languages, including C, C++, Python, and JavaScript. This library is cross compiled into a Shared Object file (*.so), which can be called by C, C++, and Python on Linux machines. This library is also compiled to pure JavaScript using GopherJS, which can be called by Node.js and loaded in the browser. See the examples folder for patterns that you can use.
Formats
GSS supports the following formats.
| Format | Description |
|---|---|
| bson | Binary JSON |
| csv | Comma-Separated Values |
| hcl | HashiCorp Configuration Language |
| hcl2 | HashiCorp Configuration Language (version 2.x) |
| json | JSON |
| jsonl | JSON Lines |
| properties | Properties |
| tags | single-line key=value tags |
| toml | TOML |
| tsv | Tab-Separated Values |
| yaml | YAML |
hcl and hcl2 implementation is fragile and very much in alpha. The other formats are well-supported.
Packages
The main public api for GSS is in the gss package. However, this library does ship with lower-level packages that can be imported directly as well.
| Package | Purpose |
|---|---|
| bson | Binary JSON |
| escaper | Escape/unescape strings |
| gss | The main public API |
| inspector | Reusable functions for inspecting objects |
| iterator | Wrapper for iterable formats |
| json | JSON |
| jsonl | JSON Lines |
| properties | Properties Files |
| scanner | Scanning through a stream of bytes |
| splitter | Creating custom bufio.SplitFunc |
| sv | Separated-Values formats, i.e., CSV and TSV. |
| toml | TOML |
| yaml | YAML |
Usage
CLI
The command line tool, gss, can be used to easily covert data between formats. We currently support the following platforms.
| GOOS | GOARCH |
|---|---|
| darwin | amd64 |
| linux | amd64 |
| windows | amd64 |
| linux | arm64 |
Pull requests to support other platforms are welcome! See the examples section below for usage.
Go
You can install the go-simple-serializer packages with.
go get -u -d github.com/spatialcurrent/go-simple-serializer/...
You can then import the main public API with import "github.com/spatialcurrent/go-simple-serializer/pkg/gss" or one of the underlying packages, e.g., import "github.com/spatialcurrent/go-simple-serializer/pkg/tags".
See go-simple-serializer in GoDoc for API documentation and examples.
Node
GSS is built as a module. In modern JavaScript, the module can be imported using destructuring assignment.
const { serialize, deserialize, convert, formats } = require('./dist/gss.global.min.js');
In legacy JavaScript, you can use the gss.main.js file that simply adds gss to the global scope.
Android
The go-simple-serializer code is available for use in Android applications under com.spatialcurrent.gss. For example,
import com.spatialcurrent.gss.Gss;
...
String output_format = Gss.convert(input_string, input_format, input_header, input_comment, output_format, verbose);
...
C
A variant of the Convert function is exported in a Shared Object file (*.so), which can be called by C, C++, and Python programs on Linux machines. For complete patterns for C, C++, and Python, see the examples folder in this repo.
Releases
go-simple-serializer is currently in alpha. See releases at https://github.com/spatialcurrent/go-simple-serializer/releases.
Examples
CLI
.gitignore file to jsonl
cat .gitignore | gss -i csv --input-header path -o json
Get language from CircleCI config.
cat .circleci/config.yml | gss -i yaml -o json -c '#' | jq -r .version
Convert list of files to JSON Lines
find . -name '*.go' | gss -i csv --input-header path -o jsonl
Go
See the examples in GoDoc.
C
See the examples/c/main.c file. You can run the example with make run_example_c.
C++
See the examples/cpp/main.cpp file. You can run the example with make run_example_cpp.
Python
See the examples/python/test.py file. You can run the example with make run_example_python.
JavaScript
See the examples/js/index.js file. You can run the example with make run_example_javascript.
Building
CLI
The make build_cli script is used to build executables for Linux and Windows.
JavaScript
You can compile GSS to pure JavaScript with the make build_javascript script.
Android
The make build_android script is used to build an Android Archive (AAR) file and associated Javadocs.
Shared Object
The make build_so script is used to build a Shared Object (*.go), which can be called by C, C++, and Python on Linux machines.
Changing Destination
The default destination for build artifacts is go-simple-serializer/bin, but you can change the destination with an environment variable. For building on a Chromebook consider saving the artifacts in /usr/local/go/bin, e.g., DEST=/usr/local/go/bin make build_cli
Testing
Go
To run Go tests use make test_go (or bash scripts/test.sh), which runs unit tests, go vet, go vet with shadow, errcheck, ineffassign, staticcheck, and misspell.
JavaScript
To run JavaScript tests, first install Jest using make deps_javascript, use Yarn, or another method. Then, build the JavaScript module with make build_javascript. To run tests, use make test_javascript. You can also use the scripts in the package.json.
Contributing
Spatial Current, Inc. is currently accepting pull requests for this repository. We'd love to have your contributions! Please see Contributing.md for how to get started.
License
This work is distributed under the MIT License. See LICENSE file.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
gss
command
gss is the command line program for go-simple-serializer (GSS).
|
gss is the command line program for go-simple-serializer (GSS). |
|
gss.global.js
command
gss.global.js is the package for go-simple-serializer (GSS) that adds GSS functions to the global scope under the "gss" variable.
|
gss.global.js is the package for go-simple-serializer (GSS) that adds GSS functions to the global scope under the "gss" variable. |
|
gss.mod.js
command
gss.mod.js is the package for go-simple-serializer (GSS) that is built as a JavaScript module.
|
gss.mod.js is the package for go-simple-serializer (GSS) that is built as a JavaScript module. |
|
pkg
|
|
|
bson
Package bson provides an API for BSON serialization.
|
Package bson provides an API for BSON serialization. |
|
cli
Package cli manages loading and testing configuration for serializing and deserializing objects from the command line.
|
Package cli manages loading and testing configuration for serializing and deserializing objects from the command line. |
|
escaper
Package escaper provides an easy API to escapae and unescape a string using a custom escape prefix.
|
Package escaper provides an easy API to escapae and unescape a string using a custom escape prefix. |
|
gss
Package gss provides simple functions for serializing/deserializing objects into common formats.
|
Package gss provides simple functions for serializing/deserializing objects into common formats. |
|
gssjs
Package gssjs includes functions for the JavaScript build of GSS.
|
Package gssjs includes functions for the JavaScript build of GSS. |
|
inspector
Package inspector is a package of utility functions for inspecting field names and keys from objects.
|
Package inspector is a package of utility functions for inspecting field names and keys from objects. |
|
iterator
Package iterator provides an easy API to create an iterator to read objects from a file.
|
Package iterator provides an easy API to create an iterator to read objects from a file. |
|
json
Package json provides an API for JSON serialization that automatically infers types.
|
Package json provides an API for JSON serialization that automatically infers types. |
|
jsonl
Package jsonl provides a simple API for reading and writing to JSON Lines (aka jsonl).
|
Package jsonl provides a simple API for reading and writing to JSON Lines (aka jsonl). |
|
properties
Package properties includes functions for reading and writing from properties files.
|
Package properties includes functions for reading and writing from properties files. |
|
scanner
Package scanner provides an interface{} for scanning through a stream of bytes.
|
Package scanner provides an interface{} for scanning through a stream of bytes. |
|
serializer
Package serializer provides a middle layer between the gss package and the lower-level packages.
|
Package serializer provides a middle layer between the gss package and the lower-level packages. |
|
splitter
Package splitter provides a wrapper for creating a custom bufio.SplitFunc.
|
Package splitter provides a wrapper for creating a custom bufio.SplitFunc. |
|
sv
Package sv includes iterators for reading from separated-values sources and writing using separated-values formats, such as CSV and TSV.
|
Package sv includes iterators for reading from separated-values sources and writing using separated-values formats, such as CSV and TSV. |
|
tags
Package tags provides a simple API for reading and writing to lines of tags.
|
Package tags provides a simple API for reading and writing to lines of tags. |
|
toml
Package toml provides an API for TOML serialization.
|
Package toml provides an API for TOML serialization. |
|
yaml
Package yaml provides an API for YAML serialization that can automatically infers types.
|
Package yaml provides an API for YAML serialization that can automatically infers types. |
|
plugins
|
|
|
gss
command
gss.so creates a shared library of Go that can be called by C, C++, or Python - https://godoc.org/cmd/cgo - https://blog.golang.org/c-go-cgo
|
gss.so creates a shared library of Go that can be called by C, C++, or Python - https://godoc.org/cmd/cgo - https://blog.golang.org/c-go-cgo |