README
¶
Codecs
The go-ipld-prime/codec package is a grouping package.
The subpackages contains some codecs which reside in this repo.
The codecs included here are our "batteries included" codecs, but they are not otherwise special.
It is not necessary for a codec to be a subpackage here to be a valid codec to use with go-ipld-prime;
anything that implements the ipld.Encoder and ipld.Decoder interfaces is fine.
Terminology
We generally refer to "codecs" as having an "encode" function and "decode" function.
We consider "encoding" to be the process of going from {Data Model} to {serial data}, and "decoding" to be the process of going from {serial data} to {Data Model}.
Codec vs Multicodec
A "codec" is any function that goes from {Data Model} to {serial data}, or vice versa.
A "multicodec" is a function which does that and is also specifically recognized and described in the tables in https://github.com/multiformats/multicodec/ .
Multicodecs generally leave no further room for customization and configuration, because their entire behavior is supposed to be specified by a multicodec indicator code number.
Our codecs, in the child packages of this one, usually offer configuration options. They also usually offer exactly one function, which does not allow configuration, which is supplying a multicodec-compatible behavior. You'll see this marked in the docs on those functions.
Marshal vs Encode
It's common to see the terms "marshal" and "unmarshal" used in golang.
Those terms are usually describing when structured data is transformed into linearized, tokenized data (and then, perhaps, all the way to serially encoded data), or vice versa.
We would use the words the same way... except we don't end up using them, because that feature doesn't really come up in our codec layer.
In IPLD, we would describe mapping some typed data into Data Model as "marshalling".
(It's one step shy of tokenizing, but barely: Data Model does already have defined ordering for every element of data.)
And we do have systems that do this:
bindnode and our codegen systems both do this, implicitly, when they give you an ipld.Node of the representation of some data.
We just don't end up talking about it as "marshalling" because of how it's done implicitly by those systems. As a result, all of our features relating to codecs only end up speaking about "encoding" and "decoding".
Legacy code
There are some appearances of the words "marshal" and "unmarshal" in some of our subpackages here.
That verbiage is generally on the way out. For functions and structures with those names, you'll notice their docs marking them as deprecated.
Why have "batteries-included" codecs?
These codecs live in this repo because they're commonly used, highly supported, and general-purpose codecs that we recommend for widespread usage in new developments.
Also, it's just plain nice to have something in-repo for development purposes. It makes sure that if we try to make any API changes, we immediately see if they'd make codecs harder to implement. We also use the batteries-included codecs for debugging, for test fixtures, and for benchmarking.
Further yet, the batteries-included codecs let us offer getting-started APIs. For example, we offer some helper APIs which use codecs like e.g. JSON to give consumers of the libraries one-step helper methods that "do the right thing" with zero config... so long as they happen to use that codec. Even for consumers who don't use those codecs, such functions then serve as natural documentation and examples for what to do to put their codec of choice to work.
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Marshal ¶
Marshal is a deprecated function. Please consider switching to one of the Encode functions of one of the subpackages instead.
Marshal provides a very general node-to-tokens marshalling feature. It can handle either cbor or json by being combined with a refmt TokenSink.
It is valid for all the data model types except links, which are only supported if the nodes are typed and provide additional information to clarify how links should be encoded through their type info. (The dag-cbor and dag-json formats can be used if links are of CID implementation and need to be encoded in a schemafree way.)
func Unmarshal ¶
func Unmarshal(na ipld.NodeAssembler, tokSrc shared.TokenSource) error
Unmarshal is a deprecated function. Please consider switching to one of the Decode functions of one of the subpackages instead.
Unmarshal provides a very general tokens-to-node unmarshalling feature. It can handle either cbor or json by being combined with a refmt TokenSink.
The unmarshalled data is fed to the given NodeAssembler, which accumulates it; at the end, any error is returned from the Unmarshal method, and the user can pick up the finished Node from wherever their assembler has it. Typical usage might look like the following:
nb := basicnode.Prototype__Any{}.NewBuilder()
err := codec.Unmarshal(nb, json.Decoder(reader))
n := nb.Build()
It is valid for all the data model types except links, which are only supported if the nodes are typed and provide additional information to clarify how links should be decoded through their type info. (The dag-cbor and dag-json formats can be used if links are of CID implementation and need to be decoded in a schemafree way.)
Types ¶
type Decoder ¶ added in v0.6.0
Decoder is defined in the root ipld package; this alias is just for documentation and discoverability.
Most of the documentation for Encoder also applies wholesale to the Decoder interface.
type Encoder ¶ added in v0.6.0
Encoder is defined in the root ipld package; this alias is just for documentation and discoverability.
Encoder functions can be composed into an ipld.LinkSystem to provide a "one stop shop" API for handling content addressable storage. Encoder functions can also be used directly if you want to handle serial data streams.
Most codec packages will have a ReusableEncoder type (which contains any working memory needed by the encoder implementation, as well as any configuration options), and that type will have an Encode function matching this interface.
By convention, codec packages that have a multicodec contract will also have a package-scope exported function called Encode which also matches this interface, and is the equivalent of creating a zero-value ReusableEncoder (aka, default config) and using its Encode method. This package-scope function will typically also internally use a sync.Pool to keep some ReusableEncoder values on hand to avoid unnecesary allocations.
Note that a ReusableEncoder type that supports configuration options does not functionally expose those options when invoked by the multicodec system -- multicodec indicators do not provide room for extended configuration info. Codecs that expose configuration options are doing so for library users to enjoy; it does not mean those non-default configurations will necessarly be available in all scenarios that use codecs indirectly. There is also no standard interface for such configurations: by nature, if they exist at all, they vary per codec.
type ErrBudgetExhausted ¶ added in v0.6.0
type ErrBudgetExhausted struct{}
func (ErrBudgetExhausted) Error ¶ added in v0.6.0
func (e ErrBudgetExhausted) Error() string
type MapSortMode ¶ added in v0.11.0
type MapSortMode uint8
const ( MapSortMode_None MapSortMode = iota MapSortMode_Lexical MapSortMode_RFC7049 )
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
The dagcbor package provides a DAG-CBOR codec implementation.
|
The dagcbor package provides a DAG-CBOR codec implementation. |
|
Several groups of exported symbols are available at different levels of abstraction: - You might just want the multicodec registration! Then never deal with this package directly again.
|
Several groups of exported symbols are available at different levels of abstraction: - You might just want the multicodec registration! Then never deal with this package directly again. |
|
"jst" -- JSON Table -- is a format that's parsable as JSON, while sprucing up the display to humans using the non-significant whitespace cleverly.
|
"jst" -- JSON Table -- is a format that's parsable as JSON, while sprucing up the display to humans using the non-significant whitespace cleverly. |
|
demo
command
|
|
|
Package raw implements IPLD's raw codec, which simply writes and reads a Node which can be represented as bytes.
|
Package raw implements IPLD's raw codec, which simply writes and reads a Node which can be represented as bytes. |