Documentation
¶
Overview ¶
Package swecommon provides schema-bound encoders and decoders for the OGC SWE Common data model, covering the JSON, text, and binary encodings the OGC Connected Systems API negotiates for observation results and command payloads.
Why this package exists ¶
CS API gateways (semconnect and friends) need to advertise a per-datastream result schema and stream observation values against that schema in three media types:
- application/swe+json
- application/swe+csv
- application/swe+binary
Without a schema-bound model, gateways fall back to projecting value-only views (time + result), tagging the response with a non-conformant subset header. This package lets gateways drop the subset header and claim the SWE Common conformance classes.
Model ¶
A schema is a DataRecord of named [Field]s. Each field carries a typed DataComponent — Quantity, Count, Time, Boolean, Text, Category. The scalar types are sealed; adding a new kind requires extending this package (and is a compile-time error in every consumer until they handle it).
Values are carried as Values maps keyed by field name. The encoders are tolerant on input — a Quantity field accepts any Go numeric type, a Time accepts time.Time or RFC 3339 string. Decoders always populate the canonical Go type per the schema.
Encoding choices ¶
- JSON values: array of objects, one per row, field names as keys.
- Text values: producer-configurable token + block separators (default CSV). Decimal separator must be ".". Optional header.
- Binary values: per-record nil bitmap (ceil(nfields/8) bytes, high bit = field 0) + packed primitives. Big-endian by default. No record-count prefix; consumers read until EOF.
Round-trip stability ¶
Every encoding has a corresponding decoder in the same file and every type/kind combination is exercised by the round-trip table-driven tests in *_test.go. Producers that depend on a shape that does not appear in the test table should add a row.
Phase 1 scope (this package as introduced by [ADR-050]) ¶
Implemented: scalar components + DataRecord + JSON/text/binary encoders + decoders + schema marshal/unmarshal + media-type constants. Deferred (tracked in https://github.com/C360Studio/semstreams/issues/167): DataArray (homogeneous element record), DataChoice (discriminated union), Vector (axis-frame-bound values), per-component constraints (allowedValues, ranges), multi-reason NilValues block, nested DataRecord values, SWE XML encoding (CS API does not require it).
Index ¶
- Constants
- func EncodeBinary(w io.Writer, schema *DataRecord, rows []Values, enc BinaryEncoding) error
- func EncodeJSON(w io.Writer, schema *DataRecord, rows []Values) error
- func EncodeText(w io.Writer, schema *DataRecord, rows []Values, enc TextEncoding) error
- func MarshalBinaryRows(schema *DataRecord, rows []Values, enc BinaryEncoding) ([]byte, error)
- func MarshalJSONRows(schema *DataRecord, rows []Values) ([]byte, error)
- func MarshalSchema(r *DataRecord) ([]byte, error)
- func MarshalTextRows(schema *DataRecord, rows []Values, enc TextEncoding) ([]byte, error)
- type BinaryByteOrder
- type BinaryEncoding
- type Boolean
- type Category
- type CommonFields
- type ComponentKind
- type Count
- type DataComponent
- type DataRecord
- type Field
- type Quantity
- type Text
- type TextEncoding
- type Time
- type Values
- func DecodeBinary(r io.Reader, schema *DataRecord, enc BinaryEncoding) ([]Values, error)
- func DecodeJSON(r io.Reader, schema *DataRecord) ([]Values, error)
- func DecodeText(r io.Reader, schema *DataRecord, enc TextEncoding) ([]Values, error)
- func UnmarshalBinaryRows(data []byte, schema *DataRecord, enc BinaryEncoding) ([]Values, error)
- func UnmarshalJSONRows(data []byte, schema *DataRecord) ([]Values, error)
- func UnmarshalTextRows(data []byte, schema *DataRecord, enc TextEncoding) ([]Values, error)
Constants ¶
const ( // MediaSWEJSON — schema-bound JSON encoding. MediaSWEJSON = "application/swe+json" // MediaSWECSV — schema-bound text encoding using CSV-shaped // separators (comma token, newline block). MediaSWECSV = "application/swe+csv" // MediaSWEBinary — schema-bound packed primitives binary // encoding. MediaSWEBinary = "application/swe+binary" )
CS API media type identifiers for SWE Common encodings. These match the strings the OGC Connected Systems API uses in Accept / Content-Type negotiation. Importing consumers (e.g. CS API gateways) should reference these constants rather than redeclare the strings, so a future OGC errata that changes a suffix lands in exactly one place.
const SubsetObservationValues = "observation-values"
SubsetObservationValues is the legacy semconnect-side header value used to flag the value-only projection subset (no schema binding). Consumers that have migrated to this package's schema-bound encoders should drop the header entirely; the constant is provided so the migration code path can grep for one canonical token while it removes the workaround.
Variables ¶
This section is empty.
Functions ¶
func EncodeBinary ¶
func EncodeBinary(w io.Writer, schema *DataRecord, rows []Values, enc BinaryEncoding) error
EncodeBinary writes the rows as SWE BinaryEncoding bytes per the schema and encoding configuration. Field order follows the schema's field declaration order. Each record is preceded by a nil bitmap (see BinaryEncoding for the bitmap layout).
No record-count prefix is emitted — consumers read until EOF. This matches the streaming-friendly shape CS API uses for chunked observation responses.
func EncodeJSON ¶
func EncodeJSON(w io.Writer, schema *DataRecord, rows []Values) error
EncodeJSON writes the rows as a SWE Common JSON Encoding (22-022) document bound to the given schema. The output shape is a JSON array of objects, one per row, with field names matching the schema's field declarations.
Per-field value encoding:
- Quantity → JSON number (float64)
- Count → JSON number (int64)
- Time → JSON string (RFC 3339)
- Boolean → JSON boolean
- Text → JSON string
- Category → JSON string
nilValue rule: when a value is Go nil OR exactly equal to the schema's nilValue for the field, the output is JSON null. Decoders honor the same rule.
func EncodeText ¶
func EncodeText(w io.Writer, schema *DataRecord, rows []Values, enc TextEncoding) error
EncodeText writes the rows as SWE TextEncoding bytes per the given encoding (separators + optional header row). Field order follows the schema's field declaration order.
Nil values are emitted as the schema's nilValue token when set, otherwise as the empty string. The empty string is a legal "absent value" stand-in for SWE text streams but operators are strongly encouraged to set a nilValue for fields where a real reading could be empty (Text, Category) so decoders can tell "absent" from "empty string reading."
func MarshalBinaryRows ¶
func MarshalBinaryRows(schema *DataRecord, rows []Values, enc BinaryEncoding) ([]byte, error)
MarshalBinaryRows is the bytes-returning convenience wrapper around EncodeBinary.
func MarshalJSONRows ¶
func MarshalJSONRows(schema *DataRecord, rows []Values) ([]byte, error)
MarshalJSONRows is the bytes-returning convenience wrapper around EncodeJSON. The result has no trailing newline.
func MarshalSchema ¶
func MarshalSchema(r *DataRecord) ([]byte, error)
MarshalSchema serializes a DataRecord schema to OGC SWE Common JSON Encoding (22-022). Round-trips through UnmarshalSchema.
This is the metadata document — a datastream advertises its observation-result schema by serving this on a `/schema` endpoint (or embedding it in the datastream resource). The schema then drives the JSON / CSV / binary encoders for the values.
func MarshalTextRows ¶
func MarshalTextRows(schema *DataRecord, rows []Values, enc TextEncoding) ([]byte, error)
MarshalTextRows is the bytes-returning convenience wrapper around EncodeText.
Types ¶
type BinaryByteOrder ¶
type BinaryByteOrder uint8
BinaryByteOrder is the wire byte order for the binary encoding. Default and recommendation: big-endian (network byte order). The producer and consumer must agree; the schema's BinaryEncoding metadata block carries the choice on the wire.
const ( // BigEndian — default. Matches network byte order. BigEndian BinaryByteOrder = iota // LittleEndian — opt-in for producers that want to skip the // host-to-network byte swap on x86/ARM. LittleEndian )
type BinaryEncoding ¶
type BinaryEncoding struct {
ByteOrder BinaryByteOrder
}
BinaryEncoding configures the SWE BinaryEncoding (SWE Common §7.4.4) options. The framework Phase 1 supports the "packed primitives" variant: each field is encoded back-to-back with no padding, types per the wire format below.
Per-field wire format:
- Quantity → 8 bytes IEEE 754 float64
- Count → 8 bytes int64
- Time → uint32 length prefix + UTF-8 RFC 3339 bytes
- Boolean → 1 byte (0 = false, 1 = true)
- Text → uint32 length prefix + UTF-8 bytes
- Category → uint32 length prefix + UTF-8 bytes
Nil values: a separate nil bitmap precedes each record. The bitmap is ceil(nfields/8) bytes, big-endian bit order (high bit = field 0). A 1 bit means the field is nil; the value bytes are skipped for nil fields.
func DefaultBinaryEncoding ¶
func DefaultBinaryEncoding() BinaryEncoding
DefaultBinaryEncoding returns the network-byte-order packed primitives encoding.
type Boolean ¶
type Boolean struct {
CommonFields
}
Boolean is a true/false value (SWE Common §7.6.2). Round-trips through Go bool.
type Category ¶
type Category struct {
CommonFields
// CodeSpace is the IRI of the code list this category draws
// from. Empty = free-form token (legal but limits validation).
CodeSpace string `json:"codeSpace,omitempty"`
}
Category is a controlled-vocabulary token (SWE Common §7.6.7). CodeSpace is the IRI of the code list; values are short tokens drawn from that list. Round-trips through Go string.
type CommonFields ¶
type CommonFields struct {
LabelValue string `json:"label,omitempty"`
DefinitionValue string `json:"definition,omitempty"`
// NilValue is the wire-side stand-in token a producer emits when
// the field is absent. Decoders treat any value equal to this
// token (after format-specific normalization) as Go `nil`.
// Per SWE Common §7.5 NilValues — Phase 1 supports a single
// stand-in; the multi-reason NilValues block is deferred.
NilValue string `json:"nilValue,omitempty"`
}
CommonFields is the embedded base for every concrete component. It carries the label + definition + nilValue stand-in that every SWE component may optionally declare. JSON tag rules: omit when empty so a minimal Quantity{} round-trips to `{"type":"Quantity"}`.
func (CommonFields) Definition ¶
func (c CommonFields) Definition() string
Definition implements DataComponent.
func (CommonFields) Nil ¶
func (c CommonFields) Nil() string
Nil returns the optional wire-side nil-stand-in token.
type ComponentKind ¶
type ComponentKind string
ComponentKind is the SWE Common data-component discriminator. Every concrete DataComponent reports one of these so encoders and decoders can dispatch without reflection. The string values match the `type` discriminator field in OGC SWE Common JSON Encoding (22-022), so they double as the wire-format type tag.
const ( KindRecord ComponentKind = "DataRecord" KindQuantity ComponentKind = "Quantity" KindCount ComponentKind = "Count" KindTime ComponentKind = "Time" KindBoolean ComponentKind = "Boolean" KindText ComponentKind = "Text" KindCategory ComponentKind = "Category" )
SWE Common data-component kinds. KindRecord is the only composite kind covered in Phase 1; DataArray and DataChoice are deferred.
type Count ¶
type Count struct {
CommonFields
}
Count is a discrete integer count with no unit of measure (SWE Common §7.6.6). Round-trips through int64.
type DataComponent ¶
type DataComponent interface {
// Kind reports which SWE Common component this is.
Kind() ComponentKind
// Label is the human-readable name; empty when unset.
Label() string
// Definition is the IRI of the controlled concept that defines
// this component's semantics (e.g. a QUDT QuantityKind or a
// SWEET property concept). Empty when unset.
Definition() string
// contains filtered or unexported methods
}
DataComponent is the sealed supertype for SWE Common data components. Implementations live in this package only — adding a new kind requires updating the encoders, decoders, and the register tables in schema.go, which would not compile otherwise.
type DataRecord ¶
type DataRecord struct {
CommonFields
// Fields are the record's typed members, in declaration order.
// Order is wire-significant for CSV and binary encodings.
Fields []Field
}
DataRecord is the schema for a heterogeneous record of named typed fields (SWE Common §7.6.8). It is the framework's observation-result and command-payload model.
func UnmarshalSchema ¶
func UnmarshalSchema(data []byte) (*DataRecord, error)
UnmarshalSchema parses a SWE Common JSON Encoding schema document into a typed DataRecord. The result is structurally validated (non-empty record, unique field names).
func (*DataRecord) FieldByName ¶
func (r *DataRecord) FieldByName(name string) (Field, bool)
FieldByName returns the field with the given name and reports whether it was found. O(n) — DataRecords are typically small (a handful of fields).
func (*DataRecord) Validate ¶
func (r *DataRecord) Validate() error
Validate enforces the structural invariants the encoders rely on: non-empty record, non-empty unique field names, non-nil scalar component per field. Phase 1 rejects nested DataRecord fields at schema-validate time so consumers do not learn about the unsupported shape per-row at first encode.
type Field ¶
type Field struct {
// Name is the field's identifier within its parent record. Must
// be unique across the record's fields, non-empty, and (for
// CSV/text encoding) free of the configured token separator.
Name string
// Component is the typed schema for this field's values.
Component DataComponent
}
Field is a named member of a DataRecord. The name is the dictionary key in the record's value map (and the column header in CSV / position index in binary). Component carries the typed schema for the field's values.
type Quantity ¶
type Quantity struct {
CommonFields
// UoM is the unit of measure. Carry a UCUM symbol (e.g. "Cel",
// "m/s") via Code OR a QUDT/OGC unit IRI via Href. Both empty
// = dimensionless quantity, which is legal but lossy through
// the SWE+csv encoding.
UoMCode string `json:"uomCode,omitempty"`
UoMHref string `json:"uomHref,omitempty"`
}
Quantity is a numeric value with a unit of measure (SWE Common §7.6.4). Round-trips through float64; producers needing higher precision (int64-fidelity counts, decimal-fixed) should declare a Count or carry the value as Text instead.
type Text ¶
type Text struct {
CommonFields
}
Text is a free-text string (SWE Common §7.6.3). Round-trips through Go string; binary encoding length-prefixes the UTF-8 bytes with a uint32.
type TextEncoding ¶
type TextEncoding struct {
// TokenSeparator separates values within a record. Default ",".
TokenSeparator string
// BlockSeparator separates records. Default "\n".
BlockSeparator string
// DecimalSeparator is the decimal point. Must be ".".
DecimalSeparator string
// CollapseWhiteSpace controls whether the decoder trims
// leading/trailing whitespace from each token. Defaults true.
CollapseWhiteSpace bool
// EmitHeader controls whether the encoder emits the field-name
// header row before the first record. CS API SWE text streams
// typically omit it (the schema is advertised separately); the
// framework default is false to match.
EmitHeader bool
}
TextEncoding configures the SWE TextEncoding (SWE Common §7.4.3) separators. CS API requires the producer/consumer to agree on these; the framework defaults match the CSV media type and are safe round-trip targets.
Token separator splits values within one record (column delimiter). Block separator splits records (row delimiter). Decimal separator must be "." to round-trip JSON-compatible floats; encoders surface a configuration error otherwise.
func DefaultTextEncoding ¶
func DefaultTextEncoding() TextEncoding
DefaultTextEncoding is the CSV-shaped default used when no per-datastream encoding is configured.
type Time ¶
type Time struct {
CommonFields
// UoMHref carries the temporal reference IRI (e.g.
// "http://www.opengis.net/def/uom/ISO-8601/0/Gregorian"). The
// SWE JSON encoding emits this as uom.href on the wire.
UoMHref string `json:"uomHref,omitempty"`
// ReferenceFrame is the optional temporal frame IRI for
// non-UTC time values. Empty = UTC.
ReferenceFrame string `json:"referenceFrame,omitempty"`
}
Time is a temporal instant (SWE Common §7.6.5). Round-trips through ISO 8601 / RFC 3339 strings — Phase 1 does not encode epoch-seconds doubles. Time-zone-frame and reference-frame are carried as IRIs; missing frame is treated as UTC by convention.
type Values ¶
Values is one record's worth of typed field values, keyed by field name. Missing keys are treated as nil (encoders emit the schema's nilValue or JSON null). Per-field Go types:
- Quantity → float64
- Count → int64
- Time → time.Time (or RFC 3339 string — encoders convert)
- Boolean → bool
- Text → string
- Category → string
Encoders are tolerant on input: a Quantity field accepts any numeric type (int, int32, int64, float32, float64) and rounds through float64; a Time field accepts either time.Time or an RFC 3339 string. Decoders always populate the canonical type.
func DecodeBinary ¶
func DecodeBinary(r io.Reader, schema *DataRecord, enc BinaryEncoding) ([]Values, error)
DecodeBinary reads SWE BinaryEncoding bytes into typed rows against the schema. EOF after a complete record terminates the stream cleanly; EOF mid-record surfaces as io.ErrUnexpectedEOF.
func DecodeJSON ¶
func DecodeJSON(r io.Reader, schema *DataRecord) ([]Values, error)
DecodeJSON reads a SWE Common JSON Encoding document into typed row values bound to the schema. Unknown fields in the JSON input are ignored — operators evolving the schema can add fields without breaking old payloads. Missing fields decode to absent keys (NOT to nil values, so callers can distinguish "the producer omitted this" from "the producer asserted null").
func DecodeText ¶
func DecodeText(r io.Reader, schema *DataRecord, enc TextEncoding) ([]Values, error)
DecodeText parses SWE TextEncoding bytes back into typed rows against the schema. Field count per row must match the schema's field count exactly; mismatched rows surface as errors.
Header handling: when enc.EmitHeader is true, the first non-empty block is consumed as the header and validated against the schema's field names. When false, no header is expected.
func UnmarshalBinaryRows ¶
func UnmarshalBinaryRows(data []byte, schema *DataRecord, enc BinaryEncoding) ([]Values, error)
UnmarshalBinaryRows is the bytes-taking convenience wrapper around DecodeBinary.
func UnmarshalJSONRows ¶
func UnmarshalJSONRows(data []byte, schema *DataRecord) ([]Values, error)
UnmarshalJSONRows is the bytes-taking convenience wrapper around DecodeJSON.
func UnmarshalTextRows ¶
func UnmarshalTextRows(data []byte, schema *DataRecord, enc TextEncoding) ([]Values, error)
UnmarshalTextRows is the bytes-taking convenience wrapper around DecodeText.