swecommon

package
v1.0.0-beta.113 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 19, 2026 License: MIT Imports: 11 Imported by: 0

Documentation

Overview

Package swecommon provides schema-bound encoders and decoders for the OGC SWE Common data model, covering the JSON, text, and binary encodings the OGC Connected Systems API negotiates for observation results and command payloads.

Why this package exists

CS API gateways (semconnect and friends) need to advertise a per-datastream result schema and stream observation values against that schema in three media types:

  • application/swe+json
  • application/swe+csv
  • application/swe+binary

Without a schema-bound model, gateways fall back to projecting value-only views (time + result), tagging the response with a non-conformant subset header. This package lets gateways drop the subset header and claim the SWE Common conformance classes.

Model

A schema is a DataRecord of named [Field]s. Each field carries a typed DataComponent — Quantity, Count, Time, Boolean, Text, Category. The scalar types are sealed; adding a new kind requires extending this package (and is a compile-time error in every consumer until they handle it).

Values are carried as Values maps keyed by field name. The encoders are tolerant on input — a Quantity field accepts any Go numeric type, a Time accepts time.Time or RFC 3339 string. Decoders always populate the canonical Go type per the schema.

Encoding choices

  • JSON values: array of objects, one per row, field names as keys.
  • Text values: producer-configurable token + block separators (default CSV). Decimal separator must be ".". Optional header.
  • Binary values: per-record nil bitmap (ceil(nfields/8) bytes, high bit = field 0) + packed primitives. Big-endian by default. No record-count prefix; consumers read until EOF.

Round-trip stability

Every encoding has a corresponding decoder in the same file and every type/kind combination is exercised by the round-trip table-driven tests in *_test.go. Producers that depend on a shape that does not appear in the test table should add a row.

Phase 1 scope (this package as introduced by [ADR-050])

Implemented: scalar components + DataRecord + JSON/text/binary encoders + decoders + schema marshal/unmarshal + media-type constants. Deferred (tracked in https://github.com/C360Studio/semstreams/issues/167): DataArray (homogeneous element record), DataChoice (discriminated union), Vector (axis-frame-bound values), per-component constraints (allowedValues, ranges), multi-reason NilValues block, nested DataRecord values, SWE XML encoding (CS API does not require it).

Index

Constants

View Source
const (
	// MediaSWEJSON — schema-bound JSON encoding.
	MediaSWEJSON = "application/swe+json"

	// MediaSWECSV — schema-bound text encoding using CSV-shaped
	// separators (comma token, newline block).
	MediaSWECSV = "application/swe+csv"

	// MediaSWEBinary — schema-bound packed primitives binary
	// encoding.
	MediaSWEBinary = "application/swe+binary"
)

CS API media type identifiers for SWE Common encodings. These match the strings the OGC Connected Systems API uses in Accept / Content-Type negotiation. Importing consumers (e.g. CS API gateways) should reference these constants rather than redeclare the strings, so a future OGC errata that changes a suffix lands in exactly one place.

View Source
const SubsetObservationValues = "observation-values"

SubsetObservationValues is the legacy semconnect-side header value used to flag the value-only projection subset (no schema binding). Consumers that have migrated to this package's schema-bound encoders should drop the header entirely; the constant is provided so the migration code path can grep for one canonical token while it removes the workaround.

Variables

This section is empty.

Functions

func EncodeBinary

func EncodeBinary(w io.Writer, schema *DataRecord, rows []Values, enc BinaryEncoding) error

EncodeBinary writes the rows as SWE BinaryEncoding bytes per the schema and encoding configuration. Field order follows the schema's field declaration order. Each record is preceded by a nil bitmap (see BinaryEncoding for the bitmap layout).

No record-count prefix is emitted — consumers read until EOF. This matches the streaming-friendly shape CS API uses for chunked observation responses.

func EncodeJSON

func EncodeJSON(w io.Writer, schema *DataRecord, rows []Values) error

EncodeJSON writes the rows as a SWE Common JSON Encoding (22-022) document bound to the given schema. The output shape is a JSON array of objects, one per row, with field names matching the schema's field declarations.

Per-field value encoding:

  • Quantity → JSON number (float64)
  • Count → JSON number (int64)
  • Time → JSON string (RFC 3339)
  • Boolean → JSON boolean
  • Text → JSON string
  • Category → JSON string

nilValue rule: when a value is Go nil OR exactly equal to the schema's nilValue for the field, the output is JSON null. Decoders honor the same rule.

func EncodeText

func EncodeText(w io.Writer, schema *DataRecord, rows []Values, enc TextEncoding) error

EncodeText writes the rows as SWE TextEncoding bytes per the given encoding (separators + optional header row). Field order follows the schema's field declaration order.

Nil values are emitted as the schema's nilValue token when set, otherwise as the empty string. The empty string is a legal "absent value" stand-in for SWE text streams but operators are strongly encouraged to set a nilValue for fields where a real reading could be empty (Text, Category) so decoders can tell "absent" from "empty string reading."

func MarshalBinaryRows

func MarshalBinaryRows(schema *DataRecord, rows []Values, enc BinaryEncoding) ([]byte, error)

MarshalBinaryRows is the bytes-returning convenience wrapper around EncodeBinary.

func MarshalJSONRows

func MarshalJSONRows(schema *DataRecord, rows []Values) ([]byte, error)

MarshalJSONRows is the bytes-returning convenience wrapper around EncodeJSON. The result has no trailing newline.

func MarshalSchema

func MarshalSchema(r *DataRecord) ([]byte, error)

MarshalSchema serializes a DataRecord schema to OGC SWE Common JSON Encoding (22-022). Round-trips through UnmarshalSchema.

This is the metadata document — a datastream advertises its observation-result schema by serving this on a `/schema` endpoint (or embedding it in the datastream resource). The schema then drives the JSON / CSV / binary encoders for the values.

func MarshalTextRows

func MarshalTextRows(schema *DataRecord, rows []Values, enc TextEncoding) ([]byte, error)

MarshalTextRows is the bytes-returning convenience wrapper around EncodeText.

Types

type BinaryByteOrder

type BinaryByteOrder uint8

BinaryByteOrder is the wire byte order for the binary encoding. Default and recommendation: big-endian (network byte order). The producer and consumer must agree; the schema's BinaryEncoding metadata block carries the choice on the wire.

const (
	// BigEndian — default. Matches network byte order.
	BigEndian BinaryByteOrder = iota
	// LittleEndian — opt-in for producers that want to skip the
	// host-to-network byte swap on x86/ARM.
	LittleEndian
)

type BinaryEncoding

type BinaryEncoding struct {
	ByteOrder BinaryByteOrder
}

BinaryEncoding configures the SWE BinaryEncoding (SWE Common §7.4.4) options. The framework Phase 1 supports the "packed primitives" variant: each field is encoded back-to-back with no padding, types per the wire format below.

Per-field wire format:

  • Quantity → 8 bytes IEEE 754 float64
  • Count → 8 bytes int64
  • Time → uint32 length prefix + UTF-8 RFC 3339 bytes
  • Boolean → 1 byte (0 = false, 1 = true)
  • Text → uint32 length prefix + UTF-8 bytes
  • Category → uint32 length prefix + UTF-8 bytes

Nil values: a separate nil bitmap precedes each record. The bitmap is ceil(nfields/8) bytes, big-endian bit order (high bit = field 0). A 1 bit means the field is nil; the value bytes are skipped for nil fields.

func DefaultBinaryEncoding

func DefaultBinaryEncoding() BinaryEncoding

DefaultBinaryEncoding returns the network-byte-order packed primitives encoding.

type Boolean

type Boolean struct {
	CommonFields
}

Boolean is a true/false value (SWE Common §7.6.2). Round-trips through Go bool.

func (Boolean) Kind

func (Boolean) Kind() ComponentKind

Kind reports KindBoolean.

type Category

type Category struct {
	CommonFields

	// CodeSpace is the IRI of the code list this category draws
	// from. Empty = free-form token (legal but limits validation).
	CodeSpace string `json:"codeSpace,omitempty"`
}

Category is a controlled-vocabulary token (SWE Common §7.6.7). CodeSpace is the IRI of the code list; values are short tokens drawn from that list. Round-trips through Go string.

func (Category) Kind

func (Category) Kind() ComponentKind

Kind reports KindCategory.

type CommonFields

type CommonFields struct {
	LabelValue      string `json:"label,omitempty"`
	DefinitionValue string `json:"definition,omitempty"`

	// NilValue is the wire-side stand-in token a producer emits when
	// the field is absent. Decoders treat any value equal to this
	// token (after format-specific normalization) as Go `nil`.
	// Per SWE Common §7.5 NilValues — Phase 1 supports a single
	// stand-in; the multi-reason NilValues block is deferred.
	NilValue string `json:"nilValue,omitempty"`
}

CommonFields is the embedded base for every concrete component. It carries the label + definition + nilValue stand-in that every SWE component may optionally declare. JSON tag rules: omit when empty so a minimal Quantity{} round-trips to `{"type":"Quantity"}`.

func (CommonFields) Definition

func (c CommonFields) Definition() string

Definition implements DataComponent.

func (CommonFields) Label

func (c CommonFields) Label() string

Label implements DataComponent.

func (CommonFields) Nil

func (c CommonFields) Nil() string

Nil returns the optional wire-side nil-stand-in token.

type ComponentKind

type ComponentKind string

ComponentKind is the SWE Common data-component discriminator. Every concrete DataComponent reports one of these so encoders and decoders can dispatch without reflection. The string values match the `type` discriminator field in OGC SWE Common JSON Encoding (22-022), so they double as the wire-format type tag.

const (
	KindRecord   ComponentKind = "DataRecord"
	KindQuantity ComponentKind = "Quantity"
	KindCount    ComponentKind = "Count"
	KindTime     ComponentKind = "Time"
	KindBoolean  ComponentKind = "Boolean"
	KindText     ComponentKind = "Text"
	KindCategory ComponentKind = "Category"
)

SWE Common data-component kinds. KindRecord is the only composite kind covered in Phase 1; DataArray and DataChoice are deferred.

type Count

type Count struct {
	CommonFields
}

Count is a discrete integer count with no unit of measure (SWE Common §7.6.6). Round-trips through int64.

func (Count) Kind

func (Count) Kind() ComponentKind

Kind reports KindCount.

type DataComponent

type DataComponent interface {
	// Kind reports which SWE Common component this is.
	Kind() ComponentKind

	// Label is the human-readable name; empty when unset.
	Label() string

	// Definition is the IRI of the controlled concept that defines
	// this component's semantics (e.g. a QUDT QuantityKind or a
	// SWEET property concept). Empty when unset.
	Definition() string
	// contains filtered or unexported methods
}

DataComponent is the sealed supertype for SWE Common data components. Implementations live in this package only — adding a new kind requires updating the encoders, decoders, and the register tables in schema.go, which would not compile otherwise.

type DataRecord

type DataRecord struct {
	CommonFields

	// Fields are the record's typed members, in declaration order.
	// Order is wire-significant for CSV and binary encodings.
	Fields []Field
}

DataRecord is the schema for a heterogeneous record of named typed fields (SWE Common §7.6.8). It is the framework's observation-result and command-payload model.

func UnmarshalSchema

func UnmarshalSchema(data []byte) (*DataRecord, error)

UnmarshalSchema parses a SWE Common JSON Encoding schema document into a typed DataRecord. The result is structurally validated (non-empty record, unique field names).

func (*DataRecord) FieldByName

func (r *DataRecord) FieldByName(name string) (Field, bool)

FieldByName returns the field with the given name and reports whether it was found. O(n) — DataRecords are typically small (a handful of fields).

func (*DataRecord) Kind

func (*DataRecord) Kind() ComponentKind

Kind reports KindRecord.

func (*DataRecord) Validate

func (r *DataRecord) Validate() error

Validate enforces the structural invariants the encoders rely on: non-empty record, non-empty unique field names, non-nil scalar component per field. Phase 1 rejects nested DataRecord fields at schema-validate time so consumers do not learn about the unsupported shape per-row at first encode.

type Field

type Field struct {
	// Name is the field's identifier within its parent record. Must
	// be unique across the record's fields, non-empty, and (for
	// CSV/text encoding) free of the configured token separator.
	Name string

	// Component is the typed schema for this field's values.
	Component DataComponent
}

Field is a named member of a DataRecord. The name is the dictionary key in the record's value map (and the column header in CSV / position index in binary). Component carries the typed schema for the field's values.

type Quantity

type Quantity struct {
	CommonFields

	// UoM is the unit of measure. Carry a UCUM symbol (e.g. "Cel",
	// "m/s") via Code OR a QUDT/OGC unit IRI via Href. Both empty
	// = dimensionless quantity, which is legal but lossy through
	// the SWE+csv encoding.
	UoMCode string `json:"uomCode,omitempty"`
	UoMHref string `json:"uomHref,omitempty"`
}

Quantity is a numeric value with a unit of measure (SWE Common §7.6.4). Round-trips through float64; producers needing higher precision (int64-fidelity counts, decimal-fixed) should declare a Count or carry the value as Text instead.

func (Quantity) Kind

func (Quantity) Kind() ComponentKind

Kind reports KindQuantity.

type Text

type Text struct {
	CommonFields
}

Text is a free-text string (SWE Common §7.6.3). Round-trips through Go string; binary encoding length-prefixes the UTF-8 bytes with a uint32.

func (Text) Kind

func (Text) Kind() ComponentKind

Kind reports KindText.

type TextEncoding

type TextEncoding struct {
	// TokenSeparator separates values within a record. Default ",".
	TokenSeparator string

	// BlockSeparator separates records. Default "\n".
	BlockSeparator string

	// DecimalSeparator is the decimal point. Must be ".".
	DecimalSeparator string

	// CollapseWhiteSpace controls whether the decoder trims
	// leading/trailing whitespace from each token. Defaults true.
	CollapseWhiteSpace bool

	// EmitHeader controls whether the encoder emits the field-name
	// header row before the first record. CS API SWE text streams
	// typically omit it (the schema is advertised separately); the
	// framework default is false to match.
	EmitHeader bool
}

TextEncoding configures the SWE TextEncoding (SWE Common §7.4.3) separators. CS API requires the producer/consumer to agree on these; the framework defaults match the CSV media type and are safe round-trip targets.

Token separator splits values within one record (column delimiter). Block separator splits records (row delimiter). Decimal separator must be "." to round-trip JSON-compatible floats; encoders surface a configuration error otherwise.

func DefaultTextEncoding

func DefaultTextEncoding() TextEncoding

DefaultTextEncoding is the CSV-shaped default used when no per-datastream encoding is configured.

type Time

type Time struct {
	CommonFields

	// UoMHref carries the temporal reference IRI (e.g.
	// "http://www.opengis.net/def/uom/ISO-8601/0/Gregorian"). The
	// SWE JSON encoding emits this as uom.href on the wire.
	UoMHref string `json:"uomHref,omitempty"`

	// ReferenceFrame is the optional temporal frame IRI for
	// non-UTC time values. Empty = UTC.
	ReferenceFrame string `json:"referenceFrame,omitempty"`
}

Time is a temporal instant (SWE Common §7.6.5). Round-trips through ISO 8601 / RFC 3339 strings — Phase 1 does not encode epoch-seconds doubles. Time-zone-frame and reference-frame are carried as IRIs; missing frame is treated as UTC by convention.

func (Time) Kind

func (Time) Kind() ComponentKind

Kind reports KindTime.

type Values

type Values map[string]any

Values is one record's worth of typed field values, keyed by field name. Missing keys are treated as nil (encoders emit the schema's nilValue or JSON null). Per-field Go types:

  • Quantity → float64
  • Count → int64
  • Time → time.Time (or RFC 3339 string — encoders convert)
  • Boolean → bool
  • Text → string
  • Category → string

Encoders are tolerant on input: a Quantity field accepts any numeric type (int, int32, int64, float32, float64) and rounds through float64; a Time field accepts either time.Time or an RFC 3339 string. Decoders always populate the canonical type.

func DecodeBinary

func DecodeBinary(r io.Reader, schema *DataRecord, enc BinaryEncoding) ([]Values, error)

DecodeBinary reads SWE BinaryEncoding bytes into typed rows against the schema. EOF after a complete record terminates the stream cleanly; EOF mid-record surfaces as io.ErrUnexpectedEOF.

func DecodeJSON

func DecodeJSON(r io.Reader, schema *DataRecord) ([]Values, error)

DecodeJSON reads a SWE Common JSON Encoding document into typed row values bound to the schema. Unknown fields in the JSON input are ignored — operators evolving the schema can add fields without breaking old payloads. Missing fields decode to absent keys (NOT to nil values, so callers can distinguish "the producer omitted this" from "the producer asserted null").

func DecodeText

func DecodeText(r io.Reader, schema *DataRecord, enc TextEncoding) ([]Values, error)

DecodeText parses SWE TextEncoding bytes back into typed rows against the schema. Field count per row must match the schema's field count exactly; mismatched rows surface as errors.

Header handling: when enc.EmitHeader is true, the first non-empty block is consumed as the header and validated against the schema's field names. When false, no header is expected.

func UnmarshalBinaryRows

func UnmarshalBinaryRows(data []byte, schema *DataRecord, enc BinaryEncoding) ([]Values, error)

UnmarshalBinaryRows is the bytes-taking convenience wrapper around DecodeBinary.

func UnmarshalJSONRows

func UnmarshalJSONRows(data []byte, schema *DataRecord) ([]Values, error)

UnmarshalJSONRows is the bytes-taking convenience wrapper around DecodeJSON.

func UnmarshalTextRows

func UnmarshalTextRows(data []byte, schema *DataRecord, enc TextEncoding) ([]Values, error)

UnmarshalTextRows is the bytes-taking convenience wrapper around DecodeText.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL