Documentation
¶
Overview ¶
Package ulog reads and writes PX4 ULog streams.
Each ULog file defines the formats of its own data records. Reader resolves those definitions and subscriptions as it advances. The current record is replaced on the next successful read, while metadata, parameters, logs, and dropouts remain available through the reader's accessors. The reader accepts future file versions and ignores unknown message types, but currently rejects unknown incompatibility flags and logs marked as containing appended data.
FormatsFor, FormatFor, Register, and Decode adapt named Go structs to that dynamic model. Exported fields become lower_snake_case by default; a “ulog” struct tag can rename a field or exclude it with “-”. The adapter supports ULog primitive types, fixed-size arrays, and nested named structs. A string field requires its wire width in a tag such as ulog:"name,char[80]". Dynamic records expose character arrays as strings with trailing NUL padding removed. Scalar characters and uint8 arrays remain byte-valued.
Use Reader when records can be processed as a stream. Package dataset loads a complete file into typed columns, and package columnar converts those datasets to Apache Arrow or Parquet.
Index ¶
- func Decode[T any](record Record) (T, error)
- type DefaultParameter
- type DefaultParameterTypes
- type Dropout
- type Field
- type FieldValue
- type Format
- type Header
- type KeyValue
- type LogEntry
- type LogLevel
- type MultiInformationGroup
- type MultiInformationValue
- type RawStream
- type Reader
- func (r *Reader) DefaultParameters() []DefaultParameter
- func (r *Reader) Dropouts() []Dropout
- func (r *Reader) Err() error
- func (r *Reader) Header() Header
- func (r *Reader) Information() []KeyValue
- func (r *Reader) Logs() []LogEntry
- func (r *Reader) MultiInformation() []MultiInformationGroup
- func (r *Reader) Next() bool
- func (r *Reader) Parameters() []KeyValue
- func (r *Reader) Record() Record
- type Record
- type ScalarField
- type Stream
- type StreamOption
- type Type
- type Writer
- func (w *Writer) Close() error
- func (w *Writer) Define(format Format) error
- func (w *Writer) RegisterFormat(format Format, options ...StreamOption) (*RawStream, error)
- func (w *Writer) WriteDropout(duration time.Duration) error
- func (w *Writer) WriteInformation(name string, value any) error
- func (w *Writer) WriteLog(level LogLevel, timestamp uint64, message string) error
- func (w *Writer) WriteParameter(name string, value any) error
- type WriterOption
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Decode ¶
Decode maps the leading fields of record into T. It requires the flattened field names, order, and primitive wire types derived by FormatsFor, but does not require Record.Name to match T. Additional trailing record fields are ignored so that T can decode compatible schema extensions.
Example ¶
package main
import (
"fmt"
"os"
"github.com/sunfish-robotics/ulog"
)
func main() {
type vehicleAttitude struct {
Timestamp uint64
Q [4]float32
}
source, err := os.Open("flight.ulg")
if err != nil {
panic(err)
}
defer func() {
if err := source.Close(); err != nil {
panic(err)
}
}()
reader, err := ulog.NewReader(source)
if err != nil {
panic(err)
}
for reader.Next() {
if reader.Record().Name() != "vehicle_attitude" {
continue
}
attitude, err := ulog.Decode[vehicleAttitude](reader.Record())
if err != nil {
panic(err)
}
fmt.Printf("%d: %v\n", attitude.Timestamp, attitude.Q)
}
if err := reader.Err(); err != nil {
panic(err)
}
}
Output:
Types ¶
type DefaultParameter ¶
type DefaultParameter struct {
KeyValue
// Types contains one or more applicable default scopes.
Types DefaultParameterTypes
}
DefaultParameter is a parameter's default value for the scopes in DefaultParameter.Types. If a log has no default for a given parameter and scope, ULog defines its parameter value as the default.
type DefaultParameterTypes ¶
type DefaultParameterTypes uint8
DefaultParameterTypes identifies the independent configuration scopes to which a DefaultParameter applies. A value may apply to both scopes.
const ( // DefaultParameterSystemWide marks a system-wide default value. DefaultParameterSystemWide DefaultParameterTypes = 1 << 0 // DefaultParameterCurrentConfiguration marks a default for the current configuration. DefaultParameterCurrentConfiguration DefaultParameterTypes = 1 << 1 )
type Dropout ¶
type Dropout struct {
// Duration is the period for which logging messages were lost.
Duration time.Duration
}
Dropout describes a period during which logging messages were lost, often because the logging device could not keep up.
type Field ¶
type Field struct {
// Name is the case-sensitive field name used on the wire.
Name string
// Type is a ULog primitive type or the name of another [Format].
Type Type
// ArrayLength is the fixed element count, or zero for a scalar.
ArrayLength int
}
Field describes one member of a Format. A non-primitive Field.Type names another format. Field.ArrayLength is zero for a scalar.
type FieldValue ¶
type FieldValue struct {
// Name is the flattened field path.
Name string
// Type is the primitive wire type of Value.
Type Type
// ArrayLength is the fixed byte width of a character array, or zero for a
// scalar value.
ArrayLength int
// Value has the Go scalar type corresponding to Type. Character arrays are
// strings with trailing NUL padding removed.
Value any
}
FieldValue is one dynamically decoded value from a Record. FieldValue.Name is flattened: numeric arrays and nested formats use paths such as "q[0]" and "position.x". Character arrays remain one string-valued field.
type Format ¶
type Format struct {
// Name is the case-sensitive name used by subscriptions and nested fields.
Name string
// Fields contains at least one field in wire order.
Fields []Field
}
Format is the self-described wire schema for one kind of data record. Fields remain in wire order.
func FormatFor ¶
FormatFor derives the root ULog Format for T. Use FormatsFor when nested definitions must also be written.
func FormatsFor ¶
FormatsFor derives every ULog Format needed to encode T. T must be a named struct; nested definitions precede the root definition in the returned slice. See the package documentation for the field and struct-tag mapping.
func ParseFormat ¶
ParseFormat parses one "name:type field;" definition. It validates the grammar, names, duplicate fields, and primitive array sizes, but does not resolve nested Format names, detect cycles, or require the timestamp field needed by a subscription.
type Header ¶
type Header struct {
// Version is the file-format version declared by the log.
Version uint8
// Timestamp is when logging started, in microseconds.
Timestamp uint64
}
Header contains the ULog version and the logging start time from the fixed file header.
type KeyValue ¶
type KeyValue struct {
// Name is the case-sensitive information key or parameter name.
Name string
// Type is the primitive ULog type of Value.
Type Type
// ArrayLength is the fixed element count, or zero for a scalar.
ArrayLength int
// Value is a Go scalar, primitive slice, or string matching Type and ArrayLength.
Value any
}
KeyValue is a decoded information or parameter entry. KeyValue.Value has the Go scalar or slice type selected by KeyValue.Type; character arrays are strings. KeyValue.ArrayLength is zero for a scalar.
type LogEntry ¶
type LogEntry struct {
// Level is the message severity.
Level LogLevel
// Timestamp is the message timestamp in microseconds.
Timestamp uint64
// Message is the logged text.
Message string
// Tag is an application-defined source identifier when Tagged is true.
Tag uint16
// Tagged reports whether Tag was present on the wire.
Tagged bool
}
LogEntry is one printf-style ULog text message. LogEntry.Timestamp is in microseconds. LogEntry.Tag identifies an application-defined source only when LogEntry.Tagged is true.
type LogLevel ¶
type LogLevel uint8
LogLevel is the severity of a ULog text message.
const ( // LogLevelEmergency indicates that the system is unusable. LogLevelEmergency LogLevel = '0' // LogLevelAlert indicates that action must be taken immediately. LogLevelAlert LogLevel = '1' // LogLevelCritical indicates a critical condition. LogLevelCritical LogLevel = '2' // LogLevelError indicates an error condition. LogLevelError LogLevel = '3' // LogLevelWarning indicates a warning condition. LogLevelWarning LogLevel = '4' // LogLevelNotice indicates a normal but significant condition. LogLevelNotice LogLevel = '5' // LogLevelInfo indicates an informational message. LogLevelInfo LogLevel = '6' // LogLevelDebug indicates a debug message. LogLevelDebug LogLevel = '7' )
type MultiInformationGroup ¶ added in v0.2.0
type MultiInformationGroup struct {
// Name is the case-sensitive information key shared by Values.
Name string
// Values contains the independently typed entries in wire order.
Values []MultiInformationValue
}
MultiInformationGroup contains one ordered group of multi-information values with the same name. The first value starts the group; subsequent values were marked as continuations on the wire. Each value retains its own declared type and array length.
type MultiInformationValue ¶ added in v0.2.0
type MultiInformationValue struct {
KeyValue
// IsArray reports whether the wire declaration included an array length.
IsArray bool
}
MultiInformationValue is one independently typed value from a MultiInformationGroup. IsArray distinguishes scalar values from arrays, including PX4's empty char[0] separator values.
type RawStream ¶
type RawStream struct {
// contains filtered or unexported fields
}
RawStream writes payloads for one dynamic Format registered by Writer.RegisterFormat.
type Reader ¶
type Reader struct {
// contains filtered or unexported fields
}
Reader streams format-resolved Record values from ULog. As Reader.Next advances, it also collects information, multi-information groups, parameters, logs, and dropouts.
Example ¶
package main
import (
"fmt"
"os"
"github.com/sunfish-robotics/ulog"
)
func main() {
source, err := os.Open("flight.ulg")
if err != nil {
panic(err)
}
defer func() {
if err := source.Close(); err != nil {
panic(err)
}
}()
reader, err := ulog.NewReader(source)
if err != nil {
panic(err)
}
for reader.Next() {
record := reader.Record()
timestamp, err := record.Value("timestamp")
if err != nil {
panic(err)
}
fmt.Printf("%s[%d] timestamp=%v\n", record.Name(), record.MultiID(), timestamp)
}
if err := reader.Err(); err != nil {
panic(err)
}
}
Output:
func NewReader ¶
NewReader consumes the fixed file header from source and checks its magic bytes. It accepts later file-format versions for forward compatibility and does not close source.
func (*Reader) DefaultParameters ¶
func (r *Reader) DefaultParameters() []DefaultParameter
DefaultParameters returns independent copies of the parameter defaults encountered so far, in file order. Missing defaults are not synthesised.
func (*Reader) Dropouts ¶
Dropouts returns the periods of lost logging messages encountered so far, in file order.
func (*Reader) Err ¶
Err returns the first error encountered by Reader.Next, or nil after a clean end of stream.
func (*Reader) Header ¶
Header returns the version and logging start time consumed by NewReader. It is available before the first call to Reader.Next.
func (*Reader) Information ¶
Information returns independent copies of the typed metadata entries encountered so far, in file order.
func (*Reader) Logs ¶
Logs returns the tagged and untagged text messages encountered so far, in file order.
func (*Reader) MultiInformation ¶ added in v0.2.0
func (r *Reader) MultiInformation() []MultiInformationGroup
MultiInformation returns independent copies of the grouped multi-information values encountered so far, in the order each group started. Every value keeps the type and array length declared by its own wire message.
func (*Reader) Next ¶
Next consumes messages until it reaches the next data record. It returns false at end of stream or after the first error; call Reader.Err to distinguish the two. Definition, subscription, metadata, log, and dropout messages update the reader's state without producing a record. Unknown message types are skipped unless the log advertises an unsupported incompatibility feature.
func (*Reader) Parameters ¶
Parameters returns independent copies of the initial parameter values and later changes encountered so far, in file order.
type Record ¶
type Record struct {
// contains filtered or unexported fields
}
Record is one ULog data message paired with the subscription and Format needed to interpret it.
func (Record) Bytes ¶
Bytes returns an independent copy of the raw bytes described by Record.Format, excluding the subscription ID and enclosing message header. The payload may omit trailing top-level padding permitted by ULog.
func (Record) Fields ¶
func (r Record) Fields() []ScalarField
Fields returns every flattened, non-padding value in the selected format, in wire order. A compatible older record may omit trailing fields; Record.Value reports those fields as unavailable.
func (Record) Format ¶
Format returns an independent copy of the dynamic schema selected by the record's subscription.
func (Record) MessageID ¶
MessageID returns the runtime subscription identifier. It is only meaningful within this log.
func (Record) MultiID ¶
MultiID returns the instance identifier for the subscribed format. Zero is the first and default instance.
func (Record) Name ¶
Name returns the case-sensitive Format.Name selected by the record's subscription.
func (Record) Value ¶
Value decodes a field by its flattened path. It reports an error when the field is unknown, omitted as trailing data, or truncated.
func (Record) Values ¶
func (r Record) Values() ([]FieldValue, error)
Values decodes every available non-padding value in wire order. Missing trailing fields are omitted to support compatible schema extension.
type ScalarField ¶
type ScalarField struct {
// Name is the flattened field path.
Name string
// Type is the field's primitive wire type.
Type Type
// ArrayLength is the fixed byte width of a character array, or zero for a
// scalar value.
ArrayLength int
}
ScalarField describes one flattened, non-padding value in a Record.
type Stream ¶
type Stream[T any] struct { // contains filtered or unexported fields }
Stream writes values of T to one ULog subscription created by Register.
func Register ¶
func Register[T any](writer *Writer, options ...StreamOption) (*Stream[T], error)
Register derives the formats required by T using the same mapping as FormatsFor, defines them, then registers a typed Stream. T must contain an exported scalar field that maps to the case-sensitive ULog name "timestamp" and wire type uint64_t. Registration must finish before the data section starts. Values are encoded in field order without Go struct padding.
Example ¶
package main
import (
"bytes"
"fmt"
"github.com/sunfish-robotics/ulog"
)
func main() {
type sensorSample struct {
Timestamp uint64
Pressure float32
Valid bool
}
var destination bytes.Buffer
writer, err := ulog.NewWriter(&destination, ulog.WithStartTimestamp(1_000_000))
if err != nil {
panic(err)
}
stream, err := ulog.Register[sensorSample](writer)
if err != nil {
panic(err)
}
if err := stream.Write(sensorSample{Timestamp: 1_000_100, Pressure: 101.25, Valid: true}); err != nil {
panic(err)
}
if err := writer.Close(); err != nil {
panic(err)
}
reader, err := ulog.NewReader(bytes.NewReader(destination.Bytes()))
if err != nil {
panic(err)
}
var samples int
for reader.Next() {
samples++
}
if err := reader.Err(); err != nil {
panic(err)
}
fmt.Println(samples)
}
Output: 1
type StreamOption ¶
type StreamOption func(*streamConfig) error
StreamOption configures Register or Writer.RegisterFormat.
func WithFormatName ¶
func WithFormatName(name string) StreamOption
WithFormatName sets the root format name used by Register or Writer.RegisterFormat. For typed streams it replaces the name derived from the Go type; nested format names are unchanged.
func WithMultiID ¶
func WithMultiID(multiID uint8) StreamOption
WithMultiID identifies one of several streams using the same format. Zero is the first and default instance.
type Type ¶
type Type string
Type identifies a primitive ULog type or the name of another Format.
const ( // TypeInt8 identifies a signed 8-bit integer. TypeInt8 Type = "int8_t" // TypeUint8 identifies an unsigned 8-bit integer. TypeUint8 Type = "uint8_t" // TypeInt16 identifies a signed 16-bit integer. TypeInt16 Type = "int16_t" // TypeUint16 identifies an unsigned 16-bit integer. TypeUint16 Type = "uint16_t" // TypeInt32 identifies a signed 32-bit integer. TypeInt32 Type = "int32_t" // TypeUint32 identifies an unsigned 32-bit integer. TypeUint32 Type = "uint32_t" // TypeInt64 identifies a signed 64-bit integer. TypeInt64 Type = "int64_t" // TypeUint64 identifies an unsigned 64-bit integer. TypeUint64 Type = "uint64_t" // TypeFloat32 identifies a 32-bit IEEE-754 floating-point value. TypeFloat32 Type = "float" // TypeFloat64 identifies a 64-bit IEEE-754 floating-point value. TypeFloat64 Type = "double" // TypeBool identifies a one-byte Boolean value. TypeBool Type = "bool" // TypeChar identifies a one-byte character. TypeChar Type = "char" )
func (Type) IsPrimitive ¶
IsPrimitive reports whether t is one of ULog's built-in scalar types.
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
Writer serialises a ULog stream. All definitions, information, parameters, and stream registrations must be complete before the first data or log write. Writer methods and registered stream writes are safe for concurrent use; concurrent writes are serialised, but their relative order is unspecified.
func NewWriter ¶
func NewWriter(destination io.Writer, options ...WriterOption) (*Writer, error)
NewWriter immediately writes the ULog file header and required flag-bits message to destination. Neither Writer.Close nor a failed constructor closes destination.
func (*Writer) Close ¶
Close writes any subscriptions not yet emitted, then prevents further writes. It does not close the underlying destination and is safe to call more than once. Close returns the first destination write error, including on later calls.
func (*Writer) Define ¶
Define validates and writes a dynamic Format. Definitions must be added before the first data or log write. Repeating an identical definition is a no-op; redefining the same name differently returns an error.
func (*Writer) RegisterFormat ¶
func (w *Writer) RegisterFormat(format Format, options ...StreamOption) (*RawStream, error)
RegisterFormat validates and defines format, then registers a RawStream. The format must contain a scalar uint64_t timestamp field. Definitions referenced by nested fields must first be added with Writer.Define. Registration must finish before the data section starts.
func (*Writer) WriteDropout ¶
WriteDropout marks a period in which logging messages were lost. duration must be a non-negative whole number of milliseconds no greater than 65,535 ms. The first call starts the data section.
func (*Writer) WriteInformation ¶
WriteInformation writes one initial metadata entry before the data section. value may be a non-empty string or a scalar Go value supported by the ULog primitive mapping. Calls after the first data or log write return an error.
type WriterOption ¶
type WriterOption func(*writerConfig) error
WriterOption configures NewWriter.
func WithStartTimestamp ¶
func WithStartTimestamp(timestamp uint64) WriterOption
WithStartTimestamp records when logging started, in microseconds, in the file header. Without this option NewWriter writes zero.
Directories
¶
| Path | Synopsis |
|---|---|
|
pkg
|
|
|
columnar
Package columnar converts dataset.Dataset values to Apache Arrow and Parquet.
|
Package columnar converts dataset.Dataset values to Apache Arrow and Parquet. |
|
dataset
Package dataset loads ULog streams into eager, column-oriented views.
|
Package dataset loads ULog streams into eager, column-oriented views. |
|
wire
Package wire defines the byte-level types used by the PX4 ULog file format.
|
Package wire defines the byte-level types used by the PX4 ULog file format. |