lazyproto

package

v0.35.0 Latest Latest Go to latest Published: Jan 21, 2025 License: MIT Imports: 11 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/CrowdStrike/csproto

Links

Open Source Insights

Documentation ¶

Index ¶

Variables
type DecodeResult
- func Decode(data []byte, def Def) (DecodeResult, error)deprecated
type Decoder
- func NewDecoder(def Def, opts ...Option) (*Decoder, error)
- func (dec *Decoder) Decode(data []byte) (*DecodeResult, error)
type Def
- func NewDef(tags ...int) Def
type FieldData
type Option
type RawValueConversionErrordeprecated
- func (e *RawValueConversionError) Error() string
type WireTypeMismatchError
- func (e *WireTypeMismatchError) Error() string

Examples ¶

DecodeResult.FieldData

Constants ¶

This section is empty.

Variables ¶

View Source

var (
	// ErrNestingNotDefined is returned by [PartialDecodeResult.FieldData] when the specified tag
	// was not supplied with a nested definitions. This will also return true for errors.Is(err, ErrTagNotFound) but is
	// more specific because this means the tag nesting was not in the original decoder definition
	ErrNestingNotDefined = fmt.Errorf("%w: the requested tag was not defined as a nested tag in the decoder", ErrTagNotFound)
	// ErrTagNotDefined is returned by [PartialDecodeResult.FieldData] when the specified tag
	// was not defined in the decoder. This will also return true for errors.Is(err, ErrTagNotFound) but is
	// more specific because this means the tag was not in the original decoder definition
	ErrTagNotDefined = fmt.Errorf("%w: the requested tag was not defined in the decoder", ErrTagNotFound)
	// ErrTagNotFound is returned by [PartialDecodeResult.FieldData] when the specified tag(s) do not
	// exist in the result.
	ErrTagNotFound = fmt.Errorf("the requested tag does not exist in the partial decode result")
)

Functions ¶

This section is empty.

Types ¶

type DecodeResult ¶

type DecodeResult struct {
	// contains filtered or unexported fields
}

DecodeResult holds a (possibly nested) mapping of integer field tags to FieldData instances which can be used to retrieve typed values for specific Protobuf message fields.

func Decode deprecated

func Decode(data []byte, def Def) (DecodeResult, error)

Decode extracts the specified field tags from data without unmarshaling the entire message. The methods on the returned PartialDecodeResult can be used to retrieve the decoded values.

The def param is an optionally nested mapping of protobuf field tags declaring which values should be decoded from the message. If the value for a given tag is a nested mapping and the wire type in the encoded data is WireTypeLengthDelimited , the contents are treated as a nested message and is decoded recursively.

The purpose of this API is to avoid fully unmarshalling nested message data when only a small subset of field values are needed, so [PartialDecodeResult] and FieldData only support extracting scalar values or slices of scalar values. Consumers that need to decode entire messages will need to use [Unmarshal] instead.

Deprecated: use NewDecoder(def) and (*Decoder).Decode(data) For best performance, decoders should be initialized once per definition and reused when decoding data

func (*DecodeResult) BoolValue ¶ added in v0.34.0

func (r *DecodeResult) BoolValue(tag int) (bool, error)

BoolValue is a helper method to get the field data at the given tag and return it as a boolean. It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.BoolValue()

func (*DecodeResult) BoolValues ¶ added in v0.34.0

func (r *DecodeResult) BoolValues(tag int) ([]bool, error)

BoolValues is a helper method to get the field data at the given tag and return it as a boolean slice. It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.BoolValues()

func (*DecodeResult) BytesValue ¶ added in v0.34.0

func (r *DecodeResult) BytesValue(tag int) ([]byte, error)

BytesValue is a helper method to get the field data at the given tag and return it as a byte slice. It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.BytesValue()

func (*DecodeResult) BytesValues ¶ added in v0.34.0

func (r *DecodeResult) BytesValues(tag int) ([][]byte, error)

BytesValues is a helper method to get the field data at the given tag and return it as a []byte slice. It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.BytesValues()

func (*DecodeResult) Close ¶

func (r *DecodeResult) Close() error

Close releases all internal resources held by r.

Consumers should always call Close() on instances returned by Decode to ensure that internal resources are cleaned up.

When using with csproto.DecoderModeFast it is important that any strings, bytes, etc that were generated using any of the DecodeResult/FieldData methods have moved out of scope before closing the DecodeResult.

func (*DecodeResult) FieldData ¶

func (r *DecodeResult) FieldData(tags ...int) (*FieldData, error)

FieldData returns a FieldData instance for the specified tag "path", if it exists.

The tags parameter is a list of one or more integer field tags that act as a "path" to a particular field to support retreiving fields from nested messages. Each value is used to retreieve the field data at the corresponding level of nesting, i.e. a value of [1, 2] would return the field data for tag 2 within the nested data for tag 1 at the root.

Example ¶

// data contains the serialized bytes of an Example Protobuf message that is
// defined as:
//	message Example {
//		int64  id     = 1;
//		Nested nested = 2;
//	}
//	message Nested {
//		string name        = 1;
//		string description = 2;
//	}
data := []byte{
	// id: 1
	(1 << 3), 0x1,
	// nested: 10 bytes
	(2 << 3) | 2, 0x0A,
	// nested.name: foo
	(1 << 3) | 2, 0x03, 0x66, 0x6f, 0x6f,
	// nested.description: bar
	(2 << 3) | 2, 0x03, 0x62, 0x61, 0x72,
}
def := NewDef()
// extract tags 1 and 2 from the nested message at tag 2 in the outer message
_ = def.NestedTag(2, 1, 2)

// create a new decoder for this definition
dec, err := NewDecoder(def, WithMode(csproto.DecoderModeFast), WithMaxBufferSize(1024))
if err != nil {
	fmt.Println("unable to create new decoder:", err)
	return
}

// decode arbitrary data
res, err := dec.Decode(data)
if err != nil {
	fmt.Println("error from decode:", err)
	return
}
defer func() {
	// Only close the result after we are completely done using it and any values we retrieved from it
	if err := res.Close(); err != nil {
		fmt.Println("error from DecodeResult.Close():", err)
	}
}()
// grab the field data
nameData, err := res.FieldData(2, 1)
if err != nil {
	fmt.Println("error accessing field data for 'name':", err)
	return
}
descriptionData, err := res.FieldData(2, 2)
if err != nil {
	fmt.Println("error accessing field data for 'description':", err)
	return
}
// extract the values
name, err := nameData.StringValue()
if err != nil {
	fmt.Println("error retrieving string value for 'name':", err)
	return
}
description, err := descriptionData.StringValue()
if err != nil {
	fmt.Println("error retrieving string value for 'description':", err)
	return
}
fmt.Printf("name: %s\ndescription: %s", name, description)

Output:

name: foo
description: bar

func (*DecodeResult) Fixed32Value ¶ added in v0.34.0

func (r *DecodeResult) Fixed32Value(tag int) (uint32, error)

Fixed32Value is a helper method to get the field data at the given tag and return it as a uint32. It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.Fixed32Value()

func (*DecodeResult) Fixed32Values ¶ added in v0.34.0

func (r *DecodeResult) Fixed32Values(tag int) ([]uint32, error)

Fixed32Values is a helper method to get the field data at the given tag and return it as a []uint32. It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.Fixed32Values()

func (*DecodeResult) Fixed64Value ¶ added in v0.34.0

func (r *DecodeResult) Fixed64Value(tag int) (uint64, error)

Fixed64Value is a helper method to get the field data at the given tag and return it as a uint64. It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.Fixed64Value()

func (*DecodeResult) Fixed64Values ¶ added in v0.34.0

func (r *DecodeResult) Fixed64Values(tag int) ([]uint64, error)

Fixed64Values is a helper method to get the field data at the given tag and return it as a []uint64. It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.Fixed64Values()

func (*DecodeResult) Float32Value ¶ added in v0.34.0

func (r *DecodeResult) Float32Value(tag int) (float32, error)

Float32Value is a helper method to get the field data at the given tag and return it as a float32. It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.Float32Value()

func (*DecodeResult) Float32Values ¶ added in v0.34.0

func (r *DecodeResult) Float32Values(tag int) ([]float32, error)

Float32Values is a helper method to get the field data at the given tag and return it as a []float32. It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.Float32Values()

func (*DecodeResult) Float64Value ¶ added in v0.34.0

func (r *DecodeResult) Float64Value(tag int) (float64, error)

Float64Value is a helper method to get the field data at the given tag and return it as a float64. It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.Float64Value()

func (*DecodeResult) Float64Values ¶ added in v0.34.0

func (r *DecodeResult) Float64Values(tag int) ([]float64, error)

Float64Values is a helper method to get the field data at the given tag and return it as a []float64. It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.Float64Values()

func (*DecodeResult) GetFieldData ¶ added in v0.34.0

func (r *DecodeResult) GetFieldData(tag int) (*FieldData, error)

GetFieldData returns the raw field data object at the given tag

func (*DecodeResult) Int32Value ¶ added in v0.34.0

func (r *DecodeResult) Int32Value(tag int) (int32, error)

Int32Value is a helper method to get the field data at the given tag and return it as an int32.

Use this method to retrieve values that are defined as int32 in the Protobuf message. Fields that are defined as sint32 (and so use the [Protobuf ZigZag encoding]) should be retrieved using SInt32Value() instead.

It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.Int32Value()

func (*DecodeResult) Int32Values ¶ added in v0.34.0

func (r *DecodeResult) Int32Values(tag int) ([]int32, error)

Int32Values is a helper method to get the field data at the given tag and return it as a []int32.

Use this method to retrieve values that are defined as repeated int32 in the Protobuf message. Fields that are defined as repeated sint32 (and so use the [Protobuf ZigZag encoding]) should be retrieved using SInt32Values() instead.

It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.Int32Values()

func (*DecodeResult) Int64Value ¶ added in v0.34.0

func (r *DecodeResult) Int64Value(tag int) (int64, error)

Int64Value is a helper method to get the field data at the given tag and return it as an int64.

Use this method to retrieve values that are defined as int64 in the Protobuf message. Fields that are defined as sint64 (and so use the [Protobuf ZigZag encoding]) should be retrieved using SInt64Value() instead.

It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.Int64Value()

func (*DecodeResult) Int64Values ¶ added in v0.34.0

func (r *DecodeResult) Int64Values(tag int) ([]int64, error)

Int64Values is a helper method to get the field data at the given tag and return it as a []int64.

Use this method to retrieve values that are defined as repeated int64 in the Protobuf message. Fields that are defined as repeated sint64 (and so use the [Protobuf ZigZag encoding]) should be retrieved using SInt64Values() instead.

It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.Int64Values()

func (*DecodeResult) NestedResult ¶ added in v0.34.0

func (r *DecodeResult) NestedResult(tag int) (*DecodeResult, error)

NestedResult will return the last/only DecodeResult located at the give tag path

The tag parameter acts as a "path" to a particular field to support retrieving DecodeResult from nested messages. The value is used to retreieve the field data at the corresponding protonumber

func (*DecodeResult) NestedResults ¶ added in v0.34.0

func (r *DecodeResult) NestedResults(tag int) ([]*DecodeResult, error)

NestedResults will return all of the DecodeResult located at the give tag path

The tag parameter acts as a "path" to a particular field to support retrieving DecodeResult from nested messages. The value is used to retreieve the field data at the corresponding protonumber

func (*DecodeResult) Range ¶ added in v0.34.0

func (r *DecodeResult) Range(fn func(tag int, field *FieldData) bool)

Range will iterate over all tags in the DecodeResult.

If a field was not present in the original message fn will be called with the tag and a nil field.

Currently, Range will iterate in the order of the tags, but this is not guaranteed for future use.

func (*DecodeResult) SInt32Value ¶ added in v0.34.0

func (r *DecodeResult) SInt32Value(tag int) (int32, error)

SInt32Value is a helper method to get the field data at the given tag and return it as an int32.

Use this method to retrieve values that are defined as sint32 in the Protobuf message. Fields that are defined as int32 (and so use the [Protobuf base128 varint encoding]) should be retrieved using Int32Value() instead.

It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.SInt32Value()

func (*DecodeResult) SInt32Values ¶ added in v0.34.0

func (r *DecodeResult) SInt32Values(tag int) ([]int32, error)

SInt32Values is a helper method to get the field data at the given tag and return it as a []int32.

Use this method to retrieve values that are defined as repeated sint32 in the Protobuf message. Fields that are defined as repeated int32 (and so use the [Protobuf base128 varint encoding]) should be retrieved using Int32Values() instead.

It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.SInt32Values()

func (*DecodeResult) SInt64Value ¶ added in v0.34.0

func (r *DecodeResult) SInt64Value(tag int) (int64, error)

SInt64Value is a helper method to get the field data at the given tag and return it as an int64.

Use this method to retrieve values that are defined as sint64 in the Protobuf message. Fields that are defined as int64 (and so use the [Protobuf base128 varint encoding]) should be retrieved using Int64Value() instead.

It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.SInt32Value()

func (*DecodeResult) SInt64Values ¶ added in v0.34.0

func (r *DecodeResult) SInt64Values(tag int) ([]int64, error)

SInt64Values is a helper method to get the field data at the given tag and return it as a []int64.

Use this method to retrieve values that are defined as repeated sint64 in the Protobuf message. Fields that are defined as repeated int64 (and so use the [Protobuf base128 varint encoding]) should be retrieved using Int64Values() instead.

It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.SInt64Values()

func (*DecodeResult) StringValue ¶ added in v0.34.0

func (r *DecodeResult) StringValue(tag int) (string, error)

StringValue is a helper method to get the field data at the given tag and return it as a string. It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.StringValue()

func (*DecodeResult) StringValues ¶ added in v0.34.0

func (r *DecodeResult) StringValues(tag int) ([]string, error)

StringValues is a helper method to get the field data at the given tag and return it as a []string. It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.StringValues()

func (*DecodeResult) UInt32Value ¶ added in v0.34.0

func (r *DecodeResult) UInt32Value(tag int) (uint32, error)

UInt32Value is a helper method to get the field data at the given tag and return it as a uint32. It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.UInt32Value()

func (*DecodeResult) UInt32Values ¶ added in v0.34.0

func (r *DecodeResult) UInt32Values(tag int) ([]uint32, error)

UInt32Values is a helper method to get the field data at the given tag and return it as a []uint32. It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.UInt32Values()

func (*DecodeResult) UInt64Value ¶ added in v0.34.0

func (r *DecodeResult) UInt64Value(tag int) (uint64, error)

UInt64Value is a helper method to get the field data at the given tag and return it as a uint64. It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.UInt64Value()

func (*DecodeResult) UInt64Values ¶ added in v0.34.0

func (r *DecodeResult) UInt64Values(tag int) ([]uint64, error)

UInt64Values is a helper method to get the field data at the given tag and return it as a []uint64. It is equivalent to

fd, err := r.GetFieldData(tag)
if err != nil {
  return err
}
v, err := fd.UInt64Values()

type Decoder ¶ added in v0.34.0

type Decoder struct {
	// contains filtered or unexported fields
}

Decoder is a lazy decoder that will reuse DecodeResults after (DecodeResult).Close() is called.

A decoder is unique to a given Definition and can be reused for any protobuf message using that definition.

Decoder methods are thread safe and can be used by concurrent/parallel processes.

func NewDecoder ¶ added in v0.34.0

func NewDecoder(def Def, opts ...Option) (*Decoder, error)

NewDecoder creates a new Decoder for a given Def. See NewDef for defining a definition.

NewDecoder will return an error if the definition or options are invalid

func (*Decoder) Decode ¶ added in v0.34.0

func (dec *Decoder) Decode(data []byte) (*DecodeResult, error)

Decode will convert the raw []byte slice to a DecodeResult

type Def ¶

type Def map[int]Def

A Def is an optionally nested mapping of protobuf field tags declaring which values should be decoded from a message. If the value for given tag maps to a nested definition and the wire type in the message data is WireTypeLengthDelimited, the contents are treated as a nested message and the nested mapping is applied to the field data recursively.

Tags must be valid [Protobuf field tags] (between 1 and 19000 or between 20000 and 536870911).

Because a given tag can map to either a scalar value or a sub-mapping for decoding a nested message, we have an edge case where a consumer cannot extract both the bytes of a nested message and individual fields. For this case use a negative tag value to add a mapping that will return the raw bytes of the nested message field.

// extract both the raw bytes of the nested message at field 3 *and* the value of field 1 within
// that nested message
def := lazyproto.NewDef()
_ = def.Tags(-3)
_ = def.NestedTag(3, 1)
decodeResult, _ := lazyproto.Decode(data, def)
...
// get the raw bytes
fd, _ := decodeResult.FieldData(-3)
raw, _ := fd.BytesValue()
...
// get the value of field 1 within the nested message
fd, _ = decodeResult.FieldData(3, 1)
v, _ := fd.StringValue()
...

func NewDef ¶

func NewDef(tags ...int) Def

NewDef initializes and returns a new Def with mappings for the specified field tags.

func (Def) Get ¶

func (d Def) Get(tag int) (Def, bool)

Get returns the mapping value for tag plus a boolean indicating whether or not the mapping existed

func (Def) NestedTag ¶

func (d Def) NestedTag(tag int, nestedTags ...int) Def

NestedTag adds a mapping for tag to a nested Def with the specified field tags for the nested message, replacing any existing mapping, and returns the nested Def

func (Def) Tags ¶

func (d Def) Tags(tags ...int) Def

Tags adds one or more field tags to the mapping, replacing any existing mappings, and returns the Def.

func (Def) Validate ¶

func (d Def) Validate() error

Validate checks that d is structurally and semantically valid and returns an error if it is not.

type FieldData ¶

type FieldData struct {
	// contains filtered or unexported fields
}

FieldData is a wrapper around lazily-decoded Protobuf field data, with accessors for retrieving typed values.

All methods potentially return an error because the Protobuf encoding is not self-describing. We must instead rely on the consumer having some prior knowledge of the message data and using that knowledge to access the fields appropriately.

The XxxValue() methods convert the lazily decoded field data into a single value of the appropriate Go type. If the decoded message contained repeated data for the field, the last value is returned.

Similarly, the XxxValues() methods convert the data into a slice of the appropriate Go type. If the decoded message contained only a single value, or the field is not defined as repeated, the methods return a one-element slice. We err on the side of determinism since it's not possible to distinguish between these two scenarios based on only the encoded data.

For both XxxValue() and XxxValues(), if the source data was not of the correct Protobuf wire type for the target Go type a WireTypeMismatchError error is returned with relevant details.

A zero-valued instance is the equivalent of a varint field with no data. All methods valid for varint data will return ErrTagNotFound and all others will return a WireTypeMismatchError.

To avoid panics, any method called on a nil instance returns a zero value and ErrTagNotFound.

func (*FieldData) BoolValue ¶

func (fd *FieldData) BoolValue() (bool, error)

BoolValue converts the lazily-decoded field data into a bool.

Since Protobuf encodes boolean values as integers, any varint-encoded integer value is valid. A value of zero is treated as false and any non-zero value is treated as true.