framing

package
v0.8.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 11, 2026 License: MIT Imports: 6 Imported by: 0

Documentation

Overview

Package framing turns a captured DecodeRequest.payload into L7 application bytes.

Why this package exists

For every pcap-based capture source the pipeline hands a plugin the *complete link-layer frame*, untouched:

DecodeRequest.payload = link-layer header + IP + TCP/UDP + application bytes

The capture layer parses L2/L3/L4 only to fill in the context fields (link_type, src, dst, protocol_hint); it never trims the payload. Only the two proxy-class link types — ProxyPayload (1001) and TLSPlaintext (1002) — deliver bare L7.

Every decoder therefore needs the same two things before it can parse a single business field: strip the encapsulation selected by link_type, and reassemble the TCP byte stream so that messages spanning multiple segments survive. Both are protocol-independent, easy to get subtly wrong, and were historically re-implemented (or forgotten) by each plugin. They live here instead.

Usage

var ra = framing.NewReassembler()

func decode(req *pb.DecodeRequest, stream pb.Decoder_DecodeV2Server) error {
    seg, ok := framing.ExtractL7(req.GetPayload(), req.GetLinkType())
    if !ok || len(seg.Payload) == 0 {
        return done(stream, req) // pure ACK, handshake, or non-IP traffic
    }
    s := ra.Push(seg)
    for {
        msg, n := parseOneMessage(s.Bytes())
        if n == 0 {
            break // incomplete; wait for the next segment
        }
        s.Consume(n)
        emit(msg)
    }
    return done(stream, req)
}

See contract.yaml rules payload-framing-by-link-type, link-type-selects-framing and tcp-reassembly-required.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type FlowKey

type FlowKey struct {
	Src      netip.AddrPort
	Dst      netip.AddrPort
	Protocol string // "tcp" | "udp" | "" when unknown
}

FlowKey identifies one directional byte stream. Client→server and server→client are deliberately distinct keys: a TCP sequence space is per-direction, so reassembly state must not be shared between them.

The zero value is the "no flow" key, used for proxy-class inputs where the pipeline supplies addressing out of band.

func (FlowKey) Canonical

func (f FlowKey) Canonical() string

Canonical returns a direction-independent identifier: both halves of a conversation map to the same string. Use it to pair a request with its response; use FlowKey itself when you need per-direction state.

func (FlowKey) Reverse

func (f FlowKey) Reverse() FlowKey

Reverse returns the key of the opposite direction.

func (FlowKey) String

func (f FlowKey) String() string

String renders the flow as "proto src>dst", stable enough to use as a map key in a plugin's own bookkeeping or as a correlation_key component.

type Reassembler

type Reassembler struct {
	// contains filtered or unexported fields
}

Reassembler reassembles the per-flow TCP byte stream across captured segments so that application messages spanning multiple packets (or delivered out-of-order, or retransmitted) survive intact.

A single Reassembler is safe to share across the whole plugin: it keys state by FlowKey, one buffer per directional stream. UDP and proxy-class inputs are passed through untouched, since they carry self-contained payloads.

Concurrency: Push, Stream.Bytes, Stream.Consume, Forget and Reset are goroutine-safe, but the parse loop for one flow must stay sequential — call Bytes, parse, Consume, repeat on the same goroutine that feeds Push, because Consume acts on the bytes Bytes just handed you.

func NewReassembler

func NewReassembler() *Reassembler

NewReassembler returns an empty reassembler ready to receive segments.

func (*Reassembler) Forget

func (ra *Reassembler) Forget(key FlowKey)

Forget drops the reassembly state for one flow without waiting for a FIN. Use it when the application layer tells you a connection closed (e.g. an HTTP "Connection: close" response) so a reused 5-tuple starts clean.

func (*Reassembler) Push

func (ra *Reassembler) Push(seg Segment) Stream

Push folds one Segment into the reassembly state and returns the Stream for that flow.

  • A pure ACK, SYN, or FIN (no data) returns a Stream whose Bytes is empty; its flags are still applied (SYN seeds the sequence base, FIN retires the flow once drained).
  • A TCP segment with data returns the contiguously reassembled prefix, which may be empty if a preceding gap has not yet been filled.
  • UDP and proxy-class inputs bypass reassembly: Bytes returns the whole payload and Consume is a no-op on shared state.

func (*Reassembler) Reset

func (ra *Reassembler) Reset()

Reset clears all reassembly state. Useful when a capture run ends and a new one begins on the same plugin instance.

type Segment

type Segment struct {
	// Payload is the application-layer bytes. It may be empty for a pure ACK,
	// a handshake packet, or a FIN — that is normal, not an error.
	Payload []byte

	// Flow identifies the directional stream this chunk belongs to.
	Flow FlowKey

	// Seq is the TCP sequence number of Payload's first byte. Zero for UDP and
	// proxy-class inputs.
	Seq uint32

	// Flags are the TCP control bits. Zero value for UDP and proxy inputs.
	Flags TCPFlags

	// IsTCP reports whether this segment came from a TCP stream and therefore
	// participates in reassembly.
	IsTCP bool

	// LinkType echoes the link_type the segment was extracted with, so a
	// decoder can record framing provenance on the event it emits.
	LinkType int32
}

Segment is one L7 chunk extracted from a captured frame, together with the transport facts a reassembler or a correlator needs.

func ExtractL7

func ExtractL7(raw []byte, linkType int32) (Segment, bool)

ExtractL7 strips the encapsulation indicated by linkType and returns the application-layer bytes.

ok is false when raw carries no transport layer this package understands — ARP, ICMP, a truncated frame, or an unrecognised link type. That is a normal outcome for a mixed capture, not a decode error: respond done=true and move on.

ok is true with an empty Segment.Payload for TCP packets that carry no data (SYN, pure ACK, FIN). Pass those to a Reassembler anyway — it uses the flags to reset and retire per-flow state.

The returned Payload aliases raw; copy it if you need to retain it past the current Decode call.

type Stream

type Stream struct {
	// contains filtered or unexported fields
}

Stream is the in-order bytes currently available for one directional flow, plus the ability to mark bytes the caller has consumed.

Bytes returns the contiguous prefix that has been reassembled so far. It returns nil for proxy-class / UDP inputs that carried no payload. The slice is valid until the next Push for this flow or the next Consume, so parse it immediately and do not retain it.

Consume(n) drops the first n bytes from the front of the stream, advancing the reassembly frontier. Pass the number of bytes your parser emitted.

func (Stream) Bytes

func (s Stream) Bytes() []byte

Bytes returns the contiguous, reassembled prefix available for this flow.

func (Stream) Consume

func (s Stream) Consume(n int)

Consume drops the first n bytes from the front of the stream, advancing the reassembly frontier. A finished flow (FIN, nothing left) is retired.

type TCPFlags

type TCPFlags struct {
	FIN bool
	SYN bool
	RST bool
	PSH bool
	ACK bool
	URG bool
}

TCPFlags carries the TCP control bits of the segment a payload came from. All fields are false for UDP and proxy-class inputs.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL