wiretrace

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 4, 2026 License: Apache-2.0 Imports: 8 Imported by: 0

Documentation

Overview

Package wiretrace decodes FoundationDB client traffic from a loopback packet capture: it reassembles each TCP connection, splits it into FDB frames, and matches GRV and commit requests with their replies.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Decode

func Decode(r io.Reader, serverPorts []uint16) (*Trace, Completeness, error)

Decode reads a pcap capture and follows client traffic to serverPorts.

func ReadPcap

func ReadPcap(r io.Reader, fn func(Segment) error) error

ReadPcap calls fn for every IPv4 TCP segment in a classic pcap stream. Loopback captures on Linux use the Ethernet link type.

Types

type Chunk

type Chunk struct {
	Conn   int // connection generation: a new SYN on a reused 4-tuple starts a new one
	Packet int
	TimeNS int64
	Data   []byte
}

Chunk is in-order stream data. Packet and TimeNS identify the captured segment that made it readable: the one carrying it, or the one that filled the gap before it.

type Completeness

type Completeness struct {
	Gaps     int      // streams with a hole: a segment was never captured
	Partial  []string // streams that ended inside a connect packet or frame
	Requests int      // client requests decoded
}

Completeness reports what a capture could not decode.

func (Completeness) Complete

func (c Completeness) Complete() bool

Complete reports whether every stream was decoded to its end and held at least one request: a capture with none (wrong ports, no traffic) measures nothing.

type Endpoint

type Endpoint struct {
	IP   [4]byte
	Port uint16
}

Endpoint is an IPv4 address and TCP port.

func (Endpoint) String

func (e Endpoint) String() string

type Flow

type Flow struct{ Src, Dst Endpoint }

Flow is one direction of a TCP connection.

type Frame

type Frame struct {
	TimeNS int64 // capture time of the segment that made the frame's last byte readable
	Flow   Flow
	// Segment is that segment's capture index, so frames completed by one
	// captured segment share it.
	Segment int
	Token   [16]byte
	Body    []byte
}

Frame is one FDB packet: the destination endpoint token and its body.

type FrameParser

type FrameParser struct {
	// contains filtered or unexported fields
}

FrameParser splits one flow of a plaintext FDB connection into frames, after the length-prefixed ConnectPacket that opens each direction (FlowTransport.actor.cpp, ConnectPacket and scanPackets).

func NewFrameParser

func NewFrameParser(flow Flow, emit func(Frame) error) *FrameParser

NewFrameParser parses the flow's data in stream order.

func (*FrameParser) Add

func (p *FrameParser) Add(c Chunk) error

Add takes the flow's next chunk.

func (*FrameParser) Finish

func (p *FrameParser) Finish() error

Finish reports a ConnectPacket or frame cut off by the end of the stream.

type Kind

type Kind string

Kind is a request type the analyzer follows.

const (
	KindGRV    Kind = "grv"
	KindCommit Kind = "commit"
	KindRange  Kind = "range" // storage GetKeyValues
)

type KindSummary

type KindSummary struct {
	Requests     int `json:"requests"`
	Replied      int `json:"replied"`
	Errors       int `json:"errors"`
	Transactions int `json:"transactions"` // GRV: transactions the requests asked for
	// ClientSegments is the number of client TCP segments that completed a
	// request: requests written together share one.
	ClientSegments int `json:"client_segments"`
	// Malformed counts replies that arrived but did not decode.
	Malformed int `json:"malformed"`
	// Commits only. ServerBatches is the number of distinct commit versions:
	// a commit proxy gives every transaction of one batch the same version.
	// (GRV and storage replies carry observed read versions, which separate
	// batches can share, so they are not grouped.)
	ServerBatches int `json:"server_batches"`
	// BatchSizes counts commit batches by the number of commits in them.
	BatchSizes map[int]int `json:"batch_sizes"`
	// ArrivalSpreadNS is the time from a batch's first commit to its last,
	// at the capture point; the median, 90th percentile and maximum.
	ArrivalSpreadNS [3]int64 `json:"arrival_spread_ns"`
	// LatencyNS is the time from request to reply at the capture point.
	LatencyNS [3]int64 `json:"latency_ns"`
	// ReorderedReplies counts successful replies that became readable before
	// their request did: their timing is not a latency and is left out of
	// LatencyNS.
	ReorderedReplies int `json:"reordered_replies,omitempty"`
	// Params counts range requests by their parameters; More counts replies
	// that reported more rows; ReplyBytes is their total packed size.
	Params     map[string]int `json:"params,omitempty"`
	More       int            `json:"more,omitempty"`
	ReplyBytes int            `json:"reply_bytes,omitempty"`
}

KindSummary describes one request kind over a capture.

type Reassembler

type Reassembler struct {

	// Gaps counts streams with data that never became readable: a segment was
	// missing, so a hole remained when the capture ended or was too long.
	Gaps int
	// contains filtered or unexported fields
}

Reassembler orders the payload of each flow whose SYN was captured. Flows that started before the capture are ignored, so a stream always begins at its first byte.

func NewReassembler

func NewReassembler(emit func(Flow, Chunk) error) *Reassembler

NewReassembler calls emit with each flow's data in stream order.

func (*Reassembler) Add

func (r *Reassembler) Add(seg Segment) error

Add takes the next captured segment.

func (*Reassembler) Finish

func (r *Reassembler) Finish()

Finish counts the streams with a hole at the end of the capture: data still buffered behind a gap, or a FIN beyond the delivered data.

type Request

type Request struct {
	Kind       Kind
	Flow       Flow  // client to server
	SentNS     int64 // capture time of the request's last byte
	Segment    int   // client segment number within Flow
	TxnCount   uint32
	ReplyToken [16]byte
	ReplyNS    int64 // 0: no reply captured
	Version    int64 // read or commit version from the reply
	ErrorCode  int   // nonzero: the reply was an error
	Malformed  bool  // the reply arrived but did not decode
	// Range requests only: the request's parameters and the reply's shape.
	Params     string // limit, byte limit, selector offsets and read options
	ReplyBytes int    // length of the reply's packed rows
	More       bool
}

Request is a GRV or commit request and, once seen, its reply.

type Rounds

type Rounds struct {
	Size int `json:"round_size"`
	// Whole and Split count the batches that did and did not hold a whole round.
	Whole int `json:"whole_batches"`
	Split int `json:"split_batches"`
	// FirstSplit is the index of the first batch after a whole one that did not
	// hold a whole round, -1 if none.
	FirstSplit int `json:"first_split"`
	// Runs alternates the lengths of consecutive whole and split stretches,
	// starting with a whole one (possibly of length 0).
	Runs []int `json:"runs"`
}

Rounds describes the commit batches of a closed loop whose rounds have the given size: whether each batch, in order of its first request, held a whole round.

type Segment

type Segment struct {
	Index   int // capture order, from 1
	TimeNS  int64
	Flow    Flow
	Seq     uint32
	SYN     bool
	FIN     bool
	Payload []byte
}

Segment is one captured TCP segment.

type Summary

type Summary struct {
	Flows       int                  `json:"client_flows"`
	Kinds       map[Kind]KindSummary `json:"kinds"`
	Undecodable int                  `json:"undecodable"`
	Gaps        int                  `json:"stream_gaps"`
	// Partial counts streams that ended inside a connect packet or frame.
	Partial int `json:"partial_streams"`
	// Complete is false when any stream had a gap or ended inside a frame:
	// some requests or replies are missing, so the counts are not a measurement.
	Complete bool    `json:"complete"`
	Rounds   *Rounds `json:"rounds,omitempty"`
}

Summary describes a capture.

type Trace

type Trace struct {
	Requests []*Request
	// Undecodable counts request frames with a followed file identifier that
	// did not decode, and replies that did not decode.
	Undecodable int
	// contains filtered or unexported fields
}

Trace follows the requests on client connections, which are the flows to one of the server ports.

func NewTrace

func NewTrace(serverPorts []uint16) *Trace

NewTrace follows client connections to the given server ports.

func (*Trace) Add

func (t *Trace) Add(f Frame)

Add takes the next frame in capture order.

func (*Trace) CommitRounds

func (t *Trace) CommitRounds(size int) Rounds

CommitRounds classifies the replied commit batches against a round size.

func (*Trace) Summarize

func (t *Trace) Summarize() Summary

Summarize groups the replied requests of each kind into server batches.

func (*Trace) WriteEvents

func (t *Trace) WriteEvents(w io.Writer) error

WriteEvents writes one CSV line per request, in capture order.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL