Documentation
¶
Index ¶
- Constants
- Variables
- func Anonymise(inputPath, outputPath string, rewriteAddresses bool) error
- func CleanupOldFiles(dir string, retention time.Duration) (int, error)
- type Header
- type Packet
- type Reader
- type Recorder
- type TapConn
- type Uploader
- func (u *Uploader) Close() error
- func (u *Uploader) Delete(ctx context.Context, key string) error
- func (u *Uploader) Finish(ctx context.Context, connectionUID uuid.UUID)
- func (u *Uploader) Key(connectionUID uuid.UUID, t time.Time) string
- func (u *Uploader) Open(ctx context.Context, key string) (io.ReadCloser, error)
- func (u *Uploader) Stat(ctx context.Context, key string) (int64, error)
- func (u *Uploader) SweepSpool(ctx context.Context) (int, error)
- type UploaderOptions
- type Writer
Constants ¶
const ( DirClientToServer byte = 0x00 DirServerToClient byte = 0x01 )
Packet direction constants.
const ( ProtocolOracle = "oracle" ProtocolPostgreSQL = "postgresql" ProtocolMySQL = "mysql" ProtocolMongo = "mongodb" ProtocolMSSQL = "mssql" )
Protocol identifiers.
const ContentType = "application/x-pcapng"
ContentType is the MIME type captures are served and stored with.
const FileExt = ".pcapng"
FileExt is the extension used for session capture files. Captures are plain pcapng, readable by tcpdump/Wireshark/tshark without any dbbat tooling.
Variables ¶
var ( // ErrNoSpoolDir is returned when uploading is configured without a local // spool. Captures are written to disk first and uploaded once complete — // S3 objects cannot be appended to — so there is nothing to upload // without one. ErrNoSpoolDir = errors.New("dump upload requires a local spool directory") // ErrNoRecorder is returned when no Recorder is supplied: an uploaded // capture whose key is stored nowhere can never be found again. ErrNoRecorder = errors.New("dump upload requires a key recorder") // ErrNoScheme is returned for an upload URL with no scheme. The scheme is // what selects the driver. ErrNoScheme = errors.New("dump upload URL has no scheme (expected e.g. s3://bucket/prefix)") )
Uploader configuration errors.
var ErrMissingMetadata = errors.New("capture has no dbbat session metadata")
ErrMissingMetadata is returned when a capture carries no dbbat session metadata in its Section Header Block comment.
Functions ¶
func Anonymise ¶
Anonymise reads a capture and writes an anonymised copy.
Packet payloads and their relative timing are preserved verbatim. The session metadata carried in the pcapng Section Header Block comment is reduced to the session ID and the protocol: the connection object (database, user, service name, upstream address…) is dropped, and the capture is rebased onto the Unix epoch so the wall-clock time of the session leaks nothing.
When rewriteAddresses is true — the default for the CLI — the synthesized IPv4 addresses and TCP ports are re-generated from the fake endpoints too, since the capture's server-side addressing normally encodes the real upstream host and port. Pass false to keep the original addressing.
func CleanupOldFiles ¶
CleanupOldFiles deletes .pcapng capture files older than the retention period. It also reaps leftover legacyFileExt files from before the pcapng switch, since they are otherwise unreadable and invisible to this sweep. Returns the number of files deleted.
Types ¶
type Header ¶
type Header struct {
SessionID string `json:"session_id"`
Protocol string `json:"protocol"`
StartTime time.Time `json:"start_time"`
Connection map[string]any `json:"connection"`
}
Header holds the JSON-serializable session metadata. It is stored as a JSON blob in the pcapng Section Header Block comment (opt_comment).
type Packet ¶
type Packet struct {
RelativeNs int64 // Nanoseconds since session start
Direction byte // DirClientToServer or DirServerToClient
Data []byte // Raw protocol bytes (TCP payload, synthesized headers stripped)
}
Packet represents a single captured application-layer payload.
type Reader ¶
type Reader struct {
// contains filtered or unexported fields
}
Reader reads application payloads back out of a pcapng capture, undoing the synthesized Ethernet/IPv4/TCP wrapping applied by Writer.
func OpenReader ¶
OpenReader opens a capture file and parses the session metadata carried in the pcapng Section Header Block comment.
func (*Reader) ReadPacket ¶
ReadPacket returns the next application payload. Frames without a TCP payload are skipped. Returns io.EOF at the end of the capture.
type Recorder ¶ added in v0.23.0
type Recorder interface {
SetConnectionDumpKey(ctx context.Context, connectionUID uuid.UUID, key string) error
}
Recorder persists the blob key of an uploaded capture on its connection row.
The key has to be stored: the API looks captures up by connection UID alone and cannot know which instance wrote the file or on which day, so without it every download would have to LIST the bucket. Implemented by *store.Store.
type TapConn ¶
TapConn wraps a net.Conn and captures all read/written bytes to a Writer. Reads are tagged with one direction, writes with the other.
func NewTapConn ¶
NewTapConn creates a connection wrapper that captures traffic to a dump Writer.
type Uploader ¶ added in v0.23.0
type Uploader struct {
// contains filtered or unexported fields
}
Uploader moves finished captures from the local spool to a blob bucket.
Captures are never streamed to the bucket while the session is live: S3 objects cannot be appended to, so live streaming would mean multipart uploads with 5 MiB parts, would lose the flush-per-packet behavior the writer has today, and would strand an invisible incomplete upload on a crash. Spooling to disk and uploading on close keeps the "a crash still leaves a valid partial pcapng" property.
Every method is safe on a nil *Uploader, which is what "no upload configured" is represented by — the four proxies and the API can then call through unconditionally.
func OpenUploader ¶ added in v0.23.0
func OpenUploader(ctx context.Context, opts UploaderOptions) (*Uploader, error)
OpenUploader opens the destination bucket and starts the upload workers. An empty opts.URL returns (nil, nil): local-only capture storage, the default, is expressed by a nil *Uploader rather than by a flag.
func (*Uploader) Close ¶ added in v0.23.0
Close stops accepting new work, drains what is queued and releases the bucket. Safe on a nil Uploader and safe to call twice.
func (*Uploader) Delete ¶ added in v0.23.0
Delete removes an uploaded capture. A missing object is not an error: the caller's intent (the capture should be gone) is already satisfied.
func (*Uploader) Finish ¶ added in v0.23.0
Finish schedules the finished capture of connectionUID for upload. Call it after the dump writer has been closed — the file must be complete.
It never blocks on the network: the job is queued and a worker does the upload. A full queue is not an error either, the capture simply stays in the spool for the next startup sweep. No-op on a nil Uploader.
func (*Uploader) Key ¶ added in v0.23.0
Key returns the object key a capture taken at t by this instance gets: YYYY/MM/DD/<instance>/<connection uid>.pcapng.
The date segments exist for human browsing only. They are derived from the spool file's modification time rather than from the clock, so recomputing the key on a retry — or on the next process's startup sweep — yields the same key and overwrites rather than orphaning the previous attempt.
func (*Uploader) Open ¶ added in v0.23.0
Open streams a previously uploaded capture back. The caller closes the reader. Returns fs.ErrNotExist (wrapped) when the object is gone.
func (*Uploader) Stat ¶ added in v0.23.0
Stat returns the size in bytes of a previously uploaded capture, without reading its body. Returns fs.ErrNotExist (wrapped) when the object is gone.
func (*Uploader) SweepSpool ¶ added in v0.23.0
SweepSpool queues every capture left in the spool directory for upload and returns how many it queued.
This is the crash-recovery half: a session whose process died never reached Finish, so its (valid, partial) capture is sitting in the spool with nobody to upload it. Call it once at startup, before any proxy accepts — at that moment every file in the spool is by definition finished, which is what makes a blind sweep safe.
type UploaderOptions ¶ added in v0.23.0
type UploaderOptions struct {
// URL is the destination bucket, e.g. "s3://bucket/prefix" or
// "file:///var/lib/dbbat/captures". Empty disables uploading entirely.
URL string
// SpoolDir is the local capture directory (DBB_DUMP_DIR). Finished
// captures are read from — and, once uploaded, removed from — here.
SpoolDir string
// InstanceID names the process in the object key, so replicas sharing a
// bucket stay distinguishable.
InstanceID string
// Recorder stores the resulting key on the connection row. Required:
// an uploaded capture nobody can address again is a lost capture.
Recorder Recorder
Logger *slog.Logger
}
UploaderOptions configures OpenUploader.
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
Writer writes a session capture as a pcapng file. Application payloads are wrapped in synthesized Ethernet/IPv4/TCP headers (see synth.go) and the session metadata is stored as a JSON blob in the Section Header Block comment.
func NewWriter ¶
NewWriter creates a new capture file and writes the pcapng section header (carrying the session metadata) and interface description.
Source Files