buffer

package
v1.8.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 3, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type CombinedBuffer

type CombinedBuffer struct {
	// contains filtered or unexported fields
}

CombinedBuffer is a two-tier buffer: in-memory ring for speed and file-backed buffer for durability and overflow. It implements hermod.Producer and can be consumed by the engine like other buffers.

Policy (simple/default):

  • Produce tries ring first with a short timeout; on timeout or when ring is above spillHigh watermark, it appends to the file buffer.
  • Consume drains ring first; when empty, it drains from the file buffer and forwards to the handler.
  • Close signals both tiers; file buffer is responsible for persisting state.

func NewCombinedBuffer

func NewCombinedBuffer(ringCapacity int, dir string, fileSize int, opts *CombinedOptions) (*CombinedBuffer, error)

NewCombinedBuffer constructs a CombinedBuffer. ringCapacity: size of in-memory ring channel. dir: directory for file buffer; created if missing. fileSize: logical size/backpressure window used by FileBuffer.

func (*CombinedBuffer) Close

func (b *CombinedBuffer) Close() error

Close closes both tiers.

func (*CombinedBuffer) Consume

func (b *CombinedBuffer) Consume(ctx context.Context, handler hermod.Handler) error

Consume prioritizes draining the ring; when empty, drains from the file buffer.

func (*CombinedBuffer) Produce

func (b *CombinedBuffer) Produce(ctx context.Context, msg hermod.Message) error

Produce enqueues the message; tries ring first, otherwise spills to file.

type CombinedOptions

type CombinedOptions struct {
	// SpillHighPct and SpillLowPct are percentages of ring capacity (0-100).
	SpillHighPct int
	SpillLowPct  int
	// ProduceTimeout bounds the time we wait to enqueue into the ring.
	ProduceTimeout time.Duration
	// Compressor is used for file-backed storage.
	Compressor compression.Compressor
}

CombinedOptions holds optional tuning parameters for CombinedBuffer.

type FileBuffer

type FileBuffer struct {
	// contains filtered or unexported fields
}

FileBuffer is a persistent buffer that stores messages in an append-only log.

func NewFileBuffer

func NewFileBuffer(dir string, size int) (*FileBuffer, error)

func NewFileBufferWithCompressor

func NewFileBufferWithCompressor(dir string, size int, comp compression.Compressor) (*FileBuffer, error)

func (*FileBuffer) Close

func (b *FileBuffer) Close() error

func (*FileBuffer) Consume

func (b *FileBuffer) Consume(ctx context.Context, handler hermod.Handler) error

func (*FileBuffer) Produce

func (b *FileBuffer) Produce(ctx context.Context, msg hermod.Message) (err error)

type RingBuffer

type RingBuffer struct {
	// contains filtered or unexported fields
}

RingBuffer is a high-performance, lock-free (using channels for now as a simple example, but optimized for performance) buffer that implements both Producer and Consumer.

func NewRingBuffer

func NewRingBuffer(size int) *RingBuffer

func (*RingBuffer) Close

func (b *RingBuffer) Close() error

func (*RingBuffer) Consume

func (b *RingBuffer) Consume(ctx context.Context, handler hermod.Handler) error

func (*RingBuffer) Depth

func (b *RingBuffer) Depth() (queued, capacity int)

Depth reports how many messages are waiting to be consumed and the buffer's capacity. A buffer holding messages that never drain is what a wedged pipeline looks like from the inside, and is the signal the stall watchdog uses to tell "stuck" apart from "idle".

func (*RingBuffer) Produce

func (b *RingBuffer) Produce(ctx context.Context, msg hermod.Message) error

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL