whpg

package
v1.160.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 13, 2026 License: Apache-2.0 Imports: 17 Imported by: 0

Documentation

Overview

Package whpg is the postgres input of a webhook source (#2408): the platform reads an application database and files what it reads as events, written exactly as the HTTP receiver writes a request's. Compaction into Parquet, the source's table, retention, alerts and the scripts naming the source then treat it like any other source.

A notify source files each notification's payload; PostgreSQL delivers a notification only to a session listening when it is sent, so this mode is at most once. An outbox source drains an outbox table: a row is read, filed, and deleted in one transaction, and a poll drains the rows whose notification no one heard, so a missed notification delays an event and never loses it. Rows are filed before they are deleted, so a commit that fails after the write files them again, under the same event id, which compaction keeps once.

One replica listens per source, under an advisory lock; another takes over on its next pass once the lock is free. What a payload holds is written by whoever can write the table or call pg_notify: it is data, bounded in size and never read as configuration.

Index

Constants

View Source
const LockKeyspace int32 = 4713212

LockKeyspace is the advisory-lock space a postgres source's listener is held in; the key within it is a hash of the source's name.

View Source
const Tick = 5 * time.Second

Tick is how often the manager starts the listeners of sources this replica can take and stops the ones whose source changed.

Variables

This section is empty.

Functions

func LockKey

func LockKey(source string) int32

LockKey is the key a source's listener is locked under.

Types

type Config

type Config struct {
	Sources Sources
	// Database returns the live application databases, nil on a deployment
	// that serves none.
	Database   func() Database
	Writer     Writer
	Rejections Rejections
	States     StateStore
	Lock       Lock
	Replica    string
	Logger     *slog.Logger
	Now        func() time.Time
	// Tick overrides Tick; zero is Tick.
	Tick time.Duration
}

Config assembles a manager.

type Database

type Database interface {
	Listen(ctx context.Context, connection string, channels []string) (Listener, error)
	DrainOutbox(ctx context.Context, connection, table string, limit int, write func([]postgres.OutboxRow) error) (int, error)
}

Database reads application databases: the postgres toolkit.

type Listener

type Listener interface {
	Wait(ctx context.Context) (postgres.Notification, error)
	Close()
}

Listener is one listening connection.

type Lock

type Lock func(ctx context.Context, source string) (release func(), ok bool, err error)

Lock takes the one-replica lock on a source's listener, reporting whether it was free.

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager runs the listeners of the postgres sources this replica holds.

func New

func New(cfg Config) *Manager

New returns a manager, nil when a dependency is missing.

func (*Manager) Pass

func (m *Manager) Pass(ctx context.Context) error

Pass stops the listeners whose source was removed, disabled or changed, and starts one for each enabled postgres source whose lock is free.

func (*Manager) Start

func (m *Manager) Start(ctx context.Context)

Start runs the manager until Stop.

func (*Manager) Stop

func (m *Manager) Stop()

Stop ends the manager and every listener, and waits for them.

type Rejections

type Rejections interface {
	RecordRejections(ctx context.Context, rejections []whstore.Rejection) error
}

Rejections records what a listener refused.

type Sources

type Sources interface {
	List(ctx context.Context) ([]whsource.Source, error)
}

Sources lists the stored sources.

type StateStore

type StateStore interface {
	Listening(ctx context.Context, source, replica string, since time.Time) error
	Stopped(ctx context.Context, source, replica string) error
	Filed(ctx context.Context, source string, delivered, rejected int64, at time.Time) error
	Failed(ctx context.Context, source, msg string, at time.Time) error
}

StateStore keeps each source's listener state (States).

type Writer

type Writer interface {
	Write(ctx context.Context, src whsource.Source, events []whevent.Event) error
}

Writer files events as segments (whwrite.Writer).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL