Documentation
¶
Overview ¶
Package whpg is the postgres input of a webhook source (#2408): the platform reads an application database and files what it reads as events, written exactly as the HTTP receiver writes a request's. Compaction into Parquet, the source's table, retention, alerts and the scripts naming the source then treat it like any other source.
A notify source files each notification's payload; PostgreSQL delivers a notification only to a session listening when it is sent, so this mode is at most once. An outbox source drains an outbox table: a row is read, filed, and deleted in one transaction, and a poll drains the rows whose notification no one heard, so a missed notification delays an event and never loses it. Rows are filed before they are deleted, so a commit that fails after the write files them again, under the same event id, which compaction keeps once.
One replica listens per source, under an advisory lock; another takes over on its next pass once the lock is free. What a payload holds is written by whoever can write the table or call pg_notify: it is data, bounded in size and never read as configuration.
Index ¶
Constants ¶
const LockKeyspace int32 = 4713212
LockKeyspace is the advisory-lock space a postgres source's listener is held in; the key within it is a hash of the source's name.
const Tick = 5 * time.Second
Tick is how often the manager starts the listeners of sources this replica can take and stops the ones whose source changed.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Config ¶
type Config struct {
Sources Sources
// Database returns the live application databases, nil on a deployment
// that serves none.
Database func() Database
Writer Writer
Rejections Rejections
States StateStore
Lock Lock
Replica string
Logger *slog.Logger
Now func() time.Time
// Tick overrides Tick; zero is Tick.
Tick time.Duration
}
Config assembles a manager.
type Database ¶
type Database interface {
Listen(ctx context.Context, connection string, channels []string) (Listener, error)
DrainOutbox(ctx context.Context, connection, table string, limit int, write func([]postgres.OutboxRow) error) (int, error)
}
Database reads application databases: the postgres toolkit.
type Listener ¶
type Listener interface {
Wait(ctx context.Context) (postgres.Notification, error)
Close()
}
Listener is one listening connection.
type Manager ¶
type Manager struct {
// contains filtered or unexported fields
}
Manager runs the listeners of the postgres sources this replica holds.
func (*Manager) Pass ¶
Pass stops the listeners whose source was removed, disabled or changed, and starts one for each enabled postgres source whose lock is free.
type Rejections ¶
type Rejections interface {
RecordRejections(ctx context.Context, rejections []whstore.Rejection) error
}
Rejections records what a listener refused.
type StateStore ¶
type StateStore interface {
Listening(ctx context.Context, source, replica string, since time.Time) error
Stopped(ctx context.Context, source, replica string) error
Filed(ctx context.Context, source string, delivered, rejected int64, at time.Time) error
Failed(ctx context.Context, source, msg string, at time.Time) error
}
StateStore keeps each source's listener state (States).