webhookpub

package
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 21, 2026 License: Apache-2.0 Imports: 19 Imported by: 0

Documentation

Overview

Package webhookpub publishes events from the e2a core (relay, outbound sender, HITL flow) to subscribers registered via the /v1/webhooks resource, using a durable three-layer pipeline:

Layer 1 — webhook_events (the outbox): trigger code writes the event row inside its
          own DB transaction (Outbox.PublishTx / PublishBestEffortTx), so the event
          is durable the moment the business state commits.
Layer 2 — webhook_subscriber_deliveries: the event is fanned out to one row per
          matching enabled subscriber (event type + filters). Two interchangeable
          fan-out engines drain Layer 1: the legacy in-process OutboxWorker
          (LISTEN/NOTIFY + poll, the default) and the River FanOutWorker
          (E2A_WEBHOOK_FANOUT_MODE=river). Both share fanOutEventCore.
Layer 3 — HTTP POST: internal/webhookdelivery's River DeliverWorker delivers each
          Layer 2 row with retries + HMAC signing.

This is the sole push path: the legacy per-agent agent_identities.webhook_url + agent_mode columns (and the PersistentDeliverer that served them) were removed in migration 029.

Index

Constants

View Source
const (
	EventEmailReceived = "email.received"
	EventEmailSent     = "email.sent"
	// Async-send terminal outcome (async-send-contract.md §4). With the
	// persist-first pipeline the API returns 200 `accepted` before the send
	// completes, so this event is the push signal that a send finished:
	//   - email.failed: terminal failure (retries exhausted / permanent reject /
	//     block / review-reject-or-expiry). Carries reason.
	// Emitted durably by the queue-first outbound worker as the terminal failure
	// signal.
	EventEmailFailed = "email.failed"
	// Review-hold lifecycle (unified, direction-aware — design 2026-06-22). A held
	// message fires email.review_requested (defined below); on resolution it fires
	// email.review_approved (outbound: sent; inbound: released to the agent) or
	// email.review_rejected. These replace the outbound-only pending_approval /
	// approval_accepted / approval_rejected — outbound holds now fire the same
	// direction-aware review events as inbound, distinguished by the `direction`
	// field in the payload.
	EventEmailReviewApproved = "email.review_approved"
	EventEmailReviewRejected = "email.review_rejected"
	// Sender identity (decision 4 / Slice 4): the async SES sending identity
	// for a domain reached a terminal state. Lets agents skip polling
	// GET /domains/{domain} for sending_status.
	EventDomainSendingVerified = "domain.sending_verified"
	EventDomainSendingFailed   = "domain.sending_failed"
	// Delivery feedback (decision 9 / Slice 4b): async outcome of an outbound
	// message, per recipient. domain.suppression_added is account-scoped
	// (despite the prefix) — fired when an address is auto-suppressed.
	EventEmailDelivered         = "email.delivered"
	EventEmailBounced           = "email.bounced"
	EventEmailComplained        = "email.complained"
	EventDomainSuppressionAdded = "domain.suppression_added"
	// Agent-scoped recipient consent. Beta until its payload contract graduates.
	EventAgentSuppressionAdded = "agent.suppression_added"
	// Inbound trust policy (decision 10 / Slice 7): an inbound message did not
	// match the agent's ingestion policy (allowlist/domain). It is delivered but
	// flagged — operators get a signal, nothing is dropped.
	EventEmailFlagged = "email.flagged"
	// EventEmailBlocked fires when a message is refused by screening — the applied
	// action (gate ∨ scan) is `block`. Inbound: the message is accept-then-quarantined
	// (review_rejected, dropped, no human). Outbound: the send is refused (the caller
	// also gets a synchronous 4xx). It is the disposition event for the `block` action,
	// firing for BOTH directions; `reason_source` names the producer that drove it
	// (sender_gate / recipient_gate / inbound_scan / outbound_scan), mirroring the
	// protection_events audit vocabulary so a subscriber can correlate the two.
	EventEmailBlocked = "email.blocked"
	// EventEmailReviewRequested fires when an inbound message is held for human review
	// (applied action = review → status pending_review). The same event fires for
	// outbound HITL holds (it is direction-aware) and carries the review TTL plus
	// reason_source (sender_gate / inbound_scan) so a subscriber can drive an inbound
	// review queue from push instead of polling.
	EventEmailReviewRequested = "email.review_requested"
)

Event types. Keeping these as named constants (not arbitrary strings) means typos at trigger sites fail at compile time. The handler-layer allowlist of accepted event names mirrors this list.

View Source
const SchemaVersion = 1

SchemaVersion is the current webhook envelope schema version. It is the single source of truth shared by the webhook_events.schema_version column default (see writeOutboxRow), the wire envelope (AsEnvelope), and the X-E2A-Schema-Version delivery header — so the three can never drift. Bump only on an incompatible change to the envelope's shape.

Variables

AllEventTypes is the canonical allowlist of event names. Used by the handler's event-type validation. Adding a new event type means adding a constant above AND extending this slice.

ExperimentalEventTypes is the canonical list of screening + review-hold event types that are still Beta: their payloads may change before they are declared stable. The OpenAPI spec surfaces this list machine-readably as `x-experimental-values` on every field that enumerates event types (see httpapi's stability pass). Graduating an event to stable = removing it here; the spec markers and prose follow automatically.

Functions

func DeterministicEventID

func DeterministicEventID(parts ...string) string

DeterministicEventID derives a stable event id from the trigger context. Per design §5.1, the input formula per event type is:

email.received: sha256(message_id || "|" || event_type)
email.sent:     sha256(message_id || "|" || event_type)
pending_review/review_approved/review_rejected: sha256(pending_msg_id || "|" || event_type)
future bounced/complained/delivered: sha256(message_id || "|" || event_type || "|" || ses_event_id)

The "|" delimiter prevents accidental collisions where concatenated fields could be ambiguous (e.g. ("abc","def") vs ("abcdef","")).

Returns "evt_" + first 32 hex chars of the sha256 digest (128 bits of entropy). Birthday collision probability at 1M events/day × 30 days × 5 event types is ~3e-23 — negligible.

Determinism is what makes the outbox write idempotent across MTA SMTP retries: the retried trigger produces the same id, and the outbox INSERT no-ops via ON CONFLICT (id) DO NOTHING.

func IsValidEventType

func IsValidEventType(name string) bool

IsValidEventType reports whether name is one of the catalog event types. Convenience wrapper for the handler-layer validator.

Types

type DeliveryEnqueuer

type DeliveryEnqueuer interface {
	EnqueueDeliveryTx(ctx context.Context, tx pgx.Tx, deliveryID string) (int64, error)
}

DeliveryEnqueuer enqueues a webhook delivery job transactionally. *webhookdelivery.Jobs satisfies it. Injected so webhookpub stays decoupled from the River-delivery package and the choice is a wiring decision.

type Envelope

type Envelope struct {
	Type string `json:"type"`
	ID   string `json:"id"`
	// SchemaVersion is the envelope schema version (currently "1"), carried on the
	// wire so consumers can branch on it before parsing `data`. Sourced from the
	// SchemaVersion constant — the same value the DB column default uses.
	SchemaVersion string    `json:"schema_version"`
	CreatedAt     time.Time `json:"created_at"`
	Data          any       `json:"data"`
}

Envelope is the wire shape sent in the HTTP body to webhook subscribers. Stable across retries — the event_payload JSON column stores the envelope verbatim and the delivery worker POSTs it unchanged. Type is the event-type discriminator. Named `type` on the wire to match the /v1/events REST resource (EventView.type) and the Stripe-style SDK `construct_event` helper — a webhook delivery body is the same shape as a GET /v1/events/{id} object, so consumers handle both identically.

type Event

type Event struct {
	// ID is a unique identifier for this event firing. Stable across
	// retries — receivers dedup on it.
	ID string

	// Type is one of the EventEmail* constants.
	Type string

	// CreatedAt is the time the event was published. Embedded in
	// the wire envelope so receivers can reason about staleness.
	CreatedAt time.Time

	// UserID is the owner — used to find matching webhooks. Routing
	// is strictly bounded to the owning user's subscribers; cross-
	// user delivery is impossible by construction.
	UserID string

	// AgentID, ConversationID, Labels are filter-matching keys. Each
	// is matched against the corresponding key in
	// WebhookFilters. Empty / nil here means "the event has no value
	// for this attribute" — see Publisher's null/empty semantics
	// (filters that REQUIRE a value while the event has none → skip).
	AgentID        string
	ConversationID string
	Labels         []string

	// MessageID is the originating message row, if any. May be empty
	// for events without a direct message backing (e.g.
	// email.review_requested before the held message gets promoted).
	MessageID string

	// Data is the event-specific payload. Wrapped in the envelope
	// {event, id, created_at, data} and serialized into the delivery
	// row's event_payload column.
	Data any
}

Event is the input to Publisher.Publish. Carries the routing keys (UserID, AgentID, ConversationID, Labels) needed to apply filters plus the Data payload that's serialized into the delivery row's event_payload JSONB.

MessageID is optional — set when the event has an originating message row. Persisted on the delivery row with ON DELETE SET NULL so explicit message deletion or trash purge doesn't orphan the delivery.

func NewEvent

func NewEvent(eventType, userID string, data any) Event

NewEvent constructs an Event with a fresh ID and now() timestamp. Trigger sites use this rather than building struct literals so the ID format stays consistent (evt_<32-hex>).

func (Event) AsEnvelope

func (e Event) AsEnvelope() Envelope

AsEnvelope returns the wire shape for serialization.

type FanOutArgs

type FanOutArgs struct {
	EventID string `json:"event_id"`
}

FanOutArgs drives one webhook fan-out: match the event user's enabled webhooks, insert the matching webhook_subscriber_deliveries rows, and enqueue their delivery jobs. Carries only the event id — the worker re-reads the durable webhook_events row (the three-layer pattern: the job is the trigger, the row is the source of truth), so args stay tiny and a schema change to the event never invalidates enqueued jobs.

This is the River replacement for the legacy webhookpub.OutboxWorker drain (LISTEN/NOTIFY + poll + SKIP-LOCKED lease). See docs/design/webhook-fanout-river-migration.md.

func (FanOutArgs) Kind

func (FanOutArgs) Kind() string

type FanOutEnqueuer

type FanOutEnqueuer interface {
	EnqueueFanOutTx(ctx context.Context, tx pgx.Tx, eventID string) (int64, error)
}

FanOutEnqueuer enqueues a webhook fan-out job within the caller's transaction and returns the river_job id. *FanOutJobs satisfies it. Injected (not imported) so the outbox writer stays decoupled from the River wiring — mirrors DeliveryEnqueuer.

type FanOutJobs

type FanOutJobs struct {
	// contains filtered or unexported fields
}

FanOutJobs is the webhook fan-out integration on the shared River client: a jobs.Registrar (contributes FanOutWorker + the reconcile periodic) plus the transactional enqueue entry point (EnqueueFanOutTx) that PublishTx / PublishBestEffortTx call in the event's own tx. The shared client is injected via SetEnqueuer after jobs.New builds it (two-phase wiring, same as webhookdelivery).

func NewFanOutJobs

func NewFanOutJobs(pool *pgxpool.Pool, identityStore identityReader, deliveryEnq DeliveryEnqueuer, metrics telemetry.Metrics) *FanOutJobs

NewFanOutJobs builds the integration with its dependencies (no client yet). pool backs the reconciler's scan; deliveryEnq is the SAME delivery enqueuer the legacy OutboxWorker uses — fan-out enqueues Layer-2→3 delivery jobs exactly as before.

func (*FanOutJobs) EnqueueFanOutTx

func (j *FanOutJobs) EnqueueFanOutTx(ctx context.Context, tx pgx.Tx, eventID string) (int64, error)

EnqueueFanOutTx enqueues a fan-out job WITHIN the caller's transaction — the outbox pattern between Layer 1 (webhook_events) and its fan-out: the event-row write and this job commit together, so a pending event can never exist without a job (modulo the best-effort publish path, which the reconciler backstops). Returns the river_job id for the caller to stamp on webhook_events.fanout_job_id. Routed to QueueWebhook.

func (*FanOutJobs) ReconcilePending

func (j *FanOutJobs) ReconcilePending(ctx context.Context, pool *pgxpool.Pool) (int, error)

ReconcilePending enqueues a fan-out job for every pending webhook_events row with no job yet (fanout_job_id IS NULL). Runs at startup (cutover) AND on the live schedule (FanOutReconcileWorker). Idempotent: the per-row FOR UPDATE + fanout_job_id IS NULL guard means a re-run (or a concurrent replica) never double-enqueues. Returns the number of events enqueued.

func (*FanOutJobs) RegisterJobs

func (j *FanOutJobs) RegisterJobs(w *river.Workers) []*river.PeriodicJob

RegisterJobs adds the FanOutWorker + the reconcile worker and returns the reconcile periodic. Implements jobs.Registrar.

func (*FanOutJobs) SetEnqueuer

func (j *FanOutJobs) SetEnqueuer(e jobs.Enqueuer)

SetEnqueuer injects the shared client so EnqueueFanOutTx can insert fan-out jobs.

type FanOutReconcileArgs

type FanOutReconcileArgs struct{}

FanOutReconcileArgs drives the periodic stranded-event reconciler.

func (FanOutReconcileArgs) Kind

func (FanOutReconcileArgs) Kind() string

type FanOutReconcileWorker

type FanOutReconcileWorker struct {
	river.WorkerDefaults[FanOutReconcileArgs]
	// contains filtered or unexported fields
}

FanOutReconcileWorker re-enqueues any pending event with no fan-out job. It is the LIVE backstop for the best-effort publish path (PublishBestEffortTx must not fail the caller's tx, so an event can commit with its enqueue lost) and any crash window between the event commit and the job insert — turning "recovered only on restart" into "recovered within fanOutReconcileInterval". Idempotent (fanout_job_id IS NULL guard). Mirrors webhookdelivery.ReconcileWorker.

func (*FanOutReconcileWorker) Work

type FanOutWorker

type FanOutWorker struct {
	river.WorkerDefaults[FanOutArgs]
	// contains filtered or unexported fields
}

FanOutWorker fans out one webhook_events row on River, replacing the legacy OutboxWorker drain. It re-reads the event by id, skips it if it is gone (30d GC) or no longer 'pending' (a duplicate at-least-once job — already fanned out), and otherwise runs the shared fanOutEventCore. Idempotent: the (event_id, webhook_id) unique index dedups delivery-row inserts and the status='pending' guard on the terminal UPDATE makes a re-run a no-op.

func NewFanOutWorker

func NewFanOutWorker(pool *pgxpool.Pool, identityStore identityReader, deliveryEnq DeliveryEnqueuer, metrics telemetry.Metrics) *FanOutWorker

NewFanOutWorker builds a FanOutWorker. RegisterJobs builds an identical one for the client; this is exported so tests can drive Work directly without a River client.

func (*FanOutWorker) Timeout

func (w *FanOutWorker) Timeout(*river.Job[FanOutArgs]) time.Duration

Timeout bounds a fan-out job. It is a handful of queries + N inserts — never the long synchronous drain the janitor's 60s-cut hazard came from — but a bounded timeout keeps a pathologically slow DB from pinning a QueueWebhook worker indefinitely.

func (*FanOutWorker) Work

func (w *FanOutWorker) Work(ctx context.Context, job *river.Job[FanOutArgs]) error

type FeatureFlag

type FeatureFlag interface {
	Enabled() bool
}

FeatureFlag is a boolean gate. Retained because the outbox drain (OutboxWorker) is constructed with one — StaticFlag(true) in production, since the outbox is now unconditional.

type Outbox

type Outbox interface {
	// PublishTx writes the event to webhook_events inside the
	// caller's transaction. Returns error so the caller can roll back
	// its business state if the outbox write fails.
	//
	// Used for PRE-side-effect triggers (email.received, future
	// email.bounced from SNS, email.review_requested, email.review_rejected).
	// If the outbox write fails the caller's tx rolls back; on
	// retry, the deterministic event id makes the second outbox
	// INSERT a no-op via ON CONFLICT (id) DO NOTHING.
	PublishTx(ctx context.Context, tx pgx.Tx, e Event) error

	// PublishBestEffortTx attempts the outbox write inside the
	// caller's transaction but never returns an error. On failure,
	// logs and lets the caller's tx commit anyway (a future slice may pipe these
	// to a webhook_publish_failures table — not yet built).
	//
	// Returns `wrote=true` iff the row was actually written (flag
	// enabled AND writeOutboxRow succeeded). The wrote signal is retained for
	// observability; with the outbox unconditional there is no legacy fallback
	// path left to gate.
	//
	// Used for POST-side-effect triggers (email.sent, email.review_approved)
	// where the irreversible action (SES.Send) has already happened
	// and rolling back the business state would orphan an SES
	// delivery.
	PublishBestEffortTx(ctx context.Context, tx pgx.Tx, e Event) (wrote bool)

	// DeleteExpiredWebhookEvents drops rows past their 30-day retention.
	// Called from the hourly cleanup loop in cmd/e2a/main.go.
	DeleteExpiredWebhookEvents(ctx context.Context) (int, error)

	// Enabled reports whether the outbox writes durable event rows for this
	// deployment. Now unconditional (StaticFlag(true) in production) — the events
	// API gates its list/get/redeliver endpoints on this, and the outbox writer
	// no-ops when false. Retained as a seam; there is no longer a legacy
	// fan-out path to fall back to.
	Enabled() bool

	// SetFanOutEnqueuer wires the River fan-out enqueuer (two-phase, called after the
	// shared client exists). When set (E2A_WEBHOOK_FANOUT_MODE=river), PublishTx /
	// PublishBestEffortTx enqueue a webhook_fanout job in the event's own tx and stamp
	// fanout_job_id — the River FanOutWorker then does the fan-out, replacing the
	// legacy in-process OutboxWorker drain. nil (default/legacy) ⇒ the event row is
	// written as before and the OutboxWorker fans it out via LISTEN/NOTIFY + poll.
	SetFanOutEnqueuer(e FanOutEnqueuer)
}

Outbox is the Stripe-tier publisher entry point. Triggers write the webhook_events row inside the same transaction as their business state (e.g. the messages row for email.received). A separate publisher worker (slice 2) consumes pending rows and fans out into webhook_subscriber_deliveries.

Outbox.PublishTx writes ONE row to webhook_events inside the trigger's transaction and arranges NOTIFY; the OutboxWorker (slice 2) fans it out into webhook_subscriber_deliveries and enqueues a River delivery job per row. This is now the sole event path — the legacy in-process fan-out publisher is retired and River is the sole delivery engine.

See docs/design/2026-06-01-stripe-tier-webhooks.md §4.2 and Appendix A.

func NewOutbox

func NewOutbox(pool *pgxpool.Pool, flag FeatureFlag) Outbox

NewOutbox constructs the Stripe-tier outbox writer. The FeatureFlag gates writes: when disabled, PublishTx is a no-op (returns nil with no DB write). Production passes StaticFlag(true) — the outbox is now the unconditional, sole event path.

type OutboxPublisher

type OutboxPublisher struct {
	// contains filtered or unexported fields
}

OutboxPublisher is the sole Publisher: it writes events to the durable outbox (webhook_events) in their own transaction. Used for post-side-effect standalone events that have no business tx to join — domain.sending_* (senderidentity), email.delivered/bounced/complained + domain.suppression_added (SNS delivery feedback), and hitlworker TTL-resolution events. Routing these through the outbox is what closes the webhook-delivery→River gap: those sources used to bypass webhook_events via the legacy publisher, so their delivery rows got no River job and were never delivered. Now they fan out + enqueue like every other event (via the outbox drain). Best-effort by the Publisher contract (fire-after-the-fact): a write failure is logged.

func NewOutboxPublisher

func NewOutboxPublisher(outbox Outbox, pool *pgxpool.Pool) *OutboxPublisher

NewOutboxPublisher builds the adapter. outbox must be the (now unconditional) webhook_events outbox; pool opens the per-event transaction.

func (*OutboxPublisher) Publish

func (p *OutboxPublisher) Publish(ctx context.Context, e Event)

type OutboxWorker

type OutboxWorker struct {
	// contains filtered or unexported fields
}

OutboxWorker drains webhook_events. For each pending row it reads enabled webhooks for the user, applies filter matching in Go, and inserts one webhook_subscriber_deliveries row per match, enqueuing a River delivery job for each in the same transaction. The River DeliverWorker then POSTs each row to the customer endpoint. See design §4.4 for the full architecture.

Wakeup paths:

  1. LISTEN webhook_events_new — dedicated connection, sub-50ms latency from trigger COMMIT to fan-out.
  2. 1s fallback poll — catches notifications missed during deploy or LISTEN reconnect.

Multi-replica safety:

  • FOR UPDATE SKIP LOCKED + next_poll_at bump in GetPending leases rows so concurrent replicas don't fan out the same event.
  • The per-row partial unique index on (event_id, webhook_id) in webhook_subscriber_deliveries is the backstop for lease-expiry races: a slow worker B can finish after a fast worker A's lease has expired and a new worker C has taken over; per-row ON CONFLICT DO NOTHING swallows the duplicates rather than aborting B's transaction.
  • The final UPDATE outbox row uses WHERE status='pending' so a stale-and-late worker can't overwrite a fast-finisher's matched_webhook_ids snapshot.

func NewOutboxWorker

func NewOutboxWorker(pool *pgxpool.Pool, store identityReader) *OutboxWorker

NewOutboxWorker constructs the legacy LISTEN/NOTIFY fan-out drain. Production wiring passes the real pool and identity.Store; tests can pass fakes.

func (*OutboxWorker) Start

func (w *OutboxWorker) Start(ctx context.Context)

Start blocks on ctx — call in its own goroutine. Spawns the LISTEN reader in a sibling goroutine and runs the tick loop on the calling goroutine. Both stop when ctx is cancelled.

func (*OutboxWorker) Tick

func (w *OutboxWorker) Tick(ctx context.Context)

Tick processes one batch of pending events. Exposed (not just the private processBatch) so integration tests can drive the worker synchronously instead of waiting on the timer.

func (*OutboxWorker) WithDeliveryEnqueuer

func (w *OutboxWorker) WithDeliveryEnqueuer(e DeliveryEnqueuer) *OutboxWorker

WithDeliveryEnqueuer wires River delivery (the production path). Without it, inserted Layer 2 rows have no delivery job — used only in tests that deliver pending rows by hand. DeliveryEnqueuer is defined in fanout_core.go (shared).

func (*OutboxWorker) WithMetrics

func (w *OutboxWorker) WithMetrics(m telemetry.Metrics) *OutboxWorker

WithMetrics swaps in a real metrics backend. Default is NoOp so tests don't have to wire anything.

type Publisher

type Publisher interface {
	Publish(ctx context.Context, e Event)
}

Publisher fires an Event at the webhook pipeline. There is exactly one implementation left — OutboxPublisher — which writes the event to the durable outbox (webhook_events) in its own transaction; the OutboxWorker drain then fans it out to matching subscribers (webhook_subscriber_deliveries) and the River DeliverWorker POSTs each row. The legacy in-process fan-out publisher (direct delivery-row inserts, drained by the hand-rolled SubscriberRetryWorker) is retired: River is the sole delivery engine and every event flows webhook_events → drain → River delivery.

Best-effort by contract (fire-after-the-fact): a write failure is logged, not returned, so a trigger's primary state change is never rolled back on a publish error. Callers that need atomicity join the message transaction via Outbox.PublishTx directly instead of going through this interface.

type StaticFlag

type StaticFlag bool

StaticFlag is a trivially-implemented FeatureFlag for tests and for production wiring that reads an env var at startup.

func (StaticFlag) Enabled

func (f StaticFlag) Enabled() bool

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL