Help us shape the future of Go! Take the 2026 Go Developer Survey

whsource

package
v1.152.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 29, 2026 License: Apache-2.0 Imports: 16 Imported by: 0

Documentation

Overview

Package whsource defines an inbound webhook source: the administrator-managed record that says who may post to /hooks/{name}, how a request proves it, how the body becomes events, and how long what lands is kept (#1870).

It holds the model, its defaults and validation, and the PostgreSQL store. The receiver, the compactor and the admin API all read a Source through this package; none of them owns its shape.

Index

Constants

View Source
const (
	InputHTTP = "http"
	InputIMAP = "imap"
)

The inputs a source takes events from (#2247). An HTTP source is posted to; an IMAP source is a mailbox the platform polls, each message one event. Everything after the input (segments, compaction, the table, retention, alerts) is the same for both.

View Source
const (
	IMAPTLS      = "tls"
	IMAPStartTLS = "starttls"
)

The ways an IMAP source reaches its server. Plain IMAP is not one of them: the password would cross the network in the clear.

View Source
const (
	DefaultIMAPPollSeconds        = 60
	DefaultIMAPMaxMessageBytes    = 10 << 20
	DefaultIMAPMaxAttachmentBytes = 25 << 20
)

Defaults an IMAP source gets for every setting it leaves at zero.

View Source
const (
	PresetStandardWebhooks = hmacsig.PresetStandardWebhooks
	PresetGitHub           = hmacsig.PresetGitHub
	PresetStripe           = hmacsig.PresetStripe
	PresetPlatform         = hmacsig.PresetPlatform
	PresetSlack            = hmacsig.PresetSlack
	PresetZoom             = hmacsig.PresetZoom
	PresetTeams            = hmacsig.PresetTeams
	PresetShopify          = hmacsig.PresetShopify
	PresetLinear           = hmacsig.PresetLinear
	PresetGitLab           = hmacsig.PresetGitLab
	PresetJira             = hmacsig.PresetJira
	PresetMattermost       = "mattermost"
	PresetDiscord          = "discord"
	PresetMonday           = "monday"
	PresetMicrosoftGraph   = "microsoft_graph"
)

Presets a source names its sender by (#2223). Each sets the mode, the headers, the signed bytes, the encoding and the setup check that sender uses; a setting the source sets itself is kept. The HMAC conventions are hmacsig's, the ones the platform's outbound hmac connections sign with.

View Source
const (
	// AuthHMAC checks a signature over the body, or over timestamp + "." +
	// body, carried in a header.
	AuthHMAC = "hmac"
	// AuthHeaderToken compares a header's value with the secret.
	AuthHeaderToken = "header_token"
	// AuthBasic checks HTTP Basic credentials: a username and the secret as
	// the password.
	AuthBasic = "basic"
	// AuthPathToken takes the secret as an extra path segment,
	// /hooks/{name}/{token}, for a sender that can set nothing but a URL.
	AuthPathToken = "path_token"
	// AuthEd25519 checks an Ed25519 signature over the signed bytes with the
	// sender's public key, which the source stores as its secret, in hex
	// (#2223: Discord).
	AuthEd25519 = "ed25519"
	// AuthBodyToken compares a value read from the body, at TokenPath, with
	// the secret (#2223: Mattermost, Microsoft Graph's clientState).
	AuthBodyToken = "body_token"
	// AuthJWT checks an HS256 JSON Web Token in a header, signed with the
	// secret, honoring its exp (#2223: monday.com).
	AuthJWT = "jwt"
)

Auth modes a source authenticates a request with.

View Source
const (
	AlgorithmSHA256 = hmacsig.AlgorithmSHA256
	AlgorithmSHA1   = hmacsig.AlgorithmSHA1
	AlgorithmSHA512 = hmacsig.AlgorithmSHA512

	EncodingHex    = hmacsig.EncodingHex
	EncodingBase64 = hmacsig.EncodingBase64

	// SignedBody signs the raw body.
	SignedBody = hmacsig.SignedBody
	// SignedTimestampBody signs the timestamp header's value, a ".", and
	// the raw body, which is what makes a replayed request with an old
	// timestamp fail even though its signature once verified.
	SignedTimestampBody = hmacsig.SignedTimestampBody
	// SignedIDTimestampBody signs the id header's value, a ".", the
	// timestamp, a ".", and the body: the Standard Webhooks form.
	SignedIDTimestampBody = hmacsig.SignedIDTimestampBody

	// HeaderFormatStripe reads the signature header as Stripe writes it,
	// "t=<timestamp>,v1=<signature>", with the timestamp inside it.
	HeaderFormatStripe = hmacsig.FormatStripe
)

HMAC settings a source chooses between. They are hmacsig's, which the platform's outbound hmac connections sign with too, so a source and a connection configured with the same values agree (#1996).

View Source
const (
	HandshakeNone = "none"
	// HandshakeCloudEvents answers the CloudEvents HTTP webhook
	// abuse-protection OPTIONS request, which a sender that implements it
	// sends before it delivers anything.
	HandshakeCloudEvents = "cloudevents"
	// HandshakeChallengeEcho answers a JSON body carrying a challenge with
	// that challenge, before verification and before anything is stored
	// (#2223: Slack's url_verification, monday.com).
	HandshakeChallengeEcho = "challenge_echo"
	// HandshakeZoom answers Zoom's endpoint.url_validation with the plain
	// token and its HMAC under the source's secret. The request is verified
	// first: an unverified answer would hand anyone the HMAC of any text.
	HandshakeZoom = "zoom"
	// HandshakeDiscord answers a verified PING with a PONG.
	HandshakeDiscord = "discord"
	// HandshakeGraphValidation answers a request carrying a validationToken
	// query parameter with the token, as plain text (Microsoft Graph).
	HandshakeGraphValidation = "graph_validation"
)

Handshakes a source can answer.

View Source
const (
	DefaultMaxBodyBytes        int64 = 1 << 20
	DefaultFlushMaxEvents            = 5000
	DefaultFlushMaxInterval          = time.Second
	DefaultFlushMaxBytes       int64 = 8 << 20
	DefaultBufferLimit               = 50000
	DefaultRawRetentionDays          = 1
	DefaultCompactEveryMinutes       = 60
	DefaultCompactedDays             = 400
	DefaultTolerance                 = 5 * time.Minute
)

Defaults a source gets for every setting it leaves at zero.

View Source
const AuthIMAP = "imap"

AuthIMAP is the auth mode of an IMAP source: Username and Secret are the mailbox's login, Secret encrypted at rest like every source secret.

View Source
const DefaultChallengeField = "challenge"

DefaultChallengeField is the body field a challenge_echo source reads.

View Source
const DefaultJWTHeader = "Authorization"

DefaultJWTHeader is the header a jwt source reads its token from.

View Source
const FormPrefix = "form-"

FormPrefix starts the name of every form's source.

View Source
const InputForm = "form"

InputForm is a source whose events are a form's accepted submissions (#2178). The platform creates one when a form script is bound to an endpoint; it is never posted to, and no administrator creates one by hand.

View Source
const SecretBase64 = hmacsig.SecretBase64

SecretBase64 reads an hmac source's secret as base64, the key being the bytes it decodes to.

Variables

View Source
var ErrExists = errors.New("a webhook source with that name already exists")

ErrExists is returned when a source is created under a name already taken.

View Source
var ErrInvalid = errors.New("invalid webhook source")

ErrInvalid wraps every refusal Validate returns, so a surface can answer 400 for any of them and show the sentence as written.

View Source
var ErrNotFound = errors.New("webhook source not found")

ErrNotFound is returned for a name no source is stored under.

Functions

func AuthModes added in v1.152.2

func AuthModes() []string

AuthModes lists the auth modes a source can name, in the order the portal offers them. Every mode has a check in modeChecks, which a test holds them to, and #2354's criterion sends a request in each.

func FormSourceName added in v1.152.0

func FormSourceName(endpoint string) string

FormSourceName is the name of the source a form bound at endpoint gets: FormPrefix and the endpoint's name with each "_" read as "-", which is the one character an endpoint name takes that a source name does not.

func KnownPreset added in v1.152.1

func KnownPreset(name string) bool

KnownPreset reports whether name is a preset a source can name.

func PresetNames added in v1.152.1

func PresetNames() []string

PresetNames lists the presets a source can name, in the order the portal offers them.

func Refusal

func Refusal(format string, args ...any) error

Refusal is invalid for a caller that refuses a source on grounds this package cannot see, such as a persona the platform does not define.

func TableName

func TableName(source string) string

TableName is the name readers query the source by, in the scratch schema of its connection.

func Validate

func Validate(s Source) error

Validate refuses a source that could not be served as configured. It is called on the source with its defaults applied, so a zero setting has already been replaced and what is checked is what the receiver would use.

Types

type Auth

type Auth struct {
	Mode string `json:"mode"`
	// Preset names a sender's whole convention (#2223); the settings it
	// leaves unset are filled from it.
	Preset         string    `json:"preset,omitempty"`
	Secret         string    `json:"secret,omitempty"`
	PreviousSecret string    `json:"previous_secret,omitempty"`
	PreviousUntil  time.Time `json:"previous_until,omitzero"`

	// HMAC.
	Algorithm        string `json:"algorithm,omitempty"`
	SignatureHeader  string `json:"signature_header,omitempty"`
	Encoding         string `json:"encoding,omitempty"`
	Prefix           string `json:"prefix,omitempty"`
	TimestampHeader  string `json:"timestamp_header,omitempty"`
	ToleranceSeconds int    `json:"tolerance_seconds,omitempty"`
	Signed           string `json:"signed,omitempty"`
	// IDHeader carries the delivery id id.timestamp.body signs.
	IDHeader string `json:"id_header,omitempty"`
	// HeaderFormat is empty for a prefix and a signature, or "stripe".
	HeaderFormat string `json:"header_format,omitempty"`
	// HeaderPattern lays the signature header out over {signature} and
	// {timestamp}, as in "{timestamp}|{signature}" (#2223).
	HeaderPattern string `json:"header_pattern,omitempty"`
	// SecretEncoding is empty for the secret as written, or "base64".
	SecretEncoding string `json:"secret_encoding,omitempty"`
	// TimestampPath reads the timestamp from the body rather than a header,
	// in seconds or milliseconds (Linear's webhookTimestamp).
	TimestampPath string `json:"timestamp_path,omitempty"`

	// BodyToken: where in the body the token is. A path through "[*]"
	// requires every element to carry it.
	TokenPath string `json:"token_path,omitempty"`

	// HeaderToken.
	Header string `json:"header,omitempty"`

	// Basic.
	Username string `json:"username,omitempty"`
}

Auth is how a request to the source proves it came from the sender.

Secret and PreviousSecret are plaintext in memory and encrypted at rest; the admin API never returns either. PreviousSecret is accepted until PreviousUntil, which is how a secret is rotated without a window in which the sender's requests fail.

func (Auth) ChecksTimestamp added in v1.152.1

func (a Auth) ChecksTimestamp() bool

ChecksTimestamp reports whether a signed request's timestamp is checked against the tolerance window: one read from a header, from the body, or from the signature header itself (the Stripe format, or a header pattern naming {timestamp}).

func (Auth) Rotate

func (a Auth) Rotate(secret string, overlap time.Duration, now time.Time) Auth

Rotate replaces the secret, keeping the old one valid for overlap. An overlap of zero ends the old secret now.

func (Auth) Scheme added in v1.138.3

func (a Auth) Scheme() hmacsig.Scheme

Scheme is the signing convention an hmac source verifies.

func (Auth) Secrets

func (a Auth) Secrets(now time.Time) []string

Secrets returns the secrets a request may be verified with at now: the current one, and the previous one while its overlap lasts.

func (Auth) Tolerance

func (a Auth) Tolerance() time.Duration

Tolerance is how far a signed timestamp may be from the receiver's clock.

type Config

type Config struct {
	// Input is "http" (the default, posted to), "imap" (a polled mailbox,
	// configured under IMAP) or "form" (a form's submissions, under Form).
	Input string `json:"input,omitempty"`
	IMAP  *IMAP  `json:"imap,omitempty"`
	Form  *Form  `json:"form,omitempty"`

	Handshake string `json:"handshake,omitempty"`
	// ChallengeField is the body field a challenge_echo handshake reads;
	// empty is "challenge".
	ChallengeField string `json:"challenge_field,omitempty"`
	MaxBodyBytes   int64  `json:"max_body_bytes,omitempty"`
	Split          string `json:"split,omitempty"`
	EventIDPath    string `json:"event_id_path,omitempty"`
	EventTypePath  string `json:"event_type_path,omitempty"`
	KeyPath        string `json:"key_path,omitempty"`

	FlushMaxEvents     int   `json:"flush_max_events,omitempty"`
	FlushMaxIntervalMS int64 `json:"flush_max_interval_ms,omitempty"`
	FlushMaxBytes      int64 `json:"flush_max_bytes,omitempty"`
	BufferLimit        int   `json:"buffer_limit,omitempty"`

	// RateLimitPerMinute is zero for no limit. The limit is on the source,
	// not on a client address: a sender's requests arrive from whatever
	// addresses its infrastructure has.
	RateLimitPerMinute int `json:"rate_limit_per_minute,omitempty"`
	RateLimitBurst     int `json:"rate_limit_burst,omitempty"`

	// Persona is the persona whose members see the source's compacted windows
	// in Resources and search. Empty is the administrator persona, which keeps
	// them to administrators. Querying
	// the table is governed by the Trino connection, not by this. It is set
	// when the source is created and does not change: windows already written
	// stay where they were written.
	Persona string `json:"persona,omitempty"`

	// CompactEveryMinutes is the length of the window a source's events are
	// partitioned and compacted by: 60, the default, compacts each hour once
	// it has ended; a shorter window compacts sooner, into more files. It
	// divides an hour evenly, so windows start on the hour and an hour is
	// always a whole number of them.
	CompactEveryMinutes int `json:"compact_every_minutes,omitempty"`

	// RawRetentionDays is how long a raw segment is kept once the window it
	// belongs to is compacted. A compacted window's events are all in its
	// Parquet file, and a recompaction reads that file rather than the
	// segments, so they are a safety margin and nothing else; every day kept
	// is objects the raw table and retention list (#2134).
	RawRetentionDays int `json:"raw_retention_days,omitempty"`
	// CompactedRetentionDays is how long a compacted window is kept. Nil is
	// the default; zero keeps it forever.
	CompactedRetentionDays *int `json:"compacted_retention_days,omitempty"`

	// Index is how search indexes the source's compacted windows (#2111):
	// "full", "metadata" or "none". Empty leaves it to the operator's
	// resources.index rules, which can match these writes by origin webhook.
	// It applies to windows written from the next compaction on.
	Index string `json:"index,omitempty"`
}

Config is everything about a source other than its authentication.

func (Config) Challenge added in v1.152.1

func (c Config) Challenge() string

Challenge is the body field a challenge_echo handshake reads.

func (Config) CompactedRetention

func (c Config) CompactedRetention() time.Duration

CompactedRetention is how long a compacted window is kept, and zero for forever.

func (Config) FlushInterval

func (c Config) FlushInterval() time.Duration

FlushInterval is the longest an event waits in the buffer before its segment is written.

func (Config) RawRetention

func (c Config) RawRetention() time.Duration

RawRetention is how long a compacted window's raw segments are kept.

func (Config) Window

func (c Config) Window() time.Duration

Window is the length of the window the source's events are partitioned and compacted by.

type Encryptor

type Encryptor interface {
	Encrypt(plaintext string) (string, error)
	Decrypt(ciphertext string) (string, error)
}

Encryptor encrypts and decrypts one secret. fieldcrypt.RestFieldEncryptor satisfies it; a nil one stores secrets as written.

type Form added in v1.152.0

type Form struct {
	// ScriptID is the form script's id: the source follows the script, so an
	// endpoint renamed later still stores into the source made for it.
	ScriptID string `json:"script_id"`
	// Endpoint is the endpoint the form was bound to when its source was
	// made, which its name was derived from.
	Endpoint string `json:"endpoint"`
	// DecryptPersonas are the personas a script run may decrypt the form's
	// sensitive fields as (platform.decrypt, #2179); an administrator always
	// may, and so does the owner on the submissions view.
	DecryptPersonas []string `json:"decrypt_personas,omitempty"`
}

Form is the form a form source collects for.

type IMAP added in v1.150.0

type IMAP struct {
	Host string `json:"host"`
	Port int    `json:"port,omitempty"`
	// TLS is "tls" (IMAPS, the default) or "starttls".
	TLS string `json:"tls,omitempty"`
	// Folders are the folders read, each with its own watermark. Required:
	// there is no all-folders default.
	Folders []string `json:"folders"`
	// AllowSenders are the addresses and @domain entries whose mail becomes
	// an event. Mail from anyone else is counted and never stored.
	AllowSenders []string `json:"allow_senders"`
	// RequireAuthPass trusts an allowed sender only when the receiving
	// server recorded an aligned SPF, DKIM or DMARC pass. Nil is true.
	RequireAuthPass *bool `json:"require_auth_pass,omitempty"`
	// TrustedAuthServID is the receiving server's authserv-id; only an
	// Authentication-Results header carrying it is read.
	TrustedAuthServID string `json:"trusted_authserv_id,omitempty"`
	// TLSCABundlePEM are certificates to trust for a private CA.
	TLSCABundlePEM     string `json:"tls_ca_bundle_pem,omitempty"`
	PollSeconds        int    `json:"poll_seconds,omitempty"`
	MaxMessageBytes    int64  `json:"max_message_bytes,omitempty"`
	MaxAttachmentBytes int64  `json:"max_attachment_bytes,omitempty"`
}

IMAP is what an IMAP source polls: a server, the folders it reads, and whose mail becomes events.

func (IMAP) AuthRequired added in v1.150.0

func (m IMAP) AuthRequired() bool

AuthRequired reports whether an IMAP source requires an authenticated sender.

type Source

type Source struct {
	Name    string `json:"name"`
	Enabled bool   `json:"enabled"`
	Auth    Auth   `json:"auth"`
	Config  Config `json:"config"`
	// Connection is the Trino connection whose scratch catalog holds the
	// source's table. That catalog must read the managed-resources store,
	// because both the raw segments and the compacted windows are written
	// there.
	Connection string    `json:"connection"`
	CreatedBy  string    `json:"created_by"`
	CreatedAt  time.Time `json:"created_at"`
	UpdatedAt  time.Time `json:"updated_at"`
}

Source is one inbound webhook source.

func (Source) CompactedTableName

func (s Source) CompactedTableName() string

CompactedTableName is the table over the compacted windows, underneath the view.

func (Source) IsForm added in v1.152.0

func (s Source) IsForm() bool

IsForm reports whether the source collects a form's submissions.

func (Source) IsIMAP added in v1.150.0

func (s Source) IsIMAP() bool

IsIMAP reports whether the source polls a mailbox rather than being posted to.

func (Source) RawTableName

func (s Source) RawTableName() string

RawTableName is the table over the raw segments, underneath the view.

func (Source) TableName

func (s Source) TableName() string

TableName is the view readers name for this source.

func (Source) WithDefaults

func (s Source) WithDefaults() Source

WithDefaults returns the source with every zero setting replaced by its default.

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store persists sources in webhook_sources.

func NewStore

func NewStore(db *sql.DB, enc Encryptor) *Store

NewStore creates a source store. enc may be nil.

func (*Store) Create

func (s *Store) Create(ctx context.Context, src Source) error

Create inserts a new source, or returns ErrExists.

func (*Store) Delete

func (s *Store) Delete(ctx context.Context, name string) error

Delete removes a source. Its windows, counts and rejections go with it.

func (*Store) Get

func (s *Store) Get(ctx context.Context, name string) (Source, error)

Get returns one source, or ErrNotFound.

func (*Store) List

func (s *Store) List(ctx context.Context) ([]Source, error)

List returns every source in name order, secrets decrypted.

func (*Store) Update

func (s *Store) Update(ctx context.Context, src Source) (time.Time, error)

Update replaces a source's settings and returns the time it stamped on the row. The name, the connection and the creator are not changed: the table was created on that connection under a name derived from the source's.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL