Help us shape the future of Go! Take the 2026 Go Developer Survey

whimap

package
v1.150.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 23, 2026 License: Apache-2.0 Imports: 25 Imported by: 0

Documentation

Overview

Package whimap is the imap input of a webhook source (#2247): the platform polls a mailbox and files each new message from an allowed, authenticated sender as one event, written exactly as the HTTP receiver writes a request's events. Compaction into Parquet, the source's table, retention and alerts then treat it like any other source. Attachments are written as managed resources the event names.

Nothing here is a tool. The mail reaches a model or a script only through the source's table and the resources, like every webhook's data.

Index

Constants

View Source
const EventType = "mail"

EventType is the event_type of every message an imap source files.

View Source
const LockKeyspace int32 = 4713211

LockKeyspace is the advisory-lock space an imap source's poll is held in; the key within it is a hash of the source's name.

View Source
const Tick = 15 * time.Second

Tick is how often the poller looks for a source whose poll is due.

Variables

This section is empty.

Functions

func AttachmentsFolder

func AttachmentsFolder(source string) string

AttachmentsFolder is the library folder every attachment a source saved sits under, removed with the source.

func LockKey

func LockKey(source string) int32

LockKey is the key a source's poll is locked under.

Types

type Attachment

type Attachment struct {
	Name        string `json:"name"`
	ContentType string `json:"content_type"`
	Size        int64  `json:"size"`
	URI         string `json:"uri,omitempty"`
	ResourceID  string `json:"resource_id,omitempty"`
	Skipped     string `json:"skipped,omitempty"`
}

Attachment is one file a message carried: the resource it was saved as, or why it was not.

type Config

type Config struct {
	Sources    Sources
	Objects    Objects
	Bucket     string
	Recorder   Recorder
	RawWindows RawWindows
	Marks      MarkStore
	Files      Files
	Lock       Lock
	Replica    string
	Logger     *slog.Logger
	// Open opens a session; nil is imapmail.Open.
	Open func(ctx context.Context, a imapmail.Account) (*imapmail.Session, error)
	Now  func() time.Time
}

Config assembles a poller.

type External

type External struct {
	From    string   `json:"from"`
	To      []string `json:"to"`
	Cc      []string `json:"cc,omitempty"`
	Subject string   `json:"subject"`
	Body    string   `json:"body"`
}

External is what the message's sender wrote, cleaned of invisible and reordering characters, hidden HTML removed from the body.

type File

type File struct {
	Folder      string
	Name        string
	ContentType string
	Data        []byte
}

File is one attachment to save: the library folder it goes in, its name, type and bytes.

type Files

type Files interface {
	PutFile(ctx context.Context, src whsource.Source, f File) (uri, id string, err error)
}

Files saves an attachment as a managed resource in the source's library.

type Lock

type Lock func(ctx context.Context, source string) (release func(), ok bool, err error)

Lock takes the one-replica lock on a source's poll, reporting whether it was free.

type Mailbox

type Mailbox struct {
	// contains filtered or unexported fields
}

Mailbox is what the admin service asks of an imap source: whether its mailbox can be read as configured, and how each folder's polls are going.

func NewMailbox

func NewMailbox(marks *Marks) *Mailbox

NewMailbox returns the admin side of the imap input.

func (*Mailbox) Check

func (m *Mailbox) Check(ctx context.Context, src whsource.Source) error

Check logs in and opens each folder read-only, so a source whose login or folders are wrong is refused when it is saved rather than failing every poll.

func (*Mailbox) Marks

func (m *Mailbox) Marks(ctx context.Context, source string) ([]Mark, error)

Marks lists a source's folders as its polls left them.

type Mark

type Mark struct {
	Source      string `json:"-"`
	Folder      string `json:"folder"`
	UIDValidity uint32 `json:"uid_validity"`
	LastUID     uint32 `json:"last_uid"`
	Stored      int64  `json:"stored"`
	// HiddenNotAllowed and HiddenUnauthenticated count the messages the
	// sender rules withheld. Nothing else of them is kept.
	HiddenNotAllowed      int64      `json:"hidden_not_allowed"`
	HiddenUnauthenticated int64      `json:"hidden_unauthenticated"`
	LastPolledAt          *time.Time `json:"last_polled_at,omitempty"`
	LastError             string     `json:"last_error,omitempty"`
}

Mark is where one folder of a source has been read to, and what its polls found.

type MarkStore

type MarkStore interface {
	Get(ctx context.Context, source, folder string) (Mark, error)
	Put(ctx context.Context, mk Mark) error
}

MarkStore keeps each folder's watermark.

type Marks

type Marks struct {
	// contains filtered or unexported fields
}

Marks stores the marks in webhook_imap_marks.

func NewMarks

func NewMarks(db *sql.DB) *Marks

NewMarks returns the store, nil without a database.

func (*Marks) Get

func (m *Marks) Get(ctx context.Context, source, folder string) (Mark, error)

Get returns a folder's mark, zero when it has never been read.

func (*Marks) List

func (m *Marks) List(ctx context.Context, source string) ([]Mark, error)

List returns a source's marks, by folder.

func (*Marks) Put

func (m *Marks) Put(ctx context.Context, mk Mark) error

Put writes a folder's mark.

type Objects

type Objects interface {
	PutObject(ctx context.Context, bucket, key string, data []byte, contentType string) error
}

Objects writes a segment to the managed-resources bucket.

type Payload

type Payload struct {
	Folder      string `json:"folder"`
	UID         uint32 `json:"uid"`
	UIDValidity uint32 `json:"uid_validity"`
	MessageID   string `json:"message_id,omitempty"`
	InReplyTo   string `json:"in_reply_to,omitempty"`
	// References are the thread's earlier Message-IDs.
	References []string `json:"references,omitempty"`
	Date       string   `json:"date,omitempty"`
	Arrived    string   `json:"arrived"`
	// FromAddress is the sender's address as the From header names it, and
	// SenderAuth what the receiving server recorded of it.
	FromAddress string `json:"from_address"`
	SenderAuth  string `json:"sender_auth"`
	Size        int64  `json:"size"`
	BodyFormat  string `json:"body_format"`
	// BodySkipped says why the body was not read, empty when it was.
	BodySkipped string       `json:"body_skipped,omitempty"`
	Attachments []Attachment `json:"attachments"`
	External    External     `json:"external"`
	Notice      string       `json:"notice"`
}

Payload is one message as its event's payload: what the platform read of it and, under External, everything its sender wrote.

type Poller

type Poller struct {
	// contains filtered or unexported fields
}

Poller polls every imap source on its interval.

func New

func New(cfg Config) *Poller

New returns a poller, nil when a dependency is missing.

func (*Poller) Pass

func (p *Poller) Pass(ctx context.Context) error

Pass polls every imap source whose interval has passed.

func (*Poller) Poll

func (p *Poller) Poll(ctx context.Context, src whsource.Source) error

Poll reads every folder of one source once.

func (*Poller) Start

func (p *Poller) Start(ctx context.Context)

Start runs the poll loop until Stop.

func (*Poller) Stop

func (p *Poller) Stop()

Stop ends the loop and waits for a poll in progress.

type RawWindows

type RawWindows interface {
	EnsureRawWindow(ctx context.Context, src whsource.Source, start time.Time) error
}

RawWindows registers a window's raw partition so its events are queryable before compaction.

type Recorder

type Recorder interface {
	MarkSegment(ctx context.Context, source string, start time.Time, length time.Duration, bytes int64) error
}

Recorder records a window a segment was written to.

type Sources

type Sources interface {
	List(ctx context.Context) ([]whsource.Source, error)
}

Sources lists the stored sources, secrets decrypted.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL