worker

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 19, 2026 License: AGPL-3.0 Imports: 24 Imported by: 0

Documentation

Overview

Package worker runs one supervisor per mail account: it connects, watches the account's folder and feeds each arrival to the pipeline, strictly in UID order.

Index

Constants

View Source
const (
	CheckRunning = "running" // the real run-through is still going
	CheckReady   = "ready"   // it is through; the rows are kept, waiting for Sort or Discard
	CheckFailed  = "failed"  // the mail server or the model failed
	CheckStale   = "stale"   // the rules changed since the check: Sort is refused until a new one
)

Check statuses, as the API reports them.

View Source
const (
	StatusLive         = "live"
	StatusReconnecting = "reconnecting"
	StatusAuthFailed   = "auth_failed"
	// StatusCertChanged: the server presents a certificate that is not the one accepted for
	// the account, or, with none accepted, one the system does not trust. It waits until the
	// person accepts the new certificate.
	StatusCertChanged = "cert_changed"
	StatusError       = "error"
	StatusPaused      = "paused" // set by the user; Run is not started for a paused account
	// StatusReconnectNeeded: a mailbox that signs in with OAuth (one-click sign-in) can no
	// longer get in (mail.ErrReconnect): the person revoked access, or the provider refused
	// the token; or the server refused the password of a mailbox a module's sign-in form
	// connected. It waits until the person signs in to the provider again.
	StatusReconnectNeeded = "reconnect_needed"
	// StatusKeyUnavailable: the key service that holds the tenant's key (ext.Module.Keys)
	// cannot be reached, so the mailbox's password cannot be opened (crypto.ErrKeyUnavailable).
	// Unlike auth_failed nothing is wrong with the mailbox: it is retried, with the same
	// growing delay as reconnecting, until the service answers.
	StatusKeyUnavailable = "key_unavailable"
)

Account statuses the supervisor sets, as stored in accounts.status.

View Source
const (
	// NoticeDays is the least time between a warning and the deletion it announces.
	NoticeDays = 7
)

The end-of-plan schedule, in days from the moment a tenant's plan ended (store.PlanState .EndedAt). Warning 1 goes on the day; warning 2 NoticeDays before the mailbox passwords and tokens and the model keys go (store.EndedSecretsDays); warning 3 NoticeDays before everything else goes (store.EndedRemovalDays).

View Source
const QueueSize = 1000

QueueSize is how many arrivals may wait between the watcher and the processor.

Variables

View Source
var ErrCleanupRunning = errors.New("a cleanup is already running for this account")

ErrCleanupRunning means the account is already being sorted.

View Source
var ErrNotConnected = errors.New("account is not connected")

ErrNotConnected means the account has no open connection right now.

Functions

This section is empty.

Types

type Check

type Check struct {
	ID          string
	AccountID   int64
	Folder      string
	Since       int64   // 0 = all mail
	Limit       int     // 0 = no limit
	RuleIDs     []int64 // the rules the check was limited to; nil = every enabled rule (MAI-43)
	RouteOnly   []int64 // the unpicked rules a sender rule routed to as the check began
	Fingerprint string
	StartedAt   int64
	// contains filtered or unexported fields
}

Check is one account's current cleanup check and its progress. Read it with State.

func (*Check) MarkStale

func (c *Check) MarkStale()

MarkStale moves a ready check to stale: the rules changed since it ran, so Sort must wait for a new check. It leaves a running or failed check as it is.

func (*Check) SetExclude

func (c *Check) SetExclude(ids []int) bool

SetExclude saves the user's selection: the unticked row indices, stored sorted and without repeats. It reports false when the check is not ready, since a running, failed or stale check has no table to tick. Whether the indices name selectable rows is the caller's to check.

func (*Check) Started added in v0.1.3

func (c *Check) Started() CheckState

Started returns the check as it was when it began: running, with no progress. The answer to the request that started it uses this, so it never depends on how far the run in the background got in the meantime.

func (*Check) State

func (c *Check) State() CheckState

State returns the whole check, rows included.

type CheckFunc

type CheckFunc func(ctx context.Context, report func(composer.CheckProgress)) ([]composer.CheckRow, error)

CheckFunc runs the read-only check and returns its rows. The daemon calls it on the account's background context, so a check outlives the browser request that began it.

type CheckState

type CheckState struct {
	ID          string
	AccountID   int64
	Folder      string
	Since       int64
	Limit       int
	RuleIDs     []int64 // nil = every enabled rule
	RouteOnly   []int64 // the unpicked rules a sender rule routed to as the check began
	Fingerprint string
	Status      string
	Done        int
	Total       int
	Matched     int // emails in the chosen range before the limit; more than Total when it cut the range
	ModelCalls  int
	Tokens      int
	CostUSD     float64
	Error       string
	Rows        []composer.CheckRow
	// Exclude is the saved selection: the unticked selectable row indices, ascending. It
	// lives only as long as the check (MAI-44) and is never written anywhere else.
	Exclude []int
}

CheckState is a check read back: a copy safe to hand around and render. Rows are present only when the check is ready or stale.

type CheckStore

type CheckStore struct {
	// contains filtered or unexported fields
}

CheckStore keeps the current cleanup check of each account in the daemon's memory, one per account, guarded by one mutex.

MAI-44 knowingly overrides the working agreement that no state lives in process memory a restart or a second instance would break. Tilak decided (2026-10-06) that a cleanup check — the real, paid run-through of a mailbox that Sort then applies without asking the model again — is held here, not in the database: one check per account, a new check replacing (and cancelling) the previous one, deleted once Sort has used it or the user discards it, and lost on a daemon restart (accepted, no time-based expiry). Its rows carry display fields (sender, subject) that must never be logged or written to the database.

func NewCheckStore

func NewCheckStore() *CheckStore

NewCheckStore returns an empty store.

func (*CheckStore) All added in v0.1.2

func (s *CheckStore) All() []*Check

All returns every account's current check, by account id.

func (*CheckStore) Discard

func (s *CheckStore) Discard(accountID int64) *Check

Discard removes and cancels the account's current check and returns it, or nil.

func (*CheckStore) Get

func (s *CheckStore) Get(accountID int64) *Check

Get returns the account's current check, or nil.

func (*CheckStore) Take

func (s *CheckStore) Take(accountID int64, id string) *Check

Take removes the account's current check, but only when it is the one with id, so a newer check that replaced it since is left alone. It returns the removed check, or nil. Sort calls it once it has used a check's rows.

type Cleanup

type Cleanup struct {
	AccountID int64
	Folder    string
	Since     time.Time // only mail received from this time on; zero = all of it
	Limit     int       // only the newest Limit emails; 0 = no limit
	// RuleIDs limits the check to these rules, checked in their usual order; nil = every
	// enabled rule. Sender rules apply either way (MAI-43).
	RuleIDs []int64
	// RouteOnly are the rules outside RuleIDs that a sender rule routed mail to when the
	// check began; nil when every rule is checked. A check stays in step with their content.
	RouteOnly []int64
}

Cleanup describes which mail a cleanup check, and the Sort that follows it, covers.

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager runs the supervisors and is how the action executor reaches an account (it implements actions.Accounts).

func (*Manager) Checks

func (m *Manager) Checks() *CheckStore

Checks is the account cleanup checks held in the daemon's memory (MAI-44). It is created on first use, so a Manager needs no constructor.

func (*Manager) Lock

func (m *Manager) Lock(accountID int64) (unlock func())

Lock waits until the account's processor is between messages and holds it there until unlock is called. Undo and corrections move mail back into the watched folder; holding the lock until that is on record keeps the processor from sorting it again as new mail.

func (*Manager) Mailbox

func (m *Manager) Mailbox(accountID int64) (mail.Mailbox, error)

Mailbox returns the account's open connection, or ErrNotConnected.

func (*Manager) Sort

func (m *Manager) Sort(ctx context.Context, sr SortRun) (store.Batch, error)

Sort starts applying a check's kept rows in the background and returns the run's batch at once; the run goes on until it is through, the account's supervisor stops or the daemon does. Each email is verified still where the check found it (BODY.PEEK, never marking it read) and then its saved decision is handed to the executor, exactly as a live email's is, so dry-run is honoured and every action is on record; all of them belong to the one batch, which makes the run undoable as a whole. One Sort per account at a time. ctx only covers opening the batch.

func (*Manager) SortAll added in v0.1.2

func (m *Manager) SortAll(ctx context.Context, runs []SortRun) ([]store.Batch, error)

SortAll starts the Sort of several mailboxes together, one batch each, and returns the batches in the order of runs. They are all started or none is: when a mailbox is not connected or already being sorted, nothing starts and the error says so. More than one batch makes a manual run over several mailboxes: they share a store.Batch.RunID, so they read back as one run, and each is undone on its own like any cleanup batch (MAI-43). A mailbox may be named only once.

func (*Manager) Start

func (m *Manager) Start(ctx context.Context, s *Supervisor)

Start runs a supervisor until ctx is done or Stop is called for its account. An account that is already running is stopped first. An account whose secret was deleted (store.Account.SecretGone) is only stopped: it cannot log in until its owner enters a new password, which starts it again.

func (*Manager) StartCheck

func (m *Manager) StartCheck(c Cleanup, fingerprint string, run CheckFunc) (*Check, error)

StartCheck begins a cleanup check of an account in the background, keyed in the store as the account's current check (cancelling any previous one). It returns the check at once, running; progress and the finished rows arrive as check.progress events and through CheckStore.Get. fingerprint is the signature of the rules the check was run against, which Sort compares with the rules in force to refuse a stale check.

func (*Manager) Stop

func (m *Manager) Stop(accountID int64)

Stop ends an account's supervisor and waits until it has: mail already queued gets the supervisor's drain time to finish. Stopping an account that is not running does nothing.

func (*Manager) Wait

func (m *Manager) Wait()

Wait blocks until every supervisor and every cleanup run has stopped.

type PlanEnd added in v0.3.0

type PlanEnd struct {
	Store *store.Store
	// Stop stops an account's supervisor and waits until it has; stopping one that is not
	// running does nothing. Start starts one again, for a mailbox stopped as a plan was
	// bought at that moment (ext.Host.SetPlan starts them once the plan is stored, which may
	// be just before the job stopped them).
	Stop  func(accountID int64)
	Start func(store.Account)
	Hub   *events.Hub // tells the tenant's open screens its mailboxes lost their secrets; may be nil
	// Sender sends the warnings (the summary email's mail server); the daemon does not start
	// a build that sells plans without one. A warning that reaches no member is not recorded
	// and is tried again next tick, so no deletion comes before a warning was delivered.
	Sender    mailer.Sender
	PublicURL string           // where links in the warnings lead (MAILRULES_PUBLIC_URL)
	Now       func() time.Time // nil = time.Now
	Every     time.Duration    // 0 = every minute
}

PlanEnd is the end-of-plan job of a build that sells plans (ext.Module.Billing). For each tenant but the self-host one whose plan has ended (store.TenantPlan) it keeps its mailboxes stopped, emails its members the three warnings, deletes its mailbox passwords and tokens and its model keys after store.EndedSecretsDays, and removes the tenant after store.EndedRemovalDays. Each step is recorded on the tenant's plan, so it is taken once, across ticks and restarts; a plan bought again clears them, and stops the schedule.

A warning missed (the daemon was down) is sent late only while no later one is due: only the latest due warning goes, so a tenant first seen 25 days after its end gets warning 2, not 1 then 2. And a deletion never comes sooner than NoticeDays after the warning that announces it was sent (warning 2 for the secrets, 3 for the removal; 3 announces both when 2 was skipped): a tenant first seen late is warned first, and its deletions wait the notice the warning gives. The dates each warning states are the ones the job will keep to.

func (PlanEnd) Once added in v0.3.0

func (j PlanEnd) Once(ctx context.Context) (steps int, err error)

Once takes, for every tenant whose plan has ended, the steps due by now, and returns how many it took. A tenant that fails does not keep the others from their turn; the failures are returned joined.

func (PlanEnd) Run added in v0.3.0

func (j PlanEnd) Run(ctx context.Context)

Run runs the job now and then every Every, until ctx is done.

type Removal added in v0.3.0

type Removal struct {
	Store *store.Store
	// Stop stops an account's supervisor and waits until it has; Start starts one again,
	// for a mailbox whose removal did not happen after all (the person signed in again
	// between the two).
	Stop  func(accountID int64)
	Start func(store.Account)
	Hub   *events.Hub      // tells the team's open screens when its rules were rewritten; may be nil
	Now   func() time.Time // nil = time.Now
	Every time.Duration    // 0 = hourly
}

Removal is the job that removes what MailRules kept of the people a sign-in module forgot (store.ForgetIdentity) once store.ForgetDays have passed. What is waiting lives in the database, so a restart only delays a removal to the job's next run.

func (Removal) Once added in v0.3.0

func (r Removal) Once(ctx context.Context) (removed int, err error)

Once removes every forgotten user whose time has come. Each one's mailboxes are stopped before their rows go. A user that fails does not keep the others from their turn; the failures are returned joined.

func (Removal) Run added in v0.3.0

func (r Removal) Run(ctx context.Context)

Run runs the job now and then every Every, until ctx is done.

type Retention

type Retention struct {
	Store *store.Store
	Days  func(ctx context.Context, tenantID int64) int // the tenant's retention_days setting in force
	Now   func() time.Time                              // nil = time.Now
	Every time.Duration                                 // 0 = hourly
}

Retention is the job that forgets: for each tenant, it blanks the snippets of messages older than that tenant's retention setting, and deletes the rows of messages older than store.KeepMessagesDays that have nothing left to undo.

func (Retention) Once

func (r Retention) Once(ctx context.Context) (blanked, deleted int64, err error)

Once runs the job one time over every tenant. A tenant that fails does not keep the others from their turn; the failures are returned joined.

func (Retention) Run

func (r Retention) Run(ctx context.Context)

Run runs the job now and then every Every, until ctx is done.

type SortRun

type SortRun struct {
	AccountID int64
	Folder    string
	Since     int64 // only mail received from this time on; 0 = all of it. Recorded on the batch.
	// Limit and Matched are the check's: the most emails it covered and how many the range
	// held before that cut. Recorded on the batch, so it can say what it covered (MAI-48).
	Limit, Matched int
	Rows           []composer.CheckRow
}

SortRun is the Sort that follows a finished check: the rows the user kept, applied with the answers the check already settled, as one undoable cleanup batch. It makes no model calls — the decisions are replayed, never asked again.

type Supervisor

type Supervisor struct {
	Account store.Account
	Store   *store.Store
	Hub     *events.Hub
	// Open connects to the account's server. It is called again after a failure.
	Open func(ctx context.Context) (mail.Mailbox, error)
	// Pipeline is the template for the account's pipeline; the supervisor fills in
	// Mailbox, Account and Rediscover.
	Pipeline pipeline.Pipeline
	// Moves, when set, looks through the account's filed mail for emails the person moved
	// outside MailRules (moved.Watcher.Sweep), every MovesEvery, on the account's connection
	// and between two emails, like the retry job. nil (the free build) = never.
	Moves      func(ctx context.Context, mb mail.Mailbox, acct store.Account) error
	MovesEvery time.Duration // 0 = 10 minutes

	// Timers. Zero means the default; tests shrink them.
	RetryEvery    time.Duration // the retry job (pipeline.RetryEvery)
	ContactsEvery time.Duration // the Sent-folder rescan (6 h)
	DrainTimeout  time.Duration // how long queued mail may take to finish at shutdown (10 s)
	BackoffMin    time.Duration // first reconnect delay (1 s), doubling up to BackoffMax (5 min)
	BackoffMax    time.Duration
	// contains filtered or unexported fields
}

Supervisor owns one account's connection, watcher and processor goroutine.

func (*Supervisor) Mailbox

func (s *Supervisor) Mailbox() mail.Mailbox

Mailbox returns the account's open connection, or nil while it is not connected.

func (*Supervisor) Run

func (s *Supervisor) Run(ctx context.Context)

Run supervises the account until ctx is done or the account needs the user: wrong credentials, a one-click sign-in to renew, a certificate to accept, another TLS failure, a watch folder that does not exist, or a server the hosted build does not connect to (mail.ErrNotPublic); or until the tenant's plan ends (pipeline.ErrHalted), leaving the status as it is: the mailbox starts again when a plan is bought. Anything else, a panic in the mailbox's work included (logged with its stack), is retried with a growing delay: in key_unavailable while the key service cannot unwrap the tenant's key, else in reconnecting. A session that panicked does not reset the delay, so a panic met again at once is not retried every second. Wrong credentials of a mailbox a module connected (store.Account.Module, such as one its sign-in form added with Bridge's password, which the person never saw) wait in reconnect_needed, for the person to sign in through the module again, rather than in auth_failed.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL