Documentation
¶
Overview ¶
Package pipeline takes one new email from arrival to outcome: ingest (once), fetch and parse, evaluate the rules, ask the decision model when rules with an intent are in play, hand the winning rule's actions to the executor or park the email in Needs review, record it all, and learn. It never changes a mailbox itself.
Index ¶
- Constants
- Variables
- func Corrections(st *store.Store, userID int64) ...
- func OwnMail(ctx context.Context, st *store.Store, a store.Account) ([]string, error)
- type Decider
- type Executor
- type Outcome
- type Pipeline
- func (p *Pipeline) Process(ctx context.Context, ref mail.MsgRef) error
- func (p *Pipeline) RetryDue(ctx context.Context) (int, error)
- func (p *Pipeline) Sort(ctx context.Context, ref mail.MsgRef) error
- func (p *Pipeline) SortSaved(ctx context.Context, ref mail.MsgRef, out Outcome) (applied bool, err error)
Constants ¶
const ( // RetryEvery is how long a failed message waits before the retry job runs it again. RetryEvery = 5 * time.Minute // MaxRetries is how many times the retry job tries before the message goes to Needs review. MaxRetries = 6 // ReasonNoModel is the Needs review reason while no decision model is configured. ReasonNoModel = "No decision model is set" // ReasonUnreadable is the decision's reason when the model's answer could not be read // after every try, and the email was kept. ReasonUnreadable = "The AI's answer could not be read after 3 tries, so the email was left where it is." // ReasonOwnMail is the decision's reason for an email sent from the mailbox's own // address while the leave_own_mail setting is on (Decider.Own). ReasonOwnMail = "Sent from this mailbox's own address, so MailRules left it alone." )
Variables ¶
var ErrModel = errors.New("the decision model failed")
ErrModel marks a failure of the decision model, as opposed to the mailbox or the database.
Functions ¶
func Corrections ¶
func Corrections(st *store.Store, userID int64) func(ctx context.Context, email message.Summary, candidates []int64) []learn.Example
Corrections is the Examples of a Decider that learns from the user's corrections. A failure to read them is logged and costs the examples, never the decision.
Types ¶
type Decider ¶
type Decider struct {
Router *models.Router // nil = no decision model is set
// Override returns the router for a rule's own model, or nil when it cannot be used.
Override func(ctx context.Context, spec string) *models.Router
MinConfidence float64 // act threshold for rules that set none
Now time.Time // for age_days
// Examples returns the user's corrections to show the fallback model for this email,
// most similar first; candidates are the rule ids on offer. nil = show none.
Examples func(ctx context.Context, email message.Summary, candidates []int64) []learn.Example
// Own are the mailbox's own addresses, whose mail is left alone: no sender rule, rule or
// model is consulted for it. Empty = none (the leave_own_mail setting is off). See OwnMail.
Own []string
}
Decider settles which rule applies to one email: the rules' evaluation order first and, when rules with an intent are in play, the decision model. It changes nothing, so the live pipeline, cleanup and the rule tester all decide through it and cannot disagree.
func (Decider) Settle ¶
func (d Decider) Settle(ctx context.Context, sum message.Summary, rs []rules.Rule, senders []rules.SenderRule) (Outcome, error)
Settle runs the evaluation order for one email and asks the model when it has to. The email is decided by the model of the highest-priority candidate rule that names one (and whose model can be used), otherwise by the default router.
type Executor ¶
type Executor interface {
Apply(ctx context.Context, d actions.DecisionRecord, acts []rules.Action, batchID int64) ([]actions.ActionRecord, error)
}
Executor is the part of actions.Executor the pipeline needs.
type Outcome ¶
type Outcome struct {
rules.Result
RuleName string // the name of Result.RuleID; empty when there is none
Reason string // one sentence saying why
Asked bool // a model was asked
// NoModel: rules with an intent were in play and no model could be asked, so they were
// passed over. With a model set, this is an email the model would have decided.
NoModel bool
Calls int // how many model calls that took, counting retries and the fallback
Usage models.Usage // what asking cost, over every call made
// Probabilities is what the decision model gave each candidate, by rule id (0 = none
// of them); nil when it gives none.
Probabilities map[int64]float64
}
Outcome is what was settled for one email.
type Pipeline ¶
type Pipeline struct {
Store *store.Store
Mailbox mail.Mailbox
Account store.Account
Router *models.Router // nil = no decision model is set
Exec Executor // nil = record decisions only
Hub *events.Hub // nil = publish nothing
MinConfidence float64 // act threshold for rules that set none
BodyChars int // plain-text characters shown to models
// Live, when set, supplies Router and MinConfidence afresh for every message, so a
// change made in Settings needs no restart (settings.Settings.Live).
Live func(ctx context.Context) (router *models.Router, minConfidence float64)
// Override returns the router for a rule's own model (rules.Rule.Model), or nil when
// that model cannot be used (settings.Settings.RouterFor). nil = rules' models are ignored.
Override func(ctx context.Context, spec string) *models.Router
// Batch is the batch the actions belong to. 0 = the day's live batch; a cleanup run
// sets its own, so the whole run can be undone as one.
Batch int64
// Spent, when set, is told what each model decision cost: a cleanup run shows the total.
Spent func(tokens int, costUSD float64)
Now func() time.Time // nil = time.Now
// Rediscover re-runs folder discovery. It is called once when a step fails because a
// folder is unknown, and the step is then tried again. nil = never.
Rediscover func(ctx context.Context) error
}
Pipeline processes the mail of one account. It is not safe for concurrent use: the account's supervisor calls it from one goroutine, which keeps mail in UID order.
func (*Pipeline) Process ¶
Process handles one message the watcher reported. It is idempotent: a message seen before is not processed again, unless an earlier run was cut off before it finished, and neither is a known email that came back into the folder under a new UID (cameBack). The folder's last processed UID advances only once the message's row is committed; on an error (the daemon is stopping, or the database failed) it does not, so the watcher sends the message again after a restart.
func (*Pipeline) RetryDue ¶
RetryDue is the retry job: it re-runs every failed message of the account whose wait is over, and returns how many it ran.
func (*Pipeline) Sort ¶
Sort runs one message of existing mail through the same steps as new mail: cleanup's way in. Unlike Process it does not look at whether the message was seen before (the rules may have changed since, or dry-run been switched off), and it leaves the folder's watch position alone.
func (*Pipeline) SortSaved ¶
func (p *Pipeline) SortSaved(ctx context.Context, ref mail.MsgRef, out Outcome) (applied bool, err error)
SortSaved applies a decision a cleanup check already settled to one existing email, without asking a model: it verifies the email is still where the check found it (BODY.PEEK, never marking it read), ingests its row, records the decision carrying the check's stage, rule, model, confidence and reason — but no fresh cost, which is on the ledger once from the check — and hands the actions to the executor in the run's batch. applied is false when the email has moved or gone since the check: it is passed over.