Documentation
¶
Overview ¶
Package integration hands operations only a deployment's environment knows how to do to a script an administrator chooses (#2659): routing a hostname through the ingress, and later operations of the same shape. Each operation is a contract: the function the script defines, the items it is handed, and the status it returns for each. With no script chosen the platform does nothing more than it did before.
The mechanism is one copy for every operation. An operation's items are computed, never described by the call sites that change them: a pass computes the set now, compares it with the set the script was last handed, and the difference is the change. A pass runs when a change is announced (the tables an operation reads notify integration_changed), when an integration's run ends, and on a timer, on one replica at a time. One run of an operation is in flight at most; a change that arrives meanwhile is in the next run's difference.
Index ¶
- Constants
- Variables
- func Check(source string) error
- func CleanMessage(s string) string
- func Contracts() string
- func Entry(call map[string]any) (function string, req map[string]any, err error)
- func Handler(call map[string]any, into *json.RawMessage) (*scriptdialect.Handler, error)
- func Prepare(trigger string, opts *scriptrun.Options) (finish func(*script.RunResult), err error)
- func ReadResult(value any, keys []string) (map[string]ItemResult, error)
- func RequestKeys(s Spec, req map[string]any) (items, removed []string)
- func Returned(call map[string]any, value any) (json.RawMessage, error)
- func RunCall(s Spec, req Request) map[string]any
- type Actor
- type Applied
- type Approvals
- type BindRequest
- type Binding
- type Bindings
- type Candidate
- type Item
- type ItemResult
- type ItemStatus
- type NewRun
- type Overview
- type Request
- type RunOutcome
- type Runtime
- type Scripts
- type Service
- func (s *Service) Bind(ctx context.Context, a Actor, req BindRequest) (*Binding, error)
- func (s *Service) Candidates(ctx context.Context, a Actor, op, search string) ([]Candidate, error)
- func (s *Service) Clear(ctx context.Context, a Actor, op string) error
- func (s *Service) Overview(ctx context.Context) ([]Overview, error)
- func (s *Service) Pass(ctx context.Context) error
- func (s *Service) Retry(ctx context.Context, op string) error
- func (s *Service) RetryFailed(ctx context.Context, op string, keys []string) error
- func (s *Service) Statuses(ctx context.Context, op string, keys []string, admin bool) (map[string]ItemStatus, error)
- type Source
- type Spec
- type Store
- func (s *Store) Apply(ctx context.Context, op string, a Applied) error
- func (s *Store) Bind(ctx context.Context, op, scriptID string, intervalSeconds int, by string) (*Binding, error)
- func (s *Store) Binding(ctx context.Context, op string) (*Binding, error)
- func (s *Store) Clear(ctx context.Context, op string) (bool, error)
- func (s *Store) Items(ctx context.Context, op string) ([]StoredItem, error)
- func (s *Store) ItemsByKey(ctx context.Context, op string, keys []string) (map[string]StoredItem, error)
- func (s *Store) Lock(ctx context.Context) (release func(), ok bool, err error)
- func (s *Store) RecordRefusal(ctx context.Context, op, reason string) error
- func (s *Store) RequestRetry(ctx context.Context, op string) error
- func (s *Store) RunOutcome(ctx context.Context, runID string) (RunOutcome, error)
- func (s *Store) StartRun(ctx context.Context, op string, r NewRun) error
- type StoredItem
Constants ¶
const ( StatusPending = "pending" StatusReady = "ready" StatusFailed = "failed" )
The statuses an item reads.
const ( ReasonChange = "change" ReasonTimer = "timer" )
The reasons a run is made for.
const ( DefaultInterval = 15 * 60 MinInterval = 60 MaxInterval = 24 * 60 * 60 )
The interval bounds an operation's timer may be set within, in seconds.
const Channel = "integration_changed"
Channel is where a change to an operation's items is announced.
const LockKey int64 = 4713210405
LockKey is the advisory lock one replica's pass holds.
const ( // OpSiteRouting is every hostname the platform serves for sites and // applications, handed to a script that routes them (#2659). OpSiteRouting = "site_routing" )
The operations the platform registers.
const RetryInterval = 15 * time.Second
RetryInterval is how long after a run starts a reader's retry is refused.
Variables ¶
var ( // ErrUnknown is an operation the platform does not register. ErrUnknown = errors.New("unknown integration") // ErrNotAllowed is a caller who may not change an integration. ErrNotAllowed = errors.New("not allowed") // ErrInvalid is a request that cannot be carried out as it is. ErrInvalid = errors.New("invalid integration request") // ErrNothingFailed is a reader's retry when nothing they can see failed. ErrNothingFailed = errors.New("nothing it shows failed") // ErrTooSoon is a reader's retry within RetryInterval of the last run. ErrTooSoon = errors.New("asked too soon") )
Errors a caller reads by identity.
var ErrNotBound = errors.New("no script is chosen for this operation")
ErrNotBound is an operation with no script chosen.
Functions ¶
func Check ¶
Check refuses a source that could serve no operation: it defines none of the operations' functions. It is the handler kind's check, which the save gate asks of every script an operation is bound to.
func CleanMessage ¶
CleanMessage is a script's message as it is kept and shown: control and invisible runes removed, one line, bounded.
func Contracts ¶
func Contracts() string
Contracts is every operation's function, as the handler kind names it.
func Handler ¶
func Handler(call map[string]any, into *json.RawMessage) (*scriptdialect.Handler, error)
Handler is the call an integration's run makes: the operation's function with the request its call carries, what it returns checked against that request and kept in into. A value of the wrong shape fails the run.
func Prepare ¶
Prepare readies an integration's run (trigger integration): it calls the operation's function with the request its call carries in place of main(), and finish keeps what the function returned, checked, as the run's result. Any other run is left as it is.
func ReadResult ¶
func ReadResult(value any, keys []string) (map[string]ItemResult, error)
ReadResult reads the value the function returned against the keys of the request it was handed: a dict keyed by an item the request named, each value a dict of status ready or failed and an optional message. A key the request did not name is ignored, so a script cannot set the status of an item it was not asked about. The message is untrusted text: cleaned and bounded.
func RequestKeys ¶
RequestKeys is every key a run's request names: its items now and the ones it removes.
func Returned ¶
Returned checks what an integration run's function returned against the request in the run's call, and is the result the run keeps: one entry per item it was asked about, message cleaned. A value of the wrong shape fails the run, naming what was wrong, which only an administrator reads.
Types ¶
type Applied ¶
type Applied struct {
RunID string
Results map[string]ItemResult
Failed []string
FailWith string
Removed []string
Status string
Error string
}
Applied is what a finished run left: each item's status, and the items it removed.
type Approvals ¶
type Approvals interface {
HandlerRunAs(ctx context.Context, scriptID string) (approval.RunAs, bool)
ApproveForHandlers(ctx context.Context, actor approval.Actor, sc *script.Script, version int) error
}
Approvals is what a bound script's version is approved and run through (approval.Service): the same approval every handler kind runs on.
type BindRequest ¶
type BindRequest struct {
Operation string
ScriptID string
// IntervalSeconds is the timer; 0 keeps the current one or the default.
IntervalSeconds int
// Approve approves the script's latest version for handlers as the actor.
Approve bool
}
BindRequest chooses an operation's script.
type Binding ¶
type Binding struct {
ID string `json:"id"`
Operation string `json:"operation"`
ScriptID string `json:"script_id"`
IntervalSeconds int `json:"interval_seconds"`
CurrentRunID string `json:"current_run_id,omitempty"`
RetryRequested bool `json:"retry_requested"`
LastStartedAt *time.Time `json:"last_started_at,omitempty"`
LastRunID string `json:"last_run_id,omitempty"`
LastStatus string `json:"last_status,omitempty"`
LastReason string `json:"last_reason,omitempty"`
LastError string `json:"last_error,omitempty"`
LastFinishedAt *time.Time `json:"last_finished_at,omitempty"`
CreatedBy string `json:"created_by"`
UpdatedBy string `json:"updated_by"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
Binding is the script chosen for an operation and how its runs stand.
type Bindings ¶
type Bindings interface {
Binding(ctx context.Context, op string) (*Binding, error)
Bind(ctx context.Context, op, scriptID string, intervalSeconds int, by string) (*Binding, error)
Clear(ctx context.Context, op string) (bool, error)
RequestRetry(ctx context.Context, op string) error
Items(ctx context.Context, op string) ([]StoredItem, error)
ItemsByKey(ctx context.Context, op string, keys []string) (map[string]StoredItem, error)
StartRun(ctx context.Context, op string, r NewRun) error
RecordRefusal(ctx context.Context, op, reason string) error
RunOutcome(ctx context.Context, runID string) (RunOutcome, error)
Apply(ctx context.Context, op string, a Applied) error
// Lock takes the lock one replica's pass holds; ok is false when another
// replica holds it.
Lock(ctx context.Context) (release func(), ok bool, err error)
}
Bindings is where bindings, items and an operation's runs are kept (*Store).
type Candidate ¶
type Candidate struct {
ID string `json:"id"`
Name string `json:"name"`
OwnerEmail string `json:"owner_email"`
Version int `json:"version"`
Running int `json:"running_version"`
ApprovedBy string `json:"approved_by,omitempty"`
RunningServes bool `json:"running_serves"`
}
Candidate is a script the picker offers for an operation: its latest version defines the operation's function. Running is the version approved for handlers (0 for none) and RunningServes whether that one defines it too.
type Item ¶
Item is one thing an operation hands the script: its key and the entry the request carries for it, which holds the key under the operation's key field.
type ItemResult ¶
ItemResult is the status the script returned for one item.
type ItemStatus ¶
type ItemStatus struct {
Status string `json:"status"`
Message string `json:"message"`
CheckedAt *time.Time `json:"checked_at,omitempty"`
// RunID and RunError are the failing run's, filled for an administrator
// only, when the script itself failed.
RunID string `json:"run_id,omitempty"`
RunScriptID string `json:"run_script_id,omitempty"`
RunError string `json:"run_error,omitempty"`
// ScriptFailed says the script itself failed, so Retry is offered.
ScriptFailed bool `json:"script_failed,omitempty"`
}
ItemStatus is one item's status as the surfaces that show the item report it: under routing, with no run error, which only an administrator reads.
type NewRun ¶
type NewRun struct {
ID string
ScriptID string
VersionID string
Version int
Approver string
Roles []string
Call map[string]any
Reason string
Added []Item
Removed []string
}
NewRun is one integration run to queue.
type Overview ¶
type Overview struct {
Spec
Binding *Binding `json:"binding,omitempty"`
ScriptName string `json:"script_name,omitempty"`
Running int `json:"running_version,omitempty"`
ApprovedBy string `json:"approved_by,omitempty"`
Counts map[string]int `json:"counts"`
Items []StoredItem `json:"items"`
// Total is how many items the operation has now, whether or not a script
// is chosen.
Total int `json:"total"`
}
Overview is one operation as the settings page shows it.
type Request ¶
type Request struct {
Reason string `json:"reason"`
Added []map[string]any `json:"added"`
Removed []map[string]any `json:"removed"`
Items []map[string]any `json:"-"`
}
Request is what the script's function is handed.
type RunOutcome ¶
type RunOutcome struct {
Finished bool
Status string
Error string
Result []byte
Call map[string]any
}
RunOutcome is how an integration's run ended.
type Runtime ¶
type Runtime struct {
Service *Service
// DSN opens the LISTEN connection; empty polls only.
DSN string
// contains filtered or unexported fields
}
Runtime runs the passes on this replica: on every announcement on Channel and every passEvery.
func (*Runtime) Notify ¶
func (r *Runtime) Notify()
Notify wakes the next pass; the listener calls it on every announcement.
type Scripts ¶
type Scripts interface {
GetByID(ctx context.Context, id string) (*script.Script, error)
GetVersion(ctx context.Context, scriptID string, version int) (*script.Version, error)
}
Scripts reads a bound script and its versions.
type Service ¶
type Service struct {
Store Bindings
Scripts Scripts
Approvals Approvals
// Sources computes each operation's items; an operation with none is
// never run.
Sources map[string]Source
// contains filtered or unexported fields
}
Service binds operations to scripts and runs them.
func (*Service) Bind ¶
Bind chooses the script an operation calls. Only an administrator binds, and only a script whose version that will run defines the operation's function.
func (*Service) Candidates ¶
Candidates is the scripts an administrator may choose for an operation, matching search.
func (*Service) Clear ¶
Clear stops an operation's calls and drops every status it kept: the deployment is back to behaving as it did with no script.
func (*Service) Overview ¶
Overview is every operation with its binding and items. Only an administrator reads it: it carries the run error.
func (*Service) Pass ¶
Pass runs every bound operation once, on one replica at a time: it applies a finished run, then starts the next when the items changed, a retry was asked for, or the timer is due.
func (*Service) Retry ¶
Retry asks for a run with every item at once. The caller has been allowed it by its surface: an administrator, or someone who can see a failed item.
func (*Service) RetryFailed ¶
RetryFailed is a retry asked for by someone who is not an administrator and can see keys (#2659): honored only when one of keys failed, and not within RetryInterval of the last run's start, so a reader can ask again for what failed but cannot keep the script running.
func (*Service) Statuses ¶
func (s *Service) Statuses(ctx context.Context, op string, keys []string, admin bool) (map[string]ItemStatus, error)
Statuses is each key's status for an operation, nil when no script is chosen (the field is then omitted everywhere). A key the script has not answered for yet is pending. admin adds the run error to an item the script itself failed.
type Spec ¶
type Spec struct {
// Name is the operation's name in the API and the view's target.
Name string `json:"operation"`
// Label is how the portal names it.
Label string `json:"label"`
// Function is the top-level function the script defines, taking req.
Function string `json:"function"`
// ItemsField is the request field holding every item now, and the name
// an item is counted by.
ItemsField string `json:"items_field"`
// KeyField is the entry field an item is keyed by.
KeyField string `json:"key_field"`
// FailedMessage is what someone other than an administrator reads on an
// item when the script itself failed.
FailedMessage string `json:"-"`
}
Spec is one operation's contract.
func FromRunCall ¶
FromRunCall reads an integration run's call: its operation and request.
type Store ¶
Store keeps bindings and items in PostgreSQL.
func (*Store) Apply ¶
Apply writes a finished run's statuses, deletes the items it removed, and clears the run in flight, in one transaction.
func (*Store) Bind ¶
func (s *Store) Bind(ctx context.Context, op, scriptID string, intervalSeconds int, by string) (*Binding, error)
Bind chooses the script for an operation, or changes it or its interval. The next pass calls the script with every item.
func (*Store) ItemsByKey ¶
func (s *Store) ItemsByKey(ctx context.Context, op string, keys []string) (map[string]StoredItem, error)
ItemsByKey is the stored items among keys, by key.
func (*Store) RecordRefusal ¶
RecordRefusal records why a pass could not call the script, and holds the next try to the operation's interval.
func (*Store) RequestRetry ¶
RequestRetry asks the next pass to call the operation's script with every item, and wakes the passes.
func (*Store) RunOutcome ¶
RunOutcome reads a run; Finished is false while it is pending or running.
type StoredItem ¶
type StoredItem struct {
Key string `json:"key"`
Entry map[string]any `json:"entry"`
Status string `json:"status"`
Message string `json:"message"`
Removing bool `json:"-"`
FailedRun string `json:"failed_run,omitempty"`
CheckedAt *time.Time `json:"checked_at,omitempty"`
}
StoredItem is one item as the script was last handed it, and the status it last returned.