Documentation
¶
Overview ¶
Package workerapi is the contract between the backend and the workers: processes that talk to outside services (Withings, Strength Level, ...) for the backend and have no database. A worker asks the backend for a job, works on it while renewing its lease, and reports the result; the backend owns the queue, the credentials and every decision about what is stored.
The wire protocol is /internal/v1/worker, authenticated with a worker secret that opens nothing else (and the service secret of the web frontend does not open it). Like the rest of the internal API it is not a public interface: deploy the workers from the same release as the backend.
A job's kind is "<provider>.<action>", for example "withings.import". Nothing here names a provider: each provider package registers handlers for its kinds, and the backend has a descriptor for each provider (see pkg/backend/main). The package must not import the store, the backend or the frontend.
Index ¶
- Constants
- Variables
- func Kind(provider, action string) string
- type CheckLoginPayload
- type CheckLoginResult
- type Client
- func (c *Client) Complete(ctx context.Context, id, token string, comp Completion) error
- func (c *Client) Config(ctx context.Context) (Config, error)
- func (c *Client) Health(ctx context.Context) error
- func (c *Client) Heartbeat(ctx context.Context, id, token string, progress json.RawMessage) (cancel bool, err error)
- func (c *Client) Lease(ctx context.Context, r LeaseRequest) (*Lease, error)
- func (c *Client) Register(ctx context.Context, r RegisterRequest) (Config, error)
- func (c *Client) Release(ctx context.Context, id, token string) error
- func (c *Client) SaveCredentials(ctx context.Context, id, token string, data json.RawMessage) error
- type Completion
- type Config
- type ConnectPayload
- type Credentials
- type ExportEntry
- type ExportPayload
- type ExportProgress
- type ExportResult
- type Heartbeat
- type HeartbeatAnswer
- type ImportPayload
- type ImportResult
- type Job
- type KindConfig
- type Lease
- type LeaseRequest
- type Login
- type OAuthApp
- type RegisterRequest
Constants ¶
const ( ActionCheckLogin = "check_login" ActionDay = "day" ActionSync = "sync" )
Actions of an exporter, the part of a kind after the provider's name.
const ( ProviderWithings = "withings" ProviderStrengthLevel = "strengthlevel" )
The providers that ship with the program. A worker package and the backend's descriptor for a provider agree on its name and its kinds through these.
const ( ActionConnect = "connect" ActionImport = "import" )
Action names, the part of a kind after the provider's name.
const ( // HeaderWorker carries the worker secret. HeaderWorker = "X-Dexafix-Worker" // HeaderLease carries the lease token of the job a call is about: the secret only the worker // that leased the job knows. A call with another token fails with ErrLeaseLost. HeaderLease = "X-Dexafix-Lease" )
Header names.
const MaxWaitSeconds = 20
MaxWaitSeconds is the longest the backend holds a lease request open.
const Prefix = "/internal/v1/worker"
Prefix of the worker endpoints, below the internal API's.
Variables ¶
var ErrLeaseLost = errors.New("the job is no longer leased to this worker")
ErrLeaseLost: the worker no longer holds the job (its lease ran out and the job was handed on, canceled or finished). The worker must stop working on it and report nothing more.
Functions ¶
Types ¶
type CheckLoginPayload ¶
type CheckLoginPayload struct {
Login Login `json:"login"`
}
CheckLoginPayload is the payload of <provider>.check_login.
type CheckLoginResult ¶
type CheckLoginResult struct {
LoginOK bool `json:"login_ok"`
Message string `json:"message,omitempty"`
}
CheckLoginResult says whether the service accepted the login. Message is for the person.
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client is what a worker uses to talk to the backend. It is safe for concurrent use.
func New ¶
New returns a client for the backend at base (for example http://api:8081). hc may be nil; build one with apiclient.HTTPClient for mutual TLS.
func (*Client) Heartbeat ¶
func (c *Client) Heartbeat(ctx context.Context, id, token string, progress json.RawMessage) (cancel bool, err error)
Heartbeat renews the lease. It returns ErrLeaseLost when the job is no longer the worker's.
func (*Client) Lease ¶
Lease asks for a job and waits for one for up to r.WaitSeconds. It returns nil when there is none.
func (*Client) Release ¶
Release gives the job back without counting the attempt, for a worker that is shutting down.
func (*Client) SaveCredentials ¶
SaveCredentials hands the backend credentials that were renewed during the job.
type Completion ¶
type Completion struct {
// Outcome is "ok", "retry" (a temporary problem) or "fail" (a permanent one).
Outcome string `json:"outcome"`
Result json.RawMessage `json:"result,omitempty"`
// Error is a short sentence for the administrator and, for some kinds, the person. It must not
// contain a credential.
Error string `json:"error,omitempty"`
// RetryAfterSeconds is the provider's own request to wait (Retry-After); the backend never
// retries sooner than its own backoff, and never later than its cap.
RetryAfterSeconds int `json:"retry_after_seconds,omitempty"`
}
Completion ends an attempt.
type Config ¶
type Config struct {
Generation int64 `json:"generation"`
// LeaseSeconds is how long a lease lasts without a heartbeat; the worker renews at a third of it.
LeaseSeconds int `json:"lease_seconds"`
// Kinds holds the settings of each kind. A kind that is missing is disabled.
Kinds map[string]KindConfig `json:"kinds"`
}
Config is what the administrator set for the workers. The backend serves it; Generation grows whenever any of it changes, so a worker re-reads it cheaply and applies it without a restart.
type ConnectPayload ¶
ConnectPayload is the payload of <provider>.connect.
type Credentials ¶
type Credentials struct {
Data json.RawMessage `json:"data"`
}
Credentials hands the backend credentials the worker renewed during the job (a refreshed OAuth token pair, for instance) so they are stored at once, before they are used. Refresh tokens work once: losing the new one would disconnect the user.
type ExportEntry ¶
type ExportEntry struct {
Day string `json:"day"` // YYYY-MM-DD
WeightKg float64 `json:"weight_kg"`
FatPct *float64 `json:"fat_pct,omitempty"` // nil when the day has no body fat estimate
}
ExportEntry is the body weight and body fat of one calendar day.
type ExportPayload ¶
type ExportPayload struct {
Login Login `json:"login"`
// Entries are the days to send, newest first.
Entries []ExportEntry `json:"entries"`
// Since limits a sync to entries from this day on (YYYY-MM-DD; empty: all of them).
Since string `json:"since,omitempty"`
// Update also changes days that exist at the service with other values. Without it they are
// left as they are. Nothing is ever deleted.
Update bool `json:"update,omitempty"`
}
ExportPayload is the payload of <provider>.day and <provider>.sync.
type ExportProgress ¶
ExportProgress is what a sync reports with its heartbeats.
type ExportResult ¶
type ExportResult struct {
Added int `json:"added"`
Updated int `json:"updated"`
Unchanged int `json:"unchanged"` // already at the service with the same values (or left alone)
Failed int `json:"failed"` // days that could not be written
// FailedMessages are a few of the reasons, for the log.
FailedMessages []string `json:"failed_messages,omitempty"`
}
ExportResult is what an export did.
type Heartbeat ¶
type Heartbeat struct {
Progress json.RawMessage `json:"progress,omitempty"`
}
Heartbeat renews the lease and optionally reports progress (a JSON value for the person to see).
type HeartbeatAnswer ¶
type HeartbeatAnswer struct {
// Cancel: somebody canceled the job. Stop, and report the attempt as retry.
Cancel bool `json:"cancel"`
}
HeartbeatAnswer tells the worker whether to go on.
type ImportPayload ¶
type ImportPayload struct {
App OAuthApp `json:"app"`
// Credentials are what the user's connection stored, as the provider's worker wrote them (an
// OAuth token pair for example). They are opaque to the backend. A worker that renews them hands
// the new ones over with SaveCredentials before it uses them.
Credentials json.RawMessage `json:"credentials"`
// Since limits a bulk import to measurements at or after this time (zero: everything).
Since time.Time `json:"since,omitempty"`
// Day, when set (YYYY-MM-DD), asks for exactly that calendar day in the user's time zone Zone.
Day string `json:"day,omitempty"`
// Zone is the user's IANA time zone; it decides where a calendar day begins and ends.
Zone string `json:"zone,omitempty"`
}
ImportPayload is the payload of <provider>.import.
type ImportResult ¶
type ImportResult struct {
// Readings are the measurements, with Source set to the provider's name and ExternalID to the
// provider's id of the measurement, which makes repeated imports idempotent.
Readings []models.ScaleReading `json:"readings"`
// Seen are the ids of every measurement the provider returned, including the ones left out of
// Readings (a guest on the scale), so a day import never deletes a reading only because it is
// filtered out now.
Seen []string `json:"seen,omitempty"`
}
ImportResult is what an import found. The backend decides what to store: it compares Readings with the readings the user already has (and the manual ones it must not disturb).
type Job ¶
type Job struct {
ID string `json:"id"`
Kind string `json:"kind"`
UserID int64 `json:"user_id"`
Attempt int `json:"attempt"`
MaxAttempts int `json:"max_attempts"`
// Payload is what the handler of the kind needs: its parameters and the credentials the
// backend decided to hand over for this one job. Never log it.
Payload json.RawMessage `json:"payload"`
}
Job is a unit of work, as a worker sees it.
type KindConfig ¶
type KindConfig struct {
Enabled bool `json:"enabled"`
// Options are provider specific values, such as a pause between requests. See each provider.
Options map[string]string `json:"options,omitempty"`
}
KindConfig is the configuration of one kind of job.
func (KindConfig) OptionMillis ¶
Option returns an option as a duration in milliseconds, or def when it is missing or invalid.
type Lease ¶
type Lease struct {
Job Job `json:"job"`
// Token goes into HeaderLease of every later call about the job.
Token string `json:"token"`
// TimeoutSeconds is the hard limit of this attempt. The backend ends the attempt itself when
// it passes, whatever the worker does.
TimeoutSeconds int `json:"timeout_seconds"`
LeaseSeconds int `json:"lease_seconds"`
}
Lease is a job together with the proof of holding it.
type LeaseRequest ¶
type LeaseRequest struct {
Worker string `json:"worker"`
Kinds []string `json:"kinds"`
WaitSeconds int `json:"wait_seconds"`
}
LeaseRequest asks for the next job of one of the kinds. The call waits up to WaitSeconds for one.
type OAuthApp ¶
type OAuthApp struct {
ClientID string `json:"client_id"`
ClientSecret string `json:"client_secret"`
RedirectURL string `json:"redirect_url"`
}
OAuthApp is the application's own registration with a provider (set by an administrator).
type RegisterRequest ¶
type RegisterRequest struct {
ID string `json:"id"` // unique per process, chosen by the worker
Version string `json:"version"`
Kinds []string `json:"kinds"`
}
RegisterRequest introduces a worker process and the kinds of job it can run.