workerapi

package
v0.0.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 12, 2026 License: AGPL-3.0 Imports: 13 Imported by: 0

Documentation

Overview

Package workerapi is the contract between the backend and the workers: processes that talk to outside services (Withings, Strength Level, ...) for the backend and have no database. A worker asks the backend for a job, works on it while renewing its lease, and reports the result; the backend owns the queue, the credentials and every decision about what is stored.

The wire protocol is /internal/v1/worker, authenticated with a worker secret that opens nothing else (and the service secret of the web frontend does not open it). Like the rest of the internal API it is not a public interface: deploy the workers from the same release as the backend.

A job's kind is "<provider>.<action>", for example "withings.import". Nothing here names a provider: each provider package registers handlers for its kinds, and the backend has a descriptor for each provider (see pkg/backend/main). The package must not import the store, the backend or the frontend.

Index

Constants

View Source
const (
	ActionCheckLogin = "check_login"
	ActionDay        = "day"
	ActionSync       = "sync"
)

Actions of an exporter, the part of a kind after the provider's name.

View Source
const (
	ProviderWithings      = "withings"
	ProviderStrengthLevel = "strengthlevel"
)

The providers that ship with the program. A worker package and the backend's descriptor for a provider agree on its name and its kinds through these.

View Source
const (
	ActionConnect = "connect"
	ActionImport  = "import"
)

Action names, the part of a kind after the provider's name.

View Source
const (
	// HeaderWorker carries the worker secret.
	HeaderWorker = "X-Dexafix-Worker"
	// HeaderLease carries the lease token of the job a call is about: the secret only the worker
	// that leased the job knows. A call with another token fails with ErrLeaseLost.
	HeaderLease = "X-Dexafix-Lease"
)

Header names.

View Source
const MaxWaitSeconds = 20

MaxWaitSeconds is the longest the backend holds a lease request open.

View Source
const Prefix = "/internal/v1/worker"

Prefix of the worker endpoints, below the internal API's.

Variables

View Source
var ErrLeaseLost = errors.New("the job is no longer leased to this worker")

ErrLeaseLost: the worker no longer holds the job (its lease ran out and the job was handed on, canceled or finished). The worker must stop working on it and report nothing more.

Functions

func Kind

func Kind(provider, action string) string

Kind builds the name of a kind of job, "<provider>.<action>".

Types

type CheckLoginPayload

type CheckLoginPayload struct {
	Login Login `json:"login"`
}

CheckLoginPayload is the payload of <provider>.check_login.

type CheckLoginResult

type CheckLoginResult struct {
	LoginOK bool   `json:"login_ok"`
	Message string `json:"message,omitempty"`
}

CheckLoginResult says whether the service accepted the login. Message is for the person.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is what a worker uses to talk to the backend. It is safe for concurrent use.

func New

func New(base string, hc *http.Client, secret string) (*Client, error)

New returns a client for the backend at base (for example http://api:8081). hc may be nil; build one with apiclient.HTTPClient for mutual TLS.

func (*Client) Complete

func (c *Client) Complete(ctx context.Context, id, token string, comp Completion) error

Complete reports how the attempt ended.

func (*Client) Config

func (c *Client) Config(ctx context.Context) (Config, error)

Config returns the current configuration.

func (*Client) Health

func (c *Client) Health(ctx context.Context) error

Health asks the backend whether it is up.

func (*Client) Heartbeat

func (c *Client) Heartbeat(ctx context.Context, id, token string, progress json.RawMessage) (cancel bool, err error)

Heartbeat renews the lease. It returns ErrLeaseLost when the job is no longer the worker's.

func (*Client) Lease

func (c *Client) Lease(ctx context.Context, r LeaseRequest) (*Lease, error)

Lease asks for a job and waits for one for up to r.WaitSeconds. It returns nil when there is none.

func (*Client) Register

func (c *Client) Register(ctx context.Context, r RegisterRequest) (Config, error)

Register introduces the worker and returns the configuration.

func (*Client) Release

func (c *Client) Release(ctx context.Context, id, token string) error

Release gives the job back without counting the attempt, for a worker that is shutting down.

func (*Client) SaveCredentials

func (c *Client) SaveCredentials(ctx context.Context, id, token string, data json.RawMessage) error

SaveCredentials hands the backend credentials that were renewed during the job.

type Completion

type Completion struct {
	// Outcome is "ok", "retry" (a temporary problem) or "fail" (a permanent one).
	Outcome string          `json:"outcome"`
	Result  json.RawMessage `json:"result,omitempty"`
	// Error is a short sentence for the administrator and, for some kinds, the person. It must not
	// contain a credential.
	Error string `json:"error,omitempty"`
	// RetryAfterSeconds is the provider's own request to wait (Retry-After); the backend never
	// retries sooner than its own backoff, and never later than its cap.
	RetryAfterSeconds int `json:"retry_after_seconds,omitempty"`
}

Completion ends an attempt.

type Config

type Config struct {
	Generation int64 `json:"generation"`
	// LeaseSeconds is how long a lease lasts without a heartbeat; the worker renews at a third of it.
	LeaseSeconds int `json:"lease_seconds"`
	// Kinds holds the settings of each kind. A kind that is missing is disabled.
	Kinds map[string]KindConfig `json:"kinds"`
}

Config is what the administrator set for the workers. The backend serves it; Generation grows whenever any of it changes, so a worker re-reads it cheaply and applies it without a restart.

type ConnectPayload

type ConnectPayload struct {
	App  OAuthApp `json:"app"`
	Code string   `json:"code"`
}

ConnectPayload is the payload of <provider>.connect.

type Credentials

type Credentials struct {
	Data json.RawMessage `json:"data"`
}

Credentials hands the backend credentials the worker renewed during the job (a refreshed OAuth token pair, for instance) so they are stored at once, before they are used. Refresh tokens work once: losing the new one would disconnect the user.

type ExportEntry

type ExportEntry struct {
	Day      string   `json:"day"` // YYYY-MM-DD
	WeightKg float64  `json:"weight_kg"`
	FatPct   *float64 `json:"fat_pct,omitempty"` // nil when the day has no body fat estimate
}

ExportEntry is the body weight and body fat of one calendar day.

type ExportPayload

type ExportPayload struct {
	Login Login `json:"login"`
	// Entries are the days to send, newest first.
	Entries []ExportEntry `json:"entries"`
	// Since limits a sync to entries from this day on (YYYY-MM-DD; empty: all of them).
	Since string `json:"since,omitempty"`
	// Update also changes days that exist at the service with other values. Without it they are
	// left as they are. Nothing is ever deleted.
	Update bool `json:"update,omitempty"`
}

ExportPayload is the payload of <provider>.day and <provider>.sync.

type ExportProgress

type ExportProgress struct {
	Done  int `json:"done"`
	Total int `json:"total"`
}

ExportProgress is what a sync reports with its heartbeats.

type ExportResult

type ExportResult struct {
	Added     int `json:"added"`
	Updated   int `json:"updated"`
	Unchanged int `json:"unchanged"` // already at the service with the same values (or left alone)
	Failed    int `json:"failed"`    // days that could not be written
	// FailedMessages are a few of the reasons, for the log.
	FailedMessages []string `json:"failed_messages,omitempty"`
}

ExportResult is what an export did.

type Heartbeat

type Heartbeat struct {
	Progress json.RawMessage `json:"progress,omitempty"`
}

Heartbeat renews the lease and optionally reports progress (a JSON value for the person to see).

type HeartbeatAnswer

type HeartbeatAnswer struct {
	// Cancel: somebody canceled the job. Stop, and report the attempt as retry.
	Cancel bool `json:"cancel"`
}

HeartbeatAnswer tells the worker whether to go on.

type ImportPayload

type ImportPayload struct {
	App OAuthApp `json:"app"`
	// Credentials are what the user's connection stored, as the provider's worker wrote them (an
	// OAuth token pair for example). They are opaque to the backend. A worker that renews them hands
	// the new ones over with SaveCredentials before it uses them.
	Credentials json.RawMessage `json:"credentials"`
	// Since limits a bulk import to measurements at or after this time (zero: everything).
	Since time.Time `json:"since,omitempty"`
	// Day, when set (YYYY-MM-DD), asks for exactly that calendar day in the user's time zone Zone.
	Day string `json:"day,omitempty"`
	// Zone is the user's IANA time zone; it decides where a calendar day begins and ends.
	Zone string `json:"zone,omitempty"`
}

ImportPayload is the payload of <provider>.import.

type ImportResult

type ImportResult struct {
	// Readings are the measurements, with Source set to the provider's name and ExternalID to the
	// provider's id of the measurement, which makes repeated imports idempotent.
	Readings []models.ScaleReading `json:"readings"`
	// Seen are the ids of every measurement the provider returned, including the ones left out of
	// Readings (a guest on the scale), so a day import never deletes a reading only because it is
	// filtered out now.
	Seen []string `json:"seen,omitempty"`
}

ImportResult is what an import found. The backend decides what to store: it compares Readings with the readings the user already has (and the manual ones it must not disturb).

type Job

type Job struct {
	ID          string `json:"id"`
	Kind        string `json:"kind"`
	UserID      int64  `json:"user_id"`
	Attempt     int    `json:"attempt"`
	MaxAttempts int    `json:"max_attempts"`
	// Payload is what the handler of the kind needs: its parameters and the credentials the
	// backend decided to hand over for this one job. Never log it.
	Payload json.RawMessage `json:"payload"`
}

Job is a unit of work, as a worker sees it.

type KindConfig

type KindConfig struct {
	Enabled bool `json:"enabled"`
	// Options are provider specific values, such as a pause between requests. See each provider.
	Options map[string]string `json:"options,omitempty"`
}

KindConfig is the configuration of one kind of job.

func (KindConfig) OptionMillis

func (k KindConfig) OptionMillis(name string, def time.Duration) time.Duration

Option returns an option as a duration in milliseconds, or def when it is missing or invalid.

type Lease

type Lease struct {
	Job Job `json:"job"`
	// Token goes into HeaderLease of every later call about the job.
	Token string `json:"token"`
	// TimeoutSeconds is the hard limit of this attempt. The backend ends the attempt itself when
	// it passes, whatever the worker does.
	TimeoutSeconds int `json:"timeout_seconds"`
	LeaseSeconds   int `json:"lease_seconds"`
}

Lease is a job together with the proof of holding it.

type LeaseRequest

type LeaseRequest struct {
	Worker      string   `json:"worker"`
	Kinds       []string `json:"kinds"`
	WaitSeconds int      `json:"wait_seconds"`
}

LeaseRequest asks for the next job of one of the kinds. The call waits up to WaitSeconds for one.

type Login

type Login struct {
	Email    string `json:"email"`
	Password string `json:"password"`
}

Login is a user's login at an outside service.

type OAuthApp

type OAuthApp struct {
	ClientID     string `json:"client_id"`
	ClientSecret string `json:"client_secret"`
	RedirectURL  string `json:"redirect_url"`
}

OAuthApp is the application's own registration with a provider (set by an administrator).

type RegisterRequest

type RegisterRequest struct {
	ID      string   `json:"id"` // unique per process, chosen by the worker
	Version string   `json:"version"`
	Kinds   []string `json:"kinds"`
}

RegisterRequest introduces a worker process and the kinds of job it can run.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL