Documentation
¶
Overview ¶
Package thumbworker draws every tile the platform shows. It claims the assets, resources and collections the stores say are owed one, renders each in the headless browser beside the platform, and records what it drew or why it could not (#1787).
A tile used to be drawn in whichever portal tab happened to be open, by a JavaScript reimplementation of CSS painting, and was wrong wherever that reimplementation fell short. Here the browser itself paints it, through the same renderers the viewer uses, and a document nobody opens still gets one.
Index ¶
Constants ¶
const DefaultRendererURL = "http://127.0.0.1:9222"
DefaultRendererURL is where the platform looks for its renderer when none is configured: a headless browser beside it in the same pod.
const Renderer = 2
Renderer is the generation of the renderer this worker draws as. A tile drawn by an older generation still serves and is drawn again; raising this redraws every tile in the library.
2 draws at twice the density and gives HTML and JSX a dark tile (#1789).
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type AssetWork ¶
type AssetWork interface {
ClaimThumbnailWork(ctx context.Context, renderer int, lease time.Duration, limit int) ([]portaldomain.Asset, error)
HoldThumbnailWork(ctx context.Context, id string, hold time.Duration, attempts int) error
Update(ctx context.Context, id string, u portaldomain.AssetUpdate) error
Get(ctx context.Context, id string) (*portaldomain.Asset, error)
}
AssetWork is what the worker asks of the asset store.
type Blobs ¶
type Blobs interface {
GetObject(ctx context.Context, bucket, key string) ([]byte, string, error)
PutObject(ctx context.Context, bucket, key string, data []byte, contentType string) error
DeleteObject(ctx context.Context, bucket, key string) error
}
Blobs is object storage.
type CollectionWork ¶
type CollectionWork interface {
ClaimCollectionThumbnailWork(ctx context.Context, lease time.Duration, limit int) ([]portaldomain.CollectionThumbnailWork, error)
RecordCollectionThumbnail(ctx context.Context, id, key, source string) error
RecordCollectionThumbnailFailure(ctx context.Context, id, source, reason string) error
HoldCollectionThumbnailWork(ctx context.Context, id string, hold time.Duration, attempts int) error
}
CollectionWork is what the worker asks of the collection store.
type Config ¶
type Config struct {
Enabled *bool `yaml:"enabled"`
// RendererURL is the renderer's DevTools address, http:// or ws://.
// Empty means DefaultRendererURL.
RendererURL string `yaml:"renderer_url"`
// Concurrency is how many documents one replica draws at once. The
// renderer is one browser beside the replica, so the default is 1.
Concurrency int `yaml:"concurrency"`
// RenderTimeout bounds drawing one variant of one document. A document
// that has not reported itself drawn by then is recorded as not drawable.
RenderTimeout time.Duration `yaml:"render_timeout"`
// Batch is how many rows of each kind one pass claims. The default is
// Concurrency: a claimed row waits for the rows ahead of it, and one
// that waits is held from every other replica meanwhile.
Batch int `yaml:"batch"`
// Lease is how long a claimed row is held. It must outlast drawing the
// batch; the default is worked out from Batch, Concurrency and
// RenderTimeout, and a lease shorter than that is refused.
Lease time.Duration `yaml:"lease"`
// Poll is how long an idle worker waits before asking for work again.
Poll time.Duration `yaml:"poll"`
// MaxAttempts is how many times a document is tried before one that
// never finishes -- the renderer stopped answering while it was drawn,
// or its file could not be read -- is recorded as not drawable.
MaxAttempts int `yaml:"max_attempts"`
// RetryBackoff is how long a document is held back after its first
// attempt that did not finish; each later one waits four times longer,
// up to an hour.
RetryBackoff time.Duration `yaml:"retry_backoff"`
}
Config is the deployment's `thumbnails:` section. A tile is drawn in a headless Chrome the platform reaches over the DevTools protocol, and the platform answers every request the browser makes itself, so the renderer needs no network of its own and never reaches back to the platform.
Enabled by default (nil = enabled). With no renderer answering, tiles keep their content-type icons and the platform logs once that it cannot draw.
The rest paces the worker (#1868). Every field's zero value is its default, so a deployment sets nothing.
func (Config) EffectiveRendererURL ¶
EffectiveRendererURL is RendererURL, or the default beside the platform.
type Deps ¶
type Deps struct {
Drawer Drawer
Assets AssetWork
Refs RefLister
AssetBlobs Blobs
Collections CollectionWork
// CollectionBucket is where a collection's mosaic is stored.
CollectionBucket string
Resources resource.ThumbnailWork
ResourceBlobs Blobs
ResourceBucket string
// Routes is the platform's own HTTP routes. The page a tile is drawn from
// loads the viewer's chunks, the served slide runtime and a document's
// declared references from the platform, and those requests are answered
// by calling these routes in-process.
Routes http.Handler
// TileEntryURL and TileCSS are the tile page's script and stylesheet.
TileEntryURL string
TileCSS string
}
Deps is what the worker draws with and where it records the result. Assets is required; collections and resources are drawn when their stores are present.
type Drawer ¶
type Drawer interface {
Ping(ctx context.Context) error
Render(ctx context.Context, p headless.Page) ([]byte, error)
}
Drawer is the browser a tile is drawn in.
type RefLister ¶
type RefLister interface {
ListByAsset(ctx context.Context, assetID string) ([]assetrefs.Ref, error)
}
RefLister lists the references an asset declared.
type Tuning ¶
type Tuning struct {
// Poll is how long an idle worker waits before asking for work again.
Poll time.Duration
// Lease is how long a claimed row is held. It must outlast drawing the
// batch it was claimed in; a replica that dies mid-render loses its lease
// at this.
Lease time.Duration
// Batch is how many rows of each kind one pass claims.
Batch int
// Concurrency is how many documents are drawn at once.
Concurrency int
// RenderTimeout bounds one render. A document that has not reported itself
// drawn by then is recorded as not drawable.
RenderTimeout time.Duration
// MaxAttempts is how many claims a document is given before one that
// never finishes is recorded as not drawable (#1868).
MaxAttempts int
// RetryBackoff is the hold after a document's first unfinished attempt.
RetryBackoff time.Duration
}
Tuning paces the worker. The zero value is the defaults; Config.Tuning is how a deployment's section becomes one.
type Worker ¶
type Worker struct {
// contains filtered or unexported fields
}
Worker draws owed tiles until stopped.