Documentation
¶
Overview ¶
Package engine is a pure-Go (CGO=0) web rendering engine: it fetches a URL, parses the HTML into a DOM, applies a real CSS subset (cascade, inheritance, var(), @media, dark-mode, gradients), runs the page's JavaScript against a real DOM binding, lays the content out with a full box model (block, inline, float, flex, grid, table, position), and paints anti-aliased text, backgrounds, gradients, box-shadows, images and SVG to an image.RGBA. A settle-then-render loop re-lays-out after scripts mutate the DOM, so script-driven changes are reflected in the output; set Engine.DisableJS to render the static, no-JavaScript document instead.
This file adds an IMAGE-DOCUMENT path to Fetch. A URL whose response is a raster image (Content-Type image/png|jpeg|gif|webp|bmp|…, i.e. raw image bytes rather than an HTML page — think a direct https://i.redd.it/abc.jpg link) has no markup to parse: feeding its bytes to the HTML parser yields a garbage/empty DOM and nothing renders. A browser instead wraps a bare image in a synthetic document that centres/sizes the image; the engine does the analogue here, synthesising `<img style="width:100%">` markup so the image flows through the exact same decode + layout + paint path as any other <img>, spanning the full viewport width (upscaled if it is smaller, so a standalone image is always at least the pane width). SVG is deliberately excluded: a fetched SVG document parses as inline <svg> and already renders through the SVG path, so it is left untouched.
This file adds a hyperlink hit-map on top of the laid-out box tree: given a rendered page it reports the painted rectangle and resolved href of every <a href> anchor, so a consumer (e.g. the wasmdesk browserproxy) can turn a click at pixel (x, y) into a navigation target. It is kept deliberately in its own file — separate from engine.go and the js/ runtime work — so the hit-test can evolve without colliding with the rendering pipeline.
This file adds a last-resort OpenGraph/meta fallback: when a page renders to nothing — an un-hydratable JavaScript SPA whose content lives only in a React initial-state blob (Mastodon, X, many app-shell sites) — the engine synthesises a clean, readable card from the page's og:/meta tags and renders that instead of a blank frame. It is gated by Engine.MetaFallback and only ever replaces a genuinely empty render, so pages that render real content are completely unaffected.
This file adds STAGED progressive rendering on top of the batch pipeline. A browser paints before all resources/scripts are done and re-flows incrementally; the engine emulates that with a small number of fully-styled frames (external CSS already applied, so no unstyled flash):
- "initial": the TEXT-FIRST frame — external sheets fetched and cascaded and the page laid out, but BEFORE any image is fetched (image boxes reserve from their attr/CSS sizes). The big perceived-latency win: the styled text paints without waiting on the slow, network-bound image fetch.
- "images": after the images/background images are loaded and the page is re-laid-out with their real intrinsic sizes (deduped against "initial" by geometry — a page with no images that moved nothing yields just initial+final).
- "settle": after each settle pass that visibly changed the geometry (deduped; bounded by the settle pass cap) — the interruptible-reflow analogue.
- "final" (Final=true): the fully-settled render, byte-identical to RenderWithLinks for the same input.
Every frame carries an INDEPENDENT image snapshot (a fresh canvas per frame), so a consumer may retain it after onFrame returns.
Index ¶
- func EncodePNG(img image.Image) ([]byte, error)
- func LinkAt(links []Link, pt image.Point) (string, bool)
- func Screenshot(ctx context.Context, rawurl string, viewport image.Rectangle) ([]byte, error)
- type Document
- type Engine
- func (e *Engine) Fetch(ctx context.Context, rawurl string) (*Document, error)
- func (e *Engine) Render(ctx context.Context, rawurl string, viewport image.Rectangle) (*image.RGBA, *RenderInfo, error)
- func (e *Engine) RenderDocument(ctx context.Context, doc *Document, viewport image.Rectangle) (*image.RGBA, *RenderInfo, error)
- func (e *Engine) RenderDocumentProgressive(ctx context.Context, doc *Document, viewport image.Rectangle, ...) (*RenderInfo, error)
- func (e *Engine) RenderDocumentWithLinks(ctx context.Context, doc *Document, viewport image.Rectangle) (*image.RGBA, *RenderInfo, []Link, error)
- func (e *Engine) RenderHTML(ctx context.Context, htmlSrc, baseURL string, viewport image.Rectangle) (*image.RGBA, *RenderInfo, error)
- func (e *Engine) RenderProgressive(ctx context.Context, rawurl string, viewport image.Rectangle, ...) (*RenderInfo, error)
- func (e *Engine) RenderWithLinks(ctx context.Context, rawurl string, viewport image.Rectangle) (*image.RGBA, *RenderInfo, []Link, error)
- func (e *Engine) Screenshot(ctx context.Context, rawurl string, viewport image.Rectangle) ([]byte, error)
- type Link
- type ProgressiveFrame
- type RenderInfo
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Document ¶
type Document struct {
URL string // final URL after redirects
Title string // <title> text
Root *dom.Node // owned DOM tree
HTML string // decoded (UTF-8) source
}
Document is a fetched and parsed page.
type Engine ¶
type Engine struct {
Client *http.Client
UserAgent string
// MaxImages bounds how many RASTER (non-SVG) <img>/background images are
// fetched and decoded per render — the expensive network + large-decode work.
MaxImages int
// MaxVectorImages bounds how many VECTOR images (inline <svg> and
// <img src="*.svg"> / data:image/svg+xml) are rasterised per render. Vector
// chrome (nav/footer/social/UI icons) is cheap but plentiful — real design
// systems place many dozens ahead of the content — so it gets its OWN, more
// generous budget. Without this split a wall of decorative icons early in the
// DOM exhausts a single shared budget and starves the actual content photos
// (raster) further down (the DeepMind hero black-void bug).
MaxVectorImages int
// DisableJS turns off the JavaScript pass. Offline fixture tests that must
// stay byte-deterministic set this; the default (false) runs page scripts.
DisableJS bool
// JSTimeout bounds the total script + timer budget per render. Zero selects
// js.DefaultTimeout.
JSTimeout time.Duration
// MaxJSHeapBytes bounds process heap growth during the JavaScript settle
// stage (script execution AND the esbuild module-bundle fetch/parse). Some
// real pages ship huge module graphs (GitHub, large SPAs) whose bundling
// balloons memory into the gigabytes before any time budget trips; a watchdog
// samples the heap and, once growth from the stage's start exceeds this bound,
// cancels the stage so it falls back to the already-computed pre-script layout
// instead of driving the process toward OOM. Zero selects a sane default; a
// negative value disables the guard.
MaxJSHeapBytes int64
// JSLog, if non-nil, receives console.* and diagnostic lines from the script
// pass (used for debugging; nil discards them).
JSLog func(string)
// Backdrop is the base colour painted under a page before its own background.
// The zero value (transparent) selects the default white paper. A themed host
// (e.g. a reader in dark mode) sets it to its surface colour so a page that
// declares no background of its own — most plainly a bare image — is framed to
// match instead of flashing white. A page that DOES declare a background still
// paints its own over this, so normal sites are unchanged.
Backdrop css.Color
// MetaFallback, when true, renders a clean readable card synthesised from a
// page's OpenGraph/meta tags (og:title/og:description/og:image, <title>) as a
// LAST RESORT — only when the real render is empty (a JavaScript SPA the engine
// cannot hydrate: Mastodon, X, app-shell sites). Default false: it fabricates
// content that is not in the page's rendered DOM, so it is opt-in; a
// link-preview / reader consumer enables it, while callers wanting a faithful
// render (or to detect the blank SPA themselves) are unaffected. It never
// triggers on a page that renders any real content, so normal pages are
// byte-identical whether it is on or off.
MetaFallback bool
}
Engine holds the HTTP client and render configuration.
func New ¶
func New() *Engine
New returns an Engine with a browser-like HTTP client (Chrome TLS fingerprint, cookie jar, redirect following).
func (*Engine) Render ¶
func (e *Engine) Render(ctx context.Context, rawurl string, viewport image.Rectangle) (*image.RGBA, *RenderInfo, error)
Render fetches url and renders it into an image at the given viewport width (the height is grown to fit the full page, at least the viewport height).
func (*Engine) RenderDocument ¶
func (e *Engine) RenderDocument(ctx context.Context, doc *Document, viewport image.Rectangle) (*image.RGBA, *RenderInfo, error)
RenderDocument renders an already-fetched Document. It is the offline entry point used by fixtures and tests (image sub-resources are still fetched from the network if their src is absolute/resolvable).
func (*Engine) RenderDocumentProgressive ¶ added in v0.3.0
func (e *Engine) RenderDocumentProgressive(ctx context.Context, doc *Document, viewport image.Rectangle, onFrame func(ProgressiveFrame)) (*RenderInfo, error)
RenderDocumentProgressive renders an already-fetched Document progressively. See RenderProgressive.
func (*Engine) RenderDocumentWithLinks ¶
func (e *Engine) RenderDocumentWithLinks(ctx context.Context, doc *Document, viewport image.Rectangle) (*image.RGBA, *RenderInfo, []Link, error)
RenderDocumentWithLinks renders an already-fetched Document and returns its image, render info and anchor hit-map from a single layout pass. It runs the same cascade → layout → paint pipeline as RenderDocument.
func (*Engine) RenderHTML ¶
func (e *Engine) RenderHTML(ctx context.Context, htmlSrc, baseURL string, viewport image.Rectangle) (*image.RGBA, *RenderInfo, error)
RenderHTML renders an HTML string (with baseURL used to resolve relative image sources) into an image at the given viewport. It is the offline entry point for local fixtures and demos, running the same cascade/layout/paint pipeline as Render but without fetching the page itself.
func (*Engine) RenderProgressive ¶ added in v0.3.0
func (e *Engine) RenderProgressive(ctx context.Context, rawurl string, viewport image.Rectangle, onFrame func(ProgressiveFrame)) (*RenderInfo, error)
RenderProgressive fetches rawurl and renders it progressively, invoking onFrame for each staged frame. It runs the same cascade → JS settle → layout → paint pipeline as RenderWithLinks; the Final frame's image, links and RenderInfo are identical to what RenderWithLinks returns for the same input. onFrame is always called at least once and the last call has Final=true.
func (*Engine) RenderWithLinks ¶
func (e *Engine) RenderWithLinks(ctx context.Context, rawurl string, viewport image.Rectangle) (*image.RGBA, *RenderInfo, []Link, error)
RenderWithLinks fetches url, renders it to a full-page image and, in the same layout pass, returns the anchor hit-map. It mirrors (*Engine).Render but also threads the laid-out box tree into LinksFromBox so the image and the link rectangles are guaranteed to describe the exact same layout. It is kept here (rather than folded into engine.go's RenderDocument) so the hit-test feature stays isolated from the core pipeline and the js/ runtime work.
type Link ¶
Link is one hyperlink's painted rectangle (in full-page image pixels, the same coordinate space as the image returned by Render) and its href resolved against the page URL to an absolute URL.
func LinksFromBox ¶
LinksFromBox walks a laid-out box tree and returns, in document order, one Link per <a href> anchor that painted at least one inline atom (a word or an image). Each anchor's Rect is the union of the rectangles of all inline atoms it produced, so a multi-line link yields a single bounding box covering every line. baseURL is the page's own URL: each anchor's raw href is resolved against it, and links that resolve to a non-navigable scheme (javascript:, mailto:, an empty or pure-fragment href) are dropped.
The rectangles are in the box tree's own coordinate space, which is the full-page image space (layout origin is 0,0 at the page top-left), so a caller can hit-test a full-page click directly against them.
type ProgressiveFrame ¶ added in v0.3.0
type ProgressiveFrame struct {
Img *image.RGBA
Links []Link
Info *RenderInfo
Stage string
Final bool
}
ProgressiveFrame is one staged snapshot delivered to a RenderProgressive callback. Img is an independent allocation; Links matches RenderWithLinks's shape; Stage is "initial" | "images" | "settle" | "final"; Final is true exactly once, on the last frame.
type RenderInfo ¶
RenderInfo carries metadata about a completed render.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
render
command
Command render fetches a URL and writes a PNG screenshot of the statically rendered page.
|
Command render fetches a URL and writes a PNG screenshot of the statically rendered page. |
|
Package css implements a deliberately small but real subset of CSS: a value model, a tokenizer/parser for stylesheets and declaration blocks, tag/class/ id selectors with specificity, and a cascade with inheritance over a dom tree.
|
Package css implements a deliberately small but real subset of CSS: a value model, a tokenizer/parser for stylesheets and declaration blocks, tag/class/ id selectors with specificity, and a cascade with inheritance over a dom tree. |
|
Package dom is a small, owned DOM node tree built from golang.org/x/net/html.
|
Package dom is a small, owned DOM node tree built from golang.org/x/net/html. |
|
Package js runs a page's JavaScript against a minimal but real DOM binding on a pure-Go goja ECMAScript runtime.
|
Package js runs a page's JavaScript against a minimal but real DOM binding on a pure-Go goja ECMAScript runtime. |
|
Package layout is a block-and-inline flow engine.
|
Package layout is a block-and-inline flow engine. |
|
Package paint rasterises a laid-out box tree onto an *image.RGBA using go-opentype for anti-aliased text, go-widgets/painter for backgrounds and go-images-decoded bitmaps for <img>.
|
Package paint rasterises a laid-out box tree onto an *image.RGBA using go-opentype for anti-aliased text, go-widgets/painter for backgrounds and go-images-decoded bitmaps for <img>. |
