capx

package
v0.33.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 17, 2026 License: MIT Imports: 18 Imported by: 0

Documentation

Overview

Package capx is the headless media capture capability of the framework: deterministic frame-by-frame rendering of an in-memory HTML document with a local headless Chrome-family browser (GSAP timeline seek + screenshot into PNG frames), plus ffmpeg-based encoding and compositing helpers. It is the engine behind the agent video studio (docs/plan-agent-video-studio-architecture-2026-11-14.md §3) and any other capability that needs "HTML timeline -> frames -> MP4".

Gating contract (ADR 0014 shape, same as platform/mcpx): the capability is env-gated and silent by default. With Config.Enabled nil or false, New returns an inert runtime; every capability method on it returns ErrNotConfigured. Not configured is diagnosable, never silent — products log why the capability is off at boot. Availability of the external tools is probed once at New time; Runtime.Available reports the combined result. Health checks must NOT probe Chrome or ffmpeg: an optional capability must not degrade readiness for products that never use video (§3.1).

External dependencies: a Chrome-family browser (driven through chromedp) and the ffmpeg binary (os/exec only — no cgo). Set the Config paths to bypass auto-probing; see ProbeChrome and ProbeFFMpeg for the probe order.

Determinism contract (Renderer.RenderFrames): a rendered frame depends only on (doc.HTML, doc.TimelineID, frame index, FrameSpec) — same machine, same inputs, byte-identical PNGs. The projected document must cooperate: motion driven exclusively by GSAP timelines registered under window.__timelines (no CSS animations or transitions), fonts inlined (woff2 data URIs), images inlined (data URIs on the render path), readiness signaled via window.__player.ready, and the player using gsap.ticker.lagSmoothing(0) with paused timelines so that only pause(t) positions the playhead.

Boundary (AGENTS.md boundary clause): only the LocalChrome renderer and the ffmpeg helpers in encode.go are consumed paths today. remote.go declares the not-yet-consumed RemoteHTTP extension point (design §3.5). The vendored GSAP copy at assets/gsap.min.js (version 3.13.0) is the version authority that the video domain's projector reads and inlines — capx does not embed it into Go.

Index

Constants

View Source
const (
	DefaultPoolSize      = 1
	DefaultLaunchTimeout = 20 * time.Second
	// DefaultEncodeCRF is the x264 constant rate factor used when
	// EncodeOptions.CRF is unset (design §3.4: crf 18).
	DefaultEncodeCRF = 18
	// DefaultEncodePreset is the x264 speed preset used when unset.
	DefaultEncodePreset = "medium"
)

Defaults for the optional Config and EncodeOptions fields.

Variables

View Source
var ErrNoLibass = errors.New("capx: ffmpeg has no libass support (subtitles filter missing); degrade to soft subs (mov_text)")

ErrNoLibass is returned by BurnSubtitles when the local ffmpeg build has no libass support (the subtitles filter is absent). Callers degrade to soft subtitle packaging (mov_text) and note it in render.error (design §3.4).

View Source
var ErrNotConfigured = errors.New("capx: capability not configured (Config.Enabled is nil or false)")

ErrNotConfigured is returned by every capability method on an inert runtime (Config.Enabled nil or false). Treat it as "capability off", not as a failure: log it once at boot and route users to the setup docs.

View Source
var ErrRemoteNotWired = errors.New("capx: remote renderer is a declared extension point and is not wired in this release (use Runtime.Renderer / LocalChrome)")

ErrRemoteNotWired is returned by every RemoteHTTP method until a real implementation lands. It mirrors ErrNotConfigured: diagnosable, not silent — callers can distinguish "capability gated off" from "remote renderer declared but not implemented".

Functions

func NewStubInstance

func NewStubInstance() *browserInstance

NewStubInstance builds a no-op browserInstance for launcher stubs in tests (nothing to probe, nothing to close).

func ProbeChrome

func ProbeChrome(explicit string) (string, error)

ProbeChrome resolves the browser executable: explicit path first, then LookPath over the common Chrome-family names, then the platform default install locations (Linux relies on LookPath alone). Exported so products can diagnose availability before assembling a runtime.

func ProbeFFMpeg

func ProbeFFMpeg(explicit string) (string, error)

ProbeFFMpeg resolves the ffmpeg executable: explicit path first, then LookPath("ffmpeg"). Exported so products can diagnose availability before assembling a runtime.

Types

type Config

type Config struct {
	// Enabled gates the whole capability. nil counts as false (silent
	// degradation, metrics/mcpx contract): New returns an inert runtime and
	// every method returns ErrNotConfigured.
	Enabled *bool
	// ChromePath is an explicit path to a Chrome-family browser executable.
	// Empty = auto-probe: LookPath over "chrome", "google-chrome",
	// "chromium", "msedge", then the platform default install locations.
	ChromePath string
	// FFMpegPath is an explicit path to the ffmpeg executable. Empty =
	// auto-probe via LookPath("ffmpeg").
	FFMpegPath string
	// PoolSize is the number of warm headless browser instances kept for
	// reuse (a render job occupies one instance for its whole run; frames
	// reuse a tab inside it, never a new process). Default 1.
	PoolSize int
	// LaunchTimeout bounds a single browser instance launch. Default 20s.
	LaunchTimeout time.Duration
}

Config gates and shapes the capability.

type Doc

type Doc struct {
	// HTML is the complete document (inline gsap + player + IR JSON). It is
	// injected with Page.setDocumentContent — no HTTP serving involved: no
	// auth, no network jitter, determinism by construction.
	HTML string
	// TimelineID is the composition id the projected player registered under
	// window.__timelines. RenderFrames seeks exactly that timeline.
	TimelineID string
}

Doc is one self-contained projected document.

type EncodeOptions

type EncodeOptions struct {
	// CRF is the x264 constant rate factor (0-51, lower = better quality).
	// Zero means the default (18) — visually lossless for motion graphics
	// at a sane bitrate.
	CRF int
	// Preset is the x264 speed preset (ultrafast..veryslow). Empty = the
	// default "medium".
	Preset string
}

EncodeOptions tunes the libx264 encode.

type Frame

type Frame struct {
	// Index is the 0-based frame number.
	Index int
	// Time is Index/FPS in seconds — the exact seek position used.
	Time float64
	// PNG is the captured screenshot bytes.
	PNG []byte
}

Frame is one rendered frame handed to the FrameFunc callback.

type FrameFunc

type FrameFunc func(ctx context.Context, f Frame) error

FrameFunc consumes one frame. Returning an error aborts the render and the error is surfaced (wrapped with the frame number) by RenderFrames.

type FrameSpec

type FrameSpec struct {
	// Width and Height are the output resolution; they become the browser
	// viewport exactly (deviceScaleFactor 1, no scrollbars).
	Width, Height int
	// FPS is the frames-per-second rate of the timeline seek grid.
	FPS int
	// DurationFrames is the total frame count (= ceil(duration * fps)).
	DurationFrames int
}

FrameSpec fixes the output geometry and timing.

func (FrameSpec) Validate

func (s FrameSpec) Validate() error

Validate sanity-checks a spec for rendering or encoding. Odd dimensions are rejected: libx264 yuv420p requires even width/height and the encode would fail on the first frame (audit §4 P10). Callers that accept arbitrary dimensions should round to even before constructing the spec; the encode pipeline additionally carries a trunc-to-even scale filter as a belt-and-braces fallback.

type FrameWriter

type FrameWriter struct {
	// contains filtered or unexported fields
}

FrameWriter turns per-frame PNG writes into the length-prefixed stream EncodePNGStream consumes on stdin. Not safe for concurrent use.

func NewFrameWriter

func NewFrameWriter(w io.Writer) *FrameWriter

NewFrameWriter wraps w.

func (*FrameWriter) WriteFrame

func (f *FrameWriter) WriteFrame(png []byte) error

WriteFrame writes one PNG frame (length prefix + payload).

type LocalChromeRenderer

type LocalChromeRenderer struct {
	Runtime *Runtime
}

LocalChromeRenderer is the local headless-Chrome Renderer implementation over warm browser instances owned by its Runtime. Construct it via Runtime.Renderer (which enforces the gating contract); the exported type exists so the Renderer slot is a concrete, swappable value (the future RemoteHTTP implements the same interface).

func (*LocalChromeRenderer) RenderFrames

func (l *LocalChromeRenderer) RenderFrames(ctx context.Context, doc Doc, spec FrameSpec, onFrame FrameFunc) error

RenderFrames implements the per-frame capture flow of design §3.3: fresh tab -> setDocumentContent -> wait player ready -> for each frame: pause(t, false) -> double rAF flush -> Page.captureScreenshot -> onFrame.

type RemoteHTTP

type RemoteHTTP struct{}

RemoteHTTP marks the remote renderer extension point; see the file comment for the wiring boundary.

func (RemoteHTTP) RenderFrames

func (RemoteHTTP) RenderFrames(ctx context.Context, doc Doc, spec FrameSpec, onFrame FrameFunc) error

RenderFrames stub satisfies the Renderer interface shape so that a future implementation compiles against the same contract tests.

type Renderer

type Renderer interface {
	// RenderFrames renders spec.DurationFrames frames of doc at
	// spec.Width x spec.Height, spec.FPS frames per second, calling onFrame
	// once per frame in index order (0 .. DurationFrames-1). An error from
	// onFrame aborts the render and is returned wrapped with the frame
	// number; a failed capture or seek likewise. Canceling ctx aborts the
	// loop between frames (in-flight protocol calls return on their own).
	RenderFrames(ctx context.Context, doc Doc, spec FrameSpec, onFrame FrameFunc) error
}

Renderer is the single capture contract of the package (design §3.2): it turns a self-contained document plus a frame spec into a stream of PNG frames. The document is produced by the caller (the video domain's projection layer); capx owns only the browser mechanics.

Implementations must be deterministic per the package contract: the frame at index i depends only on (doc.HTML, doc.TimelineID, i, spec) — same machine, same inputs, byte-identical PNGs.

type Runtime

type Runtime struct {
	// contains filtered or unexported fields
}

Runtime is the assembled capability. Create it with New; the zero value is not usable. Safe for concurrent use.

func New

func New(cfg Config) (*Runtime, error)

New assembles the runtime.

Gating: with Enabled nil or false it returns an inert runtime and a nil error — no probing happens and tool absence is not reported. Every method of that runtime returns ErrNotConfigured (diagnosable, not silent).

With Enabled true it probes for both external tools immediately, so that misconfiguration surfaces at boot with a reason. On probe failure it still returns the runtime (with Available() == false) together with the error; callers that treat the capability as optional log the error and keep the runtime so that Available() stays the single liveness signal.

func (*Runtime) Available

func (r *Runtime) Available() bool

Available reports whether the runtime can render and encode. False means either "gated off" (inert runtime) or "a required external tool was missing when New probed" — the reason is diagnosable through New's error return or boot logs.

func (*Runtime) BurnSubtitles

func (r *Runtime) BurnSubtitles(ctx context.Context, videoPath, srtPath string, w io.Writer) error

BurnSubtitles burns an SRT file into videoPath and writes the result to w. Requires an ffmpeg build with libass; otherwise returns ErrNoLibass so the caller can fall back. Video is stream-copied besides the subtitle filter:

ffmpeg -i video -vf subtitles=<srt> -c:v libx264 -crf <default> \
    -pix_fmt yuv420p -c:a copy out.mp4

func (*Runtime) Close

func (r *Runtime) Close() error

Close shuts the browser pool down and closes every warm instance. It is idempotent and safe on inert runtimes. Instances currently checked out by in-flight renders are closed when those renders release them.

func (*Runtime) EncodePNGStream

func (r *Runtime) EncodePNGStream(ctx context.Context, w io.Writer, spec FrameSpec, frames io.Reader, opts EncodeOptions) error

EncodePNGStream encodes a PNG frame stream into an MP4 written to w.

The frames come from the caller through the returned FrameFunc-style sink: this method returns an error only for the encode itself; the frames are pushed by the caller via FrameWriter, whose writer end must be supplied.

Concretely, the caller passes frames through an intermediate pipe it owns and passes that pipe's READER to this method as frames — the reader must carry the length-prefixed frame protocol written with FrameWriter (8-byte LE length + PNG payload per frame). The protocol is decoded here and the raw PNG bytes are fed to ffmpeg on stdin via image2pipe, so there is no intermediate disk file:

ffmpeg -f image2pipe -framerate <fps> -i - -c:v libx264 \
    -pix_fmt yuv420p -crf <crf> -preset <preset> \
    -movflags frag_keyframe+empty_moov+default_base_moof -f mp4 -

A typical wiring pairs this with RenderFrames: the onFrame callback writes each captured PNG through a FrameWriter, and the reading end of that pipe is handed to EncodePNGStream — a fully streaming render-to-encode pipeline.

The ffmpeg process is killed when ctx is canceled. Stderr is captured and reported (tail) on failure; on success it is discarded.

func (*Runtime) ExtractFrameAt

func (r *Runtime) ExtractFrameAt(ctx context.Context, videoPath string, sec float64, w io.Writer) error

ExtractFrameAt writes the single video frame at sec seconds into w as PNG:

ffmpeg -ss <sec> -i video -frames:v 1 -c:v png -f image2 -

sec may be fractional; input seeking (-ss before -i) is fast and frame-accurate on every supported ffmpeg release. The output codec is PNG so callers get a lossless frame with a deterministic content type — ExtractThumbnail above keeps its legacy fixed-1s JPEG behavior untouched. The PNG bytes are streamed to w (stdout pipe); nothing touches disk. A sec past the end of the stream makes ffmpeg produce zero frames: the call succeeds with an empty write, and callers should treat that as "no frame at that time". The ffmpeg process is killed when ctx is canceled.

func (*Runtime) ExtractThumbnail

func (r *Runtime) ExtractThumbnail(ctx context.Context, videoPath string, w io.Writer) error

ExtractThumbnail writes a single JPEG frame taken early in videoPath to w:

ffmpeg -i video -ss 00:00:01 -frames:v 1 -q:v 2 -f image2 -

func (*Runtime) MuxAV

func (r *Runtime) MuxAV(ctx context.Context, out io.Writer, videoPath, voicePath, bgmPath string, voiceVolume, bgmVolume float64) error

MuxAV muxes a video file with optional voice-over and background music into out. voicePath or bgmPath may be empty; bgmVolume scales the music track (before mixing). Video is stream-copied, audio is mixed with amix:

-i video -i voice -i bgm \
-filter_complex "[1:a]volume=<vv>[voice];[2:a]volume=<bv>[bgm];
                 [voice][bgm]amix=inputs=2:duration=first:normalize=0,apad[a]" \
-map 0:v -map "[a]" -c:v copy out.mp4

Voice-only / bgm-only / silent variants drop the missing inputs from the graph. Design §3.4.

voiceVolume is applied (it used to be ignored — IR validated the field but the mix never consumed it, so "voice_volume: 0" still produced audible narration; delivery audit P1). normalize=0 keeps the declared gains intact: ffmpeg's amix default divides by the input count, which silently halved both tracks.

func (*Runtime) Renderer

func (r *Runtime) Renderer() (Renderer, error)

Renderer returns the Renderer view of the runtime (LocalChrome). On an inert runtime it returns ErrNotConfigured.

func (*Runtime) SetLauncherForTest

func (r *Runtime) SetLauncherForTest(l browserLauncher)

SetLauncherForTest replaces the browser-launch seam. TEST-ONLY: it exists so the pool/concurrency contract is exercisable without a real Chrome; production wiring must never call it.

func (*Runtime) SoftSubtitles

func (r *Runtime) SoftSubtitles(ctx context.Context, videoPath, srtPath string, w io.Writer) error

SoftSubtitles packages an SRT file as an embedded mov_text subtitle track inside the MP4 written to w. The video stream is copied without re-encoding (the deliverable keeps the already-encoded picture; players show the track as toggleable captions instead of burned-in pixels):

ffmpeg -i video -i srt -map 0:v -map 1:s \
    -c:v copy -c:s mov_text -f mp4 -

This is the no-libass degradation target for BurnSubtitles (design §3.4) and the native implementation of subtitle_mode=soft.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL