Documentation
¶
Overview ¶
Package mediaops implements the offload_edit_image / offload_media engines: deterministic image edits (PIL worker + GIMP design-file conversion) and ffmpeg av operations. Everything here is CPU work — NO GPU lock, no llama-swap eviction; these ops run in parallel with renders. Spec: docs/superpowers/specs/2026-07-16-edit-media-tools-design.md.
Index ¶
- Variables
- func BuildConcatList(inputs []string) (string, error)
- func BuildFFmpegArgs(r MediaRequest) ([]string, error)
- func BuildGimpScript(src, dst, layersPath string) (string, error)
- func BuildInstantiateScript(src, dst string, setText, replaceImage map[string]string) (string, error)
- func GimpArgs(script string) []string
- func ResolveEditPython(editPython, comfyDir string) string
- func ToSeconds(v string) (float64, error)
- func UsesGimp(ops []EditOp) bool
- func ValidateOps(ops []EditOp) error
- func ValidateRenditions(rs []Rendition) error
- type EditConfig
- type EditOp
- type EditRequest
- type EditResult
- type FinishSharpen
- type GradeCurve
- type GradeLevels
- type GradeWB
- type Layer
- type MaskBox
- type MediaConfig
- type MediaRequest
- type MediaResult
- type Probe
- type ProbeStream
- type Rendition
Constants ¶
This section is empty.
Variables ¶
var ErrEngineAbsent = errors.New("engine absent on this machine")
ErrEngineAbsent marks a defer-class failure: the machine simply lacks the engine (unset/missing binary). Handlers turn any error into a clean defer; this sentinel lets the reason say "engine absent" instead of "failed".
Functions ¶
func BuildConcatList ¶
BuildConcatList renders the concat demuxer list file content. Single quotes in paths use the demuxer's '\” escape. Pure.
func BuildFFmpegArgs ¶
func BuildFFmpegArgs(r MediaRequest) ([]string, error)
BuildFFmpegArgs assembles the ffmpeg argv for one media op. Pure — no filesystem, no exec — so every op's argv is unit-testable. The caller prepends the ffmpeg path.
func BuildGimpScript ¶
BuildGimpScript renders the flatten_design script-fu for (src -> dst raster, layer sidecar). Pure. src must be a .xcf or .psd.
func BuildInstantiateScript ¶
func BuildInstantiateScript(src, dst string, setText, replaceImage map[string]string) (string, error)
BuildInstantiateScript renders the instantiate_design script-fu: open the .xcf/.psd template, set named text layers' copy (gimp-text-layer-set-text), swap named pixel layers for replacement images (loaded via gimp-file-load-layer, inserted at the old layer's offsets, old removed), flatten, save the raster to dst. Pure; deterministic (layer names sorted). A layer-name mismatch errors inside GIMP (gimp-image-get-layer-by-name) and surfaces on stderr — the single most common template failure. PDB names verified against gimp-console-3.2 (2026-07-17): shipped 3.2 scripts use gimp-drawable-get-offsets (3.x rename), insert-layer parent 0.
func GimpArgs ¶
GimpArgs wraps a built script in the gimp-console batch argv (always ends with gimp-quit so the console exits).
func ResolveEditPython ¶
ResolveEditPython resolves the PIL engine's python: the explicit config value when set, else <comfyDir>/.venv/{Scripts/python.exe|bin/python}. "" = absent (the caller defers). Existence-checked — a configured-but-missing engine is absent, not an error at call time.
func UsesGimp ¶
UsesGimp reports whether the pipeline needs the GIMP engine (spec: only flatten_design and instantiate_design do, and validation pins both to ops[0]).
func ValidateOps ¶
ValidateOps checks a whole pipeline BEFORE any work starts, so a bad op defers with a message naming the offending index instead of failing mid-pipeline with a half-written file. Pure.
func ValidateRenditions ¶
ValidateRenditions checks the export matrix BEFORE any work starts. Pure. An empty set is valid (renditions are optional).
Types ¶
type EditConfig ¶
type EditConfig struct {
Python string // resolved PIL python ("" = absent)
GimpConsole string // gimp-console path ("" = absent)
Worker string // absolute path of render/edit_image.py
Timeout time.Duration // edit_timeout_sec
}
EditConfig carries this machine's edit-engine bindings (from config).
type EditOp ¶
type EditOp struct {
Op string `json:"op"`
// crop / composite / text placement
X int `json:"x,omitempty"`
Y int `json:"y,omitempty"`
Width int `json:"width,omitempty"`
Height int `json:"height,omitempty"`
// resize
KeepAspect *bool `json:"keep_aspect,omitempty"`
// convert
Format string `json:"format,omitempty"`
// composite
Overlay string `json:"overlay,omitempty"`
Opacity float64 `json:"opacity,omitempty"`
// text
Text string `json:"text,omitempty"`
Size int `json:"size,omitempty"`
Color string `json:"color,omitempty"`
Font string `json:"font,omitempty"`
Anchor string `json:"anchor,omitempty"`
// mask_boxes (white-on-black inpaint mask builder; REPLACES the working image)
Boxes []MaskBox `json:"boxes,omitempty"`
Feather int `json:"feather,omitempty"`
Pad int `json:"pad,omitempty"`
Invert bool `json:"invert,omitempty"`
// perspective_composite (overlay reused; quad = 4 [x,y] dest corners UL,UR,LR,LL)
Quad [][]float64 `json:"quad,omitempty"`
// lut_cube (.cube 3D LUT; strength 0-1 blends graded over original, nil = 1.0)
Path string `json:"path,omitempty"`
Strength *float64 `json:"strength,omitempty"`
// grade (tone/color: everything composes into ONE LUT per channel, single quantize)
Levels *GradeLevels `json:"levels,omitempty"`
Curve *GradeCurve `json:"curve,omitempty"`
WB *GradeWB `json:"wb,omitempty"`
LuminanceOnly bool `json:"luminance_only,omitempty"`
// finish (delivery sharpening — MUST be the LAST op, after any resize:
// sharpening before a resize is undone by resampling)
Sharpen *FinishSharpen `json:"sharpen,omitempty"`
Median int `json:"median,omitempty"`
// instantiate_design (FIRST op only; GIMP layered-template factory —
// layer name -> new text copy / replacement image path)
SetText map[string]string `json:"set_text,omitempty"`
ReplaceImage map[string]string `json:"replace_image,omitempty"`
}
EditOp is one step of an offload_edit_image pipeline (spec §1). Ops are applied in order by render/edit_image.py (PIL); flatten_design runs first via GIMP.
type EditRequest ¶
type EditRequest struct {
Image string
Ops []EditOp
Out string
Renditions []Rendition // optional export matrix derived from the master Out
}
EditRequest is one offload_edit_image call (ops already shaped; validated here).
type EditResult ¶
type EditResult struct {
Out string `json:"image_path"`
Width int `json:"width"`
Height int `json:"height"`
OpsApplied int `json:"ops_applied"`
Layers []Layer `json:"layers,omitempty"`
Renditions []string `json:"renditions,omitempty"`
Engine string `json:"engine"`
}
EditResult mirrors the tool's return payload (spec §1).
func RunEditImage ¶
func RunEditImage(ctx context.Context, cfg EditConfig, req EditRequest) (EditResult, error)
RunEditImage validates and executes a whole edit pipeline: optional GIMP flatten first (design files), then the PIL worker for the remaining ops.
type FinishSharpen ¶
type FinishSharpen struct {
Radius *float64 `json:"radius,omitempty"`
Percent *int `json:"percent,omitempty"`
Threshold *int `json:"threshold,omitempty"`
}
FinishSharpen tunes the finish op's unsharp mask. ABSENT fields mean "worker default" (radius 1.2, percent 80, threshold 3 — post-AI-upscale web delivery). Pointers, not values: an EXPLICIT 0 (e.g. percent 0 = no visible sharpening, the Go-path way to get a median-only finish) must survive the struct round-trip — a value field with omitempty silently dropped it and the worker re-defaulted it (review finding 2026-07-17).
type GradeCurve ¶
type GradeCurve struct {
Points [][]float64 `json:"points"`
}
GradeCurve is a piecewise-linear tone curve through [in,out] control points (0-255).
type GradeLevels ¶
type GradeLevels struct {
Black int `json:"black,omitempty"`
White int `json:"white,omitempty"`
Gamma float64 `json:"gamma,omitempty"`
}
GradeLevels is the levels sub-adjustment of a grade op. Zero values mean "worker default" (black 0, white 255, gamma 1.0) — the worker fills them in.
type GradeWB ¶
type GradeWB struct {
Mode string `json:"mode"`
R float64 `json:"r,omitempty"`
G float64 `json:"g,omitempty"`
B float64 `json:"b,omitempty"`
}
GradeWB is the white-balance sub-adjustment: mode "gray_world" (automatic) or "scale" with explicit per-channel multipliers (0 = worker default 1.0).
type Layer ¶
Layer is one entry of a design file's layer listing.
func ParseLayerList ¶
ParseLayerList parses the sidecar layer file ("LAYER:<name>|visible|hidden" per line; the name may itself contain pipes — split on the LAST one). Pure.
type MaskBox ¶
type MaskBox struct {
X int `json:"x"`
Y int `json:"y"`
Width int `json:"width"`
Height int `json:"height"`
}
MaskBox is one white rectangle of a mask_boxes op, in pixel coordinates of the working image (the same contract as offload_inpaint_image's mask: white = repaint).
type MediaConfig ¶
type MediaConfig struct {
FFmpeg string // ffmpeg_path ("" = absent)
Timeout time.Duration // edit_timeout_sec governs media ops too
}
MediaConfig carries the ffmpeg binding.
type MediaRequest ¶
type MediaRequest struct {
Op string // trim | concat | extract_frames | convert | mux_audio | probe
In string // input path (all ops except concat)
Inputs []string // concat inputs (>=2)
ListPath string // concat: path of the demuxer list file the caller wrote
Out string // output path (probe: unused)
Start string // trim: seconds or hh:mm:ss
End string // trim: absolute end time (resolved to Duration in RunMedia)
Duration string // trim: seconds
FPS float64 // extract_frames: sampling rate
Count int // extract_frames: total frames (resolved to FPS via probe in RunMedia)
Audio string // mux_audio: audio input path
Shortest bool // mux_audio: stop at the shorter input (default true at the handler)
Reencode bool // trim: re-encode for exact cuts instead of keyframe-snapped -c copy
AudioOnly bool // convert: drop video (-vn)
VideoOnly bool // convert: drop audio (-an)
}
MediaRequest is one offload_media operation. Exactly one op per call (spec §1).
type MediaResult ¶
type MediaResult struct {
MediaPath string `json:"media_path,omitempty"`
DurationSec float64 `json:"duration_sec,omitempty"`
Frames []string `json:"frames,omitempty"`
Count int `json:"count,omitempty"`
Probe *Probe `json:"probe,omitempty"`
}
MediaResult is the op-specific payload (exactly one field group is set).
func RunMedia ¶
func RunMedia(ctx context.Context, cfg MediaConfig, req MediaRequest) (MediaResult, error)
RunMedia executes one offload_media op end-to-end.
type Probe ¶
type Probe struct {
DurationSec float64 `json:"duration_sec"`
Format string `json:"format"`
Streams []ProbeStream `json:"streams"`
}
Probe is the parsed `ffmpeg -i` banner (spec §1: probe op).
func ParseProbe ¶
ParseProbe extracts duration/format/streams from `ffmpeg -i` stderr. The banner grammar is fixture-tested against the REAL imageio ffmpeg 7.1 output so a future ffmpeg bump that changes the banner fails in tests, not silently in production.
type ProbeStream ¶
type Rendition ¶
type Rendition struct {
Width int `json:"width,omitempty"`
Height int `json:"height,omitempty"`
Format string `json:"format"`
Suffix string `json:"suffix"`
}
Rendition is one entry of the optional export matrix: after the ops pipeline produces the master out, each rendition re-runs the worker with a resize+convert pair, writing <out-stem><suffix>.<format>.