mediaops

package
v0.21.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 17, 2026 License: Apache-2.0 Imports: 15 Imported by: 0

Documentation

Overview

Package mediaops implements the offload_edit_image / offload_media engines: deterministic image edits (PIL worker + GIMP design-file conversion) and ffmpeg av operations. Everything here is CPU work — NO GPU lock, no llama-swap eviction; these ops run in parallel with renders. Spec: docs/superpowers/specs/2026-07-16-edit-media-tools-design.md.

Index

Constants

This section is empty.

Variables

View Source
var ErrEngineAbsent = errors.New("engine absent on this machine")

ErrEngineAbsent marks a defer-class failure: the machine simply lacks the engine (unset/missing binary). Handlers turn any error into a clean defer; this sentinel lets the reason say "engine absent" instead of "failed".

Functions

func BuildConcatList

func BuildConcatList(inputs []string) (string, error)

BuildConcatList renders the concat demuxer list file content. Single quotes in paths use the demuxer's '\” escape. Pure.

func BuildFFmpegArgs

func BuildFFmpegArgs(r MediaRequest) ([]string, error)

BuildFFmpegArgs assembles the ffmpeg argv for one media op. Pure — no filesystem, no exec — so every op's argv is unit-testable. The caller prepends the ffmpeg path.

func BuildGimpScript

func BuildGimpScript(src, dst, layersPath string) (string, error)

BuildGimpScript renders the flatten_design script-fu for (src -> dst raster, layer sidecar). Pure. src must be a .xcf or .psd.

func BuildInstantiateScript

func BuildInstantiateScript(src, dst string, setText, replaceImage map[string]string) (string, error)

BuildInstantiateScript renders the instantiate_design script-fu: open the .xcf/.psd template, set named text layers' copy (gimp-text-layer-set-text), swap named pixel layers for replacement images (loaded via gimp-file-load-layer, inserted at the old layer's offsets, old removed), flatten, save the raster to dst. Pure; deterministic (layer names sorted). A layer-name mismatch errors inside GIMP (gimp-image-get-layer-by-name) and surfaces on stderr — the single most common template failure. PDB names verified against gimp-console-3.2 (2026-07-17): shipped 3.2 scripts use gimp-drawable-get-offsets (3.x rename), insert-layer parent 0.

func GimpArgs

func GimpArgs(script string) []string

GimpArgs wraps a built script in the gimp-console batch argv (always ends with gimp-quit so the console exits).

func ResolveEditPython

func ResolveEditPython(editPython, comfyDir string) string

ResolveEditPython resolves the PIL engine's python: the explicit config value when set, else <comfyDir>/.venv/{Scripts/python.exe|bin/python}. "" = absent (the caller defers). Existence-checked — a configured-but-missing engine is absent, not an error at call time.

func ToSeconds

func ToSeconds(v string) (float64, error)

ToSeconds parses "ss", "ss.ms", "mm:ss", or "hh:mm:ss(.ms)" into seconds.

func UsesGimp

func UsesGimp(ops []EditOp) bool

UsesGimp reports whether the pipeline needs the GIMP engine (spec: only flatten_design and instantiate_design do, and validation pins both to ops[0]).

func ValidateOps

func ValidateOps(ops []EditOp) error

ValidateOps checks a whole pipeline BEFORE any work starts, so a bad op defers with a message naming the offending index instead of failing mid-pipeline with a half-written file. Pure.

func ValidateRenditions

func ValidateRenditions(rs []Rendition) error

ValidateRenditions checks the export matrix BEFORE any work starts. Pure. An empty set is valid (renditions are optional).

Types

type EditConfig

type EditConfig struct {
	Python      string        // resolved PIL python ("" = absent)
	GimpConsole string        // gimp-console path ("" = absent)
	Worker      string        // absolute path of render/edit_image.py
	Timeout     time.Duration // edit_timeout_sec
}

EditConfig carries this machine's edit-engine bindings (from config).

type EditOp

type EditOp struct {
	Op string `json:"op"`
	// crop / composite / text placement
	X      int `json:"x,omitempty"`
	Y      int `json:"y,omitempty"`
	Width  int `json:"width,omitempty"`
	Height int `json:"height,omitempty"`
	// resize
	KeepAspect *bool `json:"keep_aspect,omitempty"`
	// convert
	Format string `json:"format,omitempty"`
	// composite
	Overlay string  `json:"overlay,omitempty"`
	Opacity float64 `json:"opacity,omitempty"`
	// text
	Text   string `json:"text,omitempty"`
	Size   int    `json:"size,omitempty"`
	Color  string `json:"color,omitempty"`
	Font   string `json:"font,omitempty"`
	Anchor string `json:"anchor,omitempty"`
	// mask_boxes (white-on-black inpaint mask builder; REPLACES the working image)
	Boxes   []MaskBox `json:"boxes,omitempty"`
	Feather int       `json:"feather,omitempty"`
	Pad     int       `json:"pad,omitempty"`
	Invert  bool      `json:"invert,omitempty"`
	// perspective_composite (overlay reused; quad = 4 [x,y] dest corners UL,UR,LR,LL)
	Quad [][]float64 `json:"quad,omitempty"`
	// lut_cube (.cube 3D LUT; strength 0-1 blends graded over original, nil = 1.0)
	Path     string   `json:"path,omitempty"`
	Strength *float64 `json:"strength,omitempty"`
	// grade (tone/color: everything composes into ONE LUT per channel, single quantize)
	Levels        *GradeLevels `json:"levels,omitempty"`
	Curve         *GradeCurve  `json:"curve,omitempty"`
	WB            *GradeWB     `json:"wb,omitempty"`
	LuminanceOnly bool         `json:"luminance_only,omitempty"`
	// finish (delivery sharpening — MUST be the LAST op, after any resize:
	// sharpening before a resize is undone by resampling)
	Sharpen *FinishSharpen `json:"sharpen,omitempty"`
	Median  int            `json:"median,omitempty"`
	// instantiate_design (FIRST op only; GIMP layered-template factory —
	// layer name -> new text copy / replacement image path)
	SetText      map[string]string `json:"set_text,omitempty"`
	ReplaceImage map[string]string `json:"replace_image,omitempty"`
}

EditOp is one step of an offload_edit_image pipeline (spec §1). Ops are applied in order by render/edit_image.py (PIL); flatten_design runs first via GIMP.

type EditRequest

type EditRequest struct {
	Image      string
	Ops        []EditOp
	Out        string
	Renditions []Rendition // optional export matrix derived from the master Out
}

EditRequest is one offload_edit_image call (ops already shaped; validated here).

type EditResult

type EditResult struct {
	Out        string   `json:"image_path"`
	Width      int      `json:"width"`
	Height     int      `json:"height"`
	OpsApplied int      `json:"ops_applied"`
	Layers     []Layer  `json:"layers,omitempty"`
	Renditions []string `json:"renditions,omitempty"`
	Engine     string   `json:"engine"`
}

EditResult mirrors the tool's return payload (spec §1).

func RunEditImage

func RunEditImage(ctx context.Context, cfg EditConfig, req EditRequest) (EditResult, error)

RunEditImage validates and executes a whole edit pipeline: optional GIMP flatten first (design files), then the PIL worker for the remaining ops.

type FinishSharpen

type FinishSharpen struct {
	Radius    *float64 `json:"radius,omitempty"`
	Percent   *int     `json:"percent,omitempty"`
	Threshold *int     `json:"threshold,omitempty"`
}

FinishSharpen tunes the finish op's unsharp mask. ABSENT fields mean "worker default" (radius 1.2, percent 80, threshold 3 — post-AI-upscale web delivery). Pointers, not values: an EXPLICIT 0 (e.g. percent 0 = no visible sharpening, the Go-path way to get a median-only finish) must survive the struct round-trip — a value field with omitempty silently dropped it and the worker re-defaulted it (review finding 2026-07-17).

type GradeCurve

type GradeCurve struct {
	Points [][]float64 `json:"points"`
}

GradeCurve is a piecewise-linear tone curve through [in,out] control points (0-255).

type GradeLevels

type GradeLevels struct {
	Black int     `json:"black,omitempty"`
	White int     `json:"white,omitempty"`
	Gamma float64 `json:"gamma,omitempty"`
}

GradeLevels is the levels sub-adjustment of a grade op. Zero values mean "worker default" (black 0, white 255, gamma 1.0) — the worker fills them in.

type GradeWB

type GradeWB struct {
	Mode string  `json:"mode"`
	R    float64 `json:"r,omitempty"`
	G    float64 `json:"g,omitempty"`
	B    float64 `json:"b,omitempty"`
}

GradeWB is the white-balance sub-adjustment: mode "gray_world" (automatic) or "scale" with explicit per-channel multipliers (0 = worker default 1.0).

type Layer

type Layer struct {
	Name    string `json:"name"`
	Visible bool   `json:"visible"`
}

Layer is one entry of a design file's layer listing.

func ParseLayerList

func ParseLayerList(content string) []Layer

ParseLayerList parses the sidecar layer file ("LAYER:<name>|visible|hidden" per line; the name may itself contain pipes — split on the LAST one). Pure.

type MaskBox

type MaskBox struct {
	X      int `json:"x"`
	Y      int `json:"y"`
	Width  int `json:"width"`
	Height int `json:"height"`
}

MaskBox is one white rectangle of a mask_boxes op, in pixel coordinates of the working image (the same contract as offload_inpaint_image's mask: white = repaint).

type MediaConfig

type MediaConfig struct {
	FFmpeg  string        // ffmpeg_path ("" = absent)
	Timeout time.Duration // edit_timeout_sec governs media ops too
}

MediaConfig carries the ffmpeg binding.

type MediaRequest

type MediaRequest struct {
	Op        string   // trim | concat | extract_frames | convert | mux_audio | probe
	In        string   // input path (all ops except concat)
	Inputs    []string // concat inputs (>=2)
	ListPath  string   // concat: path of the demuxer list file the caller wrote
	Out       string   // output path (probe: unused)
	Start     string   // trim: seconds or hh:mm:ss
	End       string   // trim: absolute end time (resolved to Duration in RunMedia)
	Duration  string   // trim: seconds
	FPS       float64  // extract_frames: sampling rate
	Count     int      // extract_frames: total frames (resolved to FPS via probe in RunMedia)
	Audio     string   // mux_audio: audio input path
	Shortest  bool     // mux_audio: stop at the shorter input (default true at the handler)
	Reencode  bool     // trim: re-encode for exact cuts instead of keyframe-snapped -c copy
	AudioOnly bool     // convert: drop video (-vn)
	VideoOnly bool     // convert: drop audio (-an)
}

MediaRequest is one offload_media operation. Exactly one op per call (spec §1).

type MediaResult

type MediaResult struct {
	MediaPath   string   `json:"media_path,omitempty"`
	DurationSec float64  `json:"duration_sec,omitempty"`
	Frames      []string `json:"frames,omitempty"`
	Count       int      `json:"count,omitempty"`
	Probe       *Probe   `json:"probe,omitempty"`
}

MediaResult is the op-specific payload (exactly one field group is set).

func RunMedia

func RunMedia(ctx context.Context, cfg MediaConfig, req MediaRequest) (MediaResult, error)

RunMedia executes one offload_media op end-to-end.

type Probe

type Probe struct {
	DurationSec float64       `json:"duration_sec"`
	Format      string        `json:"format"`
	Streams     []ProbeStream `json:"streams"`
}

Probe is the parsed `ffmpeg -i` banner (spec §1: probe op).

func ParseProbe

func ParseProbe(stderr string) (Probe, error)

ParseProbe extracts duration/format/streams from `ffmpeg -i` stderr. The banner grammar is fixture-tested against the REAL imageio ffmpeg 7.1 output so a future ffmpeg bump that changes the banner fails in tests, not silently in production.

type ProbeStream

type ProbeStream struct {
	Type       string  `json:"type"` // video | audio
	Codec      string  `json:"codec"`
	Width      int     `json:"width,omitempty"`
	Height     int     `json:"height,omitempty"`
	FPS        float64 `json:"fps,omitempty"`
	SampleRate int     `json:"sample_rate,omitempty"`
}

type Rendition

type Rendition struct {
	Width  int    `json:"width,omitempty"`
	Height int    `json:"height,omitempty"`
	Format string `json:"format"`
	Suffix string `json:"suffix"`
}

Rendition is one entry of the optional export matrix: after the ops pipeline produces the master out, each rendition re-runs the worker with a resize+convert pair, writing <out-stem><suffix>.<format>.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL