gpugen

package
v0.22.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 18, 2026 License: Apache-2.0 Imports: 12 Imported by: 0

Documentation

Overview

Package gpugen is the shared Go exec wrapper for every LOCAL GPU generation runner (image / video / audio). It shells out to a render/*.mjs (or a python TTS worker) that takes the single-slot GPU lock + drives ComfyUI/Chatterbox, and wraps that child with the hard-won lifecycle guards the bare runners lack:

  • killTree on timeout/cancel — on Windows a bare node-kill ORPHANS the spawned ComfyUI python grandchild (pinning ~8GB VRAM) and skips node's JS finally (leaking the GPU lock); we taskkill the WHOLE process tree.
  • WaitDelay — a short grace window after Cancel before the pipe is force-closed.
  • defer freeComfyVRAM — belt-and-suspenders: however the child ended (clean exit, error, or a timeout-kill that skipped its finally), force-drop any ComfyUI VRAM so a render never leaves the GPU pinned (zero-always-warm; protects the load-bearing CPU memory stack).

This was extracted from internal/imagegen.Generate so video + audio get the SAME process-tree-kill (they previously had no Go wrapper → no kill on timeout). Pure os/exec + net/http, no deps. The output-file stat is the success gate (a child can exit 0 yet produce nothing). Behavior for the image path is preserved byte-for-byte.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AsInt

func AsInt(v any) int

AsInt is the exported coercion used by thin callers assembling Args.

func ClassifyErr

func ClassifyErr(err error) string

ClassifyErr maps a gen failure to a coarse class (oom|timeout|conn_refused|other) for the ledger's ErrClass. Mirrors pipeline.classifyErr; nil => "".

func Generate

func Generate(ctx context.Context, spec Spec) (string, error)

Generate runs the spec's command, returning Out on success. A non-zero exit, a timeout (child + tree killed), or a missing/empty Out returns an error so the caller can map it to a clean defer. Never panics on a nil/absent process.

func ResolveScript added in v0.6.1

func ResolveScript(script string) (string, error)

ResolveScript resolves a configured render-script path for a GPU-gen runner. A RELATIVE path (the shipped defaults are "render/*.mjs") is resolved against the EXECUTABLE's directory, NOT the process cwd: an MCP host (e.g. the ~/.claude.json registration) spawns the server with no meaningful cwd, so a cwd-relative default made node fail with MODULE_NOT_FOUND — an instant defer on every video/voice/music call. If the resolved file does not exist, the error reads "script not found at <absolute-path>": a distinct, actionable defer reason, unlike the generic runner failure.

Types

type FootprintKey added in v0.22.0

type FootprintKey struct {
	Family string // e.g. "sdxl", "wan2.2", "whisper", "acestep"
	Quant  string // e.g. "bf16", "q8_0"; "" = node default
	Task   string // fleet task_type, e.g. "image-gen"
}

FootprintKey identifies which footprint-store entry a sampled render belongs to (mirrors the fleet contract's model_footprints identity: family + quant + task).

type Sampling added in v0.22.0

type Sampling struct {
	Footprint   *FootprintKey
	SampleFunc  func(childPid int) (float64, error)
	OnFootprint func(peakGiB float64)
}

Sampling bundles Spec's passive footprint-sampling hook fields so the thin wrappers (imagegen, rungraph) can thread them opaquely without growing three parameters each. nil = no sampling (the legacy Spec path, byte-identical).

func (*Sampling) ApplyTo added in v0.22.0

func (s *Sampling) ApplyTo(spec *Spec)

ApplyTo copies s onto spec. nil-safe: a nil receiver is a no-op, so callers thread whatever the pipeline composed without a guard.

type Spec

type Spec struct {
	// Exe is the executable ("" => "node"). Script is its first argument (the
	// render/*.mjs path, or for `node -e` the verb). Args are the remaining argv.
	Exe    string
	Script string
	Args   []string
	// Env are extra "K=V" entries appended to the current environment (e.g.
	// COMFY_DIR, MEMORY_STACK, GPU_LOCK_WAIT_MS). nil = inherit only.
	Env []string
	// Out is the file the runner must produce; a missing/empty Out after a clean
	// exit is treated as failure (the caller defers). Required.
	Out string
	// Timeout bounds the whole invocation (cold-start + render + margin).
	Timeout time.Duration
	// ComfyAPI is the ComfyUI endpoint freeComfyVRAM hits after the run. "" =>
	// the COMFY_API env or the 127.0.0.1:8188 default. Set "" to inherit; a runner
	// with no ComfyUI (TTS) can leave it — /free on a dead endpoint is a no-op.
	ComfyAPI string
	// SkipFreeComfy, when true, suppresses the post-run ComfyUI /free (the TTS/voice
	// path never starts ComfyUI, so there is nothing to free). The killTree + output
	// stat still apply — the python worker still gets process-tree-killed on timeout.
	SkipFreeComfy bool
	// Footprint, when non-nil, turns on passive per-render VRAM peak sampling for
	// the fleet-node footprint store (added 2026-07-17): while the child runs,
	// SampleFunc is polled and the max observation is reported via OnFootprint —
	// on SUCCESS only (a crashed/phantom run's peak may be partial). nil keeps the
	// legacy CombinedOutput path byte-identical.
	Footprint *FootprintKey
	// SampleFunc returns the current VRAM usage in GiB attributable to the render
	// rooted at childPid. The CALLER composes what a sample means (PDH process-tree
	// sum, or a global-delta closure) — gpugen stays dependency-free and only tracks
	// the peak. nil = no sampling (OnFootprint never fires).
	SampleFunc func(childPid int) (float64, error)
	// OnFootprint receives the observed peak (GiB) after a SUCCESSFUL render whose
	// sampled peak was > 0. nil = observations are discarded.
	OnFootprint func(peakGiB float64)
}

Spec describes one GPU-gen invocation. The caller assembles Args (the runner's CLI flags) — gpugen owns only the cross-cutting lifecycle, not the per-task arg shape.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL