transcoder

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 1, 2026 License: AGPL-3.0 Imports: 32 Imported by: 0

Documentation

Overview

Package transcoder turns an uploaded file into a ladder of encrypted HLS renditions: it reads the source, plans the ladder, hands the pieces to the nodes that ask for them (the scheduler), joins and packages what they make, and reports back. It knows nothing of libraries, folders, playback or billing: whoever submits a video gives it a Ref and implements Hooks, and the transcoder never imports them.

Index

Constants

View Source
const DefaultTemplate = "balanced"

DefaultTemplate is what a library uses until it chooses another.

View Source
const (

	// EncodeThreads is the cores one encode may use, the unit a node's capacity is counted in.
	EncodeThreads = 4
)

Measured 2026-09-30 (60 s of 1080p, VMAF against the source): faster at CRF 23 scores 97.4 at 1080p and 96.4 at 720p, encoding 4.4x and 6.5x realtime on 10 cores. veryfast lost 0.7 at the same size; medium gained 0.2 at half the speed.

Variables

View Source
var Heights = []int{240, 360, 480, 720, 1080}

Heights are the picture heights a ladder may hold, smallest first. Nothing taller is offered: 4K costs four times a 1080p encode in CPU and storage, and nothing shorter than 240 is readable.

Functions

func DecryptSegment

func DecryptSegment(data, key, iv []byte) ([]byte, error)

func Encoders

func Encoders() []string

Encoders is the video formats this machine's ffmpeg can produce: h264, hevc, av1. Empty without ffmpeg.

func EncryptRendition

func EncryptRendition(dir, name, keyURI string, key, iv []byte) error

EncryptRendition applies HLS full-segment AES-128: each segment CBC-encrypted with PKCS#7 padding, the init segment left clear. A fixed IV per asset keeps output identical across re-packaging, so republished segments never change under a player.

func Even

func Even(x float64) int

func Ffmpeg

func Ffmpeg(ctx context.Context, args ...string) error

func FfmpegProgress

func FfmpegProgress(ctx context.Context, frames int, report func(float64), args ...string) error

FfmpegProgress is Ffmpeg that also calls report with the share of frames written (of frames), every couple of seconds, from ffmpeg's own -progress lines.

func PeriodicJobs

func PeriodicJobs() []*river.PeriodicJob

func Prefix

func Prefix(ref AssetRef) string

Prefix is where everything about one video is stored: <workspace>/<video>/. Files of the transcoder (renditions, the chunks it works from under work/) and of the video module (source, captions, images) share it, so deleting a video deletes the prefix.

func Restart

func Restart(ctx context.Context, rc *river.Client[pgx.Tx], q core.TenantTx, ref AssetRef) error

Restart forgets the video's earlier pieces and queues its source to be read and encoded again.

func RunEncoder

func RunEncoder(ctx context.Context, d *core.Deps, hooks Hooks, node string, slots int, encoders []string)

RunEncoder is a node's side: until ctx ends it keeps up to slots tasks running, claiming more as each finishes. It returns once the ones already running are done; they are not cut off by ctx.

func SegmentAt

func SegmentAt(playlist []byte, t float64) (segment string, offset float64, iv []byte, err error)

SegmentAt finds the segment of a rendition playlist that holds time t, how far into it t falls, and the playlist's IV.

func Start

func Start(ctx context.Context, rc *river.Client[pgx.Tx], q core.TenantTx, ref AssetRef) error

Start queues a video's source to be read and encoded, in tx's transaction.

func ValidatePackage

func ValidatePackage(dir string, names []string, wantSeconds float64) error

ValidatePackage checks every playlist parses, ends, lists only segments that exist, and adds up to the source's duration.

func Workers

func Workers(w *river.Workers, d *core.Deps, hooks Hooks)

Workers registers the transcoder's River workers; hooks is how it asks and reports (see Hooks).

Types

type AssetRef

type AssetRef struct {
	TenantID pgtype.UUID `json:"tenant_id"`
	AssetID  pgtype.UUID `json:"asset_id"`
	JobID    pgtype.UUID `json:"job_id"`
}

AssetRef names the work: the workspace, the video it is for, and the job reporting on it. The transcoder only carries it back to Hooks. Its JSON names are what queued jobs already hold.

type Encoder

type Encoder struct {
	Codec          string `json:"codec"`
	Preset         string `json:"preset"`
	CRF            string `json:"crf"`
	Profile        string `json:"profile"`
	SegmentSeconds int    `json:"segment_seconds"`
	Encryption     string `json:"encryption"`
}

type Hooks

type Hooks interface {
	// Work reads a video; pgx.ErrNoRows when it no longer exists.
	Work(ctx context.Context, ref AssetRef) (Work, error)
	// Started says the first attempt at a video began.
	Started(ctx context.Context, ref AssetRef) error
	// Admit decides whether a video of this length may be encoded (a workspace's allowance); a non-empty code
	// refuses it.
	Admit(ctx context.Context, tx core.TenantTx, ref AssetRef, durationSeconds float64) (code string, err error)
	// Planned stores what the transcoder learned, in the same transaction that queues the pieces.
	Planned(ctx context.Context, tx core.TenantTx, ref AssetRef, p Planned) error
	// Publish runs upload (which puts the master playlist in place), then marks the video playable at stage
	// "ready" (its fast qualities) or finished at "complete". It does nothing if the video has moved on.
	Publish(ctx context.Context, ref AssetRef, stage string, upload func(ctx context.Context) error) error
	// Failed says the video cannot be encoded; code is a stable reason.
	Failed(ctx context.Context, ref AssetRef, code string) error
}

Hooks is everything the transcoder asks of whoever owns the videos, and every report it makes. The transcoder holds no video record of its own; this keeps the two modules apart. Methods that take a TenantTx run inside the transcoder's transaction.

type PackageArgs

type PackageArgs struct {
	AssetRef
	Stage string `json:"stage"` // "ready": fast rungs only; "complete": the full ladder
}

func (PackageArgs) InsertOpts

func (PackageArgs) InsertOpts() river.InsertOpts

func (PackageArgs) Kind

func (PackageArgs) Kind() string

type Plan

type Plan struct {
	FPS         string `json:"fps"`
	GOP         int    `json:"gop"`
	ChunkFrames int    `json:"chunk_frames"`
	Chunks      int    `json:"chunks"`
	Deinterlace bool   `json:"deinterlace"`
	Audio       bool   `json:"audio"`
	Rungs       []Rung `json:"rungs"`
	// Recorded at probe time so a later change to the defaults never misreports how this video was made.
	Encoder *Encoder `json:"encoder,omitempty"`
}

Plan is fixed at probe time and stored on the asset, so every retry encodes the same thing.

func MakePlan

func MakePlan(src Source, preset Preset) Plan

MakePlan decides the encode for src from the library's preset: the preset's heights that the source can fill, and at least one rung. A ladder with no fast rung makes its smallest one fast, so the video still plays early.

func (Plan) Lane

func (p Plan) Lane(lane string) []Rung

func (Plan) Tasks

func (p Plan) Tasks() []Task

type Planned

type Planned struct {
	Probe         []byte
	Duration      float64
	Width, Height int
	Plan          []byte
	Key, IV       []byte // a fresh key for this video, unsealed: the owner seals it
}

Planned is the transcoder's reading of a source and the ladder it chose, for the owner to store.

type Preset

type Preset struct {
	Heights []int   `json:"heights"`
	Quality Quality `json:"quality"`
}

Preset is what a library asks of the transcoder: which rungs to make and how carefully.

func (*Preset) Clean

func (p *Preset) Clean() bool

Clean sorts and de-duplicates the heights and reports whether the preset is usable.

type ProbeArgs

type ProbeArgs struct{ AssetRef }

func (ProbeArgs) InsertOpts

func (ProbeArgs) InsertOpts() river.InsertOpts

func (ProbeArgs) Kind

func (ProbeArgs) Kind() string

type Progress

type Progress struct {
	AssetID     pgtype.UUID
	Lane        string
	Done, Total int32
	Partial     float64 // how far the pieces being encoded now have got, in pieces (0.5 is half of one)
}

Progress is how many pieces of a lane are done.

func ProgressOf

func ProgressOf(ctx context.Context, q core.TenantTx, assets []pgtype.UUID) ([]Progress, error)

Progress counts the pieces done and in all, per video and lane.

type Quality

type Quality string

Quality is how hard the encoder works for each video: sharper and smaller costs more time.

const (
	QualityFast     Quality = "fast"     // publishes soonest, a little softer or larger
	QualityBalanced Quality = "balanced" // the default; measured below
	QualityHigh     Quality = "high"     // sharpest for its size, takes about twice as long
)

func (Quality) Valid

func (q Quality) Valid() bool

type Rung

type Rung struct {
	Name     string `json:"name"`
	Width    int    `json:"width"`
	Height   int    `json:"height"`
	MaxrateK int    `json:"maxrate_k"`
	Lane     string `json:"lane"`
	// Half encodes at half the source's frame rate: below 720p a frame rate above 30 costs double for nothing seen.
	Half bool `json:"half,omitempty"`
}

type Source

type Source struct {
	Duration       float64
	Width, Height  int
	FPSNum, FPSDen int
	Interlaced     bool
	HasAudio       bool
	Raw            []byte
}

type StoryboardArgs

type StoryboardArgs struct{ AssetRef }

func (StoryboardArgs) InsertOpts

func (StoryboardArgs) InsertOpts() river.InsertOpts

func (StoryboardArgs) Kind

func (StoryboardArgs) Kind() string

type SweepArgs

type SweepArgs struct{}

SweepArgs fails the videos whose encode tasks have run out of tries (see sweepWorker).

func (SweepArgs) InsertOpts

func (SweepArgs) InsertOpts() river.InsertOpts

func (SweepArgs) Kind

func (SweepArgs) Kind() string

type Task

type Task struct {
	Lane  string
	Chunk int
}

type TaskInfo

type TaskInfo struct {
	Lane     string
	Chunk    int32
	Done     bool
	State    string // done, running, retrying, queued or failed
	Node     string // the machine running it, or that last ran it
	Attempts int32
	Progress float32 // of the try running now, 0 to 1
	Error    string  // the last try's failure
	// Queued, last started and finished (zero when it hasn't).
	Queued, Started, Finished time.Time
}

TaskInfo is one piece of a video's encode as the scheduler sees it.

func Tasks

func Tasks(ctx context.Context, q core.TenantTx, asset pgtype.UUID) ([]TaskInfo, error)

type Template

type Template struct {
	Key         string `json:"key"`
	Name        string `json:"name"`
	Description string `json:"description"`
	Preset
}

Template is a ready-made preset with a name and a plain description of who it is for.

func TemplateByKey

func TemplateByKey(key string) (Template, bool)

TemplateByKey finds a template; the key is case-insensitive.

func Templates

func Templates() []Template

Templates lists the ready-made presets.

type Work

type Work struct {
	SourceKey       string
	DurationSeconds float64
	Plan            []byte // JSON written by Planned; empty before the video is planned
	Key, IV         []byte // the encryption key, unsealed, once planned
	Preset          Preset // which rungs to make and how carefully
}

Work is what the transcoder needs to know about a video.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL