media

package
v0.62.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 2, 2026 License: MIT Imports: 48 Imported by: 0

Documentation

Overview

Package media stores app media as a self-describing file system in one private bucket. The app declares everything in one registry at startup (Config): its kinds, their uploads, the private files derived from them and the public images at fixed names. ContentKit hard-codes no kinds, names or layouts; it owns the mechanics: uploads, the worker's generic producers, conditional manifest writes, the storage areas, the access rule, and purge, sweep and erasure. No database table records what media exists.

An item is a folder, {namespace}/{kind}/{id}/ (see media/layout), with a manifest.json: an ordered virtual file system over its private blobs.

Index

Constants

View Source
const (
	VideoLive      = ""
	VideoAnimation = "animation"
)

Video profiles (HLS.Profile, MP4.Profile).

View Source
const (
	DefaultMinAspect = 1 / 2.4
	DefaultMaxAspect = 2.4
)

Default aspect bounds: 1:2.4 vertical to 2.4:1 wide.

View Source
const (
	HLSContentType = "application/vnd.apple.mpegurl"
	VTTContentType = "text/vtt; charset=utf-8"
)

Playlist content types.

View Source
const (
	IngestPartSize    = 32 << 20
	IngestConcurrency = 3
)

Ingest defaults: parts are buffered in memory, so memory is about (IngestConcurrency+1) × IngestPartSize.

View Source
const (
	MetaLang    = "lang"    // subtitles and audio: BCP 47
	MetaLabel   = "label"   // a track's name
	MetaForced  = "forced"  // bool: a forced-narrative subtitle track
	MetaFor     = "for"     // a subtitle's video upload path; default every video of the item
	MetaCharset = "charset" // a subtitle's IANA charset, overriding detection
)

Upload meta keys ContentKit reads (put's meta).

View Source
const (
	MaxAudioTracks    = 8
	MaxSubtitleTracks = 16
)

Tracks per source a video's HLS ladder carries beyond its renditions: the probe keeps at most this many audio and text subtitle streams.

View Source
const (
	TrackVideo  = "video"
	TrackAudio  = "audio"
	TrackSubs   = "subs"
	TrackSprite = "sprite"
)

Track kinds.

View Source
const (
	MaxManifestBytes = 8 << 20

	MaxMetaBytes     = 4 << 10  // an upload's meta, as JSON
	MaxItemMetaBytes = 16 << 10 // the item's meta, as JSON
	MaxUploads       = 10000
)

Manifest bounds. MaxManifestBytes bounds a manifest's JSON on every read and write (8 MiB: a 2,000-page gallery is about 1.5 MB), so every manifest a reader may meet fits the cache. Edits stop editHeadroom below it, so a hide always fits; a commit stops when the item, processed, would pass that (Kind.Unwritten). Meta is bounded where ops set it, and an item holds at most MaxUploads uploads.

View Source
const (
	OpPut        = "put"        // add an upload, or replace the one with the same path stem
	OpEdit       = "edit"       // crop and rotate an image upload; nil clears
	OpMove       = "move"       // reorder an upload among its Upload's; its outputs follow
	OpRename     = "rename"     // rename an upload and its outputs
	OpRemove     = "remove"     // remove an upload and its outputs
	OpAttach     = "attach"     // make an unattached upload part of the item
	OpCopy       = "copy"       // copy an upload within the kind, optionally with a new edit
	OpFrame      = "frame"      // fill an upload from a frame of its Upload.Frames video
	OpMeta       = "meta"       // set the template values (download names)
	OpRegenerate = "regenerate" // redo stale outputs (of Preset), or every output with Force
)

Commit ops. Only upload paths are writable; derived paths belong to the worker.

View Source
const (
	PhaseQueued      = "queued"
	PhaseDownloading = "downloading" // fetching the original
	PhaseProbing     = "probing"
	PhaseEncoding    = "encoding" // the single ffmpeg pass: ladder, audio, subtitles, sprite
	PhaseMuxing      = "muxing"   // per-quality MP4 downloads
	PhaseUploading   = "uploading"
	PhasePublishing  = "publishing" // the manifest edit
	PhaseImages      = "images"     // item-wide, after every file: poster frame from the renditions
)

Encode phases, in order. Queued covers a job waiting for a worker and a file waiting behind another file of the same job.

View Source
const (
	StateReady      = "ready"
	StateProcessing = "processing"
	StateFailed     = "failed"
	StateFull       = "full" // the manifest is Full: work is left, stopped until a commit shrinks it
)

Processing states of an item.

View Source
const (
	MaxSinglePut = 64 << 20
	MinPartSize  = 8 << 20
	MaxPartSize  = 16 << 20
)

Upload size rules. An upload lands at a staged name in temp/ (u-{uuid}): up to MaxSinglePut as one checksum-bound PUT, larger in parts of MinPartSize up to MaxPartSize (the last may be smaller). Once committed, the media worker hashes it and places it at private/sha256-{hex} (Manifests.Place), so a blob's bytes always hash to its name.

View Source
const (
	CodeInvalid      = "invalid_request"   // 400
	CodeForbidden    = "forbidden"         // 403
	CodeNotFound     = "not_found"         // 404: unknown kind, file or upload
	CodeConflict     = "conflict"          // 409: a path taken, or a new item's folder not empty
	CodeIncomplete   = "incomplete"        // 409: multipart parts missing
	CodeNotUploaded  = "not_uploaded"      // 409: commit before the object landed
	CodeTooManyFiles = "too_many_files"    // 409: the commit would exceed an Upload's Max
	CodeTooLarge     = "too_large"         // 413: over the Upload's MaxBytes
	CodeQuota        = "quota_exceeded"    // 413: owner quota
	CodeType         = "type_not_allowed"  // 415
	CodeChecksum     = "checksum_mismatch" // 422: stored bytes differ from the declared hash
	CodeRate         = "rate_limited"      // 429
	CodeUnavailable  = "unavailable"       // 503: the bucket cannot be reached; retry

	// Image refusals (ImageError): the rules refuse the image; never a server fault.
	CodeImageTooSmall   = "image_too_small"  // 422: edited narrower than the public preset's MinWidth
	CodeImageTooLarge   = "image_too_large"  // 413: more pixels than the processor decodes
	CodeImageUnreadable = "image_unreadable" // 422: not a decodable image of its declared type

	CodeAnimationNotAllowed  = "animation_not_allowed" // 422: an animated image where the policy is AnimationReject
	CodeAnimationTooLong     = "animation_too_long"    // 422: more frames or seconds than the processor allows
	CodeAnimationUnsupported = "animation_unsupported" // 415: an AVIF/HEIF image sequence (decoded as one frame)

	// Video refusals (ImageError, recorded as the upload's Failure): its
	// Upload.Video limits refuse what the real stream would cost.
	CodeVideoTooLong    = "video_too_long"    // 422: runs longer than MaxSeconds (audio too)
	CodeVideoTooLarge   = "video_too_large"   // 422: frames larger than MaxPixels
	CodeVideoOverBudget = "video_over_budget" // 422: the planned encode is over MaxWork, or the source averages under a frame a second
)

Stable upload error codes: clients (the browser SDK) branch on Code.

View Source
const (
	AccessFull = "full" // every private file
	AccessNone = "none" // public files only (previews)
)

Access levels in ReadResult: an item's private files are all or nothing.

View Source
const CookieName = token.CookieName

CookieName is the access agent's cookie.

View Source
const DefaultQueue = "contentkit_media"

DefaultQueue is JobsConfig.Queue's default.

View Source
const ItemProgressKey = ""

ItemProgressKey carries Item in a job's reported map.

View Source
const ManifestVersion = 2

ManifestVersion is the manifest format.

View Source
const NamedPrefix = "i-"

NamedPrefix starts the names the server gives Named uploads.

View Source
const StartRung = 1080

StartRung is the short side of the variant listed first in each codec: native HLS players (Safari, iOS) start there before measuring.

View Source
const ZipRecipe = "zip-store-1"

ZipRecipe versions the Zip producer.

Variables

View Source
var (
	AspectNative = Aspect{}
	Square       = Aspect{1, 1}
)
View Source
var (
	// ErrManifestTooLarge: an edit refused because the manifest would grow
	// past its bound. A worker refused it marks the item Full (SetFull).
	ErrManifestTooLarge = errors.New("media: manifest edit refused: over its size limit")
	// ErrManifestUnreadable: a stored manifest over MaxManifestBytes, written
	// before the bound; it does not decode.
	ErrManifestUnreadable = errors.New("media: stored manifest over its size limit")
)
View Source
var (
	// ErrNotVisible hides an item the viewer may not see, or that does not exist.
	ErrNotVisible = errors.New("media: not found")
	// ErrResolve wraps a resolver failure; it always denies.
	ErrResolve = errors.New("media: resolve failed")
	// ErrInvalidRequest is a malformed read request.
	ErrInvalidRequest = errors.New("media: invalid request")
	// ErrNotAllowed is a file the viewer may not have.
	ErrNotAllowed = errors.New("media: not allowed")
)
View Source
var (
	ErrNotFound           = errors.New("media: object not found")
	ErrPreconditionFailed = errors.New("media: precondition failed")
	ErrNotModified        = errors.New("media: not modified")
	// ErrUnavailable marks a store that could not be reached or answered 5xx;
	// HTTP handlers answer 503.
	ErrUnavailable = errors.New("media: store unavailable")
	// ErrChecksumMismatch: the backend rejected a PUT whose body does not
	// match its x-amz-checksum-sha256.
	ErrChecksumMismatch = errors.New("media: checksum mismatch")
	// ErrNotImplemented: the backend does not support a request feature (501).
	ErrNotImplemented = errors.New("media: not implemented by the store")
)
View Source
var DefaultEditor = Image{Width: 2048, Height: 2048, Quality: 82}

DefaultEditor renders editor views.

View Source
var DefaultLadder = []int{2160, 1080, 480}

DefaultLadder is the HLS ladder by short side.

View Source
var DefaultVideoLimits = VideoLimits{MaxSeconds: 4 * 3600, MaxFPS: 60, MaxPixels: 8192 * 4320, MaxWork: 4e13}

DefaultVideoLimits are an upload's VideoLimits where it sets none: 4 h at up to 60 fps, frames up to DCI 8K (8192×4320; the ladder scales larger sources down to 4K), and the work of a 4 h 4K60 default ladder (2160, 1080, 480) in three codecs plus MP4 downloads, about 3e13 pixel-frames.

EncodePhases lists every phase in order.

View Source
var ErrFolderNotEmpty = errors.New("media: new item's folder is not empty")

ErrFolderNotEmpty: a new item's folder already holds objects. Item ids are never reused, so leftovers mean a host bug (a reset id, a restored database); they are never adopted. Purge them deliberately with Jobs.Purge.

View Source
var ErrJobsNotBound = errors.New("media: River jobs are not composed into a client")
View Source
var ErrManifestConflict = errors.New("media: manifest edit kept conflicting")
View Source
var ErrNoViewerKey = errors.New("media: an anonymous actor needs Actor.IP for the issuance limit")

ErrNoViewerKey is a grant the issuance limit cannot count: an anonymous actor without Actor.IP. It is the host's bug, not the viewer's limit.

View Source
var ErrQuotaReleasePending = errors.New("media: another quota release is pending for the folder")

ErrQuotaReleasePending means a different deletion must finish or be retried first.

View Source
var ErrRateLimited = errors.New("media: rate limited")

ErrRateLimited is a refusal that a later attempt will pass.

View Source
var ErrUnknownKind = errors.New("media: unknown kind")
View Source
var ImageTypes = []string{"image/jpeg", "image/png", "image/webp", "image/gif", "image/avif", "image/heic", "image/heif", "image/tiff"}

ImageTypes are the image types the image presets decode (media/image); an upload feeding one accepts no other. SVG, BMP and JPEG XL are refused: their libvips loaders are blocked as untrusted.

View Source
var MaxOutageSnoozes = 2880

MaxOutageSnoozes caps the snoozes of one job (a day at UnavailableSnooze); past it, outage errors spend attempts again.

PendingOnce dedupes a job per args while one is waiting or running.

View Source
var RedisErrors = expvar.NewInt("contentkit_media_ratelimit_redis_errors")

RedisErrors counts shared-limit Redis failures (each served by the per-process limit instead); published as expvar "contentkit_media_ratelimit_redis_errors".

View Source
var SubtitleTypes = []string{"text/vtt", "application/x-subrip", "text/x-ssa", "text/x-ass"}

SubtitleTypes are the subtitle types the Subtitles producer converts.

View Source
var UnavailableSnooze = 30 * time.Second

UnavailableSnooze is how long a job waits after the store was unreachable.

Functions

func CleanName added in v0.62.0

func CleanName(s string) string

CleanName makes a template value ({name}, {title}) safe to fill a path or a download name: "/", "\", control characters and ".." are removed, the text is NFC-normalized, trimmed and capped at 200 bytes of JSON.

func Fingerprint added in v0.62.0

func Fingerprint(parts ...string) string

Fingerprint hashes a derived file's inputs: for a producer, the source's blob, its edit, the preset's spec and the producer's recipe version. A derived file is stale exactly when its FP no longer matches.

func InsertOnce added in v0.43.0

func InsertOnce(ctx context.Context, insert InsertFunc, args RerunArgs, o river.InsertOpts) error

InsertOnce enqueues args after the caller's change to their inputs. An equal job still waiting to run absorbs it. River's uniqueness also covers running jobs, which may have read the inputs before the change, so one follow-up is queued behind a running equal job; a burst shares it. The follow-up's worker calls WaitFor first.

func NewName added in v0.62.0

func NewName() string

NewName is a fresh Named upload name: "i-{uuid}".

func NewStaged added in v0.62.0

func NewStaged() string

NewStaged is a fresh staged upload name: "u-{uuid}".

func PublicURL added in v0.62.0

func PublicURL(base, namespace, kind, id, name string) string

PublicURL is a public file's URL: {base}/v1/{namespace}/{kind}/{id}/public/{name}.

func SnoozeUnavailable added in v0.58.0

func SnoozeUnavailable(ctx context.Context, store Store, job *rivertype.JobRow, err error) error

SnoozeUnavailable turns a real store outage into a River snooze, which does not count as an attempt, so an outage longer than the retry budget never discards a job. It snoozes only when the job's own context is still live (a job that outran its timeout spends its attempt) and a fresh, bounded store.Check fails too (one bad object on a healthy bucket spends its attempt); River's snooze count in job metadata is capped by MaxOutageSnoozes. Other errors pass through.

func SpecFP added in v0.62.0

func SpecFP(src File, spec any, recipe string) string

SpecFP is the fingerprint of a spec (any JSON value) applied to upload src by a producer at recipe.

func SrcSet added in v0.62.0

func SrcSet(base, namespace, kind, id, to string, widths []int) string

SrcSet is a srcset over a public template's widths: "{url} 230w, …".

func UploadHandler

func UploadHandler(u *Uploads, o UploadHandlerOptions) http.Handler

UploadHandler serves the upload API the browser SDK calls. Every route but GET /frame is POST with a JSON body. Errors are ErrorReply with the status of its code (Retry-After on 429).

POST /presign     PresignBody -> PresignReply
POST /parts       PartsBody   -> PartsReply   presign multipart parts
POST /parts/list  TicketBody  -> PartsReply   parts that landed (resume)
POST /complete    TicketBody  -> CompleteReply
POST /abort       TicketBody  -> 204
POST /commit      CommitBody  -> CommitReply
GET  /frame?kind=&id=&path=&t=&w= -> image/jpeg   a still of a video upload (UploadOptions.Frames)

func ValidNamed added in v0.62.0

func ValidNamed(name string) bool

ValidNamed reports a server-given name ("i-{uuid}").

func WaitFor added in v0.43.0

func WaitFor(ctx context.Context, c *river.Client[pgx.Tx], id int64) error

WaitFor snoozes a follow-up (InsertOnce) while the job it follows still runs, so jobs for the same inputs do not overlap. The client is the one running the job (river.ClientFromContext).

func ZipFP added in v0.62.0

func ZipFP(inputs []File) string

ZipFP is the fingerprint of a Zip preset's current inputs.

Types

type AgentRules added in v0.62.0

type AgentRules struct {
	Namespaces []string
	Defaults   []layout.Default
}

AgentRules are the access agent's rules for an app's media host: the namespaces it serves there (MEDIA_ACCESS_HOSTS, with layout.FormatHosts) and the default public names of its kinds (MEDIA_ACCESS_DEFAULTS, with layout.FormatDefaults).

func AgentConfig added in v0.62.0

func AgentConfig(r *Registry) AgentRules

AgentConfig derives the access agent's rules from the registry.

type Animation added in v0.39.0

type Animation string

Animation is a policy for animated images (GIF, WebP; AVIF/HEIF sequences are refused as animation_unsupported, never flattened).

const (
	// AnimationAllow keeps every frame, delay and the loop count.
	AnimationAllow Animation = ""
	// AnimationReject refuses an animated upload with animation_not_allowed.
	AnimationReject Animation = "reject"
)

type Aspect added in v0.40.0

type Aspect struct{ W, H int }

Aspect is a width:height ratio in lowest terms, written "W:H" ("3:1", "9:16"). The zero value is AspectNative: the source's own shape.

func AspectOf added in v0.40.0

func AspectOf(w, h int) Aspect

AspectOf is the ratio of a w×h size, reduced (native for an empty size).

func ParseAspect added in v0.40.0

func ParseAspect(s string) (Aspect, error)

ParseAspect parses "W:H" with positive integer terms up to 10000 and reduces it ("6:2" is "3:1"); "" and "native" are AspectNative.

func Ratio added in v0.40.0

func Ratio(s string) Aspect

Ratio parses "W:H" and panics on an invalid one (for constants in code).

func (Aspect) Height added in v0.40.0

func (a Aspect) Height(width int) int

Height is width's height at a, rounded half up, at least 1 (0 when native).

func (Aspect) MarshalText added in v0.40.0

func (a Aspect) MarshalText() ([]byte, error)

MarshalText writes "W:H" (native: empty), so JSON and config carry the string.

func (Aspect) Native added in v0.40.0

func (a Aspect) Native() bool

Native reports the source's own shape.

func (Aspect) Rotated added in v0.40.0

func (a Aspect) Rotated() Aspect

Rotated is a turned a quarter (90 or 270 degrees).

func (Aspect) String added in v0.40.0

func (a Aspect) String() string

func (*Aspect) UnmarshalText added in v0.40.0

func (a *Aspect) UnmarshalText(b []byte) error

func (Aspect) Valid added in v0.40.0

func (a Aspect) Valid() bool

Valid reports a native or reduced, bounded ratio.

func (Aspect) Width added in v0.40.0

func (a Aspect) Width(height int) int

Width is height's width at a, rounded half up, at least 1 (0 when native).

type Audio added in v0.56.0

type Audio struct {
	// Loudness normalizes to this integrated loudness in LUFS (EBU R128),
	// e.g. -16; 0 keeps the source's level.
	Loudness float64 `json:"loudness,omitempty"`
}

Audio is AAC-LC: an HLS audio track ({To}audio.mp4) and a faststart M4A ({To}audio.m4a).

type Capabilities

type Capabilities struct {
	ConditionalPut bool // If-Match / If-None-Match on PUT
	ChecksumSHA256 bool // x-amz-checksum-sha256 enforced on PUT
}

Capabilities are backend features the library depends on, established by Probe.

func Probe

func Probe(ctx context.Context, s Store, prefix string) (Capabilities, error)

Probe measures the backend's capabilities with scratch objects under prefix, which it removes. Results depend on the backend release (RGW vs MinIO). Every step must succeed or be refused cleanly (412, a checksum mismatch, 501); anything else (throttling, timeouts, a proxy's 403, cancellation) is an error and nothing is recorded, so a transient failure never reads as a missing capability.

type Codec added in v0.53.0

type Codec string

Codec is a video codec of an HLS ladder: each rung is encoded in each of the worker's codecs (Track.Codec).

const (
	CodecH264 Codec = "h264"
	CodecHEVC Codec = "hevc"
	CodecAV1  Codec = "av1"
)

type CommitBody

type CommitBody struct {
	Ref RefBody `json:"ref"`
	Ops []Op    `json:"ops"`
}

CommitBody applies ops to an item in one conditional write.

type CommitReply

type CommitReply struct {
	Files []FileInfo `json:"files"`
}

CommitReply is the item's uploads as an editor reads them.

type CompleteReply

type CompleteReply struct {
	Blob string `json:"blob"`
	Type string `json:"type"`
	Size int64  `json:"size"`
}

CompleteReply is a completed multipart upload; Blob is its staged name.

type Config added in v0.62.0

type Config struct {
	// Namespace is the app's own namespace, e.g. "doujins": its kinds' items
	// live under {Namespace}/{kind}/{id}/.
	Namespace string `json:"namespace"`
	// BaseURL is the site's media origin, e.g. "https://media.doujins.ai":
	// every URL is {BaseURL}/v1/{namespace}/{kind}/{id}/{public|private}/{name}.
	BaseURL string `json:"base_url,omitempty"`
	// Kinds are the app's kinds plus the shared kinds it imports.
	Kinds []Kind `json:"kinds"`
	// Editor is how editor views render (the whole oriented source the
	// cropper draws on); zero is DefaultEditor.
	Editor Image `json:"editor,omitzero"`
	Hooks  Hooks `json:"-"`
}

Config is the app's media registry (NewRegistry).

func ParseConfig added in v0.62.0

func ParseConfig(b []byte) (Config, error)

ParseConfig reads a registry written by Registry.MarshalJSON; unknown keys are ignored.

type CopyFrom added in v0.62.0

type CopyFrom struct {
	ID   string `json:"id"`
	Path string `json:"path"`
}

CopyFrom names an upload of an item of the same kind, including this item.

type CopyOptions added in v0.40.0

type CopyOptions struct {
	IfMatch string
}

CopyOptions: IfMatch copies only while the source's ETag is this one (ErrPreconditionFailed otherwise).

type Crop added in v0.17.0

type Crop struct {
	X int `json:"x"`
	Y int `json:"y"`
	W int `json:"w"`
	H int `json:"h"`
}

Crop is a rectangle in source pixels.

type Deletion

type Deletion struct {
	Ref         contentref.ContentRef
	Owner       string
	OperationID string // required by Purge with a quota owner; stable across retries
}

Deletion is one item to delete. Owner is its quota owner (UploadGrant.Owner), "" for none; with a Limiter configured the owner's usage is released. OperationID is only for Purge; queued deletion uses its River job id.

type Delivery

type Delivery struct {
	Mode DeliveryMode
	// CookieDomain is the site's registrable domain, e.g. "doujins.ai";
	// required in cookie mode, because a host-only cookie never reaches media.
	CookieDomain string
	// SigningKey is the current key of the ring the access agent verifies.
	SigningKey token.Key
	// TTL is the minimum token lifetime (default 1h); expiries round up to
	// Window (default token.DefaultWindow, whole seconds).
	TTL, Window time.Duration
}

Delivery is the host's signing configuration; URLs are at the registry's BaseURL.

type DeliveryMode

type DeliveryMode string

DeliveryMode is how viewers with access present their item token.

const (
	// DeliverCookie (default) sets the item cookie; URLs are plain.
	DeliverCookie DeliveryMode = "cookie"
	// DeliverURL appends ?t= to every URL: apps and clients without cookies.
	DeliverURL DeliveryMode = "url"
)

type Dims added in v0.17.0

type Dims struct {
	W int `json:"w"`
	H int `json:"h"`
}

Dims is a source's size with EXIF orientation applied.

type Edit added in v0.17.0

type Edit struct {
	Crop   *Crop `json:"crop,omitempty"`
	Rotate int   `json:"rotate,omitempty"` // clockwise degrees: 0, 90, 180 or 270
}

Edit is a non-destructive image edit: Crop, in the source's pixels (EXIF orientation applied), then a clockwise Rotate. Variants derive from the source through it; the source is never modified.

func (*Edit) Check added in v0.17.0

func (e *Edit) Check(w, h int) error

Check validates the edit's shape and, with a known size (w, h > 0), that the crop lies inside it.

func (*Edit) Hash added in v0.17.0

func (e *Edit) Hash() string

Hash is the edit's stable identity; "" for no edit.

func (*Edit) Normalize added in v0.17.0

func (e *Edit) Normalize() *Edit

Normalize returns nil for an identity edit, else a copy.

func (*Edit) Size added in v0.17.0

func (e *Edit) Size(w, h int) (int, int)

Size is the edited size of a w×h source.

type EncodeProgress added in v0.29.0

type EncodeProgress struct {
	Phase         string  `json:"phase"`
	QueuePosition int     `json:"queue_position,omitempty"` // 1 = next; 0 = unknown or behind a file of the same job
	SegmentsDone  int     `json:"segments_done,omitempty"`
	SegmentsTotal int     `json:"segments_total,omitempty"`
	Percent       float64 `json:"percent"`
	Speed         float64 `json:"speed,omitempty"`
	ETA           float64 `json:"eta,omitempty"`
	At            int64   `json:"at"` // unix milliseconds of the measurement
	Stalled       bool    `json:"stalled,omitempty"`
	// Stage of Stages: each rung is published before the next one starts.
	Stage  int `json:"stage,omitempty"`
	Stages int `json:"stages,omitempty"`
}

EncodeProgress is a pending video file's live encode state. Segments count HLS segments of the source's duration; Speed is the encode's smoothed ×realtime; ETA is seconds from At to publish, including uploads. Percent never decreases within a run. Stalled marks a report older than a minute from a running job (a dead worker until River rescues the job).

type EncodeStatus added in v0.29.0

type EncodeStatus struct {
	Files  map[string]EncodeProgress
	Queued *EncodeProgress
	Item   *EncodeProgress
}

EncodeStatus is an item's encode progress: Files from the running job, Queued for pending files it does not cover (a waiting job), Item for the job's item-wide step (PhaseImages).

func (EncodeStatus) Current added in v0.29.0

func (s EncodeStatus) Current() *EncodeProgress

Current is the item's most advanced step: the item-wide one, a running file, else the wait.

type ErrorDetails added in v0.38.0

type ErrorDetails struct {
	Width      int      `json:"width,omitempty"`       // image_too_small: the edited width; image_too_large, video_too_large: the source's
	Height     int      `json:"height,omitempty"`      // image_too_large, video_too_large: the source's
	MinWidth   int      `json:"min_width,omitempty"`   // image_too_small
	MaxPixels  int      `json:"max_pixels,omitempty"`  // image_too_large, video_too_large
	Type       string   `json:"type,omitempty"`        // the declared type (image_unreadable, type_not_allowed, too_large)
	Allowed    []string `json:"allowed,omitempty"`     // type_not_allowed: the kind's types
	Size       int64    `json:"size,omitempty"`        // too_large: bytes
	MaxBytes   int64    `json:"max_bytes,omitempty"`   // too_large
	Frames     int      `json:"frames,omitempty"`      // animation_too_long, image_too_large: the animation's
	MaxFrames  int      `json:"max_frames,omitempty"`  // animation_too_long
	Seconds    float64  `json:"seconds,omitempty"`     // animation_too_long, video_too_long: running time
	MaxSeconds float64  `json:"max_seconds,omitempty"` // animation_too_long, video_too_long
}

ErrorDetails qualifies an image refusal so clients can state the rule.

type ErrorReply

type ErrorReply struct {
	Error      string        `json:"error"`
	Code       string        `json:"code"`
	RetryAfter int           `json:"retry_after,omitempty"` // seconds, with 429
	Blobs      []string      `json:"blobs,omitempty"`       // not_uploaded at commit: the blobs to upload again
	Details    *ErrorDetails `json:"details,omitempty"`     // image refusals
}

ErrorReply is the error body; Code is one of the Code* constants, "unauthorized" or "internal_error".

type Failure added in v0.62.0

type Failure struct {
	Of      string        `json:"of"` // File.Key it was recorded for
	Message string        `json:"message"`
	Code    string        `json:"code,omitempty"` // an ImageError's code
	Details *ErrorDetails `json:"details,omitempty"`
}

Failure is why an upload's blob through its edit cannot be processed.

func NewFailure added in v0.62.0

func NewFailure(f File, err error) *Failure

NewFailure records err for upload f: an ImageError keeps its code and details; the message is capped at maxFailureBytes.

type File

type File struct {
	Path string  `json:"path"`           // app path: "originals/001.png", "low-res/001.webp"
	Blob string  `json:"blob,omitempty"` // "sha256-{hex}" in private/; "" for a staged upload or a frame not grabbed yet
	Type string  `json:"type"`
	Size int64   `json:"size,omitempty"`
	W    int     `json:"w,omitempty"` // an upload's oriented size once measured; an output's size
	H    int     `json:"h,omitempty"`
	Dur  float64 `json:"dur,omitempty"`

	// Uploads:
	Staged     string         `json:"staged,omitempty"`     // "u-{uuid}" in temp/ until the worker hashes and places it at Blob
	CreateID   string         `json:"create_id,omitempty"`  // create-only put receipt; retained through placement and processing
	Edit       *Edit          `json:"edit,omitempty"`       // crop and rotate in source pixels
	Frame      *Frame         `json:"frame,omitempty"`      // grabbed from the Upload.Frames video
	Meta       map[string]any `json:"meta,omitempty"`       // lang, label, …
	Unattached bool           `json:"unattached,omitempty"` // processed on upload, not yet part of the item
	Gone       bool           `json:"gone,omitempty"`       // blob dropped (KeepOriginals false); the hash stays for provenance
	Pending    []string       `json:"pending,omitempty"`    // presets still producing from this upload
	Failed     *Failure       `json:"failed,omitempty"`     // this blob and edit cannot be processed

	// Derived files:
	From     string `json:"from,omitempty"`     // the upload's path, or a zip's prefix
	Preset   string `json:"preset,omitempty"`   // the Private preset
	FP       string `json:"fp,omitempty"`       // Fingerprint of its inputs
	Download string `json:"download,omitempty"` // the human download name
	Track    *Track `json:"track,omitempty"`    // HLS
}

File is one file: an upload (no Preset) or a derived file.

func (File) Fail added in v0.62.0

func (f File) Fail() *Failure

Fail is the upload's failure for its current blob and edit, or nil.

func (File) IsUpload added in v0.62.0

func (f File) IsUpload() bool

IsUpload reports an upload (a file with no preset).

func (File) Key added in v0.62.0

func (f File) Key() string

Key identifies the source and edit an upload's Failure applies to.

func (File) Source

func (f File) Source() string

Source is the object holding an upload's bytes: its blob, or its staged upload until placed.

type FileInfo

type FileInfo struct {
	Path     string  `json:"path"`
	Type     string  `json:"type"`
	Size     int64   `json:"size,omitempty"`
	W        int     `json:"w,omitempty"`
	H        int     `json:"h,omitempty"`
	Dur      float64 `json:"dur,omitempty"`
	Download string  `json:"download,omitempty"` // the name a download read serves it under
	URL      string  `json:"url,omitempty"`
	Locked   bool    `json:"locked,omitempty"`

	// Editors (editor reads and commit replies):
	Upload     bool            `json:"upload,omitempty"`
	Staged     bool            `json:"staged,omitempty"` // uploaded, not yet placed by the worker: no blob, views or frames yet
	From       string          `json:"from,omitempty"`
	Edit       *Edit           `json:"edit,omitempty"`
	Frame      *Frame          `json:"frame,omitempty"`
	Meta       map[string]any  `json:"meta,omitempty"`
	Unattached bool            `json:"unattached,omitempty"`
	Pending    []string        `json:"pending,omitempty"`
	Failed     *Failure        `json:"failed,omitempty"`
	EditorURL  string          `json:"editor_url,omitempty"`
	Progress   *EncodeProgress `json:"progress,omitempty"`
}

FileInfo is one file of a read.

type FitMode added in v0.62.0

type FitMode string

FitMode is how an image fits its Width×Height box.

const (
	FitInside FitMode = "" // within the box, keeping the aspect
	FitCover  FitMode = "cover"
)

type FolderNotEmptyError added in v0.45.0

type FolderNotEmptyError struct {
	Prefix string
	Keys   []string
}

FolderNotEmptyError names the folder and a few of its keys.

func (*FolderNotEmptyError) Error added in v0.45.0

func (e *FolderNotEmptyError) Error() string

func (*FolderNotEmptyError) Unwrap added in v0.45.0

func (e *FolderNotEmptyError) Unwrap() error

type Frame added in v0.62.0

type Frame struct {
	T    float64 `json:"t,omitempty"`
	Auto bool    `json:"auto,omitempty"`
	Of   string  `json:"of,omitempty"`
}

Frame is an upload grabbed from a frame of its Upload.Frames video at T seconds; Auto lets the worker choose T. Of is the video blob it was grabbed from: a new video grabs again.

type FrameGrabber added in v0.26.0

type FrameGrabber interface {
	Frame(ctx context.Context, item Item, video File, t float64, width int) ([]byte, error)
}

FrameGrabber grabs a JPEG still at t seconds (width 0 keeps the frame's) from a video upload, for the frame picker (Uploads.Frame); media/video's Frames implements it.

type GetOptions

type GetOptions struct {
	IfNoneMatch string
	Range       string
}

GetOptions: IfNoneMatch returns ErrNotModified on a match; Range is an HTTP Range value.

type Grant

type Grant struct {
	Item       Item
	Resolution access.Resolution
	Manifest   *Manifest
	Expires    time.Time
	// contains filtered or unexported fields
}

Grant is one viewer's resolved access to one item: reads and playlists sign every URL through it.

func (*Grant) Allowed

func (g *Grant) Allowed(f File) bool

Allowed reports whether a read gives this viewer f's URL: with access, every listed file with a blob. Uploads are listed only with ServeOriginals, HostOnly presets never (Grant.HostURL), unattached files and frames not grabbed yet never.

func (*Grant) Cookie

func (g *Grant) Cookie() *http.Cookie

Cookie is the item cookie for a viewer with access in cookie mode, else nil.

func (*Grant) Editor added in v0.19.0

func (g *Grant) Editor() bool

Editor reports an editor's grant.

func (*Grant) Full

func (g *Grant) Full() bool

Full reports access to the item's private files: the resolver's verdict, or an editor's.

func (*Grant) HostURL added in v0.62.0

func (g *Grant) HostURL(path string, dl bool) (string, error)

HostURL is the URL of a HostOnly preset's file for a viewer with access, always carrying the item token (the host route may not have set the cookie). HostOnly only keeps a file out of generic reads: call this from a host route after its own policy, which decides who is handed the URL.

func (*Grant) MasterPlaylist

func (g *Grant) MasterPlaylist(dir string, o MasterOptions) ([]byte, error)

MasterPlaylist is the multivariant playlist of the ladder under dir: one variant per video track (rung and codec), with the audio and subtitle groups; an audio-only ladder is one audio variant. Codecs are listed in the ladder's order, so a player that decodes the first starts on it.

func (*Grant) MediaPlaylist added in v0.62.0

func (g *Grant) MediaPlaylist(ctx context.Context, filePath string) ([]byte, error)

MediaPlaylist is the playlist of the track file at filePath: a byte-range playlist over its blob for video and audio (from its index), or one segment over a whole WebVTT file.

func (*Grant) SpriteVTT

func (g *Grant) SpriteVTT(ctx context.Context, dir string) ([]byte, error)

SpriteVTT is the ladder's seek-preview track: one cue per sprite tile, each pointing at its tile with a #xywh fragment.

func (*Grant) URL

func (g *Grant) URL(f File, dl bool) (string, error)

URL is f's URL (dl: served as a download under its name).

type HLS

type HLS struct {
	// Ladder is the rendition short sides, largest first; empty is DefaultLadder.
	Ladder []int `json:"ladder,omitempty"`
	// MinAspect and MaxAspect bound the source's display width/height; zero
	// is DefaultMinAspect and DefaultMaxAspect. A source outside fails.
	MinAspect float64 `json:"min_aspect,omitempty"`
	MaxAspect float64 `json:"max_aspect,omitempty"`
	Profile   string  `json:"profile,omitempty"` // VideoLive or VideoAnimation
}

HLS is a byte-range fMP4 ladder, one blob per rendition, with the source's audio and subtitle tracks and a seek sprite. A rung above the source is not produced.

func (*HLS) Aspects added in v0.62.0

func (h *HLS) Aspects() (lo, hi float64)

Aspects are the aspect bounds in effect.

func (*HLS) Rungs added in v0.62.0

func (h *HLS) Rungs() []int

Rungs is the ladder in effect.

type HandlerOptions

type HandlerOptions struct {
	Identity Identity
	Logger   *slog.Logger
	// Limit is the per-viewer rate limit (default 2/s, burst 120, per
	// process; set Limit.Redis to share it across replicas): every read,
	// playlist and download counts.
	// Viewers are keyed by Actor.ID, anonymous ones by Actor.IP, else by the
	// connection's address.
	Limit RateLimit
}

HandlerOptions configure the read API.

type Hooks

type Hooks struct {
	// Resolver says who may see an item: the read API, Expose and a new
	// item's first commit (Uploads) use it, and both require it.
	Resolver access.ContentResolver
	// CanUpload says who may write which upload path.
	CanUpload UploadAuthorizer
	// PurgePublic hears of public URLs overwritten, deleted or first
	// written (the CDN may hold the default), for a CDN purge.
	PurgePublic func(ctx context.Context, urls []string)
	// ItemReady reports an item whose processing settled (ready, or failed
	// with nothing processing) in a host transaction, after the worker's
	// job, through the host's media queue. It must be idempotent; an error
	// retries.
	ItemReady func(ctx context.Context, tx pgx.Tx, ref contentref.ContentRef, r Readiness) error
	// Failed reports an upload a producer cannot process, where it runs.
	Failed func(ctx context.Context, ref contentref.ContentRef, path string, err error)
}

Hooks are the app's callbacks.

type HostQueue added in v0.43.0

type HostQueue struct {
	// contains filtered or unexported fields
}

HostQueue is the media worker's handle on the host's River schema: it inserts the jobs the host runs on the worker's behalf (a folder's sweep after an edit, ItemReady and PurgePublic relays). It inserts only.

func NewHostQueue added in v0.43.0

func NewHostQueue(pool *pgxpool.Pool, reg *Registry, schema, queue string, grace time.Duration) (*HostQueue, error)

NewHostQueue targets the host's River schema ("" is the connection's search path) and media queue ("" is DefaultQueue); grace is the host's JobsConfig.Grace (default 24 h).

func (*HostQueue) ExposeTx added in v0.50.0

func (h *HostQueue) ExposeTx(ctx context.Context, tx pgx.Tx, refs ...contentref.ContentRef) error

ExposeTx enqueues the host's Expose of refs in tx, a transaction on the host database, like Jobs.ExposeTx.

func (*HostQueue) Purge added in v0.62.0

func (h *HostQueue) Purge(ctx context.Context, keys []string) error

Purge asks the host to purge public keys from its CDN (Hooks.PurgePublic).

func (*HostQueue) Ready added in v0.62.0

func (h *HostQueue) Ready(ctx context.Context, ref contentref.ContentRef) error

Ready asks the host to report ref's readiness (Hooks.ItemReady) once it has settled.

func (*HostQueue) ScheduleSweep added in v0.43.0

func (h *HostQueue) ScheduleSweep(ctx context.Context, ref contentref.ContentRef) error

ScheduleSweep implements SweepScheduler like Jobs.ScheduleSweep.

type Identity

type Identity interface {
	Actor(ctx context.Context) (access.Actor, bool)
}

Identity reads the authenticated actor the host's middleware put in the context; the same shape as content.Identity.

type Image added in v0.62.0

type Image struct {
	Width   int     `json:"width,omitempty"`
	Height  int     `json:"height,omitempty"`
	Fit     FitMode `json:"fit,omitempty"`
	Quality int     `json:"quality,omitempty"` // default 80
	Blur    float64 `json:"blur,omitempty"`
	// Aspect and MinWidth bound an upload's edit when this is its public
	// preset's image: the crop is fitted to Aspect, and an edit narrower
	// than MinWidth fails.
	Aspect    Aspect    `json:"aspect,omitzero"`
	MinWidth  int       `json:"min_width,omitempty"`
	Animation Animation `json:"animation,omitempty"`
}

Image is a WebP rendering. Zero Width and Height keep the full size; nothing is upscaled.

func Fit

func Fit(n int) *Image

Fit is an image within an n×n box.

func (Image) Resolve added in v0.62.0

func (im Image) Resolve(e *Edit, w, h int) (*Edit, error)

Resolve is the edit im applies to a w×h source (EXIF-oriented): e with its crop height fitted to Aspect, or without a crop the largest centred one at Aspect (a native aspect: the whole source). It must lie inside the source and be at least MinWidth wide once edited.

type ImageError added in v0.38.0

type ImageError struct {
	Code    string
	Message string
	Details ErrorDetails
}

ImageError is an image the rules refuse, synchronously (an edit checked against known dims) or in the image job (recorded as the upload's Failure).

func AsImageError added in v0.38.0

func AsImageError(err error) *ImageError

AsImageError returns the refusal in err's chain, or nil.

func (*ImageError) Error added in v0.38.0

func (e *ImageError) Error() string

type IngestRequest added in v0.18.0

type IngestRequest struct {
	Ref  contentref.ContentRef
	Path string
	Type string
	Body io.Reader // read once, sequentially
	Size int64     // expected size, checked when > 0; required when the actor is rate- or quota-limited
	Meta map[string]any

	// Resume continues a multipart write a previous Ingest left behind:
	// parts already stored with the same bytes are not uploaded again (the
	// body is still read and hashed). An unknown upload starts afresh.
	Resume *IngestUpload
	// OnUpload receives the multipart write once created, for the host to
	// persist for Resume. With it set a failed Ingest keeps the upload (the
	// bucket's abort-incomplete rule removes it after a day); without it the
	// upload is aborted.
	OnUpload func(IngestUpload) error

	PartSize    int64 // default IngestPartSize; at least 5 MiB
	Concurrency int   // parallel part uploads; default IngestConcurrency
}

IngestRequest streams one file of unknown or huge size from the host (a server-side import, not a browser upload) into Ref and puts it at Path.

type IngestResult added in v0.18.0

type IngestResult struct {
	Staged   string
	Size     int64
	Manifest *Manifest
}

IngestResult is the committed upload; Staged is its name in temp/ until the worker places it.

type IngestUpload added in v0.18.0

type IngestUpload struct {
	Temp     string `json:"temp"` // the staged name (u-{uuid})
	UploadID string `json:"upload_id"`
}

IngestUpload identifies an in-progress multipart ingest in temp/.

type InsertFunc added in v0.43.0

type InsertFunc func(ctx context.Context, args river.JobArgs, o *river.InsertOpts) (*rivertype.JobInsertResult, error)

InsertFunc inserts one job (a River client's Insert, or InsertTx bound to a transaction).

type Issuance added in v0.62.0

type Issuance struct {
	// PerHour is the items an account may open per hour (default 120);
	// AnonymousPerHour is per Actor.IP (an IPv6 address counts as its /64),
	// where one address may be many people (default 600). The read API
	// fills an anonymous actor's IP from the connection; a host calling
	// Reader.Grant itself must set it (ErrNoViewerKey otherwise: without it
	// every anonymous viewer would share one count).
	PerHour, AnonymousPerHour int
	Disabled                  bool
	// Exempt actors are not limited (staff); an item's editors never are.
	Exempt func(access.Actor) bool
	// Redis is a Redis or Microsoft Garnet client shared by the replicas.
	// Only SADD, SCARD, SREM, PEXPIRE and MULTI/EXEC are used (no Lua).
	Redis redis.UniversalClient
	// KeyPrefix namespaces the Redis keys (default "contentkit:media:issue:").
	KeyPrefix string
}

Issuance limits how many items a viewer is given the token of per hour: each grant with access opens one item, and a scraper needs one per item. Opening the same item again within the hour is free. With the ingress limiting each IP's downloads from the media host, this bounds what an account can pull: items per hour times what the ingress lets through.

With Redis set every replica shares the count; otherwise each process keeps its own (logged at start). Redis errors fail open to the per-process count (RedisErrors).

type Item

type Item struct {
	// contains filtered or unexported fields
}

Item is a validated item: a ref of a registered kind in its namespace, and its folder's keys.

func (Item) Blob

func (i Item) Blob(name string) (string, error)

Blob is the key of a private blob ("sha256-{hex}").

func (Item) Kind

func (i Item) Kind() *Kind

func (Item) ManifestKey

func (i Item) ManifestKey() string

func (Item) Prefix

func (i Item) Prefix() string

Prefix is the folder, "{namespace}/{kind}/{id}/".

func (Item) PrivatePrefix added in v0.48.0

func (i Item) PrivatePrefix() string

func (Item) Public

func (i Item) Public(name string) (string, error)

Public is the key of a public name.

func (Item) PublicPrefix

func (i Item) PublicPrefix() string

func (Item) Ref

func (i Item) Ref() contentref.ContentRef

func (Item) Staged added in v0.62.0

func (i Item) Staged(name string) (string, error)

Staged is the key of a staged upload ("u-{uuid}"), in temp/ until placed.

func (Item) TempPrefix added in v0.52.0

func (i Item) TempPrefix() string

type Jobs

type Jobs struct {
	// contains filtered or unexported fields
}

Jobs is media's River contribution to the host: sweep, folder deletion, Expose, and the worker's relays (ItemReady, PurgePublic). Processing runs in the media worker (media/worker).

func NewJobs

func NewJobs(cfg JobsConfig) (*Jobs, error)

func (*Jobs) DeleteItemsTx

func (j *Jobs) DeleteItemsTx(ctx context.Context, tx pgx.Tx, items ...Deletion) error

DeleteItemsTx deletes each item's folder through a job enqueued in the host's delete transaction: a host deletes every version item of a work, and erasing an account deletes its accounts/user/{id}/ item. A second pass after LateUploadWindow removes uploads that land after the first. The worker records the quota refund from the manifest before deleting.

func (*Jobs) Expose added in v0.48.0

func (j *Jobs) Expose(ctx context.Context, ref contentref.ContentRef) error

Expose brings an item's public files to its visibility, resolved for an anonymous viewer (Hooks.Resolver). Hiding deletes and purges public/, records Hidden, and deletes again what a pass wrote meanwhile; it needs no readable manifest, so a manifest over its bound still hides. Unhiding marks the public presets pending on their kept sources and asks the worker to render them. It re-resolves after writing and repeats until the state holds, so an Expose racing a visibility change ends at the newer one. An item without a manifest has no public files (its first commit resolves it).

func (*Jobs) ExposeTx added in v0.48.0

func (j *Jobs) ExposeTx(ctx context.Context, tx pgx.Tx, refs ...contentref.ContentRef) error

ExposeTx enqueues Expose for each item in the host's transaction. Call it whenever whether anonymous viewers may see an item changes: publish, unpublish, hide, hold, soft delete, restore.

func (*Jobs) Insert

Insert enqueues a job on the bound client; an empty queue means the media queue.

func (*Jobs) Manifests added in v0.62.0

func (j *Jobs) Manifests() *Manifests

Manifests is the jobs' manifest store; hosts may share it.

func (*Jobs) Purge added in v0.45.0

func (j *Jobs) Purge(ctx context.Context, d Deletion) error

Purge deletes an item's folder now: the explicit reset before deliberately recreating an item, or an operator cleanup. A quota-owned purge needs a caller-stable OperationID; retry a failed purge with it. Hosts deleting content use DeleteItemsTx.

func (*Jobs) Regenerate added in v0.62.0

func (j *Jobs) Regenerate(ctx context.Context, kind, preset string) (int, error)

Regenerate visits every item of kind (in its namespace) and asks the worker to redo its stale outputs of preset ("" for all): after a deploy changed a preset's spec or a producer's recipe. It returns the items visited.

func (*Jobs) RiverJobs

func (j *Jobs) RiverJobs() riverhelpers.Contribution

RiverJobs contributes media's workers, queue and periodic sweep pass to the host's helpers/river composition. It composes once.

func (*Jobs) ScheduleSweep

func (j *Jobs) ScheduleSweep(ctx context.Context, ref contentref.ContentRef) error

ScheduleSweep sweeps the item's folder after the grace period; a sweep already waiting for the folder absorbs it. Manifests calls it on every edit.

func (*Jobs) Sweep

func (j *Jobs) Sweep(ctx context.Context, ref contentref.ContentRef) (SweepResult, error)

Sweep collects garbage by manifest reference, with no other state:

  • private/ blobs the manifest does not reference, once the blob is older than the grace period (a job's outputs not recorded yet, editor views). Later edits never hold it back. A blob written long ago and dropped just now is old already, so a viewer mid-stream on a replaced file may lose it within the grace period;
  • public/ names no preset expects (a removed upload, a hidden item), at once, purged;
  • temp/ by age (JobsConfig.TempTTL), but for staged uploads the manifest references.

It holds the manifest lock through selection and deletion, and decides on a second listing, so a manifest written meanwhile keeps what it references.

func (*Jobs) SweepAll

func (j *Jobs) SweepAll(ctx context.Context) error

SweepAll sweeps every item folder of the registry's kinds: the periodic backstop for missed schedules and abandoned uploads.

func (*Jobs) SweepOrphans added in v0.45.0

func (j *Jobs) SweepOrphans(ctx context.Context, s OrphanSweep) (OrphanReport, error)

SweepOrphans lists the kind's folders and reports, or with Delete removes, those the host says do not exist, once past the grace period. The host runs it: only it knows which items exist.

type JobsConfig

type JobsConfig struct {
	Store     Store
	Registry  *Registry
	Locker    Locker       // serializes manifest edits and sweep deletion; required
	Processes ProcessQueue // the worker's queue (workqueue.Queue): Expose and Regenerate render through it
	// Pool is the host database Hooks.ItemReady's transaction runs on;
	// required with ItemReady.
	Pool *pgxpool.Pool
	// Grace protects job outputs not recorded yet: the sweep deletes an
	// unreferenced blob once it is this old, whatever edits follow. Default
	// 24 h.
	Grace time.Duration
	// TempTTL is how long temp/ objects (in-flight server-side writes) are
	// kept; above the bucket's 1-day abort-incomplete rule. Default 48 h.
	TempTTL time.Duration
	// SweepInterval is the periodic pass over every folder. Default 24 h.
	SweepInterval time.Duration
	// LateUploadWindow delays a folder deletion's second pass, which removes
	// uploads that land after the first; above the presign TTL and the
	// 1-day multipart rule. Default 25 h.
	LateUploadWindow time.Duration
	Limiter          QuotaReleaser // releases a deleted item's quota; optional
	Queue            string        // default DefaultQueue
	MaxWorkers       int           // default 2
	Logger           *slog.Logger
	Now              func() time.Time
}

JobsConfig configures media's River jobs in the host.

type Kind

type Kind struct {
	Name string `json:"name"`
	// Namespace is "" for Config.Namespace, or a shared one such as
	// "accounts" whose items every app importing the kind serves.
	Namespace string   `json:"namespace,omitempty"`
	Uploads   []Upload `json:"uploads"`
	// KeepOriginals keeps an upload's blob once its private outputs exist;
	// otherwise it is dropped (File.Gone). Public presets' sources are
	// always kept, so unhiding can render them again.
	KeepOriginals bool `json:"keep_originals,omitempty"`
	// ServeOriginals lists and serves uploads to viewers with access, like
	// any private file. Otherwise viewer URLs are scoped to served files,
	// not the whole private folder, and reads omit upload paths and blobs.
	ServeOriginals bool      `json:"serve_originals,omitempty"`
	Private        []Private `json:"private,omitempty"`
	Public         []Public  `json:"public,omitempty"`
	// Defaults holds the images the kind's public presets name in
	// Public.Default, so a shared kind ships its own (go:embed). A registry
	// read from JSON (the stock worker) has none; PublishDefaults needs them.
	Defaults fs.FS `json:"-"`
	// contains filtered or unexported fields
}

Kind is one kind of item: what may be uploaded, what is derived from the uploads in private/, and which public images it has.

func (*Kind) DefaultKey added in v0.62.0

func (k *Kind) DefaultKey(name string) string

DefaultKey is the key of the kind's default public image name.

func (*Kind) EditBounds added in v0.62.0

func (k *Kind) EditBounds(path string) Image

EditBounds is the image that bounds an edit of the upload at path: its first public preset's, else a native one.

func (*Kind) FramesVideo added in v0.62.0

func (k *Kind) FramesVideo(m *Manifest, path string) (File, bool)

FramesVideo is the video upload the upload at path is grabbed from (its Upload.Frames), if the manifest has one.

func (*Kind) NS added in v0.62.0

func (k *Kind) NS() string

NS is the namespace the kind's items live in.

func (*Kind) NameOf added in v0.62.0

func (k *Kind) NameOf(path string) string

NameOf is the {name} of an upload path ("originals/001.png" is "001", "cover.png" is "cover").

func (*Kind) Normalize added in v0.62.0

func (k *Kind) Normalize(m *Manifest)

Normalize brings m to its canonical form under k after an edit: the file order, the download names, and dropped originals (Gone).

func (*Kind) OutputPath added in v0.62.0

func (k *Kind) OutputPath(p *Private, path string) string

OutputPath is p's output path (a file, or a directory ending in "/") for the upload at path.

func (*Kind) Presets added in v0.62.0

func (k *Kind) Presets(path string, hidden bool) []string

Presets are the names of the presets an upload at path feeds: the private ones, then (unless hidden) the public ones. Zips are not listed; they follow their inputs.

func (*Kind) PrivateFor added in v0.62.0

func (k *Kind) PrivateFor(path string) []*Private

PrivateFor lists the private presets fed by the upload at path.

func (*Kind) PublicFor added in v0.62.0

func (k *Kind) PublicFor(path string) []*Public

PublicFor lists the public presets fed by the upload at path.

func (*Kind) PublicKept added in v0.62.0

func (k *Kind) PublicKept(m *Manifest) []string

PublicKept are the public names m vouches for; every other name in public/ is deleted. They are its attached uploads' names, none when hidden, and a preview position's only once rendered for the upload now there: a removed page's image does not stay under the next page's name.

func (*Kind) PublicNames added in v0.62.0

func (k *Kind) PublicNames(m *Manifest, p *Public, path string) []string

PublicNames are p's public names in m for the upload at path, one per width: none for an upload outside a preview's First. m may be nil for a preset without First.

func (*Kind) Readiness added in v0.62.0

func (k *Kind) Readiness(m *Manifest) Readiness

Readiness is the manifest's readiness under k.

func (*Kind) Unwritten added in v0.62.0

func (k *Kind) Unwritten(m *Manifest) int64

Unwritten estimates the JSON m gains as the worker processes it. Per upload it is the larger of a failure record and what success writes: an entry for every output its presets have yet to write (an HLS ladder's tracks one by one: a rendition per rung and codec, the sprite, and as many audio and subtitle tracks as a source may carry; an Audio preset's track and M4A), its measured fields, and a staged upload's or frame's blob. Then the zips, and the public presets' pending names an unhide adds. Commits bound the manifest with it; an item whose outputs still overrun it is marked Full by the worker.

func (*Kind) UploadOf added in v0.62.0

func (k *Kind) UploadOf(path string) (Upload, bool)

UploadOf is the Upload a file path belongs to.

func (*Kind) ZipInputs added in v0.62.0

func (k *Kind) ZipInputs(m *Manifest, p *Private) []File

ZipInputs are the files a Zip preset packs: the files under its prefix whose upload is attached, in manifest order.

func (*Kind) ZipStale added in v0.62.0

func (k *Kind) ZipStale(m *Manifest, p *Private) bool

ZipStale reports a Zip preset whose output does not pack its current inputs: missing with inputs, kept without, or another fingerprint.

type LimitError added in v0.62.0

type LimitError struct{ RetryAfter time.Duration }

LimitError is ErrRateLimited with how long to wait.

func (*LimitError) Error added in v0.62.0

func (e *LimitError) Error() string

func (*LimitError) Is added in v0.62.0

func (e *LimitError) Is(target error) bool

type Locker

type Locker interface {
	Lock(ctx context.Context, key string) (unlock func(), err error)
}

Locker serializes manifest edits across every process sharing the bucket.

func PGLocker

func PGLocker(pool *pgxpool.Pool) Locker

PGLocker serializes manifest edits with a per-manifest session advisory lock in the host database: the fallback for backends without conditional PUT. The lock holds its own connection, never one of pool's, so an edit may use the pool (quota settlement) without starving it.

type MP4 added in v0.62.0

type MP4 struct {
	Rung    int    `json:"rung"`
	Profile string `json:"profile,omitempty"`
}

MP4 is a muxed H.264 MP4 at one rung (short side), with the default audio.

func Rung added in v0.62.0

func Rung(n int) *MP4

Rung is an MP4 at short side n.

type Manifest

type Manifest struct {
	V      int  `json:"v"`
	Hidden bool `json:"hidden,omitempty"` // set by Expose; public files are then absent
	// Full: a producer could not record its outputs within the bound, so
	// private outputs stop until a commit frees Deficit bytes (in the
	// manifest and in what its uploads would still add): how far past the
	// bound the refused record went.
	Full    bool           `json:"full,omitempty"`
	Deficit int64          `json:"deficit,omitempty"`
	Meta    map[string]any `json:"meta,omitempty"` // the app's template values, e.g. title
	Files   []File         `json:"files"`
	// contains filtered or unexported fields
}

Manifest is an item's manifest.json: an ordered, app-defined virtual file system over the item's private blobs. Files is in an explicit, deterministic order: each Upload's files by name by default (reordered by commit ops), then each private preset's outputs in their uploads' order. Readers never re-sort it. Public files are never listed.

func DecodeManifest added in v0.62.0

func DecodeManifest(b []byte) (*Manifest, error)

DecodeManifest reads a stored manifest (gzip JSON, or plain JSON).

func (*Manifest) AddPending added in v0.62.0

func (m *Manifest) AddPending(path string, presets ...string)

AddPending adds presets to the Pending of the upload at path.

func (*Manifest) Blobs

func (m *Manifest) Blobs() []string

Blobs lists every blob the manifest references: kept uploads, derived files and track indexes.

func (*Manifest) ClearPending added in v0.62.0

func (m *Manifest) ClearPending(path, preset string)

ClearPending removes preset from the Pending of the upload at path.

func (*Manifest) Clone added in v0.62.0

func (m *Manifest) Clone() *Manifest

Clone deep-copies the manifest for an edit: a cached manifest is shared, so an edit never mutates it.

func (*Manifest) Find added in v0.62.0

func (m *Manifest) Find(path string) int

Find returns the index of the file at path, or -1.

func (*Manifest) Get added in v0.62.0

func (m *Manifest) Get(path string) (File, bool)

Get returns the file at path.

func (*Manifest) Outputs added in v0.62.0

func (m *Manifest) Outputs(from, preset string) []File

Outputs lists the derived files of preset from the upload (or zip prefix) from, in manifest order.

func (*Manifest) SetFailed added in v0.62.0

func (m *Manifest) SetFailed(path string, err error)

SetFailed records err on the upload at path for its current blob and edit, and clears its Pending.

func (*Manifest) SetOutputs added in v0.62.0

func (m *Manifest) SetOutputs(from, preset string, outs []File) error

SetOutputs replaces preset's outputs from the upload (or zip prefix) from with outs (whose From and Preset it sets) and clears preset from the upload's Pending. Outputs at paths taken by other files are an error.

func (*Manifest) StagedNames added in v0.62.0

func (m *Manifest) StagedNames() []string

StagedNames lists the staged uploads the manifest references (in temp/).

func (*Manifest) UploadBytes added in v0.62.0

func (m *Manifest) UploadBytes() int64

UploadBytes is the storage charged for a manifest; deleting or erasing the item releases it.

func (*Manifest) Validate

func (m *Manifest) Validate() error

Validate requires unique paths, well-formed blobs and edits, and upload and derived fields where they belong.

type ManifestOptions

type ManifestOptions struct {
	// Locker is required: every edit runs under it, and also writes with
	// If-Match once the store reports ConditionalPut. See PGLocker.
	Locker Locker
	// CacheBytes bounds the decoded manifests kept in process, revalidated
	// by ETag. A manifest costs about three times its JSON, so the largest
	// (MaxManifestBytes) costs 24 MiB; default 128 MiB, at least two of them.
	CacheBytes int64
	MaxRetries int // conditional-write attempts per edit; default 16
	// Sweeps schedules the folder's sweep after every written edit; best
	// effort (the periodic pass backs it up).
	Sweeps SweepScheduler
}

ManifestOptions configure Manifests.

type Manifests

type Manifests struct {
	// contains filtered or unexported fields
}

Manifests reads and edits item manifests.

func NewManifests

func NewManifests(store Store, reg *Registry, o ManifestOptions) (*Manifests, error)

func (*Manifests) Create added in v0.45.0

func (m *Manifests) Create(ctx context.Context, ref contentref.ContentRef) (*Manifest, error)

Create starts a new item: it writes the item's empty manifest, hidden until the first commit or Expose resolves it, and fails with ErrFolderNotEmpty if the folder already holds any object. Hosts call it when they create the item's row, so a reused id surfaces there.

func (*Manifests) DeleteUnreferenced added in v0.62.0

func (m *Manifests) DeleteUnreferenced(ctx context.Context, item Item, cur *Manifest, blobs []string) error

DeleteUnreferenced deletes the blobs among blobs that cur does not reference. A job calls it from its closing edit (the manifest lock held) for what it wrote for a source that is gone, so a taken-down upload's outputs do not come back under their old names.

func (*Manifests) DropIfDeleted added in v0.62.0

func (m *Manifests) DropIfDeleted(ctx context.Context, ref contentref.ContentRef, blobs []string) error

DropIfDeleted deletes blobs a worker wrote for ref only if its manifest is gone: the folder lock orders the check and the deletes with edits and folder deletion.

func (*Manifests) DropUnreferenced added in v0.62.0

func (m *Manifests) DropUnreferenced(ctx context.Context, ref contentref.ContentRef, u Unreferenced) error

DropUnreferenced deletes at once, under the manifest lock, what u selects and ref's manifest does not reference: a takedown's sweep, without the grace period.

func (*Manifests) Edit

func (m *Manifests) Edit(ctx context.Context, ref contentref.ContentRef, fn func(*Manifest) error) (*Manifest, error)

Edit applies fn to ref's manifest (empty if none) and writes it with If-Match on the ETag it read (If-None-Match for a new one), re-reading and re-applying fn on conflict. fn must be safe to run more than once; an error from fn aborts the edit. The result is normalized (Kind.Normalize) and validated; an unchanged manifest is not written. A folder's first manifest is refused (ErrFolderNotEmpty) over a previous item's public files, and one growing to within editHeadroom of MaxManifestBytes with ErrManifestTooLarge.

func (*Manifests) EditExisting added in v0.59.0

func (m *Manifests) EditExisting(ctx context.Context, ref contentref.ContentRef, fn func(*Manifest) error) (*Manifest, error)

EditExisting is Edit while the manifest exists (ErrNotFound otherwise): workers use it after processing, so a concurrent deletion stays deleted.

func (*Manifests) Get

Get returns ref's manifest and ETag, or ErrNotFound. Cached copies are revalidated with a conditional GET, so a read is never stale. The manifest is shared: never modify it (Clone it).

func (*Manifests) Place added in v0.40.0

func (m *Manifests) Place(ctx context.Context, ref contentref.ContentRef) (int, error)

Place moves ref's staged uploads (temp/u-{uuid}) to their content addresses, private/sha256-{hex} of the bytes the server read, and returns how many files it placed. The media worker runs it before processing:

  1. read each staged upload, hashing it. A single PUT (at most MaxSinglePut, the only kind a client can send again while its URL lives) is written from the bytes that were hashed; a multipart upload, fixed once completed, is copied server-side while its ETag is the one read. Neither is written when the blob exists: a blob is never overwritten, so its bytes always hash to its name;
  2. point the files at their blobs in one edit, which checks under the folder lock (the sweep holds it too) that each blob still exists;
  3. delete the staged objects no manifest references, and, in that edit, the blobs of uploads removed meanwhile (a takedown's stay gone).

A staged upload that is gone or not the size committed fails its file (CodeNotUploaded: upload it again). Every step is idempotent, so a crash converges on the next run.

func (*Manifests) Registry added in v0.62.0

func (m *Manifests) Registry() *Registry

Registry is the registry the manifests are read under.

func (*Manifests) SetFull added in v0.62.0

func (m *Manifests) SetFull(ctx context.Context, ref contentref.ContentRef, cause error) error

SetFull marks ref's manifest Full: a producer could not record its outputs (cause, an ErrManifestTooLarge, whose overrun becomes the Deficit). The flags fit in the headroom every other edit leaves. Producers write no private output for a Full item until a commit frees the deficit.

func (*Manifests) Store added in v0.62.0

func (m *Manifests) Store() Store

Store is the bucket.

func (*Manifests) SyncPublic added in v0.48.0

func (m *Manifests) SyncPublic(ctx context.Context, ref contentref.ContentRef) ([]string, error)

SyncPublic deletes the public names the manifest does not keep (Kind.PublicKept), under the manifest lock. Cleanup is bounded to one minute; deleted keys are returned for cache purging, including partial success on failure.

type MasterOptions

type MasterOptions struct {
	Audio, Subs []string
}

MasterOptions filter a master playlist's renditions by id or BCP 47 tag; nil keeps every track, an empty non-nil slice none.

type Multipart

type Multipart struct {
	Ticket      string
	MinPartSize int64
	MaxPartSize int64
	MaxParts    int
}

Multipart carries the opaque ticket for PresignParts, ListParts, Complete and Abort, and the part-size bounds the client adapts within.

type MultipartReply

type MultipartReply struct {
	Ticket      string `json:"ticket"`
	MinPartSize int64  `json:"min_part_size"`
	MaxPartSize int64  `json:"max_part_size"`
	MaxParts    int    `json:"max_parts"`
}

type Object

type Object struct {
	Key            string
	Size           int64
	ETag           string
	ContentType    string
	CacheControl   string
	ContentRange   string
	LastModified   time.Time
	ChecksumSHA256 []byte            // set when the backend stored a full-object SHA-256
	Metadata       map[string]string // user metadata (x-amz-meta-*), lower-case keys; Head and Get
}

Object is object metadata. For a ranged Get, Size is the body length and ContentRange is set.

type Op

type Op struct {
	Op         string         `json:"op"`
	Path       string         `json:"path,omitempty"`
	Blob       string         `json:"blob,omitempty"`      // put: the staged upload (u-{uuid}) or a blob in the folder
	CreateID   string         `json:"create_id,omitempty"` // put: create-only UUID; repeat the same ID to retry
	Index      *int           `json:"index,omitempty"`     // put, move, attach: among the attached uploads of its Upload
	Meta       map[string]any `json:"meta,omitempty"`      // put: the upload's meta; attach: merged into it; meta: the item's
	Edit       *Edit          `json:"edit,omitempty"`      // put, edit, copy, frame
	Unattached bool           `json:"unattached,omitempty"`
	To         string         `json:"to,omitempty"`   // rename, copy (default the source's path)
	From       *CopyFrom      `json:"from,omitempty"` // copy
	T          *float64       `json:"t,omitempty"`    // frame: seconds into the video
	Auto       bool           `json:"auto,omitempty"` // frame: the worker chooses
	Preset     string         `json:"preset,omitempty"`
	Force      bool           `json:"force,omitempty"`
	// Takedown (remove) also removes the frames grabbed from the upload and
	// the zips that bundled it, then deletes at once, not a grace period
	// later, the blobs the commit dropped, the removed uploads' editor views
	// and the public files no upload renders. With an exempt grant it deletes
	// every private blob the item no longer references (earlier versions and
	// every editor view too), and repeating it, the path already gone, runs
	// that sweep again: how staff complete a takedown that failed part way.
	Takedown bool `json:"takedown,omitempty"`
}

Op is one commit op.

type OrphanFolder added in v0.45.0

type OrphanFolder struct {
	Prefix  string
	ID      string
	ValidID bool
	Objects int
	Newest  time.Time
	Deleted bool
}

OrphanFolder is a folder no host item owns. ValidID false: its id is not an item id, so no item can ever address it.

type OrphanReport added in v0.45.0

type OrphanReport struct {
	Folders int
	Orphans []OrphanFolder
}

OrphanReport lists a kind's orphans; Folders counts every folder seen.

type OrphanSweep added in v0.45.0

type OrphanSweep struct {
	Kind string
	// Exists reports which of ids the host still has (a batch of at most
	// 500). An id it omits is an orphan.
	Exists func(ctx context.Context, ids []string) (map[string]bool, error)
	// Grace skips folders with an object newer than this (default the Jobs
	// grace), so an item created meanwhile is never taken for an orphan.
	Grace time.Duration
	// Delete removes orphans; otherwise they are only reported.
	Delete bool
}

OrphanSweep configures SweepOrphans.

type PGLimiter

type PGLimiter struct {
	// contains filtered or unexported fields
}

PGLimiter is the default UploadLimiter over ContentKit's Postgres schema: hourly counters per uploader (bytes/day sums the last 24), one usage total per owner and short-lived pending reservations. There are no rows per file.

func NewPGLimiter

func NewPGLimiter(pool *pgxpool.Pool, schema string, limits PGLimits) (*PGLimiter, error)

func (*PGLimiter) PrepareRelease added in v0.59.0

func (l *PGLimiter) PrepareRelease(ctx context.Context, r QuotaRelease) (bool, error)

func (*PGLimiter) Release added in v0.59.0

func (l *PGLimiter) Release(ctx context.Context, tenant, operation string) error

func (*PGLimiter) Reserve

func (l *PGLimiter) Reserve(ctx context.Context, r Reservation) error

func (*PGLimiter) Settle

func (l *PGLimiter) Settle(ctx context.Context, s Settlement) error

func (*PGLimiter) Usage

func (l *PGLimiter) Usage(ctx context.Context, tenant, owner string) (used, pending int64, err error)

Usage reports an owner's stored bytes and unexpired pending reservations.

type PGLimits

type PGLimits struct {
	FilesPerHour int
	BytesPerDay  int64
	// Quota is the owner's storage cap in bytes (<= 0 unlimited); nil disables quotas.
	Quota func(ctx context.Context, tenant, owner string) (int64, error)
	// ReservationTTL drops unsettled reservations (commit still enforces the
	// quota); default 24h.
	ReservationTTL time.Duration
}

PGLimits configure PGLimiter. Zero limits are off.

type Part

type Part struct {
	Number int32
	Size   int64
	ETag   string
	SHA256 []byte
}

Part is one uploaded multipart part.

type PartBody

type PartBody struct {
	Number int32  `json:"number"`
	Size   int64  `json:"size"`
	SHA256 string `json:"sha256"`
}

type PartReply

type PartReply struct {
	Number  int32         `json:"number"`
	Size    int64         `json:"size,omitempty"`
	SHA256  string        `json:"sha256,omitempty"`
	Request *RequestReply `json:"request,omitempty"`
}

PartReply is a presigned part (parts) or a landed part (parts/list).

type PartRequest

type PartRequest struct {
	Number int32
	Size   int64
	SHA256 []byte
}

PartRequest is one part to presign: its exact length and SHA-256.

type PartsBody

type PartsBody struct {
	Ticket string     `json:"ticket"`
	Parts  []PartBody `json:"parts"`
}

type PartsReply

type PartsReply struct {
	Parts []PartReply `json:"parts"`
}

type PresignBody

type PresignBody struct {
	Ref    RefBody `json:"ref"`
	Path   string  `json:"path"`
	Type   string  `json:"type"`
	Size   int64   `json:"size"`
	SHA256 string  `json:"sha256"`
}

PresignBody declares one upload: its path, type, size and whole-file SHA-256, which finds an identical blob already in the folder and binds a single PUT's body.

type PresignPut

type PresignPut struct {
	ContentType string
	Size        int64
	SHA256      []byte
	TTL         time.Duration
}

PresignPut binds a direct upload to its exact type, length and SHA-256.

type PresignReply

type PresignReply struct {
	Path            string          `json:"path"`
	Blob            string          `json:"blob"`
	Exists          bool            `json:"exists,omitempty"`
	Put             *RequestReply   `json:"put,omitempty"`
	Multipart       *MultipartReply `json:"multipart,omitempty"`
	ProcessOnUpload bool            `json:"process_on_upload,omitempty"`
}

PresignReply is the upload plan: Exists (commit directly), one Put, or a Multipart upload. Path is the path to commit: cleaned, with an extension, and named by the server for a Named upload. Blob is the name to commit: the folder's blob (sha256-{hex}) with Exists, else the staged upload (u-{uuid}) the PUT or parts write, which the worker places.

type PresignRequest

type PresignRequest struct {
	Ref    contentref.ContentRef
	Path   string
	Type   string
	Size   int64
	SHA256 []byte
}

PresignRequest declares one upload; SHA256 is the whole file's: it finds an identical blob already in the folder and binds a single PUT's body.

type Presigned

type Presigned struct {
	Path            string
	Blob            string
	Exists          bool
	Put             *PresignedRequest
	Multipart       *Multipart
	ProcessOnUpload bool
}

Presigned is the upload plan: Exists (commit it), a single Put, or a Multipart upload. Path is the path to commit; Blob is the name to commit: the folder's blob when Exists, else the staged upload (u-{uuid}) that the PUT or the parts write.

type PresignedPart

type PresignedPart struct {
	Number int32
	PresignedRequest
}

PresignedPart is a part URL.

type PresignedRequest

type PresignedRequest struct {
	Method  string
	URL     string
	Header  http.Header
	Expires time.Time
}

PresignedRequest is a request a browser sends as is, with exactly Header.

type Private added in v0.62.0

type Private struct {
	Name string `json:"name"`
	// From is an upload path or pattern; "" for Zip.
	From string `json:"from,omitempty"`
	// To is the output path template: a file ("low-res/{name}.webp",
	// "video/source-1080p.mp4") or, for producers with several outputs, a
	// directory ending in "/" ("hls/", "listen/{name}/").
	To string `json:"to"`
	// Download is the human name template a download read signs into the
	// URL, filled from the manifest's meta and {name}: "{title}.zip".
	Download string `json:"download,omitempty"`
	// HostOnly leaves this preset out of generic reads and playlists; a host
	// route applies its own policy and calls Grant.HostURL. It decides what
	// is listed, not what a token opens: an item's token opens every private
	// file of the item.
	HostOnly  bool       `json:"host_only,omitempty"`
	Image     *Image     `json:"image,omitempty"`
	HLS       *HLS       `json:"hls,omitempty"`
	MP4       *MP4       `json:"mp4,omitempty"`
	Zip       string     `json:"zip,omitempty"` // the files under this prefix, in manifest order: "high/"
	Audio     *Audio     `json:"audio,omitempty"`
	Subtitles *Subtitles `json:"subtitles,omitempty"`
	// Choose is an optional per-file image spec (Go only, not in the JSON
	// registry); nil keeps Image.
	Choose func(f File) *Image `json:"-"`
}

Private derives files in private/, recorded in the manifest with their provenance (File.From, Preset, FP). Exactly one producer is set.

type ProcessCanceler added in v0.43.0

type ProcessCanceler interface {
	Cancel(ctx context.Context, ref contentref.ContentRef) (int, error)
}

ProcessCanceler is a ProcessQueue that can cancel an item's queued and running jobs (workqueue.Queue): a commit removing an upload still being processed cancels them, then enqueues the item's remaining work.

type ProcessJob

type ProcessJob struct {
	Ref    contentref.ContentRef `json:"ref"`
	Place  bool                  `json:"place,omitempty"`
	Preset string                `json:"preset,omitempty"` // only this preset; "" for all
	Force  bool                  `json:"force,omitempty"`  // redo current outputs too
	Editor bool                  `json:"editor,omitempty"` // also render missing editor views
	Class  VideoJobClass         `json:"class,omitempty"`
}

ProcessJob asks the media worker to bring an item's outputs up to date: every producer recomputes its outputs' fingerprints and redoes the stale ones (and the pending ones), records them, and syncs public/. With Place, the worker first places the item's staged uploads (Manifests.Place).

type ProcessQueue

type ProcessQueue interface {
	Enqueue(ctx context.Context, job ProcessJob) error
}

ProcessQueue enqueues processing in the media worker (workqueue.Queue).

type ProgressSource added in v0.29.0

type ProgressSource interface {
	EncodeProgress(ctx context.Context, ref contentref.ContentRef) (EncodeStatus, error)
}

ProgressSource reads encode progress (workqueue.NewProgressSource). The read API asks only when a visible video file is pending.

type Public added in v0.62.0

type Public struct {
	Name   string `json:"name"`
	From   string `json:"from"`
	To     string `json:"to"` // "cover-{w}.webp", "{name}.webp", "preview-{n}.webp"
	First  int    `json:"first,omitempty"`
	Widths []int  `json:"widths,omitempty"`
	Image  Image  `json:"image"`
	// Default is a path in Kind.Defaults, rendered to the kind's _default
	// item at every width (PublishDefaults).
	Default string `json:"default,omitempty"`
}

Public is a fixed public image: {To} with {w} at each of Widths (or one file without Widths), rendered from an upload through its edit, at public/{name}. With Widths, each width is Image at that width: in the shape of its Width×Height box (both or neither), fitted per Fit, else of the edit. It is never in the manifest; the object carries its from and fp as metadata. A missing one is served Default by the access agent.

First makes it a preview: the first First attached uploads of From (a {name} pattern) in manifest order, {n} in To their position from 1 ("preview-{n}.webp"). Anyone who can see the item sees them; a position is rendered again when another upload takes it, and names past the last upload are deleted. Everything else of the item stays private.

type PutOptions

type PutOptions struct {
	ContentType    string
	CacheControl   string
	ChecksumSHA256 []byte
	IfMatch        string
	IfNoneMatch    string
	Metadata       map[string]string
}

PutOptions are conditions and headers for Put. IfNoneMatch "*" creates only.

type QuotaRelease added in v0.59.0

type QuotaRelease struct {
	Tenant    string
	Folder    string
	Owner     string
	Operation string
	Bytes     int64
}

QuotaRelease is the amount captured before deleting a folder.

type QuotaReleaser added in v0.59.0

type QuotaReleaser interface {
	PrepareRelease(ctx context.Context, r QuotaRelease) (applied bool, err error)
	Release(ctx context.Context, tenant, operation string) error
}

QuotaReleaser records a folder's refund before its objects are deleted, then applies that refund once. Operation identifies the deletion across retries.

type RateLimit added in v0.62.0

type RateLimit struct {
	PerSecond float64
	Burst     int
	// Disabled turns limiting off, e.g. when the host limits upstream.
	Disabled bool
	// Redis is a Redis or Microsoft Garnet client shared by the replicas;
	// pass the host's own. Only INCR, PEXPIRE, GET, DECR and MULTI/EXEC are
	// used (no Lua: Garnet ships with scripting off).
	Redis redis.UniversalClient
	// KeyPrefix namespaces the Redis keys (defaults "contentkit:media:rl:"
	// for viewers, "contentkit:media:commit:" for uploaders).
	KeyPrefix string
}

RateLimit is a per-key request limit: the read API's per viewer (HandlerOptions.Limit) and commits' per uploader (UploadOptions.Commits). Zero fields take the use's defaults.

With Redis set, every replica shares one limit per key; otherwise each process keeps its own (a single-replica assumption, logged at start), so N replicas allow N times the limit. Redis errors fail open to the per-process limit (see RedisErrors): the limit is abuse protection, and tokens, permissions and visibility checks still gate everything.

type ReadOptions

type ReadOptions struct {
	Prefix        string // only files under this path prefix ("low-res/")
	Offset, Limit int    // the range that gets URLs
	Download      bool   // URLs serve each file as a download under its name
	// Editor adds an editor's uploads with their edit, frame, meta,
	// pending, failure and editor view, unattached ones included.
	Editor bool
}

ReadOptions select what a read returns.

type ReadResult

type ReadResult struct {
	Access string `json:"access"`
	// Expires is when the item token (the URLs, the cookie) stops working,
	// in unix seconds. A read before then answers the same token, so a
	// client keeps what it has and reads again shortly before.
	Expires int64          `json:"expires"`
	Meta    map[string]any `json:"meta,omitempty"`
	// Previews are the item's public preview images in order (a Public
	// preset with First): every viewer who can see the item gets them.
	Previews []string `json:"previews,omitempty"`
	Total    int      `json:"total"`
	Offset   int      `json:"offset"`
	Limit    int      `json:"limit"`
	// HLS lists the item's playable ladders by output directory ("hls/"):
	// play {read API}/{kind}/{id}/hls/{dir}master.m3u8.
	HLS []string `json:"hls,omitempty"`
	// State is the item's readiness (editors).
	State string `json:"state,omitempty"`
	// Full (editors): the item's manifest cannot hold more outputs, so
	// processing stopped; remove uploads (any commit that shrinks it) to
	// resume. Uploads left unprocessed stay pending.
	Full  bool       `json:"full,omitempty"`
	Files []FileInfo `json:"files"`
	// Cookie must be set on the response (cookie delivery, with access).
	Cookie *http.Cookie `json:"-"`
}

ReadResult is the read API's answer: the files under the requested prefix in manifest order (Total of them). With access each has a URL within [offset, offset+limit); without, each is locked: its path, type and size.

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

Reader answers the read API and HLS playlists: one Resolve per item. An item's private files are all or nothing: a viewer with access gets one token that opens every one of them; anyone else gets none, and sees only the item's public files (covers, previews).

func NewReader

func NewReader(o ReaderOptions) (*Reader, error)

func (*Reader) Grant

func (r *Reader) Grant(ctx context.Context, ref contentref.ContentRef, actor access.Actor) (*Grant, error)

Grant resolves ref for actor exactly once and loads its manifest. A resolver error denies (ErrResolve); an invisible item is ErrNotVisible. A visible item without a manifest has no files. A viewer with access who has opened too many items this hour (ReaderOptions.Issuance) gets a LimitError (ErrRateLimited) instead of the item's token; an anonymous actor with access must carry Actor.IP (ErrNoViewerKey).

func (*Reader) Handler

func (r *Reader) Handler(o HandlerOptions) http.Handler

Handler serves the read API. The host mounts it under a prefix such as "/media/" after its auth middleware. Errors are JSON {"error", "code"}: 400 invalid_request, 404 not_found (also for what the viewer may not see), 429 rate_limited (Retry-After: too many requests, or too many items opened this hour), 503 unavailable, 500 internal_error (a resolver error denies this way).

GET /{kind}/{id}?prefix=low-res/&offset=0&limit=50&download&editor -> ReadResult (+ Set-Cookie mt)
GET /{kind}/{id}/hls/{dir}master.m3u8?audio=ja&subs=en (filters optional; empty = none)
GET /{kind}/{id}/hls/{path}.m3u8   a track's media playlist
GET /{kind}/{id}/hls/{dir}sprite.vtt

Every response is "private, no-store"; playlists carry the item cookie like reads. Signed responses log the viewer, item, access and expiry.

func (*Reader) Read

Read resolves ref once and answers the read API.

type ReaderOptions

type ReaderOptions struct {
	Manifests *Manifests // its Registry's BaseURL and Hooks.Resolver are required
	Delivery  Delivery
	// Progress adds live encode progress to pending uploads in editor
	// reads; optional.
	Progress ProgressSource
	// Queue renders the editor views an editor read finds missing; optional.
	Queue ProcessQueue
	// MaxLimit caps ReadOptions.Limit (default 200); DefaultLimit is used
	// when Limit is 0 (default 50).
	MaxLimit, DefaultLimit int
	// IndexCacheBytes bounds the track index blobs kept for playlists;
	// default 16 MiB.
	IndexCacheBytes int64
	// Issuance limits the items each viewer is given access to per hour
	// (default 120 per account, 600 per anonymous IP).
	Issuance Issuance
	Now      func() time.Time
}

ReaderOptions configure a Reader.

type Readiness added in v0.50.0

type Readiness struct {
	State      string   `json:"state"`
	Processing []string `json:"processing,omitempty"`
	Failed     []string `json:"failed,omitempty"`
}

Readiness is whether an item's media is processed: ready when no attached upload is pending (its public presets count only while the item is visible) and every zip packs its inputs; processing while any is; failed once nothing is processing and an upload failed for its current blob and edit; full while work is left on a Full manifest. Processing and Failed name upload paths (and zip outputs).

func (Readiness) Ready added in v0.50.0

func (r Readiness) Ready() bool

type RefBody

type RefBody struct {
	Kind string `json:"kind"`
	ID   string `json:"id"`
}

RefBody names an item of the handler's registry.

type Registry

type Registry struct {
	// contains filtered or unexported fields
}

Registry is a validated Config.

func NewRegistry

func NewRegistry(c Config) (*Registry, error)

NewRegistry validates c.

func (*Registry) Config added in v0.62.0

func (r *Registry) Config() Config

Config is the registry's configuration, normalized.

func (*Registry) EditorView added in v0.62.0

func (r *Registry) EditorView(f File) string

EditorView is the blob name of an image upload's editor view: the hash of its source and the editor spec, so a read finds it without rendering. It is the one private blob not named by its bytes: only the worker writes it, and no upload may name it (presign, put and copy refuse it). Only editor reads list it; the sweep removes it after the grace period and an editor read renders it again.

func (*Registry) Item

func (r *Registry) Item(ref contentref.ContentRef) (Item, error)

Item validates ref: a registered kind, its namespace and a UUIDv7 id, without a version (an item is the host's version).

func (*Registry) Kind

func (r *Registry) Kind(name string) (*Kind, error)

Kind returns a registered kind.

func (*Registry) MarshalJSON added in v0.62.0

func (r *Registry) MarshalJSON() ([]byte, error)

MarshalJSON is the registry as data (no hooks, defaults or Choose): the stock worker's MEDIA_KINDS_FILE.

func (*Registry) Namespace added in v0.62.0

func (r *Registry) Namespace() string

Namespace is the app's own namespace.

func (*Registry) Namespaces added in v0.62.0

func (r *Registry) Namespaces() []string

Namespaces are the namespaces the app's items live in: its own, then the shared ones it imports.

func (*Registry) PublicURL added in v0.62.0

func (r *Registry) PublicURL(ref contentref.ContentRef, name string) string

PublicURL is the URL of item ref's public name at the registry's BaseURL.

func (*Registry) Ref added in v0.62.0

func (r *Registry) Ref(kind, id string) (contentref.ContentRef, error)

Ref is the ref of item id of kind, in the kind's namespace.

func (*Registry) SrcSet added in v0.62.0

func (r *Registry) SrcSet(ref contentref.ContentRef, preset string) string

SrcSet is the srcset of ref's public preset (by name), "" when the kind has none.

type RequestReply

type RequestReply struct {
	Method  string            `json:"method"`
	URL     string            `json:"url"`
	Headers map[string]string `json:"headers"`
	Expires time.Time         `json:"expires"`
}

RequestReply is a presigned request: send exactly these headers (the browser adds Content-Length, which is signed too).

type RerunArgs added in v0.43.0

type RerunArgs interface {
	river.JobArgs
	FollowUp(id int64) river.JobArgs
}

RerunArgs are job args that can name the running job they follow.

type Reservation

type Reservation struct {
	Tenant   string
	Uploader string
	Owner    string
	Key      string
	Size     int64
}

Reservation is one presigned upload. Uploader is rate-limited; Owner ("" for none) has Size reserved against its quota until the upload is settled or the reservation expires.

type Segment

type Segment struct {
	Offset  int64
	Length  int64
	Seconds float64
}

Segment is one EXT-X-BYTERANGE segment, encoded as [offset, length, seconds].

func (Segment) MarshalJSON

func (s Segment) MarshalJSON() ([]byte, error)

func (*Segment) UnmarshalJSON

func (s *Segment) UnmarshalJSON(b []byte) error

type Settlement

type Settlement struct {
	Tenant  string
	Owner   string
	Keys    []string
	Delta   int64
	Enforce bool
}

Settlement releases reservations and moves an owner's usage. With Enforce, a positive Delta that takes usage over the owner's quota is refused with CodeQuota and changes nothing.

type Sprite

type Sprite struct {
	Cols     int     `json:"cols"`
	Rows     int     `json:"rows"`
	W        int     `json:"w"`
	H        int     `json:"h"`
	Interval float64 `json:"interval"`
}

Sprite is a seek-preview grid of Cols×Rows tiles of W×H, one per Interval seconds.

type Store

type Store interface {
	Put(ctx context.Context, key string, body io.Reader, size int64, opts PutOptions) (Object, error)
	Get(ctx context.Context, key string, opts GetOptions) (io.ReadCloser, Object, error)
	Head(ctx context.Context, key string) (Object, error)
	Delete(ctx context.Context, key string) error
	List(ctx context.Context, prefix string) iter.Seq2[Object, error]
	// Copy copies src to dst server-side, keeping its content type, cache
	// control and metadata; objects past the backend's single-copy limit
	// (5 GiB on S3) copy in parts. The returned Object has dst's size.
	Copy(ctx context.Context, src, dst string, o CopyOptions) (Object, error)

	PresignPut(ctx context.Context, key string, p PresignPut) (PresignedRequest, error)
	// PresignGet is for a worker reading an original over HTTP ranges. Its URL
	// uses the store's internal endpoint and must not be returned to browsers.
	PresignGet(ctx context.Context, key string, ttl time.Duration) (PresignedRequest, error)
	CreateMultipart(ctx context.Context, key, contentType string) (uploadID string, err error)
	PresignPart(ctx context.Context, key, uploadID string, number int32, size int64, sha256 []byte, ttl time.Duration) (PresignedRequest, error)
	// PutPart uploads one part from the server, bound to its length and SHA-256.
	PutPart(ctx context.Context, key, uploadID string, number int32, body io.Reader, size int64, sha256 []byte) (Part, error)
	ListParts(ctx context.Context, key, uploadID string) ([]Part, error)
	CompleteMultipart(ctx context.Context, key, uploadID string, parts []Part) (Object, error)
	AbortMultipart(ctx context.Context, key, uploadID string) error

	Capabilities() Capabilities
	// Check reports whether the backend answers. The first success also
	// establishes Capabilities (Probe under prefix) when they were not given.
	Check(ctx context.Context, prefix string) error
}

Store is the bucket. Keys are built by Item; implementations do not interpret them.

type Subtitles added in v0.62.0

type Subtitles struct{}

Subtitles converts SRT, SSA/ASS and WebVTT to clean UTF-8 WebVTT. A video lists the item's converted subtitles as tracks after its own.

type SweepResult

type SweepResult struct {
	Deleted []string
	Wait    time.Duration
}

SweepResult reports one folder sweep. Wait > 0 is when the next unreferenced blob comes due; sweep again then.

type SweepScheduler added in v0.43.0

type SweepScheduler interface {
	ScheduleSweep(ctx context.Context, ref contentref.ContentRef) error
}

SweepScheduler schedules a folder's sweep after an edit: the host's *Jobs, or in the media worker a *HostQueue.

type TicketBody

type TicketBody struct {
	Ticket string `json:"ticket"`
}

type Track added in v0.62.0

type Track struct {
	Kind      string `json:"kind"`             // TrackVideo, TrackAudio, TrackSubs, TrackSprite
	Codec     string `json:"codec,omitempty"`  // video: h264, hevc, av1
	Codecs    string `json:"codecs,omitempty"` // RFC 6381
	Bandwidth int    `json:"bw,omitempty"`
	Average   int    `json:"avg,omitempty"`
	ID        string `json:"id,omitempty"`
	Lang      string `json:"lang,omitempty"`
	Label     string `json:"label,omitempty"`
	Default   bool   `json:"default,omitempty"`
	Forced    bool   `json:"forced,omitempty"`
	Index     string `json:"index,omitempty"` // "sha256-{hex}": the TrackIndex
}

Track is an HLS track file. It is small: the segment table or sprite grid is its own private blob, Index (a TrackIndex), cached by hash.

type TrackIndex added in v0.62.0

type TrackIndex struct {
	Segments []Segment `json:"segments,omitempty"`
	Sprite   *Sprite   `json:"sprite,omitempty"`
}

TrackIndex is a Track's index blob: a byte-range track's segments (its init segment is bytes [0, Segments[0].Offset)), or a sprite's grid.

type Unreferenced added in v0.62.0

type Unreferenced struct {
	// All is every private blob in the folder, editor views included: a
	// worker whose unrecorded outputs go with them finds them missing when
	// it records, and produces them again.
	All bool
	// Blobs are candidates: each goes unless it is an editor view of a
	// current upload.
	Blobs []string
	// Staged are staged uploads' names.
	Staged []string
}

Unreferenced selects what DropUnreferenced deletes.

type Upload added in v0.62.0

type Upload struct {
	Path     string   `json:"path"`
	Types    []string `json:"types"`            // accepted content types
	MaxBytes int64    `json:"max_bytes"`        // per file
	Max      int      `json:"max,omitempty"`    // uploads matching Path; 0 is unlimited
	Frames   string   `json:"frames,omitempty"` // may be grabbed from a frame of this video upload (the frame op)
	Named    bool     `json:"named,omitempty"`  // the server names it ({name} is "i-{uuid}"): inline images
	// Video bounds a video or audio upload as probed from its real stream;
	// nil or zero fields take the defaults (DefaultVideoLimits).
	Video *VideoLimits `json:"video,omitempty"`
}

Upload is an app path that may be uploaded: a literal ("cover") or a pattern ending in {name} ("originals/{name}"). A file's path is the path plus its extension ("cover.png", "originals/001.png"); a put to the same stem replaces it.

type UploadAuthorizer

type UploadAuthorizer interface {
	CanUpload(ctx context.Context, actor access.Actor, t UploadTarget) (UploadGrant, error)
}

UploadAuthorizer is the app's upload permission check, run at presign and commit against what is written.

type UploadError

type UploadError struct {
	Code       string
	Message    string
	RetryAfter time.Duration // CodeRate: when the window frees
	Blobs      []string      // CodeNotUploaded at commit: the blobs to upload again
	Details    *ErrorDetails // image refusals
}

UploadError is a refused upload request. UploadLimiter implementations refuse with CodeRate or CodeQuota.

func AsUploadError

func AsUploadError(err error) (*UploadError, bool)

AsUploadError classifies err: an *UploadError, or the kind and registry sentinels. ok is false for internal failures.

func (*UploadError) Error

func (e *UploadError) Error() string

func (*UploadError) Status

func (e *UploadError) Status() int

Status is the HTTP status for Code.

type UploadGrant

type UploadGrant struct {
	Allowed bool
	Exempt  bool
	Owner   string
}

UploadGrant is the app's verdict. Exempt (trusted roles) skips the UploadLimiter; Owner is the quota owner, "" for none.

type UploadHandlerOptions

type UploadHandlerOptions struct {
	Actor  func(*http.Request) (access.Actor, bool) // the host's authenticated caller; false answers 401
	Logger *slog.Logger                             // 5xx causes; default slog.Default()
}

UploadHandlerOptions configure UploadHandler.

type UploadLimiter

type UploadLimiter interface {
	Reserve(ctx context.Context, r Reservation) error
	Settle(ctx context.Context, s Settlement) error
}

UploadLimiter is the optional anti-abuse port. Quota is enforced when bytes are committed: a commit that grows its owner's stored originals past the quota is refused. Reserve runs at presign (skipped for exempt uploaders) and refuses with an *UploadError coded CodeRate or CodeQuota before any bytes move; a reservation is only that early refusal and may expire. Settle runs at commit and abort: it drops the reservations of Keys and adds Delta (the change in stored originals, negative on removal) to the owner's usage.

type UploadOptions

type UploadOptions struct {
	Store     Store
	Manifests *Manifests    // its Registry's Hooks.CanUpload authorizes, Hooks.Resolver hides new items
	Tickets   *token.Ring   // signs multipart tickets (domain-separated from access tokens); required above MaxSinglePut
	Limiter   UploadLimiter // optional
	Queue     ProcessQueue  // places staged uploads and processes items in the media worker; required
	// PresignTTL bounds PUT and part URLs; default 15m. TicketTTL bounds a
	// multipart ticket; default 24h, the bucket's abort-incomplete rule.
	PresignTTL, TicketTTL time.Duration
	// Grace is the sweep's (JobsConfig.Grace): an existing blob is reused
	// only while the sweep cannot take it first. Default 24h.
	Grace time.Duration
	// Frames serves the frame picker (media/video.Frames); nil answers
	// not_found. FrameConcurrency bounds grabs per process; default 2.
	Frames           FrameGrabber
	FrameConcurrency int
	// ProcessOnUpload tells the SDK to commit each upload unattached as
	// soon as it lands, so the worker processes it while the user arranges
	// the rest; an attach op makes it part of the item.
	ProcessOnUpload bool
	// Commits limits each uploader's commits (default 1/s, burst 30, per
	// process; set Commits.Redis to share it); exempt grants are not
	// limited.
	Commits RateLimit
}

UploadOptions configure Uploads.

type UploadTarget added in v0.60.0

type UploadTarget struct {
	Ref  contentref.ContentRef
	Path string
}

UploadTarget is what an upload writes: an item and an upload path stem ("cover", "originals/001"); "" for ops that change no one upload (move, meta, regenerate).

type UploadedBlob added in v0.62.0

type UploadedBlob struct {
	Blob string
	Type string
	Size int64
}

UploadedBlob is a completed multipart upload; Blob is its staged name.

type Uploads

type Uploads struct {
	// contains filtered or unexported fields
}

Uploads presigns direct-to-bucket uploads and commits them. It keeps no state: a multipart upload is its S3 UploadId, carried in a signed ticket.

func NewUploads

func NewUploads(o UploadOptions) (*Uploads, error)

func (*Uploads) Abort

func (u *Uploads) Abort(ctx context.Context, actor access.Actor, sealed string) error

Abort cancels a multipart upload and drops its reservation.

func (*Uploads) Commit

func (u *Uploads) Commit(ctx context.Context, actor access.Actor, ref contentref.ContentRef, ops []Op) (*Manifest, error)

Commit applies ops to ref's manifest in one conditional write. Every op is authorized against what it writes (Hooks.CanUpload). A put names a staged upload or a blob in the folder, HEAD-checked against its Upload; copies are copied server-side first. The owner is charged the change in upload sizes; growth past its quota fails with CodeQuota (not for exempt grants). Then the worker places staged uploads and processes the item. A new item starts hidden unless anonymous viewers may see it (Hooks.Resolver). Successful removes finish public cleanup before returning; unreferenced private blobs go when a grace period old, or at once for a takedown (Op.Takedown).

func (*Uploads) Complete

func (u *Uploads) Complete(ctx context.Context, actor access.Actor, sealed string) (UploadedBlob, error)

Complete assembles the parts server-side. Missing parts answer CodeIncomplete and keep the upload; parts that cannot add up to the declared size abort it.

func (*Uploads) Frame added in v0.26.0

func (u *Uploads) Frame(ctx context.Context, actor access.Actor, ref contentref.ContentRef, path string, t float64, width int) ([]byte, error)

Frame grabs a still of the video upload at path for the frame picker.

func (*Uploads) Ingest added in v0.18.0

func (u *Uploads) Ingest(ctx context.Context, actor access.Actor, req IngestRequest) (IngestResult, error)

Ingest writes req.Body to a staged upload in temp/, one checksum-bound PUT when it fits in one part, else a multipart write, and commits a put of it: the worker places and processes it like a browser upload.

func (*Uploads) ListParts

func (u *Uploads) ListParts(ctx context.Context, actor access.Actor, sealed string) ([]Part, error)

ListParts reports the parts that landed, for resuming.

func (*Uploads) Presign

func (u *Uploads) Presign(ctx context.Context, actor access.Actor, r PresignRequest) (Presigned, error)

Presign checks an upload against its path's Upload and the app's permission, and plans it.

func (*Uploads) PresignParts

func (u *Uploads) PresignParts(ctx context.Context, actor access.Actor, sealed string, parts []PartRequest) ([]PresignedPart, error)

PresignParts signs parts of a multipart upload, each bound to its length and SHA-256. Parts re-signed after a failure replace the earlier attempt.

type VideoJobClass added in v0.58.0

type VideoJobClass string

VideoJobClass selects the priority of a new video run. An empty class is an upload; re-encodes and backfills yield to new playable videos.

const (
	VideoReencode VideoJobClass = "reencode"
	VideoBackfill VideoJobClass = "backfill"
)

type VideoLimits added in v0.62.0

type VideoLimits struct {
	MaxSeconds float64 `json:"max_seconds,omitempty"` // real running time (audio too)
	MaxFPS     float64 `json:"max_fps,omitempty"`     // output frame rate (at most 60); a faster source is encoded at it
	MaxPixels  int     `json:"max_pixels,omitempty"`  // displayed frame area
	// MaxWork bounds the planned encode: the sum over the HLS and MP4
	// presets' rungs and codecs of output pixels × output frames.
	MaxWork float64 `json:"max_work,omitempty"`
}

VideoLimits bound what one video or audio upload may cost the worker; an upload past them fails (video_too_long, video_too_large, video_over_budget) before any encode runs.

func (*VideoLimits) Limits added in v0.62.0

func (l *VideoLimits) Limits() VideoLimits

Limits are the limits in effect: l's, and the defaults for nil or zero fields.

Directories

Path Synopsis
Package agent is the media access agent run by cmd/media-access.
Package agent is the media access agent run by cmd/media-access.
Package image is the media worker's image producer (libvips, CGO): the Image presets' private WebP files, the Zip presets, the public presets' fixed names, editor views, and the kinds' default public images.
Package image is the media worker's image producer (libvips, CGO): the Image presets' private WebP files, the Zip presets, the public presets' fixed names, editor views, and the kinds' default public images.
internal
s3test
Package s3test opens the test bucket from the environment:
Package s3test opens the test bucket from the environment:
uploadtestserver command
Command uploadtestserver serves the media upload and read APIs over an isolated MinIO/RGW namespace for the browser SDK's integration tests (sdk/upload/test) and e2e specs.
Command uploadtestserver serves the media upload and read APIs over an isolated MinIO/RGW namespace for the browser SDK's integration tests (sdk/upload/test) and e2e specs.
videotest
Package videotest builds synthetic videos whose frames identify their time and orientation, and classifies decoded pixels, for poster and preview tests.
Package videotest builds synthetic videos whose frames identify their time and orientation, and classifies decoded pixels, for poster and preview tests.
wirets
Package wirets renders the upload and read API wire types as TypeScript for the browser SDK (sdk/upload/src/wire.gen.ts).
Package wirets renders the upload and read API wire types as TypeScript for the browser SDK (sdk/upload/src/wire.gen.ts).
Package layout defines media object keys, dependency-free so the access agent can classify paths without importing the media runtime:
Package layout defines media object keys, dependency-free so the access agent can classify paths without importing the media runtime:
Package s3 implements media.Store over aws-sdk-go-v2 for Ceph RGW (production) and MinIO (tests).
Package s3 implements media.Store over aws-sdk-go-v2 for Ceph RGW (production) and MinIO (tests).
Package tiered is an optional visibility policy: it maps an item's level to entitlement keys and asks a Checker which ones the actor holds.
Package tiered is an optional visibility policy: it maps an item's level to entitlement keys and asks a Checker which ones the actor holds.
Package token signs and verifies media access tokens, shared by the host signer and the access agent so the format cannot drift:
Package token signs and verifies media access tokens, shared by the host signer and the access agent so the format cannot drift:
Package video runs an item's video-family presets with ffmpeg: HLS (a byte-range fMP4 ladder with the source's audio and text tracks and a seek sprite), MP4 (a muxed H.264 file at one rung), Audio (an HLS track and an M4A) and Subtitles (clean WebVTT), and grabs the frames of uploads that declare Upload.Frames.
Package video runs an item's video-family presets with ffmpeg: HLS (a byte-range fMP4 ladder with the source's audio and text tracks and a seek sprite), MP4 (a muxed H.264 file at one rung), Audio (an HLS track and an M4A) and Subtitles (clean WebVTT), and grabs the frames of uploads that declare Upload.Frames.
Package worker is the media worker: the one process that places uploads and runs every producer, from the host's worker River schema (Config.Schema) in its database: staged uploads hashed and placed at their blobs (media.Manifests.Place), images, zips, public presets and editor views (media/image, libvips), and HLS, MP4, audio, subtitles and frames (media/video, ffmpeg).
Package worker is the media worker: the one process that places uploads and runs every producer, from the host's worker River schema (Config.Schema) in its database: staged uploads hashed and placed at their blobs (media.Manifests.Place), images, zips, public presets and editor views (media/image, libvips), and HLS, MP4, audio, subtitles and frames (media/video, ffmpeg).
Package workqueue is the host's side of the media worker (media/worker): the River schema it drains in the host database, insert-only enqueueing and cancelling of its jobs, and processing progress for the read API.
Package workqueue is the host's side of the media worker (media/worker): the River schema it drains in the host database, insert-only enqueueing and cancelling of its jobs, and processing progress for the read API.
metrics
Package metrics exports the host's media worker queue health.
Package metrics exports the host's media worker queue health.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL