Documentation
¶
Overview ¶
Package media stores app media as a self-describing file system in one private bucket. The app declares everything in one registry at startup (Config): its kinds, their uploads, the private files derived from them and the public images at fixed names. ContentKit hard-codes no kinds, names or layouts; it owns the mechanics: uploads, the worker's generic producers, conditional manifest writes, the storage areas, the access rule, and purge, sweep and erasure. No database table records what media exists.
An item is a folder, {namespace}/{kind}/{id}/ (see media/layout), with a manifest.json: an ordered virtual file system over its private blobs.
Index ¶
- Constants
- Variables
- func CleanName(s string) string
- func Fingerprint(parts ...string) string
- func InsertOnce(ctx context.Context, insert InsertFunc, args RerunArgs, o river.InsertOpts) error
- func NewName() string
- func NewStaged() string
- func PublicURL(base, namespace, kind, id, name string) string
- func SnoozeUnavailable(ctx context.Context, store Store, job *rivertype.JobRow, err error) error
- func SpecFP(src File, spec any, recipe string) string
- func SrcSet(base, namespace, kind, id, to string, widths []int) string
- func UploadHandler(u *Uploads, o UploadHandlerOptions) http.Handler
- func ValidNamed(name string) bool
- func WaitFor(ctx context.Context, c *river.Client[pgx.Tx], id int64) error
- func ZipFP(inputs []File) string
- type AgentRules
- type Animation
- type Aspect
- type Audio
- type Capabilities
- type Codec
- type CommitBody
- type CommitReply
- type CompleteReply
- type Config
- type CopyFrom
- type CopyOptions
- type Crop
- type Deletion
- type Delivery
- type DeliveryMode
- type Dims
- type Edit
- type EncodeProgress
- type EncodeStatus
- type ErrorDetails
- type ErrorReply
- type Failure
- type File
- type FileInfo
- type FitMode
- type FolderNotEmptyError
- type Frame
- type FrameGrabber
- type GetOptions
- type Grant
- func (g *Grant) Allowed(f File) bool
- func (g *Grant) Cookie() *http.Cookie
- func (g *Grant) Editor() bool
- func (g *Grant) Full() bool
- func (g *Grant) HostURL(path string, dl bool) (string, error)
- func (g *Grant) MasterPlaylist(dir string, o MasterOptions) ([]byte, error)
- func (g *Grant) MediaPlaylist(ctx context.Context, filePath string) ([]byte, error)
- func (g *Grant) SpriteVTT(ctx context.Context, dir string) ([]byte, error)
- func (g *Grant) URL(f File, dl bool) (string, error)
- type HLS
- type HandlerOptions
- type Hooks
- type HostQueue
- func (h *HostQueue) ExposeTx(ctx context.Context, tx pgx.Tx, refs ...contentref.ContentRef) error
- func (h *HostQueue) Purge(ctx context.Context, keys []string) error
- func (h *HostQueue) Ready(ctx context.Context, ref contentref.ContentRef) error
- func (h *HostQueue) ScheduleSweep(ctx context.Context, ref contentref.ContentRef) error
- type Identity
- type Image
- type ImageError
- type IngestRequest
- type IngestResult
- type IngestUpload
- type InsertFunc
- type Issuance
- type Item
- func (i Item) Blob(name string) (string, error)
- func (i Item) Kind() *Kind
- func (i Item) ManifestKey() string
- func (i Item) Prefix() string
- func (i Item) PrivatePrefix() string
- func (i Item) Public(name string) (string, error)
- func (i Item) PublicPrefix() string
- func (i Item) Ref() contentref.ContentRef
- func (i Item) Staged(name string) (string, error)
- func (i Item) TempPrefix() string
- type Jobs
- func (j *Jobs) DeleteItemsTx(ctx context.Context, tx pgx.Tx, items ...Deletion) error
- func (j *Jobs) Expose(ctx context.Context, ref contentref.ContentRef) error
- func (j *Jobs) ExposeTx(ctx context.Context, tx pgx.Tx, refs ...contentref.ContentRef) error
- func (j *Jobs) Insert(ctx context.Context, args river.JobArgs, o *river.InsertOpts) (*rivertype.JobInsertResult, error)
- func (j *Jobs) Manifests() *Manifests
- func (j *Jobs) Purge(ctx context.Context, d Deletion) error
- func (j *Jobs) Regenerate(ctx context.Context, kind, preset string) (int, error)
- func (j *Jobs) RiverJobs() riverhelpers.Contribution
- func (j *Jobs) ScheduleSweep(ctx context.Context, ref contentref.ContentRef) error
- func (j *Jobs) Sweep(ctx context.Context, ref contentref.ContentRef) (SweepResult, error)
- func (j *Jobs) SweepAll(ctx context.Context) error
- func (j *Jobs) SweepOrphans(ctx context.Context, s OrphanSweep) (OrphanReport, error)
- type JobsConfig
- type Kind
- func (k *Kind) DefaultKey(name string) string
- func (k *Kind) EditBounds(path string) Image
- func (k *Kind) FramesVideo(m *Manifest, path string) (File, bool)
- func (k *Kind) NS() string
- func (k *Kind) NameOf(path string) string
- func (k *Kind) Normalize(m *Manifest)
- func (k *Kind) OutputPath(p *Private, path string) string
- func (k *Kind) Presets(path string, hidden bool) []string
- func (k *Kind) PrivateFor(path string) []*Private
- func (k *Kind) PublicFor(path string) []*Public
- func (k *Kind) PublicKept(m *Manifest) []string
- func (k *Kind) PublicNames(m *Manifest, p *Public, path string) []string
- func (k *Kind) Readiness(m *Manifest) Readiness
- func (k *Kind) Unwritten(m *Manifest) int64
- func (k *Kind) UploadOf(path string) (Upload, bool)
- func (k *Kind) ZipInputs(m *Manifest, p *Private) []File
- func (k *Kind) ZipStale(m *Manifest, p *Private) bool
- type LimitError
- type Locker
- type MP4
- type Manifest
- func (m *Manifest) AddPending(path string, presets ...string)
- func (m *Manifest) Blobs() []string
- func (m *Manifest) ClearPending(path, preset string)
- func (m *Manifest) Clone() *Manifest
- func (m *Manifest) Find(path string) int
- func (m *Manifest) Get(path string) (File, bool)
- func (m *Manifest) Outputs(from, preset string) []File
- func (m *Manifest) SetFailed(path string, err error)
- func (m *Manifest) SetOutputs(from, preset string, outs []File) error
- func (m *Manifest) StagedNames() []string
- func (m *Manifest) UploadBytes() int64
- func (m *Manifest) Validate() error
- type ManifestOptions
- type Manifests
- func (m *Manifests) Create(ctx context.Context, ref contentref.ContentRef) (*Manifest, error)
- func (m *Manifests) DeleteUnreferenced(ctx context.Context, item Item, cur *Manifest, blobs []string) error
- func (m *Manifests) DropIfDeleted(ctx context.Context, ref contentref.ContentRef, blobs []string) error
- func (m *Manifests) DropUnreferenced(ctx context.Context, ref contentref.ContentRef, u Unreferenced) error
- func (m *Manifests) Edit(ctx context.Context, ref contentref.ContentRef, fn func(*Manifest) error) (*Manifest, error)
- func (m *Manifests) EditExisting(ctx context.Context, ref contentref.ContentRef, fn func(*Manifest) error) (*Manifest, error)
- func (m *Manifests) Get(ctx context.Context, ref contentref.ContentRef) (*Manifest, string, error)
- func (m *Manifests) Place(ctx context.Context, ref contentref.ContentRef) (int, error)
- func (m *Manifests) Registry() *Registry
- func (m *Manifests) SetFull(ctx context.Context, ref contentref.ContentRef, cause error) error
- func (m *Manifests) Store() Store
- func (m *Manifests) SyncPublic(ctx context.Context, ref contentref.ContentRef) ([]string, error)
- type MasterOptions
- type Multipart
- type MultipartReply
- type Object
- type Op
- type OrphanFolder
- type OrphanReport
- type OrphanSweep
- type PGLimiter
- func (l *PGLimiter) PrepareRelease(ctx context.Context, r QuotaRelease) (bool, error)
- func (l *PGLimiter) Release(ctx context.Context, tenant, operation string) error
- func (l *PGLimiter) Reserve(ctx context.Context, r Reservation) error
- func (l *PGLimiter) Settle(ctx context.Context, s Settlement) error
- func (l *PGLimiter) Usage(ctx context.Context, tenant, owner string) (used, pending int64, err error)
- type PGLimits
- type Part
- type PartBody
- type PartReply
- type PartRequest
- type PartsBody
- type PartsReply
- type PresignBody
- type PresignPut
- type PresignReply
- type PresignRequest
- type Presigned
- type PresignedPart
- type PresignedRequest
- type Private
- type ProcessCanceler
- type ProcessJob
- type ProcessQueue
- type ProgressSource
- type Public
- type PutOptions
- type QuotaRelease
- type QuotaReleaser
- type RateLimit
- type ReadOptions
- type ReadResult
- type Reader
- type ReaderOptions
- type Readiness
- type RefBody
- type Registry
- func (r *Registry) Config() Config
- func (r *Registry) EditorView(f File) string
- func (r *Registry) Item(ref contentref.ContentRef) (Item, error)
- func (r *Registry) Kind(name string) (*Kind, error)
- func (r *Registry) MarshalJSON() ([]byte, error)
- func (r *Registry) Namespace() string
- func (r *Registry) Namespaces() []string
- func (r *Registry) PublicURL(ref contentref.ContentRef, name string) string
- func (r *Registry) Ref(kind, id string) (contentref.ContentRef, error)
- func (r *Registry) SrcSet(ref contentref.ContentRef, preset string) string
- type RequestReply
- type RerunArgs
- type Reservation
- type Segment
- type Settlement
- type Sprite
- type Store
- type Subtitles
- type SweepResult
- type SweepScheduler
- type TicketBody
- type Track
- type TrackIndex
- type Unreferenced
- type Upload
- type UploadAuthorizer
- type UploadError
- type UploadGrant
- type UploadHandlerOptions
- type UploadLimiter
- type UploadOptions
- type UploadTarget
- type UploadedBlob
- type Uploads
- func (u *Uploads) Abort(ctx context.Context, actor access.Actor, sealed string) error
- func (u *Uploads) Commit(ctx context.Context, actor access.Actor, ref contentref.ContentRef, ops []Op) (*Manifest, error)
- func (u *Uploads) Complete(ctx context.Context, actor access.Actor, sealed string) (UploadedBlob, error)
- func (u *Uploads) Frame(ctx context.Context, actor access.Actor, ref contentref.ContentRef, ...) ([]byte, error)
- func (u *Uploads) Ingest(ctx context.Context, actor access.Actor, req IngestRequest) (IngestResult, error)
- func (u *Uploads) ListParts(ctx context.Context, actor access.Actor, sealed string) ([]Part, error)
- func (u *Uploads) Presign(ctx context.Context, actor access.Actor, r PresignRequest) (Presigned, error)
- func (u *Uploads) PresignParts(ctx context.Context, actor access.Actor, sealed string, parts []PartRequest) ([]PresignedPart, error)
- type VideoJobClass
- type VideoLimits
Constants ¶
const ( VideoLive = "" VideoAnimation = "animation" )
Video profiles (HLS.Profile, MP4.Profile).
const ( DefaultMinAspect = 1 / 2.4 DefaultMaxAspect = 2.4 )
Default aspect bounds: 1:2.4 vertical to 2.4:1 wide.
const ( HLSContentType = "application/vnd.apple.mpegurl" VTTContentType = "text/vtt; charset=utf-8" )
Playlist content types.
const ( IngestPartSize = 32 << 20 IngestConcurrency = 3 )
Ingest defaults: parts are buffered in memory, so memory is about (IngestConcurrency+1) × IngestPartSize.
const ( MetaLang = "lang" // subtitles and audio: BCP 47 MetaLabel = "label" // a track's name MetaForced = "forced" // bool: a forced-narrative subtitle track MetaFor = "for" // a subtitle's video upload path; default every video of the item MetaCharset = "charset" // a subtitle's IANA charset, overriding detection )
Upload meta keys ContentKit reads (put's meta).
const ( MaxAudioTracks = 8 MaxSubtitleTracks = 16 )
Tracks per source a video's HLS ladder carries beyond its renditions: the probe keeps at most this many audio and text subtitle streams.
const ( TrackVideo = "video" TrackAudio = "audio" TrackSubs = "subs" TrackSprite = "sprite" )
Track kinds.
const ( MaxManifestBytes = 8 << 20 MaxMetaBytes = 4 << 10 // an upload's meta, as JSON MaxItemMetaBytes = 16 << 10 // the item's meta, as JSON MaxUploads = 10000 )
Manifest bounds. MaxManifestBytes bounds a manifest's JSON on every read and write (8 MiB: a 2,000-page gallery is about 1.5 MB), so every manifest a reader may meet fits the cache. Edits stop editHeadroom below it, so a hide always fits; a commit stops when the item, processed, would pass that (Kind.Unwritten). Meta is bounded where ops set it, and an item holds at most MaxUploads uploads.
const ( OpPut = "put" // add an upload, or replace the one with the same path stem OpEdit = "edit" // crop and rotate an image upload; nil clears OpMove = "move" // reorder an upload among its Upload's; its outputs follow OpRename = "rename" // rename an upload and its outputs OpRemove = "remove" // remove an upload and its outputs OpAttach = "attach" // make an unattached upload part of the item OpCopy = "copy" // copy an upload within the kind, optionally with a new edit OpFrame = "frame" // fill an upload from a frame of its Upload.Frames video OpMeta = "meta" // set the template values (download names) OpRegenerate = "regenerate" // redo stale outputs (of Preset), or every output with Force )
Commit ops. Only upload paths are writable; derived paths belong to the worker.
const ( PhaseQueued = "queued" PhaseDownloading = "downloading" // fetching the original PhaseProbing = "probing" PhaseEncoding = "encoding" // the single ffmpeg pass: ladder, audio, subtitles, sprite PhaseMuxing = "muxing" // per-quality MP4 downloads PhaseUploading = "uploading" PhasePublishing = "publishing" // the manifest edit PhaseImages = "images" // item-wide, after every file: poster frame from the renditions )
Encode phases, in order. Queued covers a job waiting for a worker and a file waiting behind another file of the same job.
const ( StateReady = "ready" StateProcessing = "processing" StateFailed = "failed" StateFull = "full" // the manifest is Full: work is left, stopped until a commit shrinks it )
Processing states of an item.
const ( MaxSinglePut = 64 << 20 MinPartSize = 8 << 20 MaxPartSize = 16 << 20 )
Upload size rules. An upload lands at a staged name in temp/ (u-{uuid}): up to MaxSinglePut as one checksum-bound PUT, larger in parts of MinPartSize up to MaxPartSize (the last may be smaller). Once committed, the media worker hashes it and places it at private/sha256-{hex} (Manifests.Place), so a blob's bytes always hash to its name.
const ( CodeInvalid = "invalid_request" // 400 CodeForbidden = "forbidden" // 403 CodeNotFound = "not_found" // 404: unknown kind, file or upload CodeConflict = "conflict" // 409: a path taken, or a new item's folder not empty CodeIncomplete = "incomplete" // 409: multipart parts missing CodeNotUploaded = "not_uploaded" // 409: commit before the object landed CodeTooManyFiles = "too_many_files" // 409: the commit would exceed an Upload's Max CodeTooLarge = "too_large" // 413: over the Upload's MaxBytes CodeQuota = "quota_exceeded" // 413: owner quota CodeType = "type_not_allowed" // 415 CodeChecksum = "checksum_mismatch" // 422: stored bytes differ from the declared hash CodeRate = "rate_limited" // 429 // Image refusals (ImageError): the rules refuse the image; never a server fault. CodeImageTooSmall = "image_too_small" // 422: edited narrower than the public preset's MinWidth CodeImageTooLarge = "image_too_large" // 413: more pixels than the processor decodes CodeImageUnreadable = "image_unreadable" // 422: not a decodable image of its declared type CodeAnimationNotAllowed = "animation_not_allowed" // 422: an animated image where the policy is AnimationReject CodeAnimationTooLong = "animation_too_long" // 422: more frames or seconds than the processor allows CodeAnimationUnsupported = "animation_unsupported" // 415: an AVIF/HEIF image sequence (decoded as one frame) // Video refusals (ImageError, recorded as the upload's Failure): its // Upload.Video limits refuse what the real stream would cost. CodeVideoTooLong = "video_too_long" // 422: runs longer than MaxSeconds (audio too) CodeVideoTooLarge = "video_too_large" // 422: frames larger than MaxPixels CodeVideoOverBudget = "video_over_budget" // 422: the planned encode is over MaxWork, or the source averages under a frame a second )
Stable upload error codes: clients (the browser SDK) branch on Code.
const ( AccessFull = "full" // every private file AccessNone = "none" // public files only (previews) )
Access levels in ReadResult: an item's private files are all or nothing.
const CookieName = token.CookieName
CookieName is the access agent's cookie.
const DefaultQueue = "contentkit_media"
DefaultQueue is JobsConfig.Queue's default.
const ItemProgressKey = ""
ItemProgressKey carries Item in a job's reported map.
const ManifestVersion = 2
ManifestVersion is the manifest format.
const NamedPrefix = "i-"
NamedPrefix starts the names the server gives Named uploads.
const StartRung = 1080
StartRung is the short side of the variant listed first in each codec: native HLS players (Safari, iOS) start there before measuring.
const ZipRecipe = "zip-store-1"
ZipRecipe versions the Zip producer.
Variables ¶
var ( AspectNative = Aspect{} Square = Aspect{1, 1} )
var ( // ErrManifestTooLarge: an edit refused because the manifest would grow // past its bound. A worker refused it marks the item Full (SetFull). ErrManifestTooLarge = errors.New("media: manifest edit refused: over its size limit") // ErrManifestUnreadable: a stored manifest over MaxManifestBytes, written // before the bound; it does not decode. ErrManifestUnreadable = errors.New("media: stored manifest over its size limit") )
var ( // ErrNotVisible hides an item the viewer may not see, or that does not exist. ErrNotVisible = errors.New("media: not found") // ErrResolve wraps a resolver failure; it always denies. ErrResolve = errors.New("media: resolve failed") // ErrInvalidRequest is a malformed read request. ErrInvalidRequest = errors.New("media: invalid request") // ErrNotAllowed is a file the viewer may not have. ErrNotAllowed = errors.New("media: not allowed") )
var ( ErrNotFound = errors.New("media: object not found") ErrPreconditionFailed = errors.New("media: precondition failed") ErrNotModified = errors.New("media: not modified") // HTTP handlers answer 503. ErrUnavailable = errors.New("media: store unavailable") // ErrChecksumMismatch: the backend rejected a PUT whose body does not // match its x-amz-checksum-sha256. ErrChecksumMismatch = errors.New("media: checksum mismatch") // ErrNotImplemented: the backend does not support a request feature (501). ErrNotImplemented = errors.New("media: not implemented by the store") )
var DefaultEditor = Image{Width: 2048, Height: 2048, Quality: 82}
DefaultEditor renders editor views.
var DefaultLadder = []int{2160, 1080, 480}
DefaultLadder is the HLS ladder by short side.
var DefaultVideoLimits = VideoLimits{MaxSeconds: 4 * 3600, MaxFPS: 60, MaxPixels: 8192 * 4320, MaxWork: 4e13}
DefaultVideoLimits are an upload's VideoLimits where it sets none: 4 h at up to 60 fps, frames up to DCI 8K (8192×4320; the ladder scales larger sources down to 4K), and the work of a 4 h 4K60 default ladder (2160, 1080, 480) in three codecs plus MP4 downloads, about 3e13 pixel-frames.
var EncodePhases = []string{PhaseQueued, PhaseDownloading, PhaseProbing, PhaseEncoding, PhaseMuxing, PhaseUploading, PhasePublishing, PhaseImages}
EncodePhases lists every phase in order.
var ErrFolderNotEmpty = errors.New("media: new item's folder is not empty")
ErrFolderNotEmpty: a new item's folder already holds objects. Item ids are never reused, so leftovers mean a host bug (a reset id, a restored database); they are never adopted. Purge them deliberately with Jobs.Purge.
var ErrJobsNotBound = errors.New("media: River jobs are not composed into a client")
var ErrManifestConflict = errors.New("media: manifest edit kept conflicting")
var ErrNoViewerKey = errors.New("media: an anonymous actor needs Actor.IP for the issuance limit")
ErrNoViewerKey is a grant the issuance limit cannot count: an anonymous actor without Actor.IP. It is the host's bug, not the viewer's limit.
var ErrQuotaReleasePending = errors.New("media: another quota release is pending for the folder")
ErrQuotaReleasePending means a different deletion must finish or be retried first.
var ErrRateLimited = errors.New("media: rate limited")
ErrRateLimited is a refusal that a later attempt will pass.
var ErrUnknownKind = errors.New("media: unknown kind")
var ImageTypes = []string{"image/jpeg", "image/png", "image/webp", "image/gif", "image/avif", "image/heic", "image/heif", "image/tiff"}
ImageTypes are the image types the image presets decode (media/image); an upload feeding one accepts no other. SVG, BMP and JPEG XL are refused: their libvips loaders are blocked as untrusted.
var MaxOutageSnoozes = 2880
MaxOutageSnoozes caps the snoozes of one job (a day at UnavailableSnooze); past it, outage errors spend attempts again.
var PendingOnce = river.UniqueOpts{ByArgs: true, ByState: []rivertype.JobState{rivertype.JobStateAvailable, rivertype.JobStatePending, rivertype.JobStateRunning, rivertype.JobStateRetryable, rivertype.JobStateScheduled}}
PendingOnce dedupes a job per args while one is waiting or running.
var RedisErrors = expvar.NewInt("contentkit_media_ratelimit_redis_errors")
RedisErrors counts shared-limit Redis failures (each served by the per-process limit instead); published as expvar "contentkit_media_ratelimit_redis_errors".
var SubtitleTypes = []string{"text/vtt", "application/x-subrip", "text/x-ssa", "text/x-ass"}
SubtitleTypes are the subtitle types the Subtitles producer converts.
UnavailableSnooze is how long a job waits after the store was unreachable.
Functions ¶
func CleanName ¶ added in v0.62.0
CleanName makes a template value ({name}, {title}) safe to fill a path or a download name: "/", "\", control characters and ".." are removed, the text is NFC-normalized, trimmed and capped at 200 bytes of JSON.
func Fingerprint ¶ added in v0.62.0
Fingerprint hashes a derived file's inputs: for a producer, the source's blob, its edit, the preset's spec and the producer's recipe version. A derived file is stale exactly when its FP no longer matches.
func InsertOnce ¶ added in v0.43.0
func InsertOnce(ctx context.Context, insert InsertFunc, args RerunArgs, o river.InsertOpts) error
InsertOnce enqueues args after the caller's change to their inputs. An equal job still waiting to run absorbs it. River's uniqueness also covers running jobs, which may have read the inputs before the change, so one follow-up is queued behind a running equal job; a burst shares it. The follow-up's worker calls WaitFor first.
func NewName ¶ added in v0.62.0
func NewName() string
NewName is a fresh Named upload name: "i-{uuid}".
func NewStaged ¶ added in v0.62.0
func NewStaged() string
NewStaged is a fresh staged upload name: "u-{uuid}".
func PublicURL ¶ added in v0.62.0
PublicURL is a public file's URL: {base}/v1/{namespace}/{kind}/{id}/public/{name}.
func SnoozeUnavailable ¶ added in v0.58.0
SnoozeUnavailable turns a real store outage into a River snooze, which does not count as an attempt, so an outage longer than the retry budget never discards a job. It snoozes only when the job's own context is still live (a job that outran its timeout spends its attempt) and a fresh, bounded store.Check fails too (one bad object on a healthy bucket spends its attempt); River's snooze count in job metadata is capped by MaxOutageSnoozes. Other errors pass through.
func SpecFP ¶ added in v0.62.0
SpecFP is the fingerprint of a spec (any JSON value) applied to upload src by a producer at recipe.
func UploadHandler ¶
func UploadHandler(u *Uploads, o UploadHandlerOptions) http.Handler
UploadHandler serves the upload API the browser SDK calls. Every route but GET /frame is POST with a JSON body. Errors are ErrorReply with the status of its code (Retry-After on 429).
POST /presign PresignBody -> PresignReply POST /parts PartsBody -> PartsReply presign multipart parts POST /parts/list TicketBody -> PartsReply parts that landed (resume) POST /complete TicketBody -> CompleteReply POST /abort TicketBody -> 204 POST /commit CommitBody -> CommitReply GET /frame?kind=&id=&path=&t=&w= -> image/jpeg a still of a video upload (UploadOptions.Frames)
func ValidNamed ¶ added in v0.62.0
ValidNamed reports a server-given name ("i-{uuid}").
Types ¶
type AgentRules ¶ added in v0.62.0
AgentRules are the access agent's rules for an app's media host: the namespaces it serves there (MEDIA_ACCESS_HOSTS, with layout.FormatHosts) and the default public names of its kinds (MEDIA_ACCESS_DEFAULTS, with layout.FormatDefaults).
func AgentConfig ¶ added in v0.62.0
func AgentConfig(r *Registry) AgentRules
AgentConfig derives the access agent's rules from the registry.
type Animation ¶ added in v0.39.0
type Animation string
Animation is a policy for animated images (GIF, WebP; AVIF/HEIF sequences are refused as animation_unsupported, never flattened).
type Aspect ¶ added in v0.40.0
type Aspect struct{ W, H int }
Aspect is a width:height ratio in lowest terms, written "W:H" ("3:1", "9:16"). The zero value is AspectNative: the source's own shape.
func AspectOf ¶ added in v0.40.0
AspectOf is the ratio of a w×h size, reduced (native for an empty size).
func ParseAspect ¶ added in v0.40.0
ParseAspect parses "W:H" with positive integer terms up to 10000 and reduces it ("6:2" is "3:1"); "" and "native" are AspectNative.
func Ratio ¶ added in v0.40.0
Ratio parses "W:H" and panics on an invalid one (for constants in code).
func (Aspect) Height ¶ added in v0.40.0
Height is width's height at a, rounded half up, at least 1 (0 when native).
func (Aspect) MarshalText ¶ added in v0.40.0
MarshalText writes "W:H" (native: empty), so JSON and config carry the string.
func (*Aspect) UnmarshalText ¶ added in v0.40.0
type Audio ¶ added in v0.56.0
type Audio struct {
// Loudness normalizes to this integrated loudness in LUFS (EBU R128),
// e.g. -16; 0 keeps the source's level.
Loudness float64 `json:"loudness,omitempty"`
}
Audio is AAC-LC: an HLS audio track ({To}audio.mp4) and a faststart M4A ({To}audio.m4a).
type Capabilities ¶
type Capabilities struct {
ConditionalPut bool // If-Match / If-None-Match on PUT
ChecksumSHA256 bool // x-amz-checksum-sha256 enforced on PUT
}
Capabilities are backend features the library depends on, established by Probe.
func Probe ¶
Probe measures the backend's capabilities with scratch objects under prefix, which it removes. Results depend on the backend release (RGW vs MinIO). Every step must succeed or be refused cleanly (412, a checksum mismatch, 501); anything else (throttling, timeouts, a proxy's 403, cancellation) is an error and nothing is recorded, so a transient failure never reads as a missing capability.
type Codec ¶ added in v0.53.0
type Codec string
Codec is a video codec of an HLS ladder: each rung is encoded in each of the worker's codecs (Track.Codec).
type CommitBody ¶
CommitBody applies ops to an item in one conditional write.
type CommitReply ¶
type CommitReply struct {
Files []FileInfo `json:"files"`
}
CommitReply is the item's uploads as an editor reads them.
type CompleteReply ¶
type CompleteReply struct {
Blob string `json:"blob"`
Type string `json:"type"`
Size int64 `json:"size"`
}
CompleteReply is a completed multipart upload; Blob is its staged name.
type Config ¶ added in v0.62.0
type Config struct {
// Namespace is the app's own namespace, e.g. "doujins": its kinds' items
// live under {Namespace}/{kind}/{id}/.
Namespace string `json:"namespace"`
// BaseURL is the site's media origin, e.g. "https://media.doujins.ai":
// every URL is {BaseURL}/v1/{namespace}/{kind}/{id}/{public|private}/{name}.
BaseURL string `json:"base_url,omitempty"`
// Kinds are the app's kinds plus the shared kinds it imports.
Kinds []Kind `json:"kinds"`
// Editor is how editor views render (the whole oriented source the
// cropper draws on); zero is DefaultEditor.
Editor Image `json:"editor,omitzero"`
Hooks Hooks `json:"-"`
}
Config is the app's media registry (NewRegistry).
func ParseConfig ¶ added in v0.62.0
ParseConfig reads a registry written by Registry.MarshalJSON; unknown keys are ignored.
type CopyFrom ¶ added in v0.62.0
CopyFrom names an upload of an item of the same kind, including this item.
type CopyOptions ¶ added in v0.40.0
type CopyOptions struct {
IfMatch string
}
CopyOptions: IfMatch copies only while the source's ETag is this one (ErrPreconditionFailed otherwise).
type Deletion ¶
type Deletion struct {
Ref contentref.ContentRef
Owner string
OperationID string // required by Purge with a quota owner; stable across retries
}
Deletion is one item to delete. Owner is its quota owner (UploadGrant.Owner), "" for none; with a Limiter configured the owner's usage is released. OperationID is only for Purge; queued deletion uses its River job id.
type Delivery ¶
type Delivery struct {
Mode DeliveryMode
// CookieDomain is the site's registrable domain, e.g. "doujins.ai";
// required in cookie mode, because a host-only cookie never reaches media.
CookieDomain string
// SigningKey is the current key of the ring the access agent verifies.
SigningKey token.Key
// TTL is the minimum token lifetime (default 1h); expiries round up to
// Window (default token.DefaultWindow, whole seconds).
TTL, Window time.Duration
}
Delivery is the host's signing configuration; URLs are at the registry's BaseURL.
type DeliveryMode ¶
type DeliveryMode string
DeliveryMode is how viewers with access present their item token.
const ( // DeliverCookie (default) sets the item cookie; URLs are plain. DeliverCookie DeliveryMode = "cookie" // DeliverURL appends ?t= to every URL: apps and clients without cookies. DeliverURL DeliveryMode = "url" )
type Edit ¶ added in v0.17.0
type Edit struct {
Crop *Crop `json:"crop,omitempty"`
Rotate int `json:"rotate,omitempty"` // clockwise degrees: 0, 90, 180 or 270
}
Edit is a non-destructive image edit: Crop, in the source's pixels (EXIF orientation applied), then a clockwise Rotate. Variants derive from the source through it; the source is never modified.
func (*Edit) Check ¶ added in v0.17.0
Check validates the edit's shape and, with a known size (w, h > 0), that the crop lies inside it.
type EncodeProgress ¶ added in v0.29.0
type EncodeProgress struct {
Phase string `json:"phase"`
QueuePosition int `json:"queue_position,omitempty"` // 1 = next; 0 = unknown or behind a file of the same job
SegmentsDone int `json:"segments_done,omitempty"`
SegmentsTotal int `json:"segments_total,omitempty"`
Percent float64 `json:"percent"`
Speed float64 `json:"speed,omitempty"`
ETA float64 `json:"eta,omitempty"`
At int64 `json:"at"` // unix milliseconds of the measurement
Stalled bool `json:"stalled,omitempty"`
// Stage of Stages: each rung is published before the next one starts.
Stage int `json:"stage,omitempty"`
Stages int `json:"stages,omitempty"`
}
EncodeProgress is a pending video file's live encode state. Segments count HLS segments of the source's duration; Speed is the encode's smoothed ×realtime; ETA is seconds from At to publish, including uploads. Percent never decreases within a run. Stalled marks a report older than a minute from a running job (a dead worker until River rescues the job).
type EncodeStatus ¶ added in v0.29.0
type EncodeStatus struct {
Files map[string]EncodeProgress
Queued *EncodeProgress
Item *EncodeProgress
}
EncodeStatus is an item's encode progress: Files from the running job, Queued for pending files it does not cover (a waiting job), Item for the job's item-wide step (PhaseImages).
func (EncodeStatus) Current ¶ added in v0.29.0
func (s EncodeStatus) Current() *EncodeProgress
Current is the item's most advanced step: the item-wide one, a running file, else the wait.
type ErrorDetails ¶ added in v0.38.0
type ErrorDetails struct {
Width int `json:"width,omitempty"` // image_too_small: the edited width; image_too_large, video_too_large: the source's
Height int `json:"height,omitempty"` // image_too_large, video_too_large: the source's
MinWidth int `json:"min_width,omitempty"` // image_too_small
MaxPixels int `json:"max_pixels,omitempty"` // image_too_large, video_too_large
Type string `json:"type,omitempty"` // the declared type (image_unreadable, type_not_allowed, too_large)
Allowed []string `json:"allowed,omitempty"` // type_not_allowed: the kind's types
Size int64 `json:"size,omitempty"` // too_large: bytes
MaxBytes int64 `json:"max_bytes,omitempty"` // too_large
Frames int `json:"frames,omitempty"` // animation_too_long, image_too_large: the animation's
MaxFrames int `json:"max_frames,omitempty"` // animation_too_long
Seconds float64 `json:"seconds,omitempty"` // animation_too_long, video_too_long: running time
MaxSeconds float64 `json:"max_seconds,omitempty"` // animation_too_long, video_too_long
}
ErrorDetails qualifies an image refusal so clients can state the rule.
type ErrorReply ¶
type ErrorReply struct {
Error string `json:"error"`
Code string `json:"code"`
RetryAfter int `json:"retry_after,omitempty"` // seconds, with 429
Blobs []string `json:"blobs,omitempty"` // not_uploaded at commit: the blobs to upload again
Details *ErrorDetails `json:"details,omitempty"` // image refusals
}
ErrorReply is the error body; Code is one of the Code* constants, "unauthorized" or "internal_error".
type Failure ¶ added in v0.62.0
type Failure struct {
Of string `json:"of"` // File.Key it was recorded for
Message string `json:"message"`
Code string `json:"code,omitempty"` // an ImageError's code
Details *ErrorDetails `json:"details,omitempty"`
}
Failure is why an upload's blob through its edit cannot be processed.
func NewFailure ¶ added in v0.62.0
NewFailure records err for upload f: an ImageError keeps its code and details; the message is capped at maxFailureBytes.
type File ¶
type File struct {
Path string `json:"path"` // app path: "originals/001.png", "low-res/001.webp"
Blob string `json:"blob,omitempty"` // "sha256-{hex}" in private/; "" for a staged upload or a frame not grabbed yet
Type string `json:"type"`
Size int64 `json:"size,omitempty"`
W int `json:"w,omitempty"` // an upload's oriented size once measured; an output's size
H int `json:"h,omitempty"`
Dur float64 `json:"dur,omitempty"`
// Uploads:
Staged string `json:"staged,omitempty"` // "u-{uuid}" in temp/ until the worker hashes and places it at Blob
CreateID string `json:"create_id,omitempty"` // create-only put receipt; retained through placement and processing
Edit *Edit `json:"edit,omitempty"` // crop and rotate in source pixels
Frame *Frame `json:"frame,omitempty"` // grabbed from the Upload.Frames video
Meta map[string]any `json:"meta,omitempty"` // lang, label, …
Unattached bool `json:"unattached,omitempty"` // processed on upload, not yet part of the item
Gone bool `json:"gone,omitempty"` // blob dropped (KeepOriginals false); the hash stays for provenance
Pending []string `json:"pending,omitempty"` // presets still producing from this upload
Failed *Failure `json:"failed,omitempty"` // this blob and edit cannot be processed
// Derived files:
From string `json:"from,omitempty"` // the upload's path, or a zip's prefix
Preset string `json:"preset,omitempty"` // the Private preset
FP string `json:"fp,omitempty"` // Fingerprint of its inputs
Download string `json:"download,omitempty"` // the human download name
Track *Track `json:"track,omitempty"` // HLS
}
File is one file: an upload (no Preset) or a derived file.
func (File) Fail ¶ added in v0.62.0
Fail is the upload's failure for its current blob and edit, or nil.
type FileInfo ¶
type FileInfo struct {
Path string `json:"path"`
Type string `json:"type"`
Size int64 `json:"size,omitempty"`
W int `json:"w,omitempty"`
H int `json:"h,omitempty"`
Dur float64 `json:"dur,omitempty"`
Download string `json:"download,omitempty"` // the name a download read serves it under
URL string `json:"url,omitempty"`
Locked bool `json:"locked,omitempty"`
// Editors (editor reads and commit replies):
Upload bool `json:"upload,omitempty"`
Staged bool `json:"staged,omitempty"` // uploaded, not yet placed by the worker: no blob, views or frames yet
From string `json:"from,omitempty"`
Edit *Edit `json:"edit,omitempty"`
Frame *Frame `json:"frame,omitempty"`
Meta map[string]any `json:"meta,omitempty"`
Unattached bool `json:"unattached,omitempty"`
Pending []string `json:"pending,omitempty"`
Failed *Failure `json:"failed,omitempty"`
EditorURL string `json:"editor_url,omitempty"`
Progress *EncodeProgress `json:"progress,omitempty"`
}
FileInfo is one file of a read.
type FitMode ¶ added in v0.62.0
type FitMode string
FitMode is how an image fits its Width×Height box.
type FolderNotEmptyError ¶ added in v0.45.0
FolderNotEmptyError names the folder and a few of its keys.
func (*FolderNotEmptyError) Error ¶ added in v0.45.0
func (e *FolderNotEmptyError) Error() string
func (*FolderNotEmptyError) Unwrap ¶ added in v0.45.0
func (e *FolderNotEmptyError) Unwrap() error
type Frame ¶ added in v0.62.0
type Frame struct {
T float64 `json:"t,omitempty"`
Auto bool `json:"auto,omitempty"`
Of string `json:"of,omitempty"`
}
Frame is an upload grabbed from a frame of its Upload.Frames video at T seconds; Auto lets the worker choose T. Of is the video blob it was grabbed from: a new video grabs again.
type FrameGrabber ¶ added in v0.26.0
type FrameGrabber interface {
Frame(ctx context.Context, item Item, video File, t float64, width int) ([]byte, error)
}
FrameGrabber grabs a JPEG still at t seconds (width 0 keeps the frame's) from a video upload, for the frame picker (Uploads.Frame); media/video's Frames implements it.
type GetOptions ¶
GetOptions: IfNoneMatch returns ErrNotModified on a match; Range is an HTTP Range value.
type Grant ¶
type Grant struct {
Item Item
Resolution access.Resolution
Manifest *Manifest
Expires time.Time
// contains filtered or unexported fields
}
Grant is one viewer's resolved access to one item: reads and playlists sign every URL through it.
func (*Grant) Allowed ¶
Allowed reports whether a read gives this viewer f's URL: with access, every listed file with a blob. Uploads are listed only with ServeOriginals, HostOnly presets never (Grant.HostURL), unattached files and frames not grabbed yet never.
func (*Grant) Full ¶
Full reports access to the item's private files: the resolver's verdict, or an editor's.
func (*Grant) HostURL ¶ added in v0.62.0
HostURL is the URL of a HostOnly preset's file for a viewer with access, always carrying the item token (the host route may not have set the cookie). HostOnly only keeps a file out of generic reads: call this from a host route after its own policy, which decides who is handed the URL.
func (*Grant) MasterPlaylist ¶
func (g *Grant) MasterPlaylist(dir string, o MasterOptions) ([]byte, error)
MasterPlaylist is the multivariant playlist of the ladder under dir: one variant per video track (rung and codec), with the audio and subtitle groups; an audio-only ladder is one audio variant. Codecs are listed in the ladder's order, so a player that decodes the first starts on it.
func (*Grant) MediaPlaylist ¶ added in v0.62.0
MediaPlaylist is the playlist of the track file at filePath: a byte-range playlist over its blob for video and audio (from its index), or one segment over a whole WebVTT file.
type HLS ¶
type HLS struct {
// Ladder is the rendition short sides, largest first; empty is DefaultLadder.
Ladder []int `json:"ladder,omitempty"`
// MinAspect and MaxAspect bound the source's display width/height; zero
// is DefaultMinAspect and DefaultMaxAspect. A source outside fails.
MinAspect float64 `json:"min_aspect,omitempty"`
MaxAspect float64 `json:"max_aspect,omitempty"`
Profile string `json:"profile,omitempty"` // VideoLive or VideoAnimation
}
HLS is a byte-range fMP4 ladder, one blob per rendition, with the source's audio and subtitle tracks and a seek sprite. A rung above the source is not produced.
type HandlerOptions ¶
type HandlerOptions struct {
Identity Identity
Logger *slog.Logger
// Limit is the per-viewer rate limit (default 2/s, burst 120, per
// process; set Limit.Redis to share it across replicas): every read,
// playlist and download counts.
// Viewers are keyed by Actor.ID, anonymous ones by Actor.IP, else by the
// connection's address.
Limit RateLimit
}
HandlerOptions configure the read API.
type Hooks ¶
type Hooks struct {
// Resolver says who may see an item: the read API, Expose and a new
// item's first commit (Uploads) use it, and both require it.
Resolver access.ContentResolver
// CanUpload says who may write which upload path.
CanUpload UploadAuthorizer
// PurgePublic hears of public URLs overwritten, deleted or first
// written (the CDN may hold the default), for a CDN purge.
PurgePublic func(ctx context.Context, urls []string)
// ItemReady reports an item whose processing settled (ready, or failed
// with nothing processing) in a host transaction, after the worker's
// job, through the host's media queue. It must be idempotent; an error
// retries.
ItemReady func(ctx context.Context, tx pgx.Tx, ref contentref.ContentRef, r Readiness) error
// Failed reports an upload a producer cannot process, where it runs.
Failed func(ctx context.Context, ref contentref.ContentRef, path string, err error)
}
Hooks are the app's callbacks.
type HostQueue ¶ added in v0.43.0
type HostQueue struct {
// contains filtered or unexported fields
}
HostQueue is the media worker's handle on the host's River schema: it inserts the jobs the host runs on the worker's behalf (a folder's sweep after an edit, ItemReady and PurgePublic relays). It inserts only.
func NewHostQueue ¶ added in v0.43.0
func NewHostQueue(pool *pgxpool.Pool, reg *Registry, schema, queue string, grace time.Duration) (*HostQueue, error)
NewHostQueue targets the host's River schema ("" is the connection's search path) and media queue ("" is DefaultQueue); grace is the host's JobsConfig.Grace (default 24 h).
func (*HostQueue) ExposeTx ¶ added in v0.50.0
func (h *HostQueue) ExposeTx(ctx context.Context, tx pgx.Tx, refs ...contentref.ContentRef) error
ExposeTx enqueues the host's Expose of refs in tx, a transaction on the host database, like Jobs.ExposeTx.
func (*HostQueue) Purge ¶ added in v0.62.0
Purge asks the host to purge public keys from its CDN (Hooks.PurgePublic).
func (*HostQueue) Ready ¶ added in v0.62.0
func (h *HostQueue) Ready(ctx context.Context, ref contentref.ContentRef) error
Ready asks the host to report ref's readiness (Hooks.ItemReady) once it has settled.
func (*HostQueue) ScheduleSweep ¶ added in v0.43.0
func (h *HostQueue) ScheduleSweep(ctx context.Context, ref contentref.ContentRef) error
ScheduleSweep implements SweepScheduler like Jobs.ScheduleSweep.
type Identity ¶
Identity reads the authenticated actor the host's middleware put in the context; the same shape as content.Identity.
type Image ¶ added in v0.62.0
type Image struct {
Width int `json:"width,omitempty"`
Height int `json:"height,omitempty"`
Fit FitMode `json:"fit,omitempty"`
Quality int `json:"quality,omitempty"` // default 80
Blur float64 `json:"blur,omitempty"`
// Aspect and MinWidth bound an upload's edit when this is its public
// preset's image: the crop is fitted to Aspect, and an edit narrower
// than MinWidth fails.
Aspect Aspect `json:"aspect,omitzero"`
MinWidth int `json:"min_width,omitempty"`
Animation Animation `json:"animation,omitempty"`
}
Image is a WebP rendering. Zero Width and Height keep the full size; nothing is upscaled.
func (Image) Resolve ¶ added in v0.62.0
Resolve is the edit im applies to a w×h source (EXIF-oriented): e with its crop height fitted to Aspect, or without a crop the largest centred one at Aspect (a native aspect: the whole source). It must lie inside the source and be at least MinWidth wide once edited.
type ImageError ¶ added in v0.38.0
type ImageError struct {
Code string
Message string
Details ErrorDetails
}
ImageError is an image the rules refuse, synchronously (an edit checked against known dims) or in the image job (recorded as the upload's Failure).
func AsImageError ¶ added in v0.38.0
func AsImageError(err error) *ImageError
AsImageError returns the refusal in err's chain, or nil.
func (*ImageError) Error ¶ added in v0.38.0
func (e *ImageError) Error() string
type IngestRequest ¶ added in v0.18.0
type IngestRequest struct {
Ref contentref.ContentRef
Path string
Type string
Body io.Reader // read once, sequentially
Size int64 // expected size, checked when > 0; required when the actor is rate- or quota-limited
Meta map[string]any
// Resume continues a multipart write a previous Ingest left behind:
// parts already stored with the same bytes are not uploaded again (the
// body is still read and hashed). An unknown upload starts afresh.
Resume *IngestUpload
// OnUpload receives the multipart write once created, for the host to
// persist for Resume. With it set a failed Ingest keeps the upload (the
// bucket's abort-incomplete rule removes it after a day); without it the
// upload is aborted.
OnUpload func(IngestUpload) error
PartSize int64 // default IngestPartSize; at least 5 MiB
Concurrency int // parallel part uploads; default IngestConcurrency
}
IngestRequest streams one file of unknown or huge size from the host (a server-side import, not a browser upload) into Ref and puts it at Path.
type IngestResult ¶ added in v0.18.0
IngestResult is the committed upload; Staged is its name in temp/ until the worker places it.
type IngestUpload ¶ added in v0.18.0
type IngestUpload struct {
Temp string `json:"temp"` // the staged name (u-{uuid})
UploadID string `json:"upload_id"`
}
IngestUpload identifies an in-progress multipart ingest in temp/.
type InsertFunc ¶ added in v0.43.0
type InsertFunc func(ctx context.Context, args river.JobArgs, o *river.InsertOpts) (*rivertype.JobInsertResult, error)
InsertFunc inserts one job (a River client's Insert, or InsertTx bound to a transaction).
type Issuance ¶ added in v0.62.0
type Issuance struct {
// PerHour is the items an account may open per hour (default 120);
// AnonymousPerHour is per Actor.IP (an IPv6 address counts as its /64),
// where one address may be many people (default 600). The read API
// fills an anonymous actor's IP from the connection; a host calling
// Reader.Grant itself must set it (ErrNoViewerKey otherwise: without it
// every anonymous viewer would share one count).
PerHour, AnonymousPerHour int
Disabled bool
// Exempt actors are not limited (staff); an item's editors never are.
Exempt func(access.Actor) bool
// Redis is a Redis or Microsoft Garnet client shared by the replicas.
// Only SADD, SCARD, SREM, PEXPIRE and MULTI/EXEC are used (no Lua).
Redis redis.UniversalClient
// KeyPrefix namespaces the Redis keys (default "contentkit:media:issue:").
KeyPrefix string
}
Issuance limits how many items a viewer is given the token of per hour: each grant with access opens one item, and a scraper needs one per item. Opening the same item again within the hour is free. With the ingress limiting each IP's downloads from the media host, this bounds what an account can pull: items per hour times what the ingress lets through.
With Redis set every replica shares the count; otherwise each process keeps its own (logged at start). Redis errors fail open to the per-process count (RedisErrors).
type Item ¶
type Item struct {
// contains filtered or unexported fields
}
Item is a validated item: a ref of a registered kind in its namespace, and its folder's keys.
func (Item) ManifestKey ¶
func (Item) PrivatePrefix ¶ added in v0.48.0
func (Item) PublicPrefix ¶
func (Item) Ref ¶
func (i Item) Ref() contentref.ContentRef
func (Item) Staged ¶ added in v0.62.0
Staged is the key of a staged upload ("u-{uuid}"), in temp/ until placed.
func (Item) TempPrefix ¶ added in v0.52.0
type Jobs ¶
type Jobs struct {
// contains filtered or unexported fields
}
Jobs is media's River contribution to the host: sweep, folder deletion, Expose, and the worker's relays (ItemReady, PurgePublic). Processing runs in the media worker (media/worker).
func NewJobs ¶
func NewJobs(cfg JobsConfig) (*Jobs, error)
func (*Jobs) DeleteItemsTx ¶
DeleteItemsTx deletes each item's folder through a job enqueued in the host's delete transaction: a host deletes every version item of a work, and erasing an account deletes its accounts/user/{id}/ item. A second pass after LateUploadWindow removes uploads that land after the first. The worker records the quota refund from the manifest before deleting.
func (*Jobs) Expose ¶ added in v0.48.0
func (j *Jobs) Expose(ctx context.Context, ref contentref.ContentRef) error
Expose brings an item's public files to its visibility, resolved for an anonymous viewer (Hooks.Resolver). Hiding deletes and purges public/, records Hidden, and deletes again what a pass wrote meanwhile; it needs no readable manifest, so a manifest over its bound still hides. Unhiding marks the public presets pending on their kept sources and asks the worker to render them. It re-resolves after writing and repeats until the state holds, so an Expose racing a visibility change ends at the newer one. An item without a manifest has no public files (its first commit resolves it).
func (*Jobs) ExposeTx ¶ added in v0.48.0
func (j *Jobs) ExposeTx(ctx context.Context, tx pgx.Tx, refs ...contentref.ContentRef) error
ExposeTx enqueues Expose for each item in the host's transaction. Call it whenever whether anonymous viewers may see an item changes: publish, unpublish, hide, hold, soft delete, restore.
func (*Jobs) Insert ¶
func (j *Jobs) Insert(ctx context.Context, args river.JobArgs, o *river.InsertOpts) (*rivertype.JobInsertResult, error)
Insert enqueues a job on the bound client; an empty queue means the media queue.
func (*Jobs) Manifests ¶ added in v0.62.0
Manifests is the jobs' manifest store; hosts may share it.
func (*Jobs) Purge ¶ added in v0.45.0
Purge deletes an item's folder now: the explicit reset before deliberately recreating an item, or an operator cleanup. A quota-owned purge needs a caller-stable OperationID; retry a failed purge with it. Hosts deleting content use DeleteItemsTx.
func (*Jobs) Regenerate ¶ added in v0.62.0
Regenerate visits every item of kind (in its namespace) and asks the worker to redo its stale outputs of preset ("" for all): after a deploy changed a preset's spec or a producer's recipe. It returns the items visited.
func (*Jobs) RiverJobs ¶
func (j *Jobs) RiverJobs() riverhelpers.Contribution
RiverJobs contributes media's workers, queue and periodic sweep pass to the host's helpers/river composition. It composes once.
func (*Jobs) ScheduleSweep ¶
func (j *Jobs) ScheduleSweep(ctx context.Context, ref contentref.ContentRef) error
ScheduleSweep sweeps the item's folder after the grace period; a sweep already waiting for the folder absorbs it. Manifests calls it on every edit.
func (*Jobs) Sweep ¶
func (j *Jobs) Sweep(ctx context.Context, ref contentref.ContentRef) (SweepResult, error)
Sweep collects garbage by manifest reference, with no other state:
- private/ blobs the manifest does not reference, once the blob is older than the grace period (a job's outputs not recorded yet, editor views). Later edits never hold it back. A blob written long ago and dropped just now is old already, so a viewer mid-stream on a replaced file may lose it within the grace period;
- public/ names no preset expects (a removed upload, a hidden item), at once, purged;
- temp/ by age (JobsConfig.TempTTL), but for staged uploads the manifest references.
It holds the manifest lock through selection and deletion, and decides on a second listing, so a manifest written meanwhile keeps what it references.
func (*Jobs) SweepAll ¶
SweepAll sweeps every item folder of the registry's kinds: the periodic backstop for missed schedules and abandoned uploads.
func (*Jobs) SweepOrphans ¶ added in v0.45.0
func (j *Jobs) SweepOrphans(ctx context.Context, s OrphanSweep) (OrphanReport, error)
SweepOrphans lists the kind's folders and reports, or with Delete removes, those the host says do not exist, once past the grace period. The host runs it: only it knows which items exist.
type JobsConfig ¶
type JobsConfig struct {
Store Store
Registry *Registry
Locker Locker // serializes manifest edits and sweep deletion; required
Processes ProcessQueue // the worker's queue (workqueue.Queue): Expose and Regenerate render through it
// Pool is the host database Hooks.ItemReady's transaction runs on;
// required with ItemReady.
Pool *pgxpool.Pool
// Grace protects job outputs not recorded yet: the sweep deletes an
// unreferenced blob once it is this old, whatever edits follow. Default
// 24 h.
Grace time.Duration
// TempTTL is how long temp/ objects (in-flight server-side writes) are
// kept; above the bucket's 1-day abort-incomplete rule. Default 48 h.
TempTTL time.Duration
// SweepInterval is the periodic pass over every folder. Default 24 h.
SweepInterval time.Duration
// LateUploadWindow delays a folder deletion's second pass, which removes
// uploads that land after the first; above the presign TTL and the
// 1-day multipart rule. Default 25 h.
LateUploadWindow time.Duration
Limiter QuotaReleaser // releases a deleted item's quota; optional
Queue string // default DefaultQueue
MaxWorkers int // default 2
Logger *slog.Logger
Now func() time.Time
}
JobsConfig configures media's River jobs in the host.
type Kind ¶
type Kind struct {
Name string `json:"name"`
// Namespace is "" for Config.Namespace, or a shared one such as
// "accounts" whose items every app importing the kind serves.
Namespace string `json:"namespace,omitempty"`
Uploads []Upload `json:"uploads"`
// KeepOriginals keeps an upload's blob once its private outputs exist;
// otherwise it is dropped (File.Gone). Public presets' sources are
// always kept, so unhiding can render them again.
KeepOriginals bool `json:"keep_originals,omitempty"`
// ServeOriginals lists and serves uploads to viewers with access, like
// any private file. Otherwise viewer URLs are scoped to served files,
// not the whole private folder, and reads omit upload paths and blobs.
ServeOriginals bool `json:"serve_originals,omitempty"`
Private []Private `json:"private,omitempty"`
Public []Public `json:"public,omitempty"`
// Defaults holds the images the kind's public presets name in
// Public.Default, so a shared kind ships its own (go:embed). A registry
// read from JSON (the stock worker) has none; PublishDefaults needs them.
Defaults fs.FS `json:"-"`
// contains filtered or unexported fields
}
Kind is one kind of item: what may be uploaded, what is derived from the uploads in private/, and which public images it has.
func (*Kind) DefaultKey ¶ added in v0.62.0
DefaultKey is the key of the kind's default public image name.
func (*Kind) EditBounds ¶ added in v0.62.0
EditBounds is the image that bounds an edit of the upload at path: its first public preset's, else a native one.
func (*Kind) FramesVideo ¶ added in v0.62.0
FramesVideo is the video upload the upload at path is grabbed from (its Upload.Frames), if the manifest has one.
func (*Kind) NameOf ¶ added in v0.62.0
NameOf is the {name} of an upload path ("originals/001.png" is "001", "cover.png" is "cover").
func (*Kind) Normalize ¶ added in v0.62.0
Normalize brings m to its canonical form under k after an edit: the file order, the download names, and dropped originals (Gone).
func (*Kind) OutputPath ¶ added in v0.62.0
OutputPath is p's output path (a file, or a directory ending in "/") for the upload at path.
func (*Kind) Presets ¶ added in v0.62.0
Presets are the names of the presets an upload at path feeds: the private ones, then (unless hidden) the public ones. Zips are not listed; they follow their inputs.
func (*Kind) PrivateFor ¶ added in v0.62.0
PrivateFor lists the private presets fed by the upload at path.
func (*Kind) PublicFor ¶ added in v0.62.0
PublicFor lists the public presets fed by the upload at path.
func (*Kind) PublicKept ¶ added in v0.62.0
PublicKept are the public names m vouches for; every other name in public/ is deleted. They are its attached uploads' names, none when hidden, and a preview position's only once rendered for the upload now there: a removed page's image does not stay under the next page's name.
func (*Kind) PublicNames ¶ added in v0.62.0
PublicNames are p's public names in m for the upload at path, one per width: none for an upload outside a preview's First. m may be nil for a preset without First.
func (*Kind) Unwritten ¶ added in v0.62.0
Unwritten estimates the JSON m gains as the worker processes it. Per upload it is the larger of a failure record and what success writes: an entry for every output its presets have yet to write (an HLS ladder's tracks one by one: a rendition per rung and codec, the sprite, and as many audio and subtitle tracks as a source may carry; an Audio preset's track and M4A), its measured fields, and a staged upload's or frame's blob. Then the zips, and the public presets' pending names an unhide adds. Commits bound the manifest with it; an item whose outputs still overrun it is marked Full by the worker.
type LimitError ¶ added in v0.62.0
LimitError is ErrRateLimited with how long to wait.
func (*LimitError) Error ¶ added in v0.62.0
func (e *LimitError) Error() string
func (*LimitError) Is ¶ added in v0.62.0
func (e *LimitError) Is(target error) bool
type Locker ¶
Locker serializes manifest edits across every process sharing the bucket.
type MP4 ¶ added in v0.62.0
MP4 is a muxed H.264 MP4 at one rung (short side), with the default audio.
type Manifest ¶
type Manifest struct {
V int `json:"v"`
Hidden bool `json:"hidden,omitempty"` // set by Expose; public files are then absent
// Full: a producer could not record its outputs within the bound, so
// private outputs stop until a commit frees Deficit bytes (in the
// manifest and in what its uploads would still add): how far past the
// bound the refused record went.
Full bool `json:"full,omitempty"`
Deficit int64 `json:"deficit,omitempty"`
Meta map[string]any `json:"meta,omitempty"` // the app's template values, e.g. title
Files []File `json:"files"`
// contains filtered or unexported fields
}
Manifest is an item's manifest.json: an ordered, app-defined virtual file system over the item's private blobs. Files is in an explicit, deterministic order: each Upload's files by name by default (reordered by commit ops), then each private preset's outputs in their uploads' order. Readers never re-sort it. Public files are never listed.
func DecodeManifest ¶ added in v0.62.0
DecodeManifest reads a stored manifest (gzip JSON, or plain JSON).
func (*Manifest) AddPending ¶ added in v0.62.0
AddPending adds presets to the Pending of the upload at path.
func (*Manifest) Blobs ¶
Blobs lists every blob the manifest references: kept uploads, derived files and track indexes.
func (*Manifest) ClearPending ¶ added in v0.62.0
ClearPending removes preset from the Pending of the upload at path.
func (*Manifest) Clone ¶ added in v0.62.0
Clone deep-copies the manifest for an edit: a cached manifest is shared, so an edit never mutates it.
func (*Manifest) Outputs ¶ added in v0.62.0
Outputs lists the derived files of preset from the upload (or zip prefix) from, in manifest order.
func (*Manifest) SetFailed ¶ added in v0.62.0
SetFailed records err on the upload at path for its current blob and edit, and clears its Pending.
func (*Manifest) SetOutputs ¶ added in v0.62.0
SetOutputs replaces preset's outputs from the upload (or zip prefix) from with outs (whose From and Preset it sets) and clears preset from the upload's Pending. Outputs at paths taken by other files are an error.
func (*Manifest) StagedNames ¶ added in v0.62.0
StagedNames lists the staged uploads the manifest references (in temp/).
func (*Manifest) UploadBytes ¶ added in v0.62.0
UploadBytes is the storage charged for a manifest; deleting or erasing the item releases it.
type ManifestOptions ¶
type ManifestOptions struct {
// Locker is required: every edit runs under it, and also writes with
// If-Match once the store reports ConditionalPut. See PGLocker.
Locker Locker
// CacheBytes bounds the decoded manifests kept in process, revalidated
// by ETag. A manifest costs about three times its JSON, so the largest
// (MaxManifestBytes) costs 24 MiB; default 128 MiB, at least two of them.
CacheBytes int64
MaxRetries int // conditional-write attempts per edit; default 16
// Sweeps schedules the folder's sweep after every written edit; best
// effort (the periodic pass backs it up).
Sweeps SweepScheduler
}
ManifestOptions configure Manifests.
type Manifests ¶
type Manifests struct {
// contains filtered or unexported fields
}
Manifests reads and edits item manifests.
func NewManifests ¶
func NewManifests(store Store, reg *Registry, o ManifestOptions) (*Manifests, error)
func (*Manifests) Create ¶ added in v0.45.0
func (m *Manifests) Create(ctx context.Context, ref contentref.ContentRef) (*Manifest, error)
Create starts a new item: it writes the item's empty manifest, hidden until the first commit or Expose resolves it, and fails with ErrFolderNotEmpty if the folder already holds any object. Hosts call it when they create the item's row, so a reused id surfaces there.
func (*Manifests) DeleteUnreferenced ¶ added in v0.62.0
func (m *Manifests) DeleteUnreferenced(ctx context.Context, item Item, cur *Manifest, blobs []string) error
DeleteUnreferenced deletes the blobs among blobs that cur does not reference. A job calls it from its closing edit (the manifest lock held) for what it wrote for a source that is gone, so a taken-down upload's outputs do not come back under their old names.
func (*Manifests) DropIfDeleted ¶ added in v0.62.0
func (m *Manifests) DropIfDeleted(ctx context.Context, ref contentref.ContentRef, blobs []string) error
DropIfDeleted deletes blobs a worker wrote for ref only if its manifest is gone: the folder lock orders the check and the deletes with edits and folder deletion.
func (*Manifests) DropUnreferenced ¶ added in v0.62.0
func (m *Manifests) DropUnreferenced(ctx context.Context, ref contentref.ContentRef, u Unreferenced) error
DropUnreferenced deletes at once, under the manifest lock, what u selects and ref's manifest does not reference: a takedown's sweep, without the grace period.
func (*Manifests) Edit ¶
func (m *Manifests) Edit(ctx context.Context, ref contentref.ContentRef, fn func(*Manifest) error) (*Manifest, error)
Edit applies fn to ref's manifest (empty if none) and writes it with If-Match on the ETag it read (If-None-Match for a new one), re-reading and re-applying fn on conflict. fn must be safe to run more than once; an error from fn aborts the edit. The result is normalized (Kind.Normalize) and validated; an unchanged manifest is not written. A folder's first manifest is refused (ErrFolderNotEmpty) over a previous item's public files, and one growing to within editHeadroom of MaxManifestBytes with ErrManifestTooLarge.
func (*Manifests) EditExisting ¶ added in v0.59.0
func (m *Manifests) EditExisting(ctx context.Context, ref contentref.ContentRef, fn func(*Manifest) error) (*Manifest, error)
EditExisting is Edit while the manifest exists (ErrNotFound otherwise): workers use it after processing, so a concurrent deletion stays deleted.
func (*Manifests) Get ¶
func (m *Manifests) Get(ctx context.Context, ref contentref.ContentRef) (*Manifest, string, error)
Get returns ref's manifest and ETag, or ErrNotFound. Cached copies are revalidated with a conditional GET, so a read is never stale. The manifest is shared: never modify it (Clone it).
func (*Manifests) Place ¶ added in v0.40.0
func (m *Manifests) Place(ctx context.Context, ref contentref.ContentRef) (int, error)
Place moves ref's staged uploads (temp/u-{uuid}) to their content addresses, private/sha256-{hex} of the bytes the server read, and returns how many files it placed. The media worker runs it before processing:
- read each staged upload, hashing it. A single PUT (at most MaxSinglePut, the only kind a client can send again while its URL lives) is written from the bytes that were hashed; a multipart upload, fixed once completed, is copied server-side while its ETag is the one read. Neither is written when the blob exists: a blob is never overwritten, so its bytes always hash to its name;
- point the files at their blobs in one edit, which checks under the folder lock (the sweep holds it too) that each blob still exists;
- delete the staged objects no manifest references, and, in that edit, the blobs of uploads removed meanwhile (a takedown's stay gone).
A staged upload that is gone or not the size committed fails its file (CodeNotUploaded: upload it again). Every step is idempotent, so a crash converges on the next run.
func (*Manifests) Registry ¶ added in v0.62.0
Registry is the registry the manifests are read under.
func (*Manifests) SetFull ¶ added in v0.62.0
func (m *Manifests) SetFull(ctx context.Context, ref contentref.ContentRef, cause error) error
SetFull marks ref's manifest Full: a producer could not record its outputs (cause, an ErrManifestTooLarge, whose overrun becomes the Deficit). The flags fit in the headroom every other edit leaves. Producers write no private output for a Full item until a commit frees the deficit.
func (*Manifests) SyncPublic ¶ added in v0.48.0
func (m *Manifests) SyncPublic(ctx context.Context, ref contentref.ContentRef) ([]string, error)
SyncPublic deletes the public names the manifest does not keep (Kind.PublicKept), under the manifest lock. Cleanup is bounded to one minute; deleted keys are returned for cache purging, including partial success on failure.
type MasterOptions ¶
type MasterOptions struct {
Audio, Subs []string
}
MasterOptions filter a master playlist's renditions by id or BCP 47 tag; nil keeps every track, an empty non-nil slice none.
type Multipart ¶
Multipart carries the opaque ticket for PresignParts, ListParts, Complete and Abort, and the part-size bounds the client adapts within.
type MultipartReply ¶
type Object ¶
type Object struct {
Key string
Size int64
ETag string
ContentType string
CacheControl string
ContentRange string
LastModified time.Time
ChecksumSHA256 []byte // set when the backend stored a full-object SHA-256
Metadata map[string]string // user metadata (x-amz-meta-*), lower-case keys; Head and Get
}
Object is object metadata. For a ranged Get, Size is the body length and ContentRange is set.
type Op ¶
type Op struct {
Op string `json:"op"`
Path string `json:"path,omitempty"`
Blob string `json:"blob,omitempty"` // put: the staged upload (u-{uuid}) or a blob in the folder
CreateID string `json:"create_id,omitempty"` // put: create-only UUID; repeat the same ID to retry
Index *int `json:"index,omitempty"` // put, move, attach: among the attached uploads of its Upload
Meta map[string]any `json:"meta,omitempty"` // put: the upload's meta; attach: merged into it; meta: the item's
Edit *Edit `json:"edit,omitempty"` // put, edit, copy, frame
Unattached bool `json:"unattached,omitempty"`
To string `json:"to,omitempty"` // rename, copy (default the source's path)
From *CopyFrom `json:"from,omitempty"` // copy
T *float64 `json:"t,omitempty"` // frame: seconds into the video
Auto bool `json:"auto,omitempty"` // frame: the worker chooses
Preset string `json:"preset,omitempty"`
Force bool `json:"force,omitempty"`
// Takedown (remove) also removes the frames grabbed from the upload and
// the zips that bundled it, then deletes at once, not a grace period
// later, the blobs the commit dropped, the removed uploads' editor views
// and the public files no upload renders. With an exempt grant it deletes
// every private blob the item no longer references (earlier versions and
// every editor view too), and repeating it, the path already gone, runs
// that sweep again: how staff complete a takedown that failed part way.
Takedown bool `json:"takedown,omitempty"`
}
Op is one commit op.
type OrphanFolder ¶ added in v0.45.0
type OrphanFolder struct {
Prefix string
ID string
ValidID bool
Objects int
Newest time.Time
Deleted bool
}
OrphanFolder is a folder no host item owns. ValidID false: its id is not an item id, so no item can ever address it.
type OrphanReport ¶ added in v0.45.0
type OrphanReport struct {
Folders int
Orphans []OrphanFolder
}
OrphanReport lists a kind's orphans; Folders counts every folder seen.
type OrphanSweep ¶ added in v0.45.0
type OrphanSweep struct {
Kind string
// Exists reports which of ids the host still has (a batch of at most
// 500). An id it omits is an orphan.
Exists func(ctx context.Context, ids []string) (map[string]bool, error)
// Grace skips folders with an object newer than this (default the Jobs
// grace), so an item created meanwhile is never taken for an orphan.
Grace time.Duration
// Delete removes orphans; otherwise they are only reported.
Delete bool
}
OrphanSweep configures SweepOrphans.
type PGLimiter ¶
type PGLimiter struct {
// contains filtered or unexported fields
}
PGLimiter is the default UploadLimiter over ContentKit's Postgres schema: hourly counters per uploader (bytes/day sums the last 24), one usage total per owner and short-lived pending reservations. There are no rows per file.
func NewPGLimiter ¶
func (*PGLimiter) PrepareRelease ¶ added in v0.59.0
type PGLimits ¶
type PGLimits struct {
FilesPerHour int
BytesPerDay int64
// Quota is the owner's storage cap in bytes (<= 0 unlimited); nil disables quotas.
Quota func(ctx context.Context, tenant, owner string) (int64, error)
// ReservationTTL drops unsettled reservations (commit still enforces the
// quota); default 24h.
ReservationTTL time.Duration
}
PGLimits configure PGLimiter. Zero limits are off.
type PartReply ¶
type PartReply struct {
Number int32 `json:"number"`
Size int64 `json:"size,omitempty"`
SHA256 string `json:"sha256,omitempty"`
Request *RequestReply `json:"request,omitempty"`
}
PartReply is a presigned part (parts) or a landed part (parts/list).
type PartRequest ¶
PartRequest is one part to presign: its exact length and SHA-256.
type PartsReply ¶
type PartsReply struct {
Parts []PartReply `json:"parts"`
}
type PresignBody ¶
type PresignBody struct {
Ref RefBody `json:"ref"`
Path string `json:"path"`
Type string `json:"type"`
Size int64 `json:"size"`
SHA256 string `json:"sha256"`
}
PresignBody declares one upload: its path, type, size and whole-file SHA-256, which finds an identical blob already in the folder and binds a single PUT's body.
type PresignPut ¶
PresignPut binds a direct upload to its exact type, length and SHA-256.
type PresignReply ¶
type PresignReply struct {
Path string `json:"path"`
Blob string `json:"blob"`
Exists bool `json:"exists,omitempty"`
Put *RequestReply `json:"put,omitempty"`
Multipart *MultipartReply `json:"multipart,omitempty"`
ProcessOnUpload bool `json:"process_on_upload,omitempty"`
}
PresignReply is the upload plan: Exists (commit directly), one Put, or a Multipart upload. Path is the path to commit: cleaned, with an extension, and named by the server for a Named upload. Blob is the name to commit: the folder's blob (sha256-{hex}) with Exists, else the staged upload (u-{uuid}) the PUT or parts write, which the worker places.
type PresignRequest ¶
type PresignRequest struct {
Ref contentref.ContentRef
Path string
Type string
Size int64
SHA256 []byte
}
PresignRequest declares one upload; SHA256 is the whole file's: it finds an identical blob already in the folder and binds a single PUT's body.
type Presigned ¶
type Presigned struct {
Path string
Blob string
Exists bool
Put *PresignedRequest
Multipart *Multipart
ProcessOnUpload bool
}
Presigned is the upload plan: Exists (commit it), a single Put, or a Multipart upload. Path is the path to commit; Blob is the name to commit: the folder's blob when Exists, else the staged upload (u-{uuid}) that the PUT or the parts write.
type PresignedPart ¶
type PresignedPart struct {
Number int32
PresignedRequest
}
PresignedPart is a part URL.
type PresignedRequest ¶
PresignedRequest is a request a browser sends as is, with exactly Header.
type Private ¶ added in v0.62.0
type Private struct {
Name string `json:"name"`
// From is an upload path or pattern; "" for Zip.
From string `json:"from,omitempty"`
// To is the output path template: a file ("low-res/{name}.webp",
// "video/source-1080p.mp4") or, for producers with several outputs, a
// directory ending in "/" ("hls/", "listen/{name}/").
To string `json:"to"`
// Download is the human name template a download read signs into the
// URL, filled from the manifest's meta and {name}: "{title}.zip".
Download string `json:"download,omitempty"`
// HostOnly leaves this preset out of generic reads and playlists; a host
// route applies its own policy and calls Grant.HostURL. It decides what
// is listed, not what a token opens: an item's token opens every private
// file of the item.
HostOnly bool `json:"host_only,omitempty"`
Image *Image `json:"image,omitempty"`
HLS *HLS `json:"hls,omitempty"`
MP4 *MP4 `json:"mp4,omitempty"`
Zip string `json:"zip,omitempty"` // the files under this prefix, in manifest order: "high/"
Audio *Audio `json:"audio,omitempty"`
Subtitles *Subtitles `json:"subtitles,omitempty"`
// Choose is an optional per-file image spec (Go only, not in the JSON
// registry); nil keeps Image.
Choose func(f File) *Image `json:"-"`
}
Private derives files in private/, recorded in the manifest with their provenance (File.From, Preset, FP). Exactly one producer is set.
type ProcessCanceler ¶ added in v0.43.0
type ProcessCanceler interface {
Cancel(ctx context.Context, ref contentref.ContentRef) (int, error)
}
ProcessCanceler is a ProcessQueue that can cancel an item's queued and running jobs (workqueue.Queue): a commit removing an upload still being processed cancels them, then enqueues the item's remaining work.
type ProcessJob ¶
type ProcessJob struct {
Ref contentref.ContentRef `json:"ref"`
Place bool `json:"place,omitempty"`
Preset string `json:"preset,omitempty"` // only this preset; "" for all
Force bool `json:"force,omitempty"` // redo current outputs too
Editor bool `json:"editor,omitempty"` // also render missing editor views
Class VideoJobClass `json:"class,omitempty"`
}
ProcessJob asks the media worker to bring an item's outputs up to date: every producer recomputes its outputs' fingerprints and redoes the stale ones (and the pending ones), records them, and syncs public/. With Place, the worker first places the item's staged uploads (Manifests.Place).
type ProcessQueue ¶
type ProcessQueue interface {
Enqueue(ctx context.Context, job ProcessJob) error
}
ProcessQueue enqueues processing in the media worker (workqueue.Queue).
type ProgressSource ¶ added in v0.29.0
type ProgressSource interface {
EncodeProgress(ctx context.Context, ref contentref.ContentRef) (EncodeStatus, error)
}
ProgressSource reads encode progress (workqueue.NewProgressSource). The read API asks only when a visible video file is pending.
type Public ¶ added in v0.62.0
type Public struct {
Name string `json:"name"`
From string `json:"from"`
To string `json:"to"` // "cover-{w}.webp", "{name}.webp", "preview-{n}.webp"
First int `json:"first,omitempty"`
Widths []int `json:"widths,omitempty"`
Image Image `json:"image"`
// Default is a path in Kind.Defaults, rendered to the kind's _default
// item at every width (PublishDefaults).
Default string `json:"default,omitempty"`
}
Public is a fixed public image: {To} with {w} at each of Widths (or one file without Widths), rendered from an upload through its edit, at public/{name}. With Widths, each width is Image at that width: in the shape of its Width×Height box (both or neither), fitted per Fit, else of the edit. It is never in the manifest; the object carries its from and fp as metadata. A missing one is served Default by the access agent.
First makes it a preview: the first First attached uploads of From (a {name} pattern) in manifest order, {n} in To their position from 1 ("preview-{n}.webp"). Anyone who can see the item sees them; a position is rendered again when another upload takes it, and names past the last upload are deleted. Everything else of the item stays private.
type PutOptions ¶
type PutOptions struct {
ContentType string
CacheControl string
ChecksumSHA256 []byte
IfMatch string
IfNoneMatch string
Metadata map[string]string
}
PutOptions are conditions and headers for Put. IfNoneMatch "*" creates only.
type QuotaRelease ¶ added in v0.59.0
QuotaRelease is the amount captured before deleting a folder.
type QuotaReleaser ¶ added in v0.59.0
type QuotaReleaser interface {
PrepareRelease(ctx context.Context, r QuotaRelease) (applied bool, err error)
Release(ctx context.Context, tenant, operation string) error
}
QuotaReleaser records a folder's refund before its objects are deleted, then applies that refund once. Operation identifies the deletion across retries.
type RateLimit ¶ added in v0.62.0
type RateLimit struct {
PerSecond float64
Burst int
// Disabled turns limiting off, e.g. when the host limits upstream.
Disabled bool
// Redis is a Redis or Microsoft Garnet client shared by the replicas;
// pass the host's own. Only INCR, PEXPIRE, GET, DECR and MULTI/EXEC are
// used (no Lua: Garnet ships with scripting off).
Redis redis.UniversalClient
// KeyPrefix namespaces the Redis keys (defaults "contentkit:media:rl:"
// for viewers, "contentkit:media:commit:" for uploaders).
KeyPrefix string
}
RateLimit is a per-key request limit: the read API's per viewer (HandlerOptions.Limit) and commits' per uploader (UploadOptions.Commits). Zero fields take the use's defaults.
With Redis set, every replica shares one limit per key; otherwise each process keeps its own (a single-replica assumption, logged at start), so N replicas allow N times the limit. Redis errors fail open to the per-process limit (see RedisErrors): the limit is abuse protection, and tokens, permissions and visibility checks still gate everything.
type ReadOptions ¶
type ReadOptions struct {
Prefix string // only files under this path prefix ("low-res/")
Offset, Limit int // the range that gets URLs
Download bool // URLs serve each file as a download under its name
// Editor adds an editor's uploads with their edit, frame, meta,
// pending, failure and editor view, unattached ones included.
Editor bool
}
ReadOptions select what a read returns.
type ReadResult ¶
type ReadResult struct {
Access string `json:"access"`
// Expires is when the item token (the URLs, the cookie) stops working,
// in unix seconds. A read before then answers the same token, so a
// client keeps what it has and reads again shortly before.
Expires int64 `json:"expires"`
Meta map[string]any `json:"meta,omitempty"`
// Previews are the item's public preview images in order (a Public
// preset with First): every viewer who can see the item gets them.
Previews []string `json:"previews,omitempty"`
Total int `json:"total"`
Offset int `json:"offset"`
Limit int `json:"limit"`
// HLS lists the item's playable ladders by output directory ("hls/"):
// play {read API}/{kind}/{id}/hls/{dir}master.m3u8.
HLS []string `json:"hls,omitempty"`
// State is the item's readiness (editors).
State string `json:"state,omitempty"`
// Full (editors): the item's manifest cannot hold more outputs, so
// processing stopped; remove uploads (any commit that shrinks it) to
// resume. Uploads left unprocessed stay pending.
Full bool `json:"full,omitempty"`
Files []FileInfo `json:"files"`
// Cookie must be set on the response (cookie delivery, with access).
Cookie *http.Cookie `json:"-"`
}
ReadResult is the read API's answer: the files under the requested prefix in manifest order (Total of them). With access each has a URL within [offset, offset+limit); without, each is locked: its path, type and size.
type Reader ¶
type Reader struct {
// contains filtered or unexported fields
}
Reader answers the read API and HLS playlists: one Resolve per item. An item's private files are all or nothing: a viewer with access gets one token that opens every one of them; anyone else gets none, and sees only the item's public files (covers, previews).
func NewReader ¶
func NewReader(o ReaderOptions) (*Reader, error)
func (*Reader) Grant ¶
func (r *Reader) Grant(ctx context.Context, ref contentref.ContentRef, actor access.Actor) (*Grant, error)
Grant resolves ref for actor exactly once and loads its manifest. A resolver error denies (ErrResolve); an invisible item is ErrNotVisible. A visible item without a manifest has no files. A viewer with access who has opened too many items this hour (ReaderOptions.Issuance) gets a LimitError (ErrRateLimited) instead of the item's token; an anonymous actor with access must carry Actor.IP (ErrNoViewerKey).
func (*Reader) Handler ¶
func (r *Reader) Handler(o HandlerOptions) http.Handler
Handler serves the read API. The host mounts it under a prefix such as "/media/" after its auth middleware. Errors are JSON {"error", "code"}: 400 invalid_request, 404 not_found (also for what the viewer may not see), 429 rate_limited (Retry-After: too many requests, or too many items opened this hour), 503 unavailable, 500 internal_error (a resolver error denies this way).
GET /{kind}/{id}?prefix=low-res/&offset=0&limit=50&download&editor -> ReadResult (+ Set-Cookie mt)
GET /{kind}/{id}/hls/{dir}master.m3u8?audio=ja&subs=en (filters optional; empty = none)
GET /{kind}/{id}/hls/{path}.m3u8 a track's media playlist
GET /{kind}/{id}/hls/{dir}sprite.vtt
Every response is "private, no-store"; playlists carry the item cookie like reads. Signed responses log the viewer, item, access and expiry.
func (*Reader) Read ¶
func (r *Reader) Read(ctx context.Context, ref contentref.ContentRef, actor access.Actor, o ReadOptions) (*ReadResult, error)
Read resolves ref once and answers the read API.
type ReaderOptions ¶
type ReaderOptions struct {
Manifests *Manifests // its Registry's BaseURL and Hooks.Resolver are required
Delivery Delivery
// Progress adds live encode progress to pending uploads in editor
// reads; optional.
Progress ProgressSource
// Queue renders the editor views an editor read finds missing; optional.
Queue ProcessQueue
// MaxLimit caps ReadOptions.Limit (default 200); DefaultLimit is used
// when Limit is 0 (default 50).
MaxLimit, DefaultLimit int
// IndexCacheBytes bounds the track index blobs kept for playlists;
// default 16 MiB.
IndexCacheBytes int64
// Issuance limits the items each viewer is given access to per hour
// (default 120 per account, 600 per anonymous IP).
Issuance Issuance
Now func() time.Time
}
ReaderOptions configure a Reader.
type Readiness ¶ added in v0.50.0
type Readiness struct {
State string `json:"state"`
Processing []string `json:"processing,omitempty"`
Failed []string `json:"failed,omitempty"`
}
Readiness is whether an item's media is processed: ready when no attached upload is pending (its public presets count only while the item is visible) and every zip packs its inputs; processing while any is; failed once nothing is processing and an upload failed for its current blob and edit; full while work is left on a Full manifest. Processing and Failed name upload paths (and zip outputs).
type Registry ¶
type Registry struct {
// contains filtered or unexported fields
}
Registry is a validated Config.
func (*Registry) EditorView ¶ added in v0.62.0
EditorView is the blob name of an image upload's editor view: the hash of its source and the editor spec, so a read finds it without rendering. It is the one private blob not named by its bytes: only the worker writes it, and no upload may name it (presign, put and copy refuse it). Only editor reads list it; the sweep removes it after the grace period and an editor read renders it again.
func (*Registry) Item ¶
func (r *Registry) Item(ref contentref.ContentRef) (Item, error)
Item validates ref: a registered kind, its namespace and a UUIDv7 id, without a version (an item is the host's version).
func (*Registry) MarshalJSON ¶ added in v0.62.0
MarshalJSON is the registry as data (no hooks, defaults or Choose): the stock worker's MEDIA_KINDS_FILE.
func (*Registry) Namespaces ¶ added in v0.62.0
Namespaces are the namespaces the app's items live in: its own, then the shared ones it imports.
func (*Registry) PublicURL ¶ added in v0.62.0
func (r *Registry) PublicURL(ref contentref.ContentRef, name string) string
PublicURL is the URL of item ref's public name at the registry's BaseURL.
func (*Registry) Ref ¶ added in v0.62.0
func (r *Registry) Ref(kind, id string) (contentref.ContentRef, error)
Ref is the ref of item id of kind, in the kind's namespace.
func (*Registry) SrcSet ¶ added in v0.62.0
func (r *Registry) SrcSet(ref contentref.ContentRef, preset string) string
SrcSet is the srcset of ref's public preset (by name), "" when the kind has none.
type RequestReply ¶
type RequestReply struct {
Method string `json:"method"`
URL string `json:"url"`
Headers map[string]string `json:"headers"`
Expires time.Time `json:"expires"`
}
RequestReply is a presigned request: send exactly these headers (the browser adds Content-Length, which is signed too).
type Reservation ¶
Reservation is one presigned upload. Uploader is rate-limited; Owner ("" for none) has Size reserved against its quota until the upload is settled or the reservation expires.
type Segment ¶
Segment is one EXT-X-BYTERANGE segment, encoded as [offset, length, seconds].
func (Segment) MarshalJSON ¶
func (*Segment) UnmarshalJSON ¶
type Settlement ¶
Settlement releases reservations and moves an owner's usage. With Enforce, a positive Delta that takes usage over the owner's quota is refused with CodeQuota and changes nothing.
type Sprite ¶
type Sprite struct {
Cols int `json:"cols"`
Rows int `json:"rows"`
W int `json:"w"`
H int `json:"h"`
Interval float64 `json:"interval"`
}
Sprite is a seek-preview grid of Cols×Rows tiles of W×H, one per Interval seconds.
type Store ¶
type Store interface {
Put(ctx context.Context, key string, body io.Reader, size int64, opts PutOptions) (Object, error)
Get(ctx context.Context, key string, opts GetOptions) (io.ReadCloser, Object, error)
Head(ctx context.Context, key string) (Object, error)
Delete(ctx context.Context, key string) error
List(ctx context.Context, prefix string) iter.Seq2[Object, error]
// Copy copies src to dst server-side, keeping its content type, cache
// control and metadata; objects past the backend's single-copy limit
// (5 GiB on S3) copy in parts. The returned Object has dst's size.
Copy(ctx context.Context, src, dst string, o CopyOptions) (Object, error)
PresignPut(ctx context.Context, key string, p PresignPut) (PresignedRequest, error)
// PresignGet is for a worker reading an original over HTTP ranges. Its URL
// uses the store's internal endpoint and must not be returned to browsers.
PresignGet(ctx context.Context, key string, ttl time.Duration) (PresignedRequest, error)
CreateMultipart(ctx context.Context, key, contentType string) (uploadID string, err error)
PresignPart(ctx context.Context, key, uploadID string, number int32, size int64, sha256 []byte, ttl time.Duration) (PresignedRequest, error)
// PutPart uploads one part from the server, bound to its length and SHA-256.
PutPart(ctx context.Context, key, uploadID string, number int32, body io.Reader, size int64, sha256 []byte) (Part, error)
ListParts(ctx context.Context, key, uploadID string) ([]Part, error)
CompleteMultipart(ctx context.Context, key, uploadID string, parts []Part) (Object, error)
AbortMultipart(ctx context.Context, key, uploadID string) error
Capabilities() Capabilities
// Check reports whether the backend answers. The first success also
// establishes Capabilities (Probe under prefix) when they were not given.
Check(ctx context.Context, prefix string) error
}
Store is the bucket. Keys are built by Item; implementations do not interpret them.
type Subtitles ¶ added in v0.62.0
type Subtitles struct{}
Subtitles converts SRT, SSA/ASS and WebVTT to clean UTF-8 WebVTT. A video lists the item's converted subtitles as tracks after its own.
type SweepResult ¶
SweepResult reports one folder sweep. Wait > 0 is when the next unreferenced blob comes due; sweep again then.
type SweepScheduler ¶ added in v0.43.0
type SweepScheduler interface {
ScheduleSweep(ctx context.Context, ref contentref.ContentRef) error
}
SweepScheduler schedules a folder's sweep after an edit: the host's *Jobs, or in the media worker a *HostQueue.
type TicketBody ¶
type TicketBody struct {
Ticket string `json:"ticket"`
}
type Track ¶ added in v0.62.0
type Track struct {
Kind string `json:"kind"` // TrackVideo, TrackAudio, TrackSubs, TrackSprite
Codec string `json:"codec,omitempty"` // video: h264, hevc, av1
Codecs string `json:"codecs,omitempty"` // RFC 6381
Bandwidth int `json:"bw,omitempty"`
Average int `json:"avg,omitempty"`
ID string `json:"id,omitempty"`
Lang string `json:"lang,omitempty"`
Label string `json:"label,omitempty"`
Default bool `json:"default,omitempty"`
Forced bool `json:"forced,omitempty"`
Index string `json:"index,omitempty"` // "sha256-{hex}": the TrackIndex
}
Track is an HLS track file. It is small: the segment table or sprite grid is its own private blob, Index (a TrackIndex), cached by hash.
type TrackIndex ¶ added in v0.62.0
type TrackIndex struct {
Segments []Segment `json:"segments,omitempty"`
Sprite *Sprite `json:"sprite,omitempty"`
}
TrackIndex is a Track's index blob: a byte-range track's segments (its init segment is bytes [0, Segments[0].Offset)), or a sprite's grid.
type Unreferenced ¶ added in v0.62.0
type Unreferenced struct {
// All is every private blob in the folder, editor views included: a
// worker whose unrecorded outputs go with them finds them missing when
// it records, and produces them again.
All bool
// Blobs are candidates: each goes unless it is an editor view of a
// current upload.
Blobs []string
// Staged are staged uploads' names.
Staged []string
}
Unreferenced selects what DropUnreferenced deletes.
type Upload ¶ added in v0.62.0
type Upload struct {
Path string `json:"path"`
Types []string `json:"types"` // accepted content types
MaxBytes int64 `json:"max_bytes"` // per file
Max int `json:"max,omitempty"` // uploads matching Path; 0 is unlimited
Frames string `json:"frames,omitempty"` // may be grabbed from a frame of this video upload (the frame op)
Named bool `json:"named,omitempty"` // the server names it ({name} is "i-{uuid}"): inline images
// Video bounds a video or audio upload as probed from its real stream;
// nil or zero fields take the defaults (DefaultVideoLimits).
Video *VideoLimits `json:"video,omitempty"`
}
Upload is an app path that may be uploaded: a literal ("cover") or a pattern ending in {name} ("originals/{name}"). A file's path is the path plus its extension ("cover.png", "originals/001.png"); a put to the same stem replaces it.
type UploadAuthorizer ¶
type UploadAuthorizer interface {
CanUpload(ctx context.Context, actor access.Actor, t UploadTarget) (UploadGrant, error)
}
UploadAuthorizer is the app's upload permission check, run at presign and commit against what is written.
type UploadError ¶
type UploadError struct {
Code string
Message string
RetryAfter time.Duration // CodeRate: when the window frees
Blobs []string // CodeNotUploaded at commit: the blobs to upload again
Details *ErrorDetails // image refusals
}
UploadError is a refused upload request. UploadLimiter implementations refuse with CodeRate or CodeQuota.
func AsUploadError ¶
func AsUploadError(err error) (*UploadError, bool)
AsUploadError classifies err: an *UploadError, or the kind and registry sentinels. ok is false for internal failures.
func (*UploadError) Error ¶
func (e *UploadError) Error() string
type UploadGrant ¶
UploadGrant is the app's verdict. Exempt (trusted roles) skips the UploadLimiter; Owner is the quota owner, "" for none.
type UploadHandlerOptions ¶
type UploadHandlerOptions struct {
Actor func(*http.Request) (access.Actor, bool) // the host's authenticated caller; false answers 401
Logger *slog.Logger // 5xx causes; default slog.Default()
}
UploadHandlerOptions configure UploadHandler.
type UploadLimiter ¶
type UploadLimiter interface {
Reserve(ctx context.Context, r Reservation) error
Settle(ctx context.Context, s Settlement) error
}
UploadLimiter is the optional anti-abuse port. Quota is enforced when bytes are committed: a commit that grows its owner's stored originals past the quota is refused. Reserve runs at presign (skipped for exempt uploaders) and refuses with an *UploadError coded CodeRate or CodeQuota before any bytes move; a reservation is only that early refusal and may expire. Settle runs at commit and abort: it drops the reservations of Keys and adds Delta (the change in stored originals, negative on removal) to the owner's usage.
type UploadOptions ¶
type UploadOptions struct {
Store Store
Manifests *Manifests // its Registry's Hooks.CanUpload authorizes, Hooks.Resolver hides new items
Tickets *token.Ring // signs multipart tickets (domain-separated from access tokens); required above MaxSinglePut
Limiter UploadLimiter // optional
Queue ProcessQueue // places staged uploads and processes items in the media worker; required
// PresignTTL bounds PUT and part URLs; default 15m. TicketTTL bounds a
// multipart ticket; default 24h, the bucket's abort-incomplete rule.
PresignTTL, TicketTTL time.Duration
// Grace is the sweep's (JobsConfig.Grace): an existing blob is reused
// only while the sweep cannot take it first. Default 24h.
Grace time.Duration
// Frames serves the frame picker (media/video.Frames); nil answers
// not_found. FrameConcurrency bounds grabs per process; default 2.
Frames FrameGrabber
FrameConcurrency int
// ProcessOnUpload tells the SDK to commit each upload unattached as
// soon as it lands, so the worker processes it while the user arranges
// the rest; an attach op makes it part of the item.
ProcessOnUpload bool
// Commits limits each uploader's commits (default 1/s, burst 30, per
// process; set Commits.Redis to share it); exempt grants are not
// limited.
Commits RateLimit
}
UploadOptions configure Uploads.
type UploadTarget ¶ added in v0.60.0
type UploadTarget struct {
Ref contentref.ContentRef
Path string
}
UploadTarget is what an upload writes: an item and an upload path stem ("cover", "originals/001"); "" for ops that change no one upload (move, meta, regenerate).
type UploadedBlob ¶ added in v0.62.0
UploadedBlob is a completed multipart upload; Blob is its staged name.
type Uploads ¶
type Uploads struct {
// contains filtered or unexported fields
}
Uploads presigns direct-to-bucket uploads and commits them. It keeps no state: a multipart upload is its S3 UploadId, carried in a signed ticket.
func NewUploads ¶
func NewUploads(o UploadOptions) (*Uploads, error)
func (*Uploads) Commit ¶
func (u *Uploads) Commit(ctx context.Context, actor access.Actor, ref contentref.ContentRef, ops []Op) (*Manifest, error)
Commit applies ops to ref's manifest in one conditional write. Every op is authorized against what it writes (Hooks.CanUpload). A put names a staged upload or a blob in the folder, HEAD-checked against its Upload; copies are copied server-side first. The owner is charged the change in upload sizes; growth past its quota fails with CodeQuota (not for exempt grants). Then the worker places staged uploads and processes the item. A new item starts hidden unless anonymous viewers may see it (Hooks.Resolver). Successful removes finish public cleanup before returning; unreferenced private blobs go when a grace period old, or at once for a takedown (Op.Takedown).
func (*Uploads) Complete ¶
func (u *Uploads) Complete(ctx context.Context, actor access.Actor, sealed string) (UploadedBlob, error)
Complete assembles the parts server-side. Missing parts answer CodeIncomplete and keep the upload; parts that cannot add up to the declared size abort it.
func (*Uploads) Frame ¶ added in v0.26.0
func (u *Uploads) Frame(ctx context.Context, actor access.Actor, ref contentref.ContentRef, path string, t float64, width int) ([]byte, error)
Frame grabs a still of the video upload at path for the frame picker.
func (*Uploads) Ingest ¶ added in v0.18.0
func (u *Uploads) Ingest(ctx context.Context, actor access.Actor, req IngestRequest) (IngestResult, error)
Ingest writes req.Body to a staged upload in temp/, one checksum-bound PUT when it fits in one part, else a multipart write, and commits a put of it: the worker places and processes it like a browser upload.
func (*Uploads) Presign ¶
func (u *Uploads) Presign(ctx context.Context, actor access.Actor, r PresignRequest) (Presigned, error)
Presign checks an upload against its path's Upload and the app's permission, and plans it.
func (*Uploads) PresignParts ¶
func (u *Uploads) PresignParts(ctx context.Context, actor access.Actor, sealed string, parts []PartRequest) ([]PresignedPart, error)
PresignParts signs parts of a multipart upload, each bound to its length and SHA-256. Parts re-signed after a failure replace the earlier attempt.
type VideoJobClass ¶ added in v0.58.0
type VideoJobClass string
VideoJobClass selects the priority of a new video run. An empty class is an upload; re-encodes and backfills yield to new playable videos.
const ( VideoReencode VideoJobClass = "reencode" VideoBackfill VideoJobClass = "backfill" )
type VideoLimits ¶ added in v0.62.0
type VideoLimits struct {
MaxSeconds float64 `json:"max_seconds,omitempty"` // real running time (audio too)
MaxFPS float64 `json:"max_fps,omitempty"` // output frame rate (at most 60); a faster source is encoded at it
MaxPixels int `json:"max_pixels,omitempty"` // displayed frame area
// MaxWork bounds the planned encode: the sum over the HLS and MP4
// presets' rungs and codecs of output pixels × output frames.
MaxWork float64 `json:"max_work,omitempty"`
}
VideoLimits bound what one video or audio upload may cost the worker; an upload past them fails (video_too_long, video_too_large, video_over_budget) before any encode runs.
func (*VideoLimits) Limits ¶ added in v0.62.0
func (l *VideoLimits) Limits() VideoLimits
Limits are the limits in effect: l's, and the defaults for nil or zero fields.
Source Files
¶
- aspect.go
- config.go
- edit.go
- expose.go
- folders.go
- hls.go
- ingest.go
- issuance.go
- jobs.go
- kind.go
- limiter.go
- lock.go
- manifest.go
- manifests.go
- ops.go
- paths.go
- place.go
- process.go
- progress.go
- provenance.go
- public.go
- ratelimit.go
- read.go
- read_http.go
- registry.go
- river.go
- store.go
- sweep.go
- unavailable.go
- upload.go
- upload_errors.go
- upload_http.go
- video_support.go
- wire.go
Directories
¶
| Path | Synopsis |
|---|---|
|
Package agent is the media access agent run by cmd/media-access.
|
Package agent is the media access agent run by cmd/media-access. |
|
Package image is the media worker's image producer (libvips, CGO): the Image presets' private WebP files, the Zip presets, the public presets' fixed names, editor views, and the kinds' default public images.
|
Package image is the media worker's image producer (libvips, CGO): the Image presets' private WebP files, the Zip presets, the public presets' fixed names, editor views, and the kinds' default public images. |
|
internal
|
|
|
s3test
Package s3test opens the test bucket from the environment:
|
Package s3test opens the test bucket from the environment: |
|
uploadtestserver
command
Command uploadtestserver serves the media upload and read APIs over an isolated MinIO/RGW namespace for the browser SDK's integration tests (sdk/upload/test) and e2e specs.
|
Command uploadtestserver serves the media upload and read APIs over an isolated MinIO/RGW namespace for the browser SDK's integration tests (sdk/upload/test) and e2e specs. |
|
videotest
Package videotest builds synthetic videos whose frames identify their time and orientation, and classifies decoded pixels, for poster and preview tests.
|
Package videotest builds synthetic videos whose frames identify their time and orientation, and classifies decoded pixels, for poster and preview tests. |
|
wirets
Package wirets renders the upload and read API wire types as TypeScript for the browser SDK (sdk/upload/src/wire.gen.ts).
|
Package wirets renders the upload and read API wire types as TypeScript for the browser SDK (sdk/upload/src/wire.gen.ts). |
|
Package layout defines media object keys, dependency-free so the access agent can classify paths without importing the media runtime:
|
Package layout defines media object keys, dependency-free so the access agent can classify paths without importing the media runtime: |
|
Package s3 implements media.Store over aws-sdk-go-v2 for Ceph RGW (production) and MinIO (tests).
|
Package s3 implements media.Store over aws-sdk-go-v2 for Ceph RGW (production) and MinIO (tests). |
|
Package tiered is an optional visibility policy: it maps an item's level to entitlement keys and asks a Checker which ones the actor holds.
|
Package tiered is an optional visibility policy: it maps an item's level to entitlement keys and asks a Checker which ones the actor holds. |
|
Package token signs and verifies media access tokens, shared by the host signer and the access agent so the format cannot drift:
|
Package token signs and verifies media access tokens, shared by the host signer and the access agent so the format cannot drift: |
|
Package video runs an item's video-family presets with ffmpeg: HLS (a byte-range fMP4 ladder with the source's audio and text tracks and a seek sprite), MP4 (a muxed H.264 file at one rung), Audio (an HLS track and an M4A) and Subtitles (clean WebVTT), and grabs the frames of uploads that declare Upload.Frames.
|
Package video runs an item's video-family presets with ffmpeg: HLS (a byte-range fMP4 ladder with the source's audio and text tracks and a seek sprite), MP4 (a muxed H.264 file at one rung), Audio (an HLS track and an M4A) and Subtitles (clean WebVTT), and grabs the frames of uploads that declare Upload.Frames. |
|
Package worker is the media worker: the one process that places uploads and runs every producer, from the host's worker River schema (Config.Schema) in its database: staged uploads hashed and placed at their blobs (media.Manifests.Place), images, zips, public presets and editor views (media/image, libvips), and HLS, MP4, audio, subtitles and frames (media/video, ffmpeg).
|
Package worker is the media worker: the one process that places uploads and runs every producer, from the host's worker River schema (Config.Schema) in its database: staged uploads hashed and placed at their blobs (media.Manifests.Place), images, zips, public presets and editor views (media/image, libvips), and HLS, MP4, audio, subtitles and frames (media/video, ffmpeg). |
|
Package workqueue is the host's side of the media worker (media/worker): the River schema it drains in the host database, insert-only enqueueing and cancelling of its jobs, and processing progress for the read API.
|
Package workqueue is the host's side of the media worker (media/worker): the River schema it drains in the host database, insert-only enqueueing and cancelling of its jobs, and processing progress for the read API. |
|
metrics
Package metrics exports the host's media worker queue health.
|
Package metrics exports the host's media worker queue health. |