nodeinput

package
v0.3.5 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 7, 2026 License: GPL-3.0 Imports: 17 Imported by: 0

README

Worker-local encoder input

Pure Go input composition, wired by clusterplayback. Public protocol routes remain unchanged. The Go nodesource.Client holds verified mTLS credentials and opaque rotating source bearer. FFmpeg gets only the loopback capability URL and local X-Ether-Encoder-Input scope header (which also activates the existing encoder command-log redaction). No caller URL, source/provider header, pathname, or cluster credential reaches FFmpeg.

Host composition:

  1. Construct New(Options) with the verified *nodesource.Client, lifetime admission capacity, concurrent request/connection limit, and request timeout. Construction binds TCP4 127.0.0.1:0; host runs Serve(ctx) and owns Close.
  2. Resolve a stable Start.Grant handle through authoritative placement/source policy. Call Admit(ctx, start, grant, exactSize). It opens its own Reader and checks HEAD size. Scope must match controller/worker boot, session, positive generation without overflow; full stable-node IDs, viewer and revision remain bound by nodesource authority/Peer. Boot equality alone is not authorization. Grant expiry must cover start under the host's conservative clock policy; timestamp comparison here is not a cross-host clock guarantee.
  3. Return remotetranscode.Source{Gateway: input}. No ordinary URL/Lease may be mixed in. Return a nonnil input even on admission/Open error so the worker can seal and retain its responsibility. The resolver must never mint an unused second lease. GatewayInput validates exact Identity and stable handle; worker independently validates literal-loopback URL shape and retains leases on errors.
  4. Encoder claims once and obtains scope headers only for the exact URL. Local HTTP additionally checks exact listener Host/origin/path, scope header, and rejects duplicate/ambiguous headers/URL forms. Only HEAD/GET, one bounded byte range (including suffix/open end), 206 and 416 are supported. No redirects.
  5. Drive Reconcile during activity and drain, even after cancellation/expiry. SealInput atomically seals admissions and cancels Reader requests without waiting. Producer registration and seal share a mutex. HTTP handlers, Open, explicit renewal, and Reader work are registered before starting. A blocked writer stays pending until it exits (write/request timeout bounds network writers); Close closes listener and connections. No read-wrapper goroutines, renewal loops or polling goroutines are created here.

Every reserved admission, including failed opens, consumes capacity for this Gateway lifetime. Retained tombstones are never evicted to manufacture proof. Reconcile releases Reader ownership before InputDone/callback publication; callback runs once outside locks. Accounting.Pending=0 means no retained producers, not that callbacks have finished. Callback is immutable after seal and must perform only bounded accounting. InputErr is pending until local proof, then nil.

Local proof only: InputDone/InputErr and Reader.Done never attest remote source retirement. Remote source handler holds and durable placement capacity need source owner sealing/producer/SourceRetirement proof independently. The offline fixture explicitly keeps a delayed remote producer after gateway input has drained.

Input.Renew(ctx, exactRenewal, authority) calls authority-backed Reader.Renew, which verifies unchanged complete Scope, revision, size and HEAD identity before rotating the bearer. The Reader, URL, stable handle and InputLease stay identical. Returned RenewalEvidence echoes the exact request, identity, handle, Scope, size, SourceUntil and lease for composing ValidateRenewal. Placement must be separately verified using its authoritative clock and skew policy. Successful source extension can outlive rejected worker acceptance or stop and is never evidence of worker CAS acceptance. Successful source-operation replays fail closed: historical evidence is never returned as current source authority. Reconcile the worker's accepted history after uncertainty; if the source operation succeeded but worker acceptance did not, retire the generation. Source-validation replay support is unavailable. Only one renewal validates at a time; contention returns an error without waiting. Operation/version and preceding deadline must match source history exactly; lower or changed same-operation payloads fail before authority work. A failed attempt can retry its exact payload using fresh authority. Commit rechecks the captured Reader, history and seal after external validation. No raw expiry setter/source swap/ encoder start is available. Parent owns schedule, worker renewal and reconciliation.

Input optionally implements core.RecoverySourceReader on the same claimed lease. RecoverySourceIdentity returns the authenticated Scope revision and immutable Reader size, crosschecked against admission. This cached identity is not a fresh authority check. ReadRecoveryAt requires that exact revision and a complete, positive range of at most 64 KiB within the source. Unknown/empty identity or size, unclaimed/sealed inputs, overflow, cancellation and mismatches fail closed. Each read shares the HTTP request limit and registers with the existing tracker under the sealing mutex before invoking the actual Reader. The request deadline is capped by RequestTimeout and composed with Input lifetime cancellation. Reads run synchronously and Reconcile joins their registered work before InputDone. Errors return zero bytes with ErrDenied or standard context cancellation/deadline errors; callers must discard the buffer on error.

This capability supplies only same-source bytes for the separately bounded, strict nonfragmented AAC-LC MP4 index. It does not attest previously retained access units/output, authorize a second encoder, prove source-host equivalence, or enable application/transport recovery. Ordinary inputs remain unchanged.

Limitations: host still supplies authority, stable-handle lookup, placement and source-owner proof, certificate lifecycle, clock policy and durable recovery. Bounds apply within one gateway boot; capacity exhaustion requires host lifecycle handling without dropping unresolved ownership. Arbitrary custom trusted-gateway interface implementations are the host's trust responsibility, not wire input. No actual FFmpeg/GPU/Plex execution or deployment is claimed by the offline fixture.

Source failover is wired by clusterplayback through Options.Recover. Each network chunk has a two-second worker read bound, and recovery work has a 30-second per-chunk bound (three recovery attempts per input lifetime). Normal reads remain concurrent; source changes wait for readers and renewal. Continuous FFmpeg GETs renew their write deadline as bytes progress, rather than expiring at a fixed time after opening the file.

A failed or timed-out chunk is never exposed, even when its HTTP body was partially received. The complete undelivered chunk is fetched at its original byte offset. Replacement HEAD and range responses authenticate the same revision and size. The source generation is separate from the unchanged encoder execution fence; the controller commits a durable route CAS before issuing a replacement grant. Source authority rechecks that generation, including after stalled reads. The gateway rejects returned bytes from cancelled operations and closes the old reader at replacement. No credentials or new URLs reach FFmpeg.

The exact initial preparation (release, viewer, title, caps, revision and size) is recoverable without contacting the failed source. Only source preparations captured through the distributed prepare/bind flow support cross-host recovery; local callers without that descriptor fail closed. The stream engine's postedFile.locate and plan layout map arbitrary offsets to article ranges; opening another source may probe archive headers/indexes, but does not stream all preceding payload bytes. Revision/size checks are identity checks, not a whole-file content hash. Source providers must preserve immutable release bytes.

All old source routes remain retirement responsibilities. An unreachable source cannot be reported as successfully closed or used to release placement capacity. Offline gateway and controller fixtures prove byte continuity, fencing and stable encoder identity; they do not claim real FFmpeg or Kubernetes node-loss acceptance.

Documentation

Overview

Package nodeinput composes verified Go source transport with worker-local encoder input. It supplies no remote source retirement proof.

Index

Constants

View Source
const MaxRecoveryRead = 64 << 10

MaxRecoveryRead bounds each optional index read. Larger indexes must use multiple bounded calls; the caller separately bounds total bytes and work.

Variables

View Source
var ErrDenied = errors.New("nodeinput: unavailable input")

Functions

This section is empty.

Types

type Accounting

type Accounting struct{ Retained, Pending, Requests int }

type Gateway

type Gateway struct {
	// contains filtered or unexported fields
}

Gateway owns its loopback listener. Host runs Serve and drives Reconcile even during drain. Capacity includes all retained admissions, including failed opens; records are never evicted during this gateway lifetime.

func New

func New(o Options) (*Gateway, error)

func (*Gateway) Admit

func (g *Gateway) Admit(ctx context.Context, s remotetranscode.Start, grant nodesource.Grant, size int64) (*Input, error)

Admit opens one exact Reader itself: callers cannot substitute a Reader/source. A nonnil Input is responsibility even on error; SealInput and Reconcile it. Authority expiry must cover Start on the composing host's conservative clock policy. This local comparison does not establish cross-host clock agreement.

func (*Gateway) Close

func (g *Gateway) Close() error

Close seals all admissions, cancels reads, closes sockets (including blocked writers), and returns without waiting for producers. Reconcile retains proof.

func (*Gateway) GoString

func (*Gateway) GoString() string

func (*Gateway) Reconcile

func (g *Gateway) Reconcile() Accounting

Reconcile releases registry ownership before signaling Done/callback outside locks. Pending=0 means no retained producers, not callback completion. A sealed callback is immutable; concurrent reconciliation publishes it once. Tombstones retain only bounded identity/capability state, never a Reader.

func (*Gateway) Serve

func (g *Gateway) Serve(ctx context.Context) error

Serve blocks until Close or host cancellation. No internal polling/renewal loop.

func (*Gateway) ServeHTTP

func (g *Gateway) ServeHTTP(w http.ResponseWriter, r *http.Request)

func (*Gateway) String

func (*Gateway) String() string

type Input

type Input struct {
	// contains filtered or unexported fields
}

Input is an opaque, exact lease and trusted gateway capability. Never serialize it. String formatting deliberately hides its URL, grant, and scope.

func (*Input) ClaimInput

func (in *Input) ClaimInput(callback func(error)) error

func (*Input) GatewayInput

func (in *Input) GatewayInput(id remotetranscode.Identity, handle string) (string, core.EncoderInputLease, error)

GatewayInput binds the exact remote execution and stable authority handle.

func (*Input) GoString

func (*Input) GoString() string

func (*Input) HeadersForInput

func (in *Input) HeadersForInput(raw string) (map[string]string, error)

func (*Input) InputDone

func (in *Input) InputDone() <-chan struct{}

func (*Input) InputErr

func (in *Input) InputErr() error

func (*Input) ReadRecoveryAt

func (in *Input) ReadRecoveryAt(ctx context.Context, expectedRevision string, p []byte, off int64) (int, error)

ReadRecoveryAt reads only a complete bounded range of the same claimed input. On error n is zero and callers must discard p. Errors expose no source details. It creates no read goroutine: cancellation is not completion, and the existing Input tracker retains this call until the actual Reader operation returns.

func (*Input) RecoverySourceIdentity

func (in *Input) RecoverySourceIdentity() (revision string, size int64, err error)

RecoverySourceIdentity describes the already authenticated, immutable source of this claimed input. It is not current remote authority, retained output/AU evidence, or permission to claim another input or start another encoder.

func (*Input) Renew

Renew may conservatively extend source authority even if stop or worker CAS later rejects acceptance. It never attests worker acceptance or placement. One operation may validate at a time; contention fails closed without waiting. Successful source operations are never replayed as current authority evidence: if worker acceptance is uncertain, reconcile worker history. If unaccepted, retire this generation; source-validation replay is deliberately unsupported. Failed attempts may retry only their exact payload against fresh authority.

func (*Input) SealInput

func (in *Input) SealInput()

func (*Input) String

func (*Input) String() string

type Options

type Options struct {
	// ValidateScope checks current execution generation at the local byte consumer.
	ValidateScope         func(context.Context, nodesource.Scope) error
	WatchAuthority        func(string, func()) func()
	Client                *nodesource.Client
	Recover               func(context.Context, nodesource.Grant) (*nodesource.Client, nodesource.Grant, error)
	ReadTimeout           time.Duration
	MaxRecoveries         int
	Capacity, MaxRequests int
	RequestTimeout        time.Duration
}

type RenewalEvidence

type RenewalEvidence struct {
	Renewal     remotetranscode.Renewal
	Identity    remotetranscode.Identity
	Handle      string
	Scope       nodesource.Scope
	Size        int64
	SourceUntil time.Time
	InputLease  core.EncoderInputLease
}

RenewalEvidence is local evidence for composing ValidateRenewal, not placement proof. No bearer is exposed and no raw expiry setter exists.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL