kindstream

package
v0.3.8 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 11, 2026 License: GPL-3.0 Imports: 19 Imported by: 0

Documentation

Overview

Package kindstream is Ether's unified byte-acquisition and streaming layer for plugin-provided content kinds (spec §4, §5, §15.1). Every plugin media item — comic page, ebook file, audiobook chapter, podcast episode, or video part — resolves through kindstream into a random-access ByteSource.

Index

Constants

View Source 
const (
	PatternRange     = "range"
	PatternDownload  = "download"
	PatternDirect    = "direct"
	PatternContainer = "container"
	PatternRelease   = "release"
)

Retrieval pattern constants (spec §4.1, §4.3).

Variables

View Source 
var (
	// ErrMemberUndecodable is returned when an archive member is compressed or
	// solid in a format/method that internal/archive cannot decode sequentially.
	ErrMemberUndecodable = errors.New("kindstream: archive member not sequentially decodable")

	// ErrHostNotAllowed is returned when a source URL's host is rejected by the
	// plugin's declared network allowlist.
	ErrHostNotAllowed = errors.New("kindstream: host not in plugin allowlist")

	// ErrSizeLimitExceeded is returned when a download or range-demoted source
	// exceeds its configured maxBytes or library cache quota.
	ErrSizeLimitExceeded = errors.New("kindstream: source exceeds download size limit")

	// ErrMIMEMismatch is returned when sniffed magic bytes disagree with the
	// plugin-declared MIME at the family level (e.g. text/html error page for an
	// image/* source).
	ErrMIMEMismatch = errors.New("kindstream: sniffed MIME family disagrees with declared MIME")
)
View Source 
var (
	// ErrInvalidTemplate is returned when a URL template or secret placeholder
	// is malformed or expands to a disallowed URL.
	ErrInvalidTemplate = errors.New("kindstream: invalid URL template")
)

Functions

func ExpandHeaders 

func ExpandHeaders(ctx context.Context, headers map[string]string, secretFn func(ctx context.Context, name string) (string, error)) (map[string]string, error)

ExpandHeaders resolves {secret:name} placeholders in header values via secretFn. Resolved values must never be logged by callers.

func Handler 

func Handler(src ByteSource) http.Handler

Handler returns an http.Handler that serves a ByteSource with full HTTP Range support (206 Partial Content, Accept-Ranges: bytes, Content-Type, and Content-Length).

func MediaSource 

func MediaSource(src ByteSource, releaseID string) core.MediaSource

MediaSource adapts a ByteSource into a core.MediaSource so any pipeline accepting core.MediaSource can consume a kindstream ByteSource directly.

func Seeker 

func Seeker(src ByteSource) io.ReadSeekCloser

Seeker wraps any ByteSource as an io.ReadSeekCloser so it can be passed to http.ServeContent, standard library readers, or transcode pipelines.

func ServeLoopback 

func ServeLoopback(src ByteSource) (urlStr string, closeFn func() error, err error)

ServeLoopback starts a loopback HTTP server (http://127.0.0.1:<port>/<name>) backed by src so local ffmpeg / ffprobe sessions in internal/transcode can seek and stream any ByteSource over HTTP Range (spec §15.1). Calling closeFn shuts down the loopback server without closing src.

Types

type ByteSource 

type ByteSource interface {
	io.ReaderAt
	Size() int64  // total byte length; always > 0 once Open returns
	Name() string // filename for Content-Disposition and container sniffing
	MIME() string // MIME type (plugin-declared, verified/refined by sniffing first 512 B)
	Close() error
}

ByteSource is the uniform random-access stream interface produced by kindstream for all plugin content kinds (spec §5.1).

func ExtractMember 

func ExtractMember(
	ctx context.Context,
	c core.ContainerRef,
	format archive.Format,
	selected archive.Member,
	nonDirIdx int,
	declaredMIME string,
	maxBytes int64,
	deps Deps,
) (ByteSource, error)

ExtractMember downloads the container's volume(s) into deps.Cache (if not already local) and sequentially decodes the target member into a cached file in deps.Cache, returning a random-access ByteSource over that file.

If the archive format or compression method is not sequentially decodable by internal/archive, ExtractMember returns ErrMemberUndecodable.

func Open 

func Open(ctx context.Context, src core.PlayableSource, deps Deps) (ByteSource, error)

Open resolves a single core.PlayableSource into a random-access ByteSource according to its retrieval pattern (spec §4.3, §5.1).

func OpenContainer 

func OpenContainer(ctx context.Context, c core.ContainerRef, declaredMIME string, maxBytes int64, deps Deps) (ByteSource, error)

OpenContainer opens an archive container (RAR4/RAR5/7z/ZIP) over one or more VolumeRefs and returns a ByteSource for the requested inner member (spec §5.3).

Store-mode members (including encrypted store-mode RAR/7z when PasswordSecret is supplied) are served directly via archive.NewMemberReader without extracting the archive. Compressed ZIP members are sequentially decoded into deps.Cache via ExtractMember; undecodable compressed archives (such as compressed RAR) fail with ErrMemberUndecodable so the caller can fall back to the next core.PlayableSource.

func OpenSources 

func OpenSources(ctx context.Context, sources []core.PlayableSource, deps Deps) (ByteSource, core.PlayableSource, error)

OpenSources tries each candidate core.PlayableSource in order, returning the first ByteSource that opens and validates cleanly. When all candidates fail (or sources is empty), OpenSources returns an error wrapping core.ErrSourceUnavailable (spec §4.5).

type Deps 

type Deps struct {
	HTTP             *http.Client
	Cache            *blobcache.Namespace
	OpenRelease      func(ctx context.Context, r core.ReleaseHint) (core.MediaSource, error)
	Secret           func(ctx context.Context, name string) (string, error)
	AllowHost        func(host string) bool
	MaxDownloadBytes int64
	ItemIndex        int
	Backoff          *HostBackoff
}

Deps supplies the runtime dependencies needed by Open and OpenSources (spec §5.1).

type DirectSource 

type DirectSource struct {
	// contains filtered or unexported fields
}

DirectSource represents a validated "direct" core.PlayableSource. Callers that support client redirects (spec §4.3) can extract the target URL via DirectURL() and issue a 307 Temporary Redirect without proxying bytes; callers that require a ByteSource stream (such as tests or local transcode bridges) can call ReadAt, which lazily opens the underlying HTTP source.

func AsDirectSource 

func AsDirectSource(bs ByteSource) (*DirectSource, bool)

AsDirectSource extracts the *DirectSource if bs was opened from a "direct" core.PlayableSource.

func (*DirectSource) Close 

func (d *DirectSource) Close() error

func (*DirectSource) DirectHeaders 

func (d *DirectSource) DirectHeaders() map[string]string

DirectHeaders returns the resolved request headers for the direct URL.

func (*DirectSource) DirectURL 

func (d *DirectSource) DirectURL() string

DirectURL returns the validated, template-expanded target URL for a 307 redirect.

func (*DirectSource) MIME 

func (d *DirectSource) MIME() string

func (*DirectSource) Name 

func (d *DirectSource) Name() string

func (*DirectSource) ReadAt 

func (d *DirectSource) ReadAt(p []byte, off int64) (int, error)

func (*DirectSource) Size 

func (d *DirectSource) Size() int64

type HTTPReaderAt 

type HTTPReaderAt struct {
	// contains filtered or unexported fields
}

HTTPReaderAt implements ByteSource (and io.ReaderAt) over an HTTP Range or range-templated URL source.

func (*HTTPReaderAt) Close 

func (r *HTTPReaderAt) Close() error

Close is a no-op for stateless HTTP range readers.

func (*HTTPReaderAt) MIME 

func (r *HTTPReaderAt) MIME() string

MIME returns the resource's MIME type.

func (*HTTPReaderAt) Name 

func (r *HTTPReaderAt) Name() string

Name returns the resource's filename.

func (*HTTPReaderAt) ReadAt 

func (r *HTTPReaderAt) ReadAt(p []byte, off int64) (int, error)

ReadAt reads len(p) bytes starting at byte offset off.

func (*HTTPReaderAt) Size 

func (r *HTTPReaderAt) Size() int64

Size returns the total byte size of the resource.

func (*HTTPReaderAt) SniffBytes 

func (r *HTTPReaderAt) SniffBytes() []byte

SniffBytes returns up to the first 512 bytes captured during the initial probe.

type HostBackoff 

type HostBackoff struct {
	// contains filtered or unexported fields
}

HostBackoff tracks per-host exponential backoff and Retry-After windows on 429 and 503 responses.

func NewHostBackoff 

func NewHostBackoff() *HostBackoff

NewHostBackoff creates a per-host backoff tracker.

type Leaf 

type Leaf struct {
	Name string
	MIME string
	Size int64
	Open func(ctx context.Context) (ByteSource, error)
}

Leaf describes one part/page/chapter in a multi-part Sequence (spec §4.4, §5.1).

type Sequence 

type Sequence struct {
	// contains filtered or unexported fields
}

Sequence presents an ordered list of Leaf sources both as discrete addressable leaves (Leaf(i)) and as a single concatenated ByteSource (ReadAt) when every leaf has a known positive Size (spec §4.4, §5.1).

Sequence opens leaves lazily on demand, holds at most 2 leaves open at a time (current + next), and prefetches the next leaf when a read crosses 90% of the current leaf.

func NewSequence 

func NewSequence(leaves []Leaf) *Sequence

NewSequence constructs a Sequence over leaves.

func (*Sequence) Close 

func (s *Sequence) Close() error

Close closes any currently open leaves.

func (*Sequence) IsLeafOpen 

func (s *Sequence) IsLeafOpen(i int) bool

IsLeafOpen reports whether leaf i is currently open in the sequence pool.

func (*Sequence) Leaf 

func (s *Sequence) Leaf(i int) (ByteSource, error)

Leaf opens (or returns the already-open) ByteSource for leaf i (0-based). The returned ByteSource shares the sequence's open-leaf pool; calling Close on the returned leaf handle is a no-op (Sequence.Close closes all open leaves).

func (*Sequence) LeafContext 

func (s *Sequence) LeafContext(ctx context.Context, i int) (ByteSource, error)

LeafContext is Leaf with an explicit context for opening the leaf on cache miss.

func (*Sequence) Len 

func (s *Sequence) Len() int

Len returns the number of leaves in the sequence.

func (*Sequence) MIME 

func (s *Sequence) MIME() string

MIME returns the sequence's MIME type.

func (*Sequence) Name 

func (s *Sequence) Name() string

Name returns the sequence's display name.

func (*Sequence) OpenLeafCount 

func (s *Sequence) OpenLeafCount() int

OpenLeafCount returns the number of currently open leaves (at most 2).

func (*Sequence) ReadAt 

func (s *Sequence) ReadAt(p []byte, off int64) (int, error)

ReadAt reads len(p) bytes at offset off across the concatenated leaves.

func (*Sequence) Size 

func (s *Sequence) Size() int64

Size returns the sum of all leaf sizes when known, or 0 if any leaf has unknown size.

type TemplateVars 

type TemplateVars struct {
	ItemIndex int
	ByteStart int64
	ByteEnd   int64
	HasRange  bool
}

TemplateVars holds the integer values allowed in a URL template (spec §4.2). Values are strictly non-negative integers; arbitrary string interpolation is never performed.

type URLTemplate 

type URLTemplate struct {
	// contains filtered or unexported fields
}

URLTemplate is a parsed and validated URL template.

func ParseURLTemplate 

func ParseURLTemplate(raw string, allowHost func(host string) bool) (*URLTemplate, error)

ParseURLTemplate validates that raw contains only known placeholders ({itemIndex}, {pageNumber}, {byteStart}, {byteEnd}, {byteRange}) and that a trial expansion with zero values yields a valid http/https URL with a host.

func (*URLTemplate) Expand 

func (t *URLTemplate) Expand(v TemplateVars, allowHost func(host string) bool) (string, error)

Expand substitutes integer placeholders and validates the resulting URL's scheme (http/https), non-empty host, and optional host allowlist.

func (*URLTemplate) HasByteRange 

func (t *URLTemplate) HasByteRange() bool

HasByteRange reports whether the template embeds a bounded byte range ({byteRange} or both {byteStart} and {byteEnd}) in the URL itself.

func (*URLTemplate) HasByteStartOnly 

func (t *URLTemplate) HasByteStartOnly() bool

HasByteStartOnly reports whether the template embeds {byteStart} without an end byte, indicating an upstream that seeks from {byteStart} without a Range request header (spec §4.2).

func (*URLTemplate) UsesRangePlaceholders 

func (t *URLTemplate) UsesRangePlaceholders() bool

UsesRangePlaceholders reports whether any byte-offset placeholder is present.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.