datasource

package
v0.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 23, 2026 License: MIT Imports: 22 Imported by: 0

Documentation

Overview

Package datasource fetches arbitrary data and hands it to bound components. Sources are construct-once, fetch-many — the same Source is reused across polling ticks.

Source kinds in v1:

http   GET (or other verb) a URL; parse JSON or keep as text.
exec   Run a command; capture stdout; parse it.
file   Read a file from disk.
merge  Fan out to N children, union their results, optionally tag
       each item with which child it came from. Composes any other
       source kind — works recursively (merge of merges) as long as
       the graph has no cycles (validated upstream).

All four implement the same one-method Source interface, so any component-binding code (list / table / inspector / logview) consumes them identically.

Index

Constants

View Source
const ParamCacheDefaultSize = 100

ParamCacheDefaultSize is the entry cap when CacheSpec.Size is unset. 100 is enough for the "cursor drifts over a table of a few dozen rows re-visiting the same detail" pattern without ballooning memory for a lookup table that keeps growing.

Variables

View Source
var ErrNotStreaming = errors.New("source does not support streaming")

ErrNotStreaming is the sentinel a StreamingSource's Subscribe returns to say "I can't stream under this configuration; fall back to my Fetch (polling) path." Used by merge when its children don't all implement StreamingSource — and by any future source that publishes a Subscribe method conditionally on config (e.g. exec without `follow: true`).

Real failures (dial errors, write errors, auth rejections) surface as ordinary errors; the screen pops an alert for those. ErrNotStreaming is special: it tells the screen to ignore the streaming interface for this fetch and take the polling path instead.

View Source
var ErrNotWindowed = errors.New("source does not support windowed fetch")

ErrNotWindowed is the sentinel a WindowedSource returns when it can't serve windows under its current configuration — the same role ErrNotStreaming plays for Subscribe. The caller falls back to Fetch.

Functions

func Build

func Build(sources map[string]*cfg.Source) (map[string]Source, error)

Build constructs every leaf-kind source in the given map, resolving merge sources by recursively building children first. Used by this package's own tests; production code goes through pipeline.Build, which handles operator entries too.

func FirstString

func FirstString(v any, paths []string) string

FirstString tries each path in order against v and returns the first non-empty rendered string. Empty when every path resolves to "" / nil. Used by table column / inspector field bindings that accept a fallback chain — `kubectl`-style computed fields where the authoritative value lives under different keys depending on state.

func Get

func Get(v any, path string) any

Get resolves a dot-path into a parsed JSON value. Supported syntax:

"a.b.c"        nested object keys
"a[0].b"       array index, optionally chained
"a.0.b"        array index expressed as a dotted segment
""             return v unchanged

Missing keys / out-of-range indices return nil. Type-mismatches (e.g. indexing a non-array) also return nil — callers render nil as the empty string. The function never panics.

func Iter

func Iter(v any) []any

Iter returns v as a slice when v is a JSON array, or wraps a single object in a 1-element slice. Used by list/table bindings that walk an iterable root.

func ParamsHash

func ParamsHash(params map[string]string) string

ParamsHash produces a deterministic string key from a params map. Sorted keys, `=` and `\x00` separators, SHA-1 hex → 40 chars. Cryptographic strength is not required — we're just avoiding collisions between different (k=v) tuples. SHA-1 is cheap and its collision domain is more than enough for realistic caches.

func String

func String(v any, path string) string

String resolves a path and renders the result as a display string. Strings pass through, numbers / booleans format predictably, nested values fall back to fmt.Sprint.

Types

type Event

type Event struct {
	Line string
	Err  error
}

Event is a single update from a streaming source. v1 events carry a line (any printable text) destined for a logview's buffer. A non-empty Err signals a terminal error and ends the stream.

type ParamCache

type ParamCache struct {
	// contains filtered or unexported fields
}

ParamCache is a bounded LRU with per-entry TTL, keyed on a string hash of the params tuple. Concurrent-safe. Never caches errors — a failing fetch retries on the next call so a stale error doesn't lock a user out for the rest of the TTL. Uses single-flight so N concurrent callers with the same key issue one upstream load, not N.

Not a ds.Source itself: the parameter-varying call sites (join lookup fan-out, cursor-driven detail refetch) know their params tuple at call time and drive the cache directly. Wrapping the leaf source would require plumbing the params through Fetch, which the interface doesn't carry.

func NewParamCache

func NewParamCache(spec *cfg.CacheSpec) (*ParamCache, error)

NewParamCache constructs an empty LRU from a *cfg.CacheSpec. Zero- value Size becomes ParamCacheDefaultSize; negative Size means unbounded. Nil spec (or empty TTL) yields (nil, error) — callers that want to run without caching should not call this.

func (*ParamCache) FetchOrLoad

func (c *ParamCache) FetchOrLoad(params map[string]string, load func() (any, error)) (any, error)

FetchOrLoad returns the cached value for the given params tuple if present and fresh. Otherwise it invokes load exactly once per (params, in-flight window): concurrent callers with the same params block on the shared inflight and receive whatever the winning caller got. Errors are propagated but not cached — the next call retries.

func (*ParamCache) Len

func (c *ParamCache) Len() int

Len returns the current number of live entries. Exposed for tests and future stats reporting. Cheap; takes the mutex briefly.

type Source

type Source interface {
	Fetch(ctx context.Context) (any, error)
	// Refresh is the polling interval, or 0 for fetch-once.
	Refresh() time.Duration
}

Source fetches data on demand. Implementations are responsible for any per-fetch context (HTTP request, subprocess invocation, file read) but should be safe to call concurrently — the caller usually serializes via tea.Cmd anyway.

func BuildLeaf

func BuildLeaf(s *cfg.Source, children map[string]Source) (Source, error)

BuildLeaf constructs a single leaf-kind data source from a *cfg.Source. For merge sources, the caller pre-resolves children (via the unified Build in internal/pipeline that walks the entire graph) and passes the resolved map in.

Returns an error for operator-kind sources; the caller dispatches those to internal/pipeline constructors instead.

func NewMerge

func NewMerge(d *cfg.Source, children map[string]Source) (Source, error)

NewMerge constructs a merge composer from a cfg.MergeSource and a map of resolved child sources. Exposed so the pipeline package can reuse this proven implementation for its `union` operator without duplicating the streaming / snapshot / per-child-tag machinery.

The caller is responsible for resolving children from cfg.Sources or cfg.Children (whichever shape the config uses) into the map before calling.

type StreamingSource

type StreamingSource interface {
	Source
	Subscribe(ctx context.Context) (<-chan Event, error)
}

StreamingSource pushes events over time instead of polling for full snapshots. v1 binds streams into logview only — each Event carries a line that's appended to the buffer as it arrives. (Streaming into list/table needs a keyed-update protocol that's deferred until a real use-case demands it.)

Lifecycle:

  • Subscribe returns immediately with a channel of events; the source's own goroutine pushes events as data arrives. The given context cancels the stream — when ctx.Done() fires the source stops, closes its goroutine, and closes the events channel.
  • The screen reads events one at a time via a Bubble Tea Cmd that re-issues itself (the standard "channel-to-tea.Msg" bridge — CLAUDE rule 12 in tuilib).
  • When the channel is closed (graceful end of stream) or an Event with a non-nil Err is delivered, the screen treats the stream as finished. The user may force a reconnect via the refresh key.

type WindowPage added in v0.2.1

type WindowPage struct {
	// Items are the rows in this window, already sliced by the source's
	// `root:` the same way Fetch would.
	Items []any
	// Total is the number of rows matching the query across every page,
	// or -1 when the source can't say. -1 is not an error: the table
	// then treats the end of what has loaded as the end of the set, and
	// grows as more arrives.
	Total int
}

WindowPage is what one FetchWindow returned.

type WindowQuery added in v0.2.1

type WindowQuery struct {
	// Offset is the index of the first row wanted.
	Offset int
	// Limit is how many rows are wanted.
	Limit int

	// Search is the filter text the user typed with no column scope, all
	// bare terms joined by a space. Empty when the filter is empty or
	// every term was scoped.
	Search string
	// Filters maps a column *title* to the value the user scoped it to,
	// from "title:value" terms. Titles are resolved against the bound
	// table's columns before they get here, so an implementation can
	// look them up directly.
	Filters map[string]string

	// Sort is the title of the column to order by, "" when unsorted.
	Sort string
	// Desc reverses the order.
	Desc bool
}

WindowQuery is one request's worth of parameters: which rows, matching what, in what order. It carries no transport detail — how to reach the source is the implementation's business.

type WindowedSource added in v0.2.1

type WindowedSource interface {
	Source
	FetchWindow(ctx context.Context, q WindowQuery) (WindowPage, error)
}

WindowedSource fetches a slice of a larger remote set instead of the whole thing, and lets the server answer the filter and the sort.

It is the third optional extension of Source, alongside StreamingSource, and it earns that the same way: the lifecycle is genuinely different. Source.Fetch says "give me everything"; a windowed source is one where everything is more rows than anyone wants to hold, so the question becomes "give me rows 400-499 matching author=tolkien, newest first" — and the answer changes as the user scrolls and types.

Implementations still satisfy Source: Fetch returns the first page, so `wrangl` and any non-table consumer get something sensible rather than an error. Only the bound table drives FetchWindow.

The types here deliberately mirror tuilib's pkg/source.Query and pkg/table's window setter without importing either — the data layer never imports the TUI (see AGENTS.md). internal/screen owns the translation, which is where both halves are already in scope.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL