recordresults

package
v0.1.38 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 22, 2026 License: Apache-2.0 Imports: 30 Imported by: 0

Documentation

Overview

Package recordresults serves record streams through the profile engine: one read-only `sql` profile per result type, over the sqlite index a recordstore.Indexer keeps caught up, addressed by a stream param.

One profile per type rather than per stream is the point. Every profile is a catalog entry, an OpenAPI path and a sidebar item, so a profile per stream would grow all three with every capture; a stream param grows none of them. Paging, column filters, filter value lookups, sort and export are the engine's own, applied to the index by SQL.

Index

Constants

View Source
const FollowRecheckInterval = 5 * time.Second

FollowRecheckInterval is how often a follow confirms, while nothing is appended, that its stream still exists. It bounds how long a follow of an expired or removed stream runs on; rows appended through Results.Backend arrive without waiting for it.

View Source
const MaxExportRows = 1_000_000

MaxExportRows is where an all-row export of a result stops. It is well above the engine default because a capture is read whole far more often than a table is: a trace of a busy run is tens of thousands of rows.

View Source
const (
	// ProviderType is the provider a followable result type's profile reads
	// through (ResultType.Follow). It pages, filters, looks values up and exports
	// exactly as the sqlite provider it delegates to, and it also streams: a
	// session started with follow=true tails the stream it names.
	ProviderType = "recordstore"
)

Variables

This section is empty.

Functions

func RegisterResultType

func RegisterResultType[T any](registry *Registry, resultType ResultType[T]) error

RegisterResultType declares T's kind: its columns (reflected through query.ColumnsFor) become the kind's schema and index table, and a profile <prefix>/<kind> serves its streams.

Types

type HierarchyColumns added in v0.1.38

type HierarchyColumns struct {
	ID     string
	Parent string
}

HierarchyColumns names the stored relationship used by a root-only read.

type OpenOptions

type OpenOptions struct {
	// Prefix and ConnectionName name the result profiles and the index
	// connection, as RegistryOptions does.
	Prefix         string
	ConnectionName string

	// Settings say where the files live and how long a stream is kept. With no
	// Source, Settings.Backend is the local store to open, sqlite or ndjson —
	// resolve it first (Settings.Resolve).
	Settings recordstore.Settings

	// Source is a backend the caller routes itself, typically a
	// recordstore.Router over a kv store per tenant, or nil to open
	// Settings.Backend locally. A Source is always mirrored into a derived
	// index of its own, never into a file a route could also write, so one
	// route's streams cannot be read through another. Open owns Source: Close,
	// or a failed Open, closes it.
	Source recordstore.Backend

	// Schemas is the kind catalog the index and Register share. Pass one when
	// Source opens stores of its own that need it — a route's local sqlite
	// file resolves kinds the registry declares; nil makes a fresh one.
	Schemas *recordstore.Schemas

	// Register declares the result types the registry serves.
	Register func(*Registry) error
}

OpenOptions configure Open.

type RegisteredResultType

type RegisteredResultType struct {
	Kind    string `json:"kind"`
	Title   string `json:"title"`
	Profile string `json:"profile"`

	Views []RegisteredResultView `json:"views,omitempty"`
}

RegisteredResultType is one registered result type as a caller finds it.

type RegisteredResultView added in v0.1.38

type RegisteredResultView struct {
	Name    string `json:"name"`
	Title   string `json:"title"`
	Profile string `json:"profile"`
}

RegisteredResultView is one view of a registered result type.

type Registry

type Registry struct {
	// contains filtered or unexported fields
}

Registry is the result types a server serves: a profiles.VirtualStore of their profiles, the resolver of the index connection those profiles read, and the BeforeExecute hook that catches the index up first.

func NewRegistry

func NewRegistry(options RegistryOptions) (*Registry, error)

NewRegistry validates options and returns an empty registry.

func (*Registry) BeforeExecute

func (r *Registry) BeforeExecute(ctx context.Context, reads []profiles.ReadRequest) (func(), error)

BeforeExecute is the profiles.BeforeExecuteFunc for result profiles: it catches every requested stream up, then holds one lease across the whole read batch. Revalidation under that lease closes the gap in which a sweep can remove an index stream after Ensure returns but before its query starts. A stream that does not exist, or that holds another result type, is profiles.ErrProfileDataNotFound — an empty page would say "nothing matched" about a stream nobody wrote. Every other profile passes through untouched.

func (*Registry) BeforeRead

func (r *Registry) BeforeRead(ctx context.Context, p query.Profile, params map[string]any) (func(), error)

BeforeRead is the query.RegistryOptions.BeforeRead hook for a session registry serving result profiles. A host mounts the sessions API over result profiles as:

registry := query.NewSessionRegistry(query.RegistryOptions{BeforeRead: results.Registry.BeforeRead})
service, err := sessions.New(sessions.Options{Profiles: ..., Context: ..., Registry: registry})
handler, err := service.Handler("/api/v1", profileHandler)

where Profiles overlays results.Registry on the profile store and Context carries results.Registry.ResolveConnection, as for the profile service.

It catches the index up and checks the stream exactly as BeforeExecute does for a page. A top session keeps the lease for its sample. A trace — a follow — gives it back at once: it reads for as long as it lasts, a lease held that long would hold every append off, and the follow provider takes its own lease around each read.

func (*Registry) Close

func (r *Registry) Close() error

Close stops the registry answering follows. It closes nothing it was given: the index and source belong to whoever opened them.

func (*Registry) Delete

func (r *Registry) Delete(context.Context, string) error

func (*Registry) Get

func (r *Registry) Get(_ context.Context, name string) (query.Profile, error)

Get returns the result profile named name.

func (*Registry) IsVirtual

func (r *Registry) IsVirtual(name string) bool

IsVirtual reports whether name is a result profile.

func (*Registry) List

func (r *Registry) List(context.Context) ([]query.Profile, error)

List returns every result profile by name.

func (*Registry) Peek

func (r *Registry) Peek(ctx context.Context, name string) (query.Profile, error)

Peek is Get: reading a result profile has no expiry to slide.

func (*Registry) ResolveConnection

func (r *Registry) ResolveConnection(reference string) (*models.Connection, error)

ResolveConnection is the dbcontext.ConnectionResolver for the index connection. Any other reference is not the registry's to answer, so it reports none and resolution carries on.

func (*Registry) ResultTypes

func (r *Registry) ResultTypes() []RegisteredResultType

ResultTypes lists the registered result types by kind.

func (*Registry) Save

func (*Registry) Update

type RegistryOptions

type RegistryOptions struct {
	// Prefix names every result profile <prefix>/<kind>, and namespaces the
	// index connection.
	Prefix string

	// Schemas is the kind catalog Index resolves kinds through.
	// RegisterResultType fills it.
	Schemas *recordstore.Schemas

	// Index is the sqlite file result profiles read. It must have been opened
	// with Schemas.Kind as its schema.
	Index *sqlite.Backend

	// Source is the authoritative backend result streams are written to. The
	// registry constructs the Indexer that mirrors it into Index, so preparation
	// and profile reads cannot accidentally name different indexes. A result
	// type that follows its streams (ResultType.Follow) needs a
	// *recordstore.Notifier here, which captures must append through.
	Source recordstore.Backend

	// ConnectionName names the index's virtual connection,
	// connection://<prefix>/<name>.
	ConnectionName string
}

RegistryOptions configure a Registry.

type ResultType

type ResultType[T any] struct {
	// Kind is the stream kind, and the last segment of the profile name.
	Kind string

	// Title is the human name of the result type.
	Title string

	// TimeColumn, when set, names T's datetime column: it becomes the table's
	// timestamp, the rows are ordered newest first, and the profile takes a
	// from/to time window over it in place of the column's own filter. Without
	// it rows are in seq order and there is no time window.
	TimeColumn string

	// DefaultFrom is where the time window starts when a request names no
	// from — date math such as now-12h, or RFC3339. Empty leaves the window
	// open, so a request with no from reads the whole stream. It needs a
	// TimeColumn.
	DefaultFrom string

	// KeyColumn, when set, names T's string column identifying a row within a
	// stream: a stream holds each key once, and appending a row whose key it
	// already holds skips the row (recordstore.KindOptions.Key).
	KeyColumn string

	// Retention says how long a stream of the type keeps its rows
	// (recordstore.KindOptions.Retention).
	Retention recordstore.Retention

	// Follow lets a session tail the type's streams: its profile reads through
	// the ProviderType provider, which streams each appended row, rather than
	// plain sqlite. Only a type whose rows each stand alone should follow — one
	// listed as the latest row per id would stream superseded rows.
	Follow bool

	// SearchColumns, when set, names T's text columns (string or JSON) a
	// search matches: the profile takes a search-role param q, and keeps the
	// rows where any of them contains its text, ignoring case.
	SearchColumns []string

	// Hierarchy identifies an invocation and its caller for root-only reads.
	Hierarchy *HierarchyColumns

	// Views are SQL profiles over the type's streams, each served as
	// <prefix>/<kind>/<view>. They read the index exactly as the type's own
	// profile does, but never follow, search or read roots only.
	Views []ResultView
}

ResultType declares one kind of record a stream holds, and the profile that serves it.

type ResultView added in v0.1.38

type ResultView struct {
	// Name is a SQL identifier: the last segment of the view's profile,
	// <prefix>/<kind>/<name>, and the CTE a view using it reads.
	Name  string `yaml:"name"`
	Title string `yaml:"title"`

	// Uses names views of the same type compiled in first, AS MATERIALIZED.
	Uses []string `yaml:"uses,omitempty"`

	// Query is a SELECT, optionally WITH …, over stream_rows and the views it
	// uses. A view aggregates with GROUP BY: a correlated join or anti-join over
	// a stream of a hundred thousand rows runs for minutes.
	Query string `yaml:"query"`

	// Params are the view's own, in addition to stream, afterSeq and toSeq.
	Params []query.ParamDef `yaml:"params,omitempty"`

	Columns []query.ColumnDef `yaml:"columns"`

	// Order must end in a unique column, which is what pages it.
	Order query.Order `yaml:"order"`
}

ResultView is a named SQL profile over one result type's streams. Its query reads the stream from the CTE stream_rows: the kind's table narrowed to {{.params.stream}} and the (afterSeq, toSeq] window. The registry, not the view author, owns that predicate, because the index holds every environment's streams and a view that forgot it would read them all.

type Results

type Results struct {
	// Backend is what a capture appends its rows to. Appending through it wakes
	// every session following the stream.
	Backend *recordstore.Notifier

	// Registry serves the result types over the index.
	Registry *Registry
	// contains filtered or unexported fields
}

Results is an open result store: the backend captures append to, and the registry that serves what they appended through the profile engine.

func Open

func Open(options OpenOptions) (*Results, error)

Open opens the store options describe:

Source   Settings.Backend  backend        index
nil      sqlite            records.sqlite the same file
nil      ndjson            ndjson/        index.sqlite, derived
a Router (any)             the Router     index.sqlite, derived

func (*Results) Close

func (r *Results) Close() error

Close closes everything Open opened, and the Source it was given, in reverse order.

func (*Results) DeleteStream added in v0.1.38

func (r *Results) DeleteStream(ctx context.Context, stream, kind string) error

DeleteStream removes a stream of the expected kind from its source and index.

func (*Results) Ref added in v0.1.38

func (r *Results) Ref(ctx context.Context, stream string, from, to int64) (StreamRef, error)

Ref describes the window from..to of stream from the backend's own metadata and the store the backend reports holding it. A from or to of 0 means the stream's low or high seq. A window reaching outside the rows the stream holds is refused rather than clamped: a ref promises its rows exist.

type StoreLocation added in v0.1.38

type StoreLocation struct {
	Backend recordstore.BackendKind `json:"backend"`
	Host    string                  `json:"host,omitempty"`
	File    string                  `json:"file,omitempty"`
}

StoreLocation is where a stream's rows are: the backend kind, and for a local file the host and file holding them. A kv stream has neither; it is read back through the store the environment names.

type StreamRef added in v0.1.38

type StreamRef struct {
	Stream string `json:"stream"`
	Kind   string `json:"kind"`

	// Generation is the incarnation of the stream the seqs belong to.
	Generation string `json:"generation"`

	// Low and High are the committed seq bounds when the ref was taken.
	Low  int64 `json:"low"`
	High int64 `json:"high"`

	// From and To are the inclusive seq window the ref names: a step's
	// checkpoint, or Low..High.
	From int64 `json:"from"`
	To   int64 `json:"to"`

	Total     int64      `json:"total"`
	ExpiresAt *time.Time `json:"expiresAt,omitempty"`

	Store StoreLocation `json:"store"`
}

StreamRef points at a window of a record stream as the store holding it described the stream: what a capture reports instead of carrying its rows, and what a reader replays them from after the capturing process is gone.

func (StreamRef) EventsRef added in v0.1.38

func (r StreamRef) EventsRef() *query.EventsRef

EventsRef is the ref as a session status records it. Its JSON is the ref's own, field for field.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL