hub

package
v0.83.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 13, 2026 License: MIT Imports: 14 Imported by: 0

Documentation

Overview

Package hub is the engine's search and signal planes for one tenant: keyword search and typeahead, and the ClickHouse signal plane. The discovery routes and the engine's jobs use it; hosts reach it only through ContentKit's API.

Index

Constants

View Source
const PreferenceEventID = "current"

PreferenceEventID is the stable signal identity of a subject's current preference on one axis: (tenant, canonical reference, subject, axis, "current"). A newer revision supersedes the previous one; a re-send carries the same revision and converges.

Variables

View Source
var ErrSignalPlaneDisabled = errors.New("contentkit: signal plane disabled (no ClickHouse configured)")

ErrSignalPlaneDisabled is returned by signal methods when the hub was constructed without a ClickHouse connection.

Functions

func NewEvalRunner

func NewEvalRunner(client *Client, base SearchOptions) eval.CaseRunner

NewEvalRunner adapts a Client to eval.CaseRunner so a golden suite can be executed against real search. The base options carry cross-case settings (LanguageMode and filters); each case overrides Language, ContentKinds, and Limit from its own definition.

This adapter is the single seam where the client meets the dependency-free eval package.

Types

type CandidateTrace

type CandidateTrace struct {
	Key   TraceKey `json:"key"`
	Rank  int      `json:"rank"`
	Score float32  `json:"score"`
}

CandidateTrace records one source candidate at its raw source rank.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client answers keyword search and typeahead for one tenant.

func NewClient

func NewClient(cfg ClientConfig) (*Client, error)

func (*Client) Search

func (c *Client) Search(ctx context.Context, userText string, opts SearchOptions) (SearchResult, error)

Search returns one page of content items. Documents from every searched language are grouped per work before Offset and Limit apply; each hit carries the matched document's reference and language.

func (*Client) SearchWithTrace

func (c *Client) SearchWithTrace(ctx context.Context, userText string, opts SearchOptions) (SearchResult, SearchTrace, error)

SearchWithTrace executes Search and returns opt-in retrieval provenance. On failure, the returned trace contains all work completed before the error.

func (*Client) Tenant

func (c *Client) Tenant() string

Tenant returns the tenant this client is scoped to.

func (*Client) Typeahead

func (c *Client) Typeahead(ctx context.Context, userText string, opts TypeaheadOptions) ([]TypeaheadHit, error)

Typeahead returns suggestions while a user is typing (typos/substring matching), one per content item, grouped before Limit.

type ClientConfig

type ClientConfig struct {
	Pool   *pgxpool.Pool
	Schema string
	// Tenant scopes every document, query and result. Required.
	Tenant string

	// Defaults.
	DefaultLanguage string
	DefaultLimit    int
}

ClientConfig configures the keyword search client of one tenant.

type Config

type Config struct {
	// Content plane (Postgres). PG + PGSchema (ContentKit's schema) are required.
	PG       *pgxpool.Pool
	PGSchema string

	// Content-plane defaults (as in ClientConfig).
	DefaultLanguage string
	DefaultLimit    int
	// DefaultRRFK controls deterministic popularity fusion in personalized search (default 60).
	DefaultRRFK int

	// Signal plane (ClickHouse). Optional: omit CH to run content-only
	// (signal methods return ErrSignalPlaneDisabled). CHDatabase
	// is the hub's dedicated ClickHouse database; apply
	// migrations.ClickHouse and gate startup on signal.CheckSchema.
	CH         signal.Conn
	CHDatabase string

	// Tenant is the single tenant of this embedded hub. Required: every
	// document, signal, cursor and result carries it.
	Tenant string
}

Config configures one tenant's hub.

type ContentKey

type ContentKey = contentref.ContentKey

ContentKey is the comparable form of a ContentRef.

type ContentRef

type ContentRef = contentref.ContentRef

ContentRef is the tenant-scoped reference to a content item or one of its versions. See contentref.

type ContributionTrace

type ContributionTrace struct {
	SourceIndex  int     `json:"source_index"`
	SourceRank   int     `json:"source_rank"`
	Weight       float32 `json:"weight"`
	Contribution float32 `json:"contribution"`
}

ContributionTrace records one exact source contribution to a result score.

type DocumentKey

type DocumentKey = search.DocumentKey

DocumentKey identifies one keyword document: a ContentRef in one language.

type Eligibility

type Eligibility = search.Eligibility

Eligibility is the per-document visibility join; see search.Eligibility.

type EmptyReason

type EmptyReason string

EmptyReason explains a successful empty response.

const (
	EmptyReasonNormalizedQuery EmptyReason = "normalized_query_empty"
	EmptyReasonNoCandidates    EmptyReason = "no_candidates"
)

type GroupID

type GroupID = contentref.GroupID

GroupID identifies a group (tag, artist, series, character, channel, rail).

type Hub

type Hub struct {
	// contains filtered or unexported fields
}

Hub is one tenant's search client and signal plane: the engine's internal surface the discovery routes build on.

func New

func New(cfg Config) (*Hub, error)

New builds the hub of one tenant.

func (*Hub) Attribution

func (h *Hub) Attribution(ctx context.Context, opts signal.AttributionOptions) (signal.AttributionPage, error)

Attribution exports renders at one stage with their clicks joined (see signal.Store.Attribution); the evaluation dataset source.

func (*Hub) Client

func (h *Hub) Client() *Client

Client returns the underlying content-plane client (advanced use).

func (*Hub) Content

func (h *Hub) Content(contentKind, contentID string) ContentRef

Content returns a reference to a work of this hub's tenant.

func (*Hub) EnforceErasures

func (h *Hub) EnforceErasures(ctx context.Context) (signal.ErasureReport, error)

EnforceErasures physically removes residue of every recorded erasure of this tenant (see signal.Store.EnforceErasures): schedule it and run it after every restore.

func (*Hub) EraseSubjects

func (h *Hub) EraseSubjects(ctx context.Context, subjects []signal.Subject) (signal.ErasureReport, error)

EraseSubjects permanently erases subjects from this tenant's signal plane: see signal.Store.EraseSubjects for the completion contract. Shared accounts exist in several tenants: each host erases its own tenant.

func (*Hub) Forget

func (h *Hub) Forget(ctx context.Context, subject signal.Subject, contentKind, contentID string) error

Forget erases the subject's signals for one work and its versions (contentID set) or a whole content kind (contentID empty) — host "clear my history" support.

func (*Hub) ForgetExposures

func (h *Hub) ForgetExposures(ctx context.Context, subject signal.Subject) error

ForgetExposures clears a subject's result-list exposures (search history).

func (*Hub) ForgetExposuresBefore

func (h *Hub) ForgetExposuresBefore(ctx context.Context, subject signal.Subject, before time.Time) error

ForgetExposuresBefore clears only exposures from before the host's clear request.

func (*Hub) History

func (h *Hub) History(ctx context.Context, subject signal.Subject, opts signal.HistoryOptions) ([]signal.StateRow, error)

func (*Hub) HistoryCount

func (h *Hub) HistoryCount(ctx context.Context, subject signal.Subject, opts signal.HistoryOptions) (int64, error)

HistoryCount returns the total row count History would paginate over.

func (*Hub) Inventory

func (h *Hub) Inventory(ctx context.Context) ([]signal.InventoryRow, error)

Inventory reports canonical event volume per content kind and signal type.

func (*Hub) Metrics

func (h *Hub) Metrics(ctx context.Context, refs []ContentRef, window signal.Window) (map[ContentKey]signal.ContentMetrics, error)

Metrics returns named window metrics (viewers, views, completions, feedback, ...) for the references (works or versions).

func (*Hub) Popular

func (h *Hub) Popular(ctx context.Context, contentKind string, opts signal.PopularOptions) ([]signal.PopularHit, error)

func (*Hub) PopularityFor

func (h *Hub) PopularityFor(ctx context.Context, contentKind string, ids []string, window signal.Window) (map[string]float64, error)

PopularityFor scores a fixed candidate set (work ids of one kind) by the popularity ranking, returning content_id -> score. Use to rank a host- filtered universe (e.g. "galleries of artist X by popularity").

func (*Hub) PurgeContentKinds

func (h *Hub) PurgeContentKinds(ctx context.Context, contentKinds []string) error

PurgeContentKinds irreversibly deletes whole content kinds from this tenant's signal plane (see signal.Store.PurgeContentKinds).

func (*Hub) RecordExposures

func (h *Hub) RecordExposures(ctx context.Context, exposures []signal.Exposure) error

RecordExposures logs one row per result list and stage (served, rendered, visible) so clicks can be attributed to what was actually exposed. Hosts call it once per list per stage, never per item.

func (*Hub) RecordSignals

func (h *Hub) RecordSignals(ctx context.Context, signals []signal.Signal) error

RecordSignals records the batch (see signal.Store.RecordSignals).

func (*Hub) RefreshCoEngagement

func (h *Hub) RefreshCoEngagement(ctx context.Context, opts signal.RefreshCoEngagementOptions) error

RefreshCoEngagement (re)materializes the content_pairs co-engagement rollup for this tenant (see signal.Store.RefreshCoEngagement). Run periodically.

func (*Hub) RepairProjections

func (h *Hub) RepairProjections(ctx context.Context, opts signal.RepairOptions) (signal.RepairResult, error)

RepairProjections is the bounded, host-scheduled projection repair (see signal.Store.RepairProjections): run it periodically with IngestedSince for crash repair, and with Rebuild over a window after projection changes.

func (*Hub) Search

func (h *Hub) Search(ctx context.Context, userText string, opts HubSearchOptions) (SearchResult, error)

func (*Hub) SeenIDs

func (h *Hub) SeenIDs(ctx context.Context, subject signal.Subject, contentKind string) (map[string]struct{}, error)

SeenIDs returns the subject's seen-set for one content kind (the signal-plane half of an unseen anti-join).

func (*Hub) SendPreferences

func (h *Hub) SendPreferences(ctx context.Context, page []content.Preference) error

SendPreferences writes one page into the signal plane: one event per subject × reference × axis with the row's revision, change time and value (0 = neutral / unfavorited). Erased subjects are dropped by the store. It is content's PreferenceSender.

func (*Hub) SignalPlane

func (h *Hub) SignalPlane() bool

SignalPlane reports whether the hub has a ClickHouse signal plane.

func (*Hub) Signals

func (h *Hub) Signals() *signal.Store

Signals is the signal plane's store; nil without one.

func (*Hub) States

func (h *Hub) States(ctx context.Context, subject signal.Subject, refs []ContentRef) (map[ContentKey]signal.State, error)

func (*Hub) Tenant

func (h *Hub) Tenant() string

Tenant returns the pinned tenant value.

func (*Hub) Typeahead

func (h *Hub) Typeahead(ctx context.Context, userText string, opts TypeaheadOptions) ([]TypeaheadHit, error)

type HubSearchOptions

type HubSearchOptions struct {
	SearchOptions
	Personalize *Personalization
}

HubSearchOptions extends content SearchOptions with optional signal-aware personalization.

type KeywordDocument

type KeywordDocument = search.KeywordDocument

KeywordDocument is the host's canonical search input for one document.

type LanguageMode

type LanguageMode string
const (
	// LanguageModeExact uses only the requested language.
	LanguageModeExact LanguageMode = "exact"
	// LanguageModeFallbackEnglish uses requested language first, then English.
	LanguageModeFallbackEnglish LanguageMode = "fallback_en"
)

type Personalization

type Personalization struct {
	Subject signal.Subject

	// PopularityWeight is the RRF weight of the candidate-set popularity
	// list blended with the content ranking. Defaults to 0.25.
	PopularityWeight float32
	// PopularityWindow bounds candidate popularity (zero = all time).
	PopularityWindow signal.Window

	// AffinityWeight boosts works the subject already engaged with by
	// (1 + AffinityWeight·last_score/100). 0 = off.
	AffinityWeight float32

	// DemoteSeen demotes already-seen / completed works.
	DemoteSeen bool
	// SeenPenalty multiplies seen-but-not-completed scores (default 0.85).
	SeenPenalty float32
	// CompletedPenalty multiplies completed scores (default 0.6).
	CompletedPenalty float32
	// DislikePenalty multiplies works the subject has net-negative explicit
	// feedback for (default 0.3). Always applied when view context is loaded
	// (i.e. AffinityWeight > 0 or DemoteSeen).
	DislikePenalty float32
}

Personalization fuses signal aggregates into search ranking. Recall is unchanged: a ranking-only layer over the candidate set, which discovery applies for signed-in readers.

type ResultTrace

type ResultTrace struct {
	Key           TraceKey            `json:"key"`
	Rank          int                 `json:"rank"`
	Score         float32             `json:"score"`
	ScoreKind     ScoreKind           `json:"score_kind"`
	Contributions []ContributionTrace `json:"contributions"`
}

ResultTrace records one returned item and its best document's contributions.

type RetrievalBackend

type RetrievalBackend string

RetrievalBackend identifies one candidate source.

const (
	BackendKeyword RetrievalBackend = "keyword"
)

type ScoreKind

type ScoreKind string

ScoreKind identifies the numeric domain of a candidate or result score.

const (
	ScoreKeywordMatch ScoreKind = "keyword_match"
)

type SearchHit

type SearchHit struct {
	// ContentRef names the matched document's content: the work, or the
	// version when the document was indexed per version.
	ContentRef
	// Language is the matched document's language.
	Language string
	// Score ranks the item by its best matching document in any searched
	// language, using the keyword match tier.
	Score float32
}

SearchHit is one content item, represented by its matched document.

type SearchOptions

type SearchOptions struct {
	Language string
	// Defaults to LanguageModeExact when omitted.
	LanguageMode LanguageMode

	// ContentKinds selects the kinds searched. Required.
	ContentKinds []string

	// Limit is the page size in content items; Offset skips items. Documents
	// are grouped per item before either applies.
	Limit  int
	Offset int
	// CandidateLimit is the document window requested from each retrieval
	// source (per language) before grouping. It defaults to twice Offset+Limit
	// (at least 100) and is clamped to at least Offset+Limit. Pass the same
	// value on every page when a truncated window must stay identical.
	CandidateLimit int

	// Eligibility keeps the documents the reader may see (the discovery
	// routes' visibility join); without it every document is eligible.
	Eligibility *Eligibility
}

type SearchResult

type SearchResult struct {
	Hits []SearchHit
	// HasMore is true when items follow this page in the grouped retrieval, or
	// when Truncated: documents beyond the window were never ranked, so the
	// next page may still be non-empty.
	HasMore bool
	// Truncated reports that a candidate window filled. Raise CandidateLimit
	// for complete deep pagination.
	Truncated bool
}

SearchResult is one page of items.

type SearchTrace

type SearchTrace struct {
	NormalizedQuery         string        `json:"normalized_query"`
	RequestedLanguage       string        `json:"requested_language,omitempty"`
	RequestedLanguageMode   LanguageMode  `json:"requested_language_mode,omitempty"`
	Languages               []string      `json:"languages,omitempty"`
	RequestedResultLimit    int           `json:"requested_result_limit"`
	ResultLimit             int           `json:"result_limit"`
	RequestedCandidateLimit int           `json:"requested_candidate_limit"`
	CandidateLimit          int           `json:"candidate_limit"`
	Sources                 []SourceTrace `json:"sources,omitempty"`
	Results                 []ResultTrace `json:"results,omitempty"`
	EmptyReason             EmptyReason   `json:"empty_reason,omitempty"`
	ErrorCategory           string        `json:"error_category,omitempty"`
}

SearchTrace contains opt-in effective configuration and retrieval provenance.

type SourceStatus

type SourceStatus string

SourceStatus records whether an attempted retrieval source succeeded.

const (
	SourceStatusSucceeded SourceStatus = "succeeded"
	SourceStatusFailed    SourceStatus = "failed"
)

type SourceTrace

type SourceTrace struct {
	Backend       RetrievalBackend `json:"backend"`
	Language      string           `json:"language"`
	ScoreKind     ScoreKind        `json:"score_kind"`
	Limit         int              `json:"limit"`
	Status        SourceStatus     `json:"status"`
	ErrorCategory string           `json:"error_category,omitempty"`
	Candidates    []CandidateTrace `json:"candidates,omitempty"`
}

SourceTrace records one language-specific keyword retrieval and its candidates.

type TraceKey

type TraceKey struct {
	ContentRef
	Language string `json:"language"`
}

TraceKey identifies a document in retrieval provenance.

type TypeaheadHit

type TypeaheadHit struct {
	ContentRef
	Language string
	Score    float32
}

TypeaheadHit is one suggested content item and its matched document.

type TypeaheadOptions

type TypeaheadOptions struct {
	Language string
	// Defaults to LanguageModeExact when omitted.
	LanguageMode  LanguageMode
	ContentKinds  []string
	Limit         int
	MinSimilarity float32
	// Eligibility groups suggestions per content item; see SearchOptions.
	Eligibility *Eligibility
}

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL