content

package
v0.0.0-...-4c3468c Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 2, 2026 License: AGPL-3.0 Imports: 29 Imported by: 0

Documentation

Overview

Package content owns the page-ownership inventory (Track B slice S1): which editor owns each page on a site, and whether the content column is the thing a visitor sees. It refreshes the inventory by sending the agent's content_probe command in list mode, validates everything the site returns, and serves the per-site inventory and the owner fleet report.

Index

Constants

View Source
const (
	// StateOK: the site's agent is at or above the floor.
	StateOK = "ok"
	// StateAgentUpdateNeeded: the site's agent is below the floor.
	StateAgentUpdateNeeded = "agent_update_needed"
	// StateNotConnected: the site cannot be asked right now.
	StateNotConnected = "not_connected"
)

Refresh states reported by the inventory endpoint.

View Source
const MaxInventoryPage = 200

MaxInventoryPage is the largest page the inventory endpoint returns.

View Source
const MinAgentVersionForContentProbe = "0.61.154"

MinAgentVersionForContentProbe is the first agent release that carries the content_probe command (probe v2). A site below it is shown "agent update needed", never an error, and is never sent the command.

Pinned by TestMinAgentVersionForContentProbe_PinnedAboveLiveAgent: it must stay a well-formed dotted version and must not be moved without moving the release that ships the command.

View Source
const RefreshRateWindow = 2 * time.Minute

RefreshRateWindow is how long a site's refresh request blocks another.

View Source
const (

	// RefreshTimeout bounds a whole refresh (the River job timeout).
	RefreshTimeout = maxPages*callTimeout + time.Minute
)
View Source
const SweepInterval = 24 * time.Hour

SweepInterval is how often the sweep runs. Each site is refreshed about once a day: the sweep spreads a site's refresh across the day by a per-site jitter, so the fleet is not all asked at once.

Variables

View Source
var ErrInvalidProbeResponse = errors.New("content_probe response failed validation")

ErrInvalidProbeResponse marks a reply that broke the contract. Nothing from such a reply is stored.

View Source
var ErrNotSendable = errors.New("content refresh not sent")

ErrNotSendable marks a refresh that was declined without contacting the site (agent below the floor, site not connected, paused for a scheduled run). It is an outcome, not a failure: the job does not retry it.

Functions

func AgentMeetsFloor

func AgentMeetsFloor(v string) bool

AgentMeetsFloor reports whether a site's reported agent version is at or above MinAgentVersionForContentProbe. An empty or unparseable version is below it.

func BuildDescriptors

func BuildDescriptors(in []Integration) []json.RawMessage

BuildDescriptors turns allowlist rows into the descriptors sent to the agent. A row whose descriptor does not carry a mode flag and payload keys cannot detect anything and is left out (the seeded rows are like this until their detection data lands): pages of that builder show as an unconfirmed page builder, which is the honest answer. At most 32 are sent.

func BuildIndicators

func BuildIndicators(components []byte, integrations []Integration) agentcmd.ContentProbeIndicators

BuildIndicators derives the site-level builder hints. It sends ONLY hints for builders on the platform allowlist: a plugin directory named by an allowlist row's descriptor.plugin_dir, and the active theme only when it matches an allowlist row's theme_slug. The agent treats every hint it finds as a possible builder, so an ordinary plugin or theme sent here would strip the classic verdict from every page on the site.

func EntrySHA256

func EntrySHA256(in AdminUpsertInput) (string, error)

EntrySHA256 is sha256 over the canonical JSON of a whole allowlist entry: keys sorted at every level, no insignificant whitespace, a fixed field set with nulls kept, so the same entry always hashes the same.

Types

type AdminUpsertInput

type AdminUpsertInput struct {
	ActorUserID            uuid.UUID
	IntegrationID          string
	DisplayName            string
	Enabled                bool
	Status                 string
	Descriptor             []byte
	Abilities              []byte
	MinVersion             *string
	MaxTestedVersion       *string
	MinWPVersion           *string
	IntegrationEntrySHA256 string
	ThemeSlug              *string
	// Present names the optional fields the request carried; the rest keep
	// their stored values. Nil means every field was supplied.
	Present map[string]bool
}

AdminUpsertInput is the superadmin's write. IntegrationEntrySHA256 is computed by the service, never accepted from the caller.

type Enqueuer

type Enqueuer interface {
	EnqueueRefresh(ctx context.Context, args RefreshArgs, scheduleAt time.Time) (queued bool, err error)
}

Enqueuer inserts refresh jobs. The concrete type wraps river.Client.

type FleetBuilderShare

type FleetBuilderShare struct {
	IntegrationID string
	Version       *string
	Pages         int64
	Sites         int64
}

FleetBuilderShare is one (builder, version) line of the fleet report.

type FleetReport

type FleetReport struct {
	ByVerdict []FleetVerdictShare
	ByBuilder []FleetBuilderShare
	Pages     int64
}

FleetReport is the owner report.

type FleetVerdictShare

type FleetVerdictShare struct {
	Verdict     string
	RouteNumber int16
	Pages       int64
	Sites       int64
}

FleetVerdictShare is one (verdict, route) line of the fleet report.

type Handler

type Handler struct {
	// contains filtered or unexported fields
}

Handler serves the content inventory routes.

func NewHandler

func NewHandler(svc *Service) *Handler

NewHandler builds the handler. The enqueuer and recorder are wired after River starts and audit is constructed.

func (*Handler) Register

func (h *Handler) Register(r *gin.RouterGroup)

Register mounts the per-site routes on the authenticated /api/v1 group. RequireSiteAccess gates the by-id group.

func (*Handler) RegisterAdmin

func (h *Handler) RegisterAdmin(g *gin.RouterGroup)

RegisterAdmin mounts the superadmin routes on a group that is ALREADY behind the admin package's requireSuperadmin. Nothing here re-checks the gate; the caller must pass the gated group.

func (*Handler) SetAuditRecorder

func (h *Handler) SetAuditRecorder(r *audit.Recorder)

SetAuditRecorder wires the audit recorder.

func (*Handler) SetEnqueuer

func (h *Handler) SetEnqueuer(e Enqueuer)

SetEnqueuer wires the River enqueuer.

type Integration

type Integration struct {
	ID          string
	DisplayName string
	Status      string
	Enabled     bool
	ThemeSlug   string
	Descriptor  []byte
}

Integration is one enabled row of the global allowlist, as the refresh job needs it.

type IntegrationRecord

type IntegrationRecord struct {
	IntegrationID          string
	DisplayName            string
	Enabled                bool
	Status                 string
	Descriptor             []byte
	Abilities              []byte
	MinVersion             *string
	MaxTestedVersion       *string
	MinWPVersion           *string
	IntegrationEntrySHA256 *string
	ThemeSlug              *string
	UpdatedAt              time.Time
}

IntegrationRecord is one allowlist row as the admin API returns it.

type InventoryPage

type InventoryPage struct {
	State          string
	AgentVersion   string
	MinAgent       string
	Rows           []InventoryRow
	NextAfterPost  *int64
	LastCheckedAt  *time.Time
	TitlesIncluded bool
	Truncated      bool
}

InventoryPage is one page of a site's inventory plus the site's state.

type InventoryRow

type InventoryRow struct {
	PostID             int64
	PostType           string
	PostStatus         string
	Verdict            string
	RouteNumber        int16
	RouteReason        string
	OwnerIntegrationID *string
	OwnerDisplayName   *string
	OwnerVersion       *string
	Title              *string
	CheckedAt          time.Time
}

InventoryRow is a stored page as the API returns it.

type ProbeClient

type ProbeClient interface {
	ContentProbeList(ctx context.Context, siteID uuid.UUID, siteURL string, req agentcmd.ContentProbeRequest) (agentcmd.ContentProbeListResponse, error)
}

ProbeClient is the slice of agentcmd.Client the refresh uses.

type RefreshArgs

type RefreshArgs struct {
	TenantID uuid.UUID `json:"tenant_id"`
	// SiteID is the only field in the unique key: a sweep and an operator
	// refresh of one site are the same work, whoever asked.
	SiteID    uuid.UUID `json:"site_id" river:"unique"`
	Scheduled bool      `json:"scheduled"`
}

RefreshArgs is the River payload for one site's inventory refresh.

func (RefreshArgs) InsertOpts

func (RefreshArgs) InsertOpts() river.InsertOpts

InsertOpts bounds the job: three attempts, no storm. The unique window also carries the rate limit for an operator's Refresh: a second request for the same site inside it is not enqueued.

func (RefreshArgs) Kind

func (RefreshArgs) Kind() string

Kind implements river.JobArgs.

type RefreshResult

type RefreshResult struct {
	Stored         int
	SkippedUnknown int
	Truncated      bool
	CheckedAt      time.Time
}

RefreshResult summarises one refresh.

type RefreshWorker

type RefreshWorker struct {
	river.WorkerDefaults[RefreshArgs]
	// contains filtered or unexported fields
}

RefreshWorker refreshes one site.

func NewRefreshWorker

func NewRefreshWorker(svc *Service, logger *slog.Logger) *RefreshWorker

NewRefreshWorker builds the worker.

func (*RefreshWorker) Timeout

Timeout bounds the whole job.

func (*RefreshWorker) Work

func (w *RefreshWorker) Work(ctx context.Context, job *river.Job[RefreshArgs]) error

Work refreshes one site. A declined refresh (agent below the floor, site not connected, paused) is a normal outcome and is not retried. A refusal the agent marks non-retryable, and a reply that broke the contract, are not retried either: repeating them would send the same request for the same answer. Transport errors and retryable refusals are retried by River with its backoff, at most twice more.

type Repo

type Repo interface {
	// GetSiteTarget reads the facts a refresh needs, in the tenant's scope.
	GetSiteTarget(ctx context.Context, tenantID, siteID uuid.UUID) (SiteTarget, error)
	// ListEnabledIntegrations reads the global allowlist's enabled rows.
	ListEnabledIntegrations(ctx context.Context, tenantID uuid.UUID) ([]Integration, error)
	// ReplaceInventory upserts every row with one checked_at, then deletes the
	// site's older rows, in one tenant transaction.
	// The refresh's run record is written in the same transaction, with the
	// same checked_at.
	ReplaceInventory(ctx context.Context, tenantID, siteID uuid.UUID, checkedAt time.Time, rows []Row, truncated bool, deleteStale bool) error
	// GetRun reads the site's last refresh record; nil means never refreshed.
	GetRun(ctx context.Context, p domain.Principal, siteID uuid.UUID) (*Run, error)
	// ListInventory pages a site's inventory by post id, in the caller's scope.
	ListInventory(ctx context.Context, p domain.Principal, siteID uuid.UUID, afterPostID int64, owner *string, limit int32) ([]InventoryRow, error)
	// FleetReport reads the cross-tenant counts through the database's
	// count-only SECURITY DEFINER functions. No session can read another
	// tenant's inventory rows; the caller must already be gated to the
	// platform owner.
	FleetReport(ctx context.Context, actor uuid.UUID) ([]FleetVerdictShare, []FleetBuilderShare, error)
	// ListSweepSites enumerates the connected, unpaused sites across tenants.
	ListSweepSites(ctx context.Context) ([]SweepSite, error)
	// AdminUpsertIntegration calls the superadmin-only SQL writer.
	// The stored row is read, merged by finalize and written in ONE transaction
	// under a per-integration lock, so concurrent updates cannot re-write a
	// stale row. existing is nil for a new integration.
	AdminUpsertIntegration(ctx context.Context, in AdminUpsertInput, finalize func(existing *IntegrationRecord, in AdminUpsertInput) (AdminUpsertInput, error)) (IntegrationRecord, error)
	// ListIntegrations reads every allowlist row (enabled or not).
	ListIntegrations(ctx context.Context, actor uuid.UUID) ([]IntegrationRecord, error)
}

Repo is the storage the service needs. Every method runs inside a tx helper; the repo never sets a GUC itself.

func NewRepo

func NewRepo(pool *db.Pool) Repo

NewRepo builds the Postgres repo.

type RiverEnqueuer

type RiverEnqueuer struct {
	// contains filtered or unexported fields
}

RiverEnqueuer enqueues onto River.

func NewRiverEnqueuer

func NewRiverEnqueuer(c *river.Client[pgx.Tx]) *RiverEnqueuer

NewRiverEnqueuer wraps the started River client.

func (*RiverEnqueuer) EnqueueRefresh

func (e *RiverEnqueuer) EnqueueRefresh(ctx context.Context, args RefreshArgs, scheduleAt time.Time) (bool, error)

EnqueueRefresh inserts one refresh job. queued is false when an equal job was already inside the unique window.

type Row

type Row struct {
	PostID      int64
	PostType    string
	PostStatus  string
	Verdict     string
	RouteNumber int16
	RouteReason string
	OwnerID     string
	OwnerName   string
	OwnerVer    string
	Title       string
}

Row is one validated page ready to store. Every string has been shape checked; Title has been through humantext.Clean and capped.

func ValidateListResponse

func ValidateListResponse(resp agentcmd.ContentProbeListResponse, limit int, allowedTypes []string, names map[string]string) (rows []Row, skippedUnknown int, err error)

ValidateListResponse checks a list-mode reply against the closed sets and shapes above.

A structural violation (wrong probe version or mode, too many rows, a malformed id, type, status or owner, a duplicate id) fails the whole reply with ErrInvalidProbeResponse: the reply is not trustworthy as a whole. A verdict or route reason outside the known set is a newer agent speaking a word this control plane does not know yet; that row alone is skipped and counted, so the rest of the site is still recorded.

names maps an integration id to the display name the platform allowlist gave it. A display name is never taken from the site.

type Run

type Run struct {
	CheckedAt   time.Time
	PagesStored int32
	Truncated   bool
}

Run is the record of a site's last refresh.

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service holds the inventory business logic.

func NewService

func NewService(repo Repo, agent ProbeClient, logger *slog.Logger) *Service

NewService builds the service. agent may be nil in a build without the agent command client; a refresh then reports the feature as unavailable.

func (*Service) Fleet

func (s *Service) Fleet(ctx context.Context, actor uuid.UUID) (FleetReport, error)

Fleet aggregates every tenant's inventory. The caller must already be a superadmin; nothing here checks that.

func (*Service) Inventory

func (s *Service) Inventory(ctx context.Context, p domain.Principal, siteID uuid.UUID, after int64, owner string, limit int, withTitles bool) (InventoryPage, error)

Inventory returns one page of a site's inventory. withTitles says whether the caller holds the content-read permission; without it titles are not returned at all.

func (*Service) ListIntegrations

func (s *Service) ListIntegrations(ctx context.Context, actor uuid.UUID) ([]IntegrationRecord, error)

ListIntegrations reads the whole allowlist for the admin screen.

func (*Service) Refresh

func (s *Service) Refresh(ctx context.Context, tenantID, siteID uuid.UUID, scheduled bool) (RefreshResult, error)

Refresh replaces a site's inventory with a fresh probe. scheduled runs are skipped for a paused site; an operator's Refresh never is.

func (*Service) RequestRefresh

func (s *Service) RequestRefresh(ctx context.Context, enq Enqueuer, tenantID, siteID uuid.UUID) error

RequestRefresh enqueues an operator's refresh now. It returns a rate-limit error when a refresh for the site is already inside the unique window.

func (*Service) SetClock

func (s *Service) SetClock(now func() time.Time)

SetClock overrides the time source (tests).

func (*Service) UpsertIntegration

func (s *Service) UpsertIntegration(ctx context.Context, in AdminUpsertInput) (IntegrationRecord, error)

UpsertIntegration validates and writes one allowlist row through the superadmin-only SQL writer. The caller must already have passed requireSuperadmin; the SQL function checks the actor again.

type SiteTarget

type SiteTarget struct {
	URL             string
	AgentVersion    string
	ConnectionState string
	Enrolled        bool
	Paused          bool
	Components      []byte
}

SiteTarget is what the refresh needs to know about a site.

type SweepArgs

type SweepArgs struct{}

SweepArgs is the periodic sweep payload; it has no fields.

func (SweepArgs) Kind

func (SweepArgs) Kind() string

Kind implements river.JobArgs.

type SweepSite

type SweepSite struct {
	TenantID uuid.UUID
	SiteID   uuid.UUID
}

SweepSite is one site the periodic sweep enqueues a refresh for.

type SweepWorker

type SweepWorker struct {
	river.WorkerDefaults[SweepArgs]
	// contains filtered or unexported fields
}

SweepWorker enqueues a refresh for every connected, unpaused site.

func NewSweepWorker

func NewSweepWorker(svc *Service, logger *slog.Logger) *SweepWorker

NewSweepWorker builds the sweep worker. The enqueuer is set after River starts.

func (*SweepWorker) SetEnqueuer

func (w *SweepWorker) SetEnqueuer(e Enqueuer)

SetEnqueuer wires the River enqueuer after the client has started.

func (*SweepWorker) Work

func (w *SweepWorker) Work(ctx context.Context, _ *river.Job[SweepArgs]) error

Work enumerates the sites and enqueues one jittered refresh for each. One site failing to enqueue does not stop the rest.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL