analytics

Overview ¶

Package analytics mounts the Hanzo Cloud /v1/analytics/* surface: a native-Go, per-org analytics read API over the `hanzo` datastore warehouse (the `datastore` cluster). It is the backend for the console Native Analytics module (unified-analytics.md §5) — two read lenses over one warehouse:

• LLM lens (REAL today): hanzo.cloud_usage, the live per-org usage ledger the cloud o11y path already writes (requests, tokens, spend, models, errors).
• Web/commerce lens (honest-empty until the collector emits): hanzo.events.

ONE datastore client. This package does NOT open its own connection: it reads through clients/datastore, the leaf that holds the warehouse connection for the whole binary and opens it from the environment on first use. DRY: one transport, one pool, one set of KMS-injected DATASTORE_* creds — never hard-coded, never a second design. (It used to reach the connection through ai/object's Bootstrap; that path cost 1,933 packages to call four functions, which is why the connection moved to a leaf importing only orm/datastore.)

TENANT ISOLATION is the security bar and is enforced SERVER-SIDE on every request. The org is c.Org() — the value SanitizeIdentity minted from the VALIDATED bearer owner claim (HIP-0026), never a client header — AND every request must carry a validated principal (c.User() set, which SanitizeIdentity sets ONLY for a verified bearer). This closes the Phase-1 "no-bearer + forged X-Org-Id direct-to-pod" cross-tenant read exactly as clients/s3 does. Every datastore query binds the org POSITIONALLY (query.go llmWhere/eventsWhere), so a maxpower token can NEVER read another org's analytics.

Surface (all org-scoped; /v1 only; read-only):

GET /v1/analytics/overview     per-org KPIs (llm real; web/commerce honest-empty)
GET /v1/analytics/timeseries   requests/tokens/spend over time (hour|day buckets)
GET /v1/analytics/top          top models (real) + products + behavior lenses
                               (topPages/topReferrers/topSources over the events lens)
GET /v1/analytics/health       subsystem health (datastore connectivity + lens tables)

Registered as id "analytics" with cloud.HealthOwner + order 132: it serves its OWN /v1/analytics/health (below), and cloud.HealthOwner makes serve.go skip the generic GET /v1/<name>/health so the always-ok route never shadows the real probe — the same flag the kms/paas/s3 subsystems use. Order 132 binds /v1/analytics/* before the ai subsystem's /v1/* catch-all (150).

campaign.go is the in-process CAMPAIGN-METRICS seam over the ONE analytics warehouse: the /v1/campaign plane (clients/campaign) reads a campaign's funnel from HERE rather than opening a second store. A campaign's results ARE an analytics query scoped to the campaign — the utm_campaign-tagged events in hanzo.events — so there is one metrics plane, not a parallel one.

TENANCY: identical to every other query this package builds. campaignWhere binds the org (tenant_id) AND the campaign id (utm_campaign) AND the optional variant (utm_content) POSITIONALLY — nothing user-derived is ever interpolated, so a caller can only ever read its OWN org's campaign, and the utm_campaign filter can never escape into SQL. The variant arg powers the creative-A/B evidence read (utm_content) the experiment primitive composes.

Capture (WRITE) side of the analytics plane. analytics.go serves the read lenses over hanzo.events; this file is the symmetric ingest that FILLS that table, so the web/commerce lenses stop being honest-empty. Products emit here (the ONE native front door) instead of talking to the insights capture service directly — cloud owns the tenant boundary and the warehouse schema.

Routes (all POST; org resolved SERVER-SIDE from the validated principal):

POST /v1/analytics        capture one batch of events        -> {accepted,dropped}
POST /v1/analytics/batch  alias of the above (Segment-style)
POST /v1/tracker          beacon alias — navigator.sendBeacon / fetch(keepalive)
                          on page-unload posts here; SAME handler, SAME tenant
                          gate. It is a bare route: the /v1/tracker/* issue
                          tracker (clients/tracker) owns only /v1/tracker/projects*,
                          so bare POST /v1/tracker never collides with it.

TENANCY: the row's tenant_id is ALWAYS principal.Org (the validated IAM owner slug), never a client-supplied field — a caller can only ever write into its OWN org's partition, the same isolation invariant the read side enforces. The client controls distinct_id/session_id/properties (its own visitors), never the tenant.

PRIVACY: normalizeEvent scrubs credential- and PII-shaped property keys and any email-shaped value before the row is built (scrubProps). Only user/org identifiers (distinct_id, person_id, group_id, org) are retained as identity.

ONE datastore client: writes ride ai/object.DatastoreExec — the SAME pooled, KMS-credentialed connection the read side queries through — so there is no second transport, pool, or credential path.

event.go — the ONE canonical event-ingestion front door.

POST /v1/event   body: Event | [Event] | {batch:[…]}   ->  {accepted, dropped}

ONE door, EVERY wire, EVERY auth context. The decoder (decodeIngest) is wire-tolerant: a bare canonical Event object, a bare Event array, AND the CaptureBatch envelope ({batch:[…]} | {events:[…]}) the Segment/beacon/publishable paths speak all decode onto the SAME []CaptureEvent the ONE write core (ingestEvents) consumes, into the SAME hanzo.events table. There is deliberately no /v1/event/batch — a JSON array, or a batch envelope, IS the batch.

AUTH is the orthogonal, PLUGGABLE concern on this one door (eventTenant), resolved SERVER-SIDE and FAIL-CLOSED, in strict trust order:

1. a validated IAM bearer principal — its owner org;
2. a publishable key (pk-…) — IAM resolves it to its org; it can write but SAME key publishable.go mints; folded in here so a pk_ caller uses /v1/event directly);
3. an out-of-band IAM access key (hk-/sk-…) — resolved through the ONE key seam (cloud.OrgForKey).

A caller that PRESENTED one of those credentials and did not resolve ⇒ 403 (a misconfigured key is refused, not downgraded). A caller that presented NOTHING falls through to the ANONYMOUS lane (public.go): logged-out marketing traffic is admitted and attributed to the reserved public tenant, under a restricted kind/field allowlist and its own size + rate bounds. It rejoins this pipeline at ingestDecoded, so decode, write core, and receipt are shared — only admission differs.

There is NO brand-host fallback on the canonical door (that path stays only on the deprecated aliases), so /v1/event never writes an event into a REAL tenant IAM did not vouch for. The org is NEVER read from the body — on either lane.

The site-host carve (eventWithOrg) is the ONE exception to in-handler auth: on a published site host the tenant is FORCED from the resolved Site BEFORE the handler — the same server-supplied, host-derived tenant the file/base carves trust.

Every other ingest route (/v1/ingest, /v1/analytics{,/batch}, /v1/tracker, /v1/insights/e) is a thin alias/shim that resolves org its own way and funnels through the SAME decode + write core. One write path, many doors.

forward.go is the fan-out seam of the canonical event plane. After the ONE write core (ingestEvents) commits a batch to hanzo.events, it hands a COPY of that batch to an optional downstream sink — the destinations subsystem — which translates and forwards each event to the org's connected ad/analytics platforms (GA4, Meta CAPI, …). The seam is:

• ONE-WAY. analytics never imports destinations; destinations calls SetSink from its Mount. A nil sink means no fan-out (the default when destinations is off), so this file changes nothing about ingest when the subsystem is absent.
• RAW. The sink receives the event BEFORE the warehouse privacy scrub, because a server-side Conversions-API forwarder must hash the match keys (email/phone/ click ids) the warehouse deliberately drops. The org connected the destination and owns that consent; the destination adapters SHA-256 every PII field before it leaves the process.
• FAIL-SOFT. The sink runs detached (a panic-guarded goroutine) so a slow or broken destination can never block, fail, or crash an ingest.

public.go — the ANONYMOUS lane of the ONE canonical event door.

A logged-out visitor on a marketing surface carries no bearer and no key, so eventTenant resolves nothing. This file is the lane such a request falls through to, so a pageview or a browser error from a logged-out page lands in the warehouse instead of being refused.

Anonymous input is attested by nobody. It is therefore admitted under a policy the vouched-for lane never applies, and the two lanes are SEPARATE FUNCTIONS rather than one function with a mode flag: eventTenant → ingestBody is untouched by anything in this file, and every restriction below is unreachable from it.

The policy is in two pieces and no more: the request-scoped gates (publicIngest) and the pure decision (admitPublic).

• TENANT is publicTenant, a compile-time constant. admitPublic takes no request, so no header, query, or body field can influence the tenant an anonymous row carries: an anonymous write CANNOT land in a real org's partition, and there is no input that makes it do so.
• KIND is an ALLOWLIST of two — pageview and error, what a marketing surface emits. `identify` and `group` (which name a person and a group) and every custom event are dropped, counted in the honest receipt, never stored.
• NAME is server-chosen FROM the kind ($pageview | $error), so the anonymous name space is closed to two values: an anonymous caller can introduce neither a new name into the read lenses nor unbounded cardinality into the table's ORDER BY key.
• FIELDS are a PROJECTION, not a filter: admitPublic builds a fresh CaptureEvent from the fields it names, so a field it does not name — personId, groupId, revenue, productId, refCode, signupWeek, and the entire client property bag — cannot reach the row. The only properties an anonymous row carries are the server-folded $exception and the write core's $source.
• BYTES and COUNT are bounded first, and REFUSED rather than truncated.
• RATE is capped per client IP and, independently, per socket peer.
• DNT / Sec-GPC on the wire is honored: nothing is stored and the receipt says so.

Everything admitted here flows through the SAME ONE write core (ingestEvents) into the SAME hanzo.events table. One write path; this file only decides what a caller nobody vouched for may put on it.

publishable.go — the public capture path: a PUBLISHABLE KEY (pk-…) attributes a browser beacon to its tenant and writes straight to the datastore.

POST /v1/ingest   body: {batch:[WireEvent]}   auth: pk-…   -> {accepted,dropped}
GET  /v1/errors   recent type:'error' events for the org (read lens)

ONE publishable key, and IAM issues it. pk- is publishable, sk- is secret, and there is no third thing.

This file used to mint and verify its OWN pk_ (underscore) under an HMAC of CLOUD_INGEST_KEY_SECRET, with its own mint endpoint at /v1/ingest/keys — a second publishable-key family sitting beside the one IAM already owned. The underscore was load-bearing back then: pk_ was deliberately kept OUT of isAPIKey's set, because anything isAPIKey resolved into "the same principal a JWT yields", and a key meant for a browser bundle must not read.

That is fixed at the boundary instead of routed around: IdentityFromRequest now refuses a pk- outright (cloud.IsPublishableKey), so publishable means publishable no matter which door it arrives at. A pk- stays inside APIKeyPrefixes on purpose — OrgForKey must resolve it to learn which tenant a beacon belongs to. Resolvable, not authenticating.

The tenant is whatever IAM resolves the key to, never a body or header claim, so the tenant invariant the rest of the plane enforces holds here too. Every door funnels through the SAME write core (ingestEvents) into the SAME hanzo.events table: one write path, many front doors.

Pure core of the analytics lens: SQL predicate builders, datastore value coercers, and the pure assemblers that turn raw datastore rows into the response structs. Everything here is I/O-free so the tests drive it with mock rows — no datastore needed — exactly as ai/object/cloud_usage.go proves out its Overview assembler. The handlers (analytics.go) are the thin orchestration that fetches the rows and calls these.

THE ONE TENANCY INVARIANT lives here: llmWhere / eventsWhere ALWAYS emit "… = ?" with the org bound POSITIONALLY (never interpolated), so no query this package builds can read a tenant other than the caller's, and a hostile org slug can never escape into SQL. The isolation test asserts this directly.