analytics

Overview ¶

Package analytics mounts the Hanzo Cloud /v1/analytics/* surface: a native-Go, per-org analytics read API over the `hanzo` datastore warehouse (the `datastore` cluster). It is the backend for the console Native Analytics module (unified-analytics.md §5) — two read lenses over one warehouse:

• LLM lens (REAL today): hanzo.cloud_usage, the live per-org usage ledger the cloud o11y path already writes (requests, tokens, spend, models, errors).
• Web/commerce lens (honest-empty until the collector emits): hanzo.events.

ONE datastore client. This package does NOT open a second connection: it rides the SAME datastore-go/v2 client the ai subsystem's o11y ledger opens in the shared Bootstrap (ai/object.InitDatastore → object.DatastoreQuery). DRY: one transport, one pool, one set of KMS-injected DATASTORE_* creds — never hard-coded, never a second design.

TENANT ISOLATION is the security bar and is enforced SERVER-SIDE on every request. The org is c.Org() — the value SanitizeIdentity minted from the VALIDATED bearer owner claim (HIP-0026), never a client header — AND every request must carry a validated principal (c.User() set, which SanitizeIdentity sets ONLY for a verified bearer). This closes the Phase-1 "no-bearer + forged X-Org-Id direct-to-pod" cross-tenant read exactly as clients/s3 does. Every datastore query binds the org POSITIONALLY (query.go llmWhere/eventsWhere), so a maxpower token can NEVER read another org's analytics.

Surface (all org-scoped; /v1 only; read-only):

GET /v1/analytics/overview     per-org KPIs (llm real; web/commerce honest-empty)
GET /v1/analytics/timeseries   requests/tokens/spend over time (hour|day buckets)
GET /v1/analytics/top          top models (real) + products + behavior lenses
                               (topPages/topReferrers/topSources over the events lens)
GET /v1/analytics/health       subsystem health (datastore connectivity + lens tables)

Registered as id "analytics" with cloud.HealthOwner + order 132: it serves its OWN /v1/analytics/health (below), and cloud.HealthOwner makes serve.go skip the generic GET /v1/<name>/health so the always-ok route never shadows the real probe — the same flag the kms/paas/s3 subsystems use. Order 132 binds /v1/analytics/* before the ai subsystem's /v1/* catch-all (150).

campaign.go is the in-process CAMPAIGN-METRICS seam over the ONE analytics warehouse: the /v1/campaign plane (clients/campaign) reads a campaign's funnel from HERE rather than opening a second store. A campaign's results ARE an analytics query scoped to the campaign — the utm_campaign-tagged events in hanzo.events — so there is one metrics plane, not a parallel one.

TENANCY: identical to every other query this package builds. campaignWhere binds the org (tenant_id) AND the campaign id (utm_campaign) AND the optional variant (utm_content) POSITIONALLY — nothing user-derived is ever interpolated, so a caller can only ever read its OWN org's campaign, and the utm_campaign filter can never escape into SQL. The variant arg powers the creative-A/B evidence read (utm_content) the experiment primitive composes.

Capture (WRITE) side of the analytics plane. analytics.go serves the read lenses over hanzo.events; this file is the symmetric ingest that FILLS that table, so the web/commerce lenses stop being honest-empty. Products emit here (the ONE native front door) instead of talking to the insights capture service directly — cloud owns the tenant boundary and the warehouse schema.

Routes (all POST; org resolved SERVER-SIDE from the validated principal):

POST /v1/analytics        capture one batch of events        -> {accepted,dropped}
POST /v1/analytics/batch  alias of the above (Segment-style)
POST /v1/tracker          beacon alias — navigator.sendBeacon / fetch(keepalive)
                          on page-unload posts here; SAME handler, SAME tenant
                          gate. It is a bare route: the /v1/tracker/* issue
                          tracker (clients/tracker) owns only /v1/tracker/projects*,
                          so bare POST /v1/tracker never collides with it.

TENANCY: the row's tenant_id is ALWAYS principal.Org (the validated IAM owner slug), never a client-supplied field — a caller can only ever write into its OWN org's partition, the same isolation invariant the read side enforces. The client controls distinct_id/session_id/properties (its own visitors), never the tenant.

PRIVACY: normalizeEvent scrubs credential- and PII-shaped property keys and any email-shaped value before the row is built (scrubProps). Only user/org identifiers (distinct_id, person_id, group_id, org) are retained as identity.

ONE datastore client: writes ride ai/object.DatastoreExec — the SAME pooled, KMS-credentialed connection the read side queries through — so there is no second transport, pool, or credential path.

event.go — the ONE canonical event-ingestion front door.

POST /v1/event   body: Event | [Event] | {batch:[…]}   ->  {accepted, dropped}

ONE door, EVERY wire, EVERY auth context. The decoder (decodeIngest) is wire-tolerant: a bare canonical Event object, a bare Event array, AND the CaptureBatch envelope ({batch:[…]} | {events:[…]}) the Segment/beacon/publishable paths speak all decode onto the SAME []CaptureEvent the ONE write core (ingestEvents) consumes, into the SAME hanzo.events table. There is deliberately no /v1/event/batch — a JSON array, or a batch envelope, IS the batch.

AUTH is the orthogonal, PLUGGABLE concern on this one door (eventTenant), resolved SERVER-SIDE and FAIL-CLOSED, in strict trust order:

1. a validated IAM bearer principal — its owner org;
2. a write-only publishable key (pk_…) — HMAC-verified org, no IAM/DB hop (the SAME key publishable.go mints; folded in here so a pk_ caller uses /v1/event directly);
3. an out-of-band IAM access key (hk-/sk-…) — resolved through the ONE key seam (cloud.OrgForKey).

None of the above ⇒ 403. There is NO brand-host fallback on the canonical door (that path stays only on the deprecated aliases), so /v1/event never writes an event into a tenant IAM did not vouch for. The org is NEVER read from the body.

The site-host carve (eventWithOrg) is the ONE exception to in-handler auth: on a published site host the tenant is FORCED from the resolved Site BEFORE the handler — the same server-supplied, host-derived tenant the file/base carves trust.

Every other ingest route (/v1/ingest, /v1/analytics{,/batch}, /v1/tracker, /v1/insights/e) is a thin alias/shim that resolves org its own way and funnels through the SAME decode + write core. One write path, many doors.

forward.go is the fan-out seam of the canonical event plane. After the ONE write core (ingestEvents) commits a batch to hanzo.events, it hands a COPY of that batch to an optional downstream sink — the destinations subsystem — which translates and forwards each event to the org's connected ad/analytics platforms (GA4, Meta CAPI, …). The seam is:

• ONE-WAY. analytics never imports destinations; destinations calls SetSink from its Mount. A nil sink means no fan-out (the default when destinations is off), so this file changes nothing about ingest when the subsystem is absent.
• RAW. The sink receives the event BEFORE the warehouse privacy scrub, because a server-side Conversions-API forwarder must hash the match keys (email/phone/ click ids) the warehouse deliberately drops. The org connected the destination and owns that consent; the destination adapters SHA-256 every PII field before it leaves the process.
• FAIL-SOFT. The sink runs detached (a panic-guarded goroutine) so a slow or broken destination can never block, fail, or crash an ingest.

publishable.go — the FASTEST capture path: a write-only PUBLISHABLE KEY (pk_…) that authenticates a direct-to-datastore ingest with ZERO network hop.

POST /v1/ingest        body: {batch:[WireEvent]}   auth: pk_…   -> {accepted,dropped}
POST /v1/ingest/keys   mint a pk_ for the caller's org (validated principal)
GET  /v1/errors        recent type:'error' events for the org (read lens)

WHY a distinct key from the IAM hk-/sk-/pk- family: those resolve through IAM (get-user?accessKey — a network round-trip) and mint a FULL principal that can READ. A publishable key is meant to ship in a browser bundle, so it must be write-only and cheap to verify. This key is:

• INGEST-ONLY BY CONSTRUCTION. The `pk_` (underscore) prefix is deliberately NOT in isAPIKey's set (hk-/sk-/pk-/fw_/hz_, all dash/`fw_`/`hz_`), so the identity boundary (SanitizeIdentity) and OrgForKey both REFUSE it — it can never become a bearer principal, so it can never read. Its only door is the ingest verifier below. Write-only is a property of WHICH resolver accepts the value, not a flag on a row.
• ORG-SCOPED, SIGNED, NON-FORGEABLE. The org is carried in the key but sealed under HMAC-SHA256(secret, org): a client cannot flip the org without the secret. The server stamps tenant_id from the VERIFIED org, never from the request body — the same tenant invariant the rest of the plane enforces.
• LOWEST LATENCY. Verification is one HMAC compute — no IAM call, no keys table, no DB read. This is the no-Kafka, no-bridge, direct-to-ClickHouse path; it funnels through the SAME write core (ingestEvents) into the SAME hanzo.events table as every other adapter. One write path, many front doors.

SECRET: the HMAC secret is CLOUD_INGEST_KEY_SECRET (KMS-injected by the operator). Absent ⇒ mint and verify BOTH fail closed (503 / 403) — a deployment without the secret never mints a forgeable key nor admits an unverifiable one.

Pure core of the analytics lens: SQL predicate builders, datastore value coercers, and the pure assemblers that turn raw datastore rows into the response structs. Everything here is I/O-free so the tests drive it with mock rows — no datastore needed — exactly as ai/object/cloud_usage.go proves out its Overview assembler. The handlers (analytics.go) are the thin orchestration that fetches the rows and calls these.

THE ONE TENANCY INVARIANT lives here: llmWhere / eventsWhere ALWAYS emit "… = ?" with the org bound POSITIONALLY (never interpolated), so no query this package builds can read a tenant other than the caller's, and a hostile org slug can never escape into SQL. The isolation test asserts this directly.