Documentation
¶
Index ¶
- func IsMutation(sql string) bool
- func NewOpsRouter(deps OpsDependencies) http.Handler
- func NewRouter(deps Dependencies) http.Handler
- func RequireAdmin(store policy.Source) func(http.Handler) http.Handler
- func StoreFromContext(ctx context.Context) (*settings.Store, bool)
- func TenantMW(tenants *settings.Registry) func(http.Handler) http.Handler
- func WithStore(ctx context.Context, store *settings.Store) context.Context
- type BootState
- type Check
- type DLQHandler
- type Dependencies
- type HealthHandler
- type IngestFormat
- type IngestHandler
- type InsertChecker
- type OpsDependencies
- type PipesHandler
- type PolicySource
- type QueryHandler
- type RecordValidator
- type RegistrySource
- type SchemaHandler
- type SettingsHandler
- type StreamHandler
- type StructuredQueryHandler
- type VersionHandler
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func IsMutation ¶ added in v0.2.0
IsMutation reports whether sql's leading statement is a non-SELECT — i.e. one that returns no result set and must go through Exec, not Query. Leading whitespace and comments are skipped as ClickHouse's lexer skips them, then the first bareword is matched whole, case-insensitively, against mutationVerbs. After a WITH list ClickHouse parses only SELECT, a FROM-first SELECT or INSERT INTO, so a WITH-led statement is a write exactly when it holds INSERT INTO at the top level (hasTopLevelInsertInto). An `EXECUTE AS <user>` prefix is looked through to the statement it runs. A write classified as a read goes through Query, which runs it and then fails the call, so a client that retries the error writes again.
func NewOpsRouter ¶ added in v0.2.0
func NewOpsRouter(deps OpsDependencies) http.Handler
NewOpsRouter creates the router of a process without the api role. Every other route — the tenant routes and the rest of /v1/ops — answers 404: the handlers behind them are not wired here. The reload is gated by the operator key alone, as the ops tree is over a nested directory.
func NewRouter ¶
func NewRouter(deps Dependencies) http.Handler
NewRouter creates the chi router with all routes.
func RequireAdmin ¶
RequireAdmin restricts a route to the policy admin role (policy.AdminRole — configurable via admin_role, "admin" by default). The role established by the auth middleware and the live policy are read per request, so an admin_role change applies on reload. A nil policy (policies.json empty) admits nobody via a role — IsAdmin(nil) is false — so no token can re-open a locked-out deployment through this gate. The exception is the operator key: auth.IsOperator passes this gate even under a nil policy, so an operator can still trigger a settings reload after fixing the files (break-glass). A nil store is the same gate with nothing to read — the operator key alone — which is what NewRouter mounts over a nested settings directory.
Authentication is decoupled from this gate: a missing/invalid/expired token resolves to an empty (non-admin) role and is denied here. Denials go through writeAuthzDenied, so a present-but-invalid token fails loud (401 + token reason) rather than as a bare 403.
func StoreFromContext ¶ added in v0.2.0
StoreFromContext returns the store TenantMW resolved for this request. Handlers call it once and pass the store down as an argument; nothing below a handler reads it from the context.
func TenantMW ¶ added in v0.2.0
TenantMW resolves the request's tenant before authentication runs and stores it in the request context: the tenant requestTenant names. A malformed id is a 400; an id that cannot be served is resolveStore's 404 or 503. Every answer, the refusals included, carries Vary: X-Tenant-ID so a shared cache cannot replay one tenant's response to another — added, not set, so the CORS Vary: Origin survives.
Types ¶
type BootState ¶
type BootState struct {
// contains filtered or unexported fields
}
BootState tracks a one-shot startup diagnostic surfaced by /livez. While Err() returns non-nil the binary is considered to be in degraded-boot mode: /livez responds 503 with the diagnostic message instead of 200, so an operator can curl the endpoint to learn why the gateway isn't accepting traffic yet. Once boot work (today: the first tenant's ClickHouse schema discovery) succeeds, Set(nil) flips /livez back to 200.
BootState is safe for concurrent use.
func NewBootState ¶
NewBootState returns a BootState seeded with initialErr. Pass nil if the binary is fully ready at construction time; pass a non-nil error to start in degraded mode (the goroutine that performs boot work calls Set(nil) on success).
type Check ¶ added in v0.2.0
type Check struct {
Name string
// Run reports why the dependency cannot do its part, nil while it can.
Run func(context.Context) error
}
Check is one dependency Readiness consults, named for the 503 body: the ClickHouse pools, the external message queue.
type DLQHandler ¶
type DLQHandler struct {
Counts mq.DeadLetterStats
}
DLQHandler exposes Dead Letter Queue statistics.
func NewDLQHandler ¶
func NewDLQHandler(stats mq.DeadLetterStats) *DLQHandler
func (*DLQHandler) Stats ¶
func (h *DLQHandler) Stats(w http.ResponseWriter, r *http.Request)
Stats returns per-table message counts on one tenant's dead-letter queue: the tenant ?tenant= names (read strictly, as every ops read does — opsTenant), tenant.Default without it. The queue is the MQ's, not the settings', so it is looked up there: a tenant whose folder was rejected or removed is read like one being served, for as long as its queue is kept, and an id with no queue is a 404. Supports optional ?table= query parameter to filter by table name.
type Dependencies ¶
type Dependencies struct {
Ingest *IngestHandler
Query *QueryHandler
SSE *StreamHandler
Health *HealthHandler
Version *VersionHandler
Schema *SchemaHandler
DLQ *DLQHandler
Pipes *PipesHandler
StructuredQuery *StructuredQueryHandler
// Settings, if non-nil, mounts POST /v1/ops/settings/reload — the API
// trigger for reloading the settings directory. Nil when no settings
// directory is configured (nothing to reload).
Settings *SettingsHandler
AuthMW func(http.Handler) http.Handler
// Tenants resolves the tenant.Header of every tenant route to that
// tenant's settings store (TenantMW), ahead of AuthMW. The probes,
// /version, the metrics path, and /v1/ops/* are tenant-exempt.
Tenants *settings.Registry
// PolicySource backs the RequireAdmin gate: the admin role (policy.AdminRole)
// is read live from the adopted policy, so admin_role changes apply on reload.
// NewRouter ignores it when Tenants is nested, whatever was wired here: the
// ops gate then admits the operator key alone (see NewRouter).
PolicySource policy.Source
// CORSOrigins yields one tenant's allowed CORS origins, read per request
// so a settings reload applies immediately ((*settings.Store).CORSOrigins
// in production); which tenant's list answers a request is corsOrigins'
// rule. An empty or nil list — including a nil func, or no Tenants —
// denies every browser origin; ["*"] is the only allow-all spelling.
CORSOrigins func(*settings.Store) []string
// MetricsHandler, if non-nil, is mounted at MetricsPath as an unauthenticated
// endpoint (Prometheus convention). Wired by internal/app from the OTel Prometheus
// exporter when observability.metrics.prometheus.enabled is true AND port is 0.
MetricsHandler http.Handler
MetricsPath string
}
Dependencies holds all handler dependencies.
type HealthHandler ¶
type HealthHandler struct {
// Checks are Readiness's dependency checks, every one run on every call
// so the 503 names each one that failed, not just the first. Which
// dependencies a process has is internal/app's to wire: the ClickHouse
// pools (chconn.Pools.Ping: every open pool at once, ready at the first
// answer) and the external NATS; a handler with none is ready once
// booted.
Checks []Check
// Boot is consulted by both Liveness and Readiness. When non-nil and
// its Err() is non-nil, both endpoints report 503 with the diagnostic
// — used while boot-time schema discovery is still failing in the
// retry loop. A nil Boot preserves the pre-retry-loop behaviour
// (Liveness always 200; Readiness 503 only on a failed check).
Boot *BootState
}
HealthHandler provides liveness and readiness probes.
func NewHealthHandler ¶
func NewHealthHandler(checks ...Check) *HealthHandler
func (*HealthHandler) Liveness ¶
func (h *HealthHandler) Liveness(w http.ResponseWriter, _ *http.Request)
func (*HealthHandler) Online ¶
func (h *HealthHandler) Online(w http.ResponseWriter, _ *http.Request)
Online is a content-free public liveness ping served at /v1/health for the SDK's "is this server reachable / accepting data" check (and for picking among servers in a distributed setup). It mirrors Liveness's status logic — 200 once boot completes, 503 while boot-time schema discovery is still failing — but writes no body: the caller only branches on the status code, so there's nothing to JSON-encode or cache per request.
It deliberately lives under /v1 rather than reusing /livez: /livez (and /readyz, /healthz) are Kubernetes probe paths an operator may filter out at the reverse proxy, whereas /v1/health is documented public API surface the SDK can rely on staying reachable. It does NOT ping ClickHouse — readiness- based load balancing is the proxy/LB's job (via /readyz), not the client's.
func (*HealthHandler) Readiness ¶
func (h *HealthHandler) Readiness(w http.ResponseWriter, r *http.Request)
type IngestFormat ¶ added in v0.2.0
type IngestFormat int
IngestFormat is the wire format of an ingest request body. It comes from the request's declared Content-Type and nothing else: the body never overrides what the client said it sent, so a caller can always tell how their bytes will be read without knowing what the first one happens to be.
const ( // FormatJSON is the application/json family: one flat object, or a // top-level array of them. Which of the two is the body's own business — // the first non-whitespace byte picks it — because both are the same // format, differing only in arity. FormatJSON IngestFormat = iota // FormatNDJSON is newline-delimited JSON: one flat object per line. A line // that is not a JSON object is a per-record error, never a reason to // re-read the body as something else. FormatNDJSON )
func (IngestFormat) String ¶ added in v0.2.0
func (f IngestFormat) String() string
String renders a format for error messages and logs.
type IngestHandler ¶
type IngestHandler struct {
// Registry yields the request tenant's schema registry.
Registry RegistrySource
// Dedup resolves the request tenant's deduplicator — the tenant's own
// store, picked off the store the handler already holds (#583 story 7;
// dedupe.Stores in production). nil when no dedupe store is wired (tests).
Dedup func(store *settings.Store) dedupe.Deduplicator
// DedupeSettings resolves the effective dedupe settings for a table of the
// request's tenant ((*settings.Store).DedupeFor in production). Called
// once per record so a settings reload lands at a record boundary — one
// record never mixes two documents' values. Dedup is skipped when nil.
DedupeSettings func(store *settings.Store, table string) settings.Dedupe
// DedupeLease is how long a record's claimed id stays pending while it is
// published; 0 means dedupe.DefaultLease.
DedupeLease time.Duration
Publisher mq.Publisher
PolicySource PolicySource
// Validator and Checker are the per-record seams a native type layer will
// take over (see ingest_seams.go). Both are optional: nil means the default
// implementation, which is today's behavior unchanged.
Validator RecordValidator
Checker InsertChecker
// contains filtered or unexported fields
}
IngestHandler handles POST /v1/ingest?table={table}
func NewIngestHandler ¶
func NewIngestHandler(registry RegistrySource, pub mq.Publisher) *IngestHandler
func (*IngestHandler) Handle ¶
func (h *IngestHandler) Handle(w http.ResponseWriter, r *http.Request)
type InsertChecker ¶ added in v0.2.0
InsertChecker decides whether a record's value satisfies a policy check clause. Matches answers the scalar `_eq` form (the required value), InSet the `_in` form (set membership). It never sees a record as a whole: the auto-injection of a missing check value stays in prepareRecord, where the ordering against validation and canonicalization is load-bearing.
type OpsDependencies ¶ added in v0.2.0
type OpsDependencies struct {
Health *HealthHandler
Version *VersionHandler
// Settings mounts POST /v1/ops/settings/reload.
Settings *SettingsHandler
// AuthMW authenticates the reload. It runs no token verifier (those are
// the api role's), so the operator key is the one credential that passes.
AuthMW func(http.Handler) http.Handler
MetricsHandler http.Handler
MetricsPath string
}
OpsDependencies is what the ops-only listener serves: a process that runs no api role still answers its probes, /version, the same-port metrics endpoint, and the settings reload, so the control plane drives every process's tenant tree the same way.
type PipesHandler ¶
type PipesHandler struct {
// Source yields a tenant's pipes (the store itself in production).
Source func(*settings.Store) pipes.Source
PolicySource PolicySource // resolves empty role to default_role; may be nil
// Tenants resolves the tenant the admin reads (List, Get) serve: /v1/ops
// is tenant-exempt, so they carry no request tenant and read the one
// ?tenant= names, the default one without it (opsStore).
Tenants *settings.Registry
// CHConn yields the request tenant's connection (chconn.Pools.For in
// production); nil is a tenant on no pool, a 503.
CHConn func(*settings.Store) driver.Conn
Cache cache.Cache
// contains filtered or unexported fields
}
PipesHandler serves named query pipes: execution for callers, read-only listing for admins. Pipes are defined in the settings directory's pipes.json and read per request, so a reload applies immediately.
func NewPipesHandler ¶
func (*PipesHandler) Execute ¶
func (h *PipesHandler) Execute(w http.ResponseWriter, r *http.Request)
Execute runs a named query with the provided parameters.
func (*PipesHandler) Get ¶
func (h *PipesHandler) Get(w http.ResponseWriter, r *http.Request)
Get returns a specific named query of the ?tenant= (admin endpoint).
func (*PipesHandler) List ¶
func (h *PipesHandler) List(w http.ResponseWriter, r *http.Request)
List returns all named queries of the ?tenant= (admin endpoint).
type PolicySource ¶ added in v0.2.0
PolicySource yields the access-control policy of the request's tenant, read per request so a settings reload applies to the next one ((*settings.Store).Policy in production). A nil policy from a wired source is a deliberate lockout; a nil source means policy filtering is not wired at all, which only tests do.
type QueryHandler ¶
type QueryHandler struct {
// Tenants resolves the tenant the proxy queries: /v1/ops is
// tenant-exempt, so the request carries no tenant and the query runs
// against the one ?tenant= names, the default one without it (opsStore).
Tenants *settings.Registry
// contains filtered or unexported fields
}
QueryHandler handles POST /v1/ops/query.
Authorization is enforced at the router (the /v1/ops/* RequireAdmin gate in NewRouter). The handler trusts any caller that reaches it. See internal/api/router.go for the role-gate rationale.
Implementation: a thin proxy to ClickHouse's HTTP interface. The SQL string is forwarded verbatim with `default_format=JSON`, ClickHouse returns either a `{"meta":..., "data":[...], ...}` JSON object for read queries or an empty body for mutations, and the handler emits just the `data` array (or `[]` for mutations) back to the caller.
Why a proxy instead of clickhouse-go's native Query/Exec:
- ClickHouse classifies statements natively, so any single statement (arbitrary DDL/DML verbs, current and future) and inline FORMAT directives all just work without WaveHouse-side parsing. Multi-statement input (`SELECT 1; TRUNCATE t`) also works when the upstream ClickHouse has multi-query enabled, which is the default in recent versions; older or restrictively-configured servers may reject the second statement with a clear error.
- There is no IsMutation heuristic to maintain — no leading-verb table, no comment stripper, no CTE-aware paren scanner, no class of bug where a future ClickHouse verb routes the wrong way.
- ClickHouse's own error messages reach the admin verbatim, which is exactly what they want from an escape hatch.
- The cache (TieredCache + singleflight) was already removed in an earlier commit; this completes the simplification.
func NewQueryHandler ¶
func NewQueryHandler(target func(*settings.Store) chconn.Target, queryTimeout func(*settings.Store) time.Duration) *QueryHandler
NewQueryHandler builds a handler that proxies to ClickHouse over HTTP. target supplies the base URL (`http://host:8123`), the username/password forwarded via ClickHouse's `X-ClickHouse-User` / `X-ClickHouse-Key` headers (matching the ingest worker's convention in internal/ingest), and the database set as the `?database=` query-string parameter when non-empty.
The HTTP client itself has no Timeout — every request gets a queryTimeout deadline from a context derived from the inbound request (see Handle), which bounds the whole exchange including body read. Setting `Timeout` here too would just duplicate that bound (and silently truncate any inbound context longer than queryTimeout).
func (*QueryHandler) Handle ¶
func (h *QueryHandler) Handle(w http.ResponseWriter, r *http.Request)
type RecordValidator ¶ added in v0.2.0
type RecordValidator interface {
Validate(schema *discovery.TableSchema, record map[string]any) error
CanonicalizeTimestamps(schema *discovery.TableSchema, record map[string]any)
}
RecordValidator covers the two schema-driven steps of the record pipeline. Validate rejects a record the table's schema cannot accept; CanonicalizeTimestamps rewrites DateTime/DateTime64 values to the canonical wire form in place.
CanonicalizeTimestamps returns nothing, matching discovery's function: it is deliberately fail-open (#372) — a value it cannot read is passed through for ClickHouse to judge, and the row filter is what enforces. Giving it an error return would invite a caller to change that.
The two are one interface because they are one contract — "what this schema says about this record" — evaluated at two points in prepareRecord that must stay apart: the insert-check block sits between them deliberately, so checks keep pre-#372 semantics.
type RegistrySource ¶ added in v0.2.0
type RegistrySource func(*settings.Store) *discovery.SchemaRegistry
RegistrySource yields a tenant's schema registry, its own since #583 story 6 (the per-tenant discovery map in internal/app). Nil for a tenant whose registry is not built yet — the beat between its adoption and the hook that builds it — which reads as a schema not loaded yet.
type SchemaHandler ¶
type SchemaHandler struct {
Registry RegistrySource
// Tenants resolves the tenant the reads and the refresh serve: /v1/ops
// is tenant-exempt, so they carry no request tenant and read the one
// ?tenant= names, the default one without it (opsStore).
Tenants *settings.Registry
}
SchemaHandler exposes the discovered ClickHouse table schemas.
func NewSchemaHandler ¶
func NewSchemaHandler(registry RegistrySource) *SchemaHandler
func (*SchemaHandler) Get ¶
func (h *SchemaHandler) Get(w http.ResponseWriter, r *http.Request)
Get returns the schema for a single table of the ?tenant=, every table's without ?table=.
func (*SchemaHandler) List ¶
func (h *SchemaHandler) List(w http.ResponseWriter, r *http.Request)
List returns all discovered table schemas of the ?tenant=.
func (*SchemaHandler) Refresh ¶
func (h *SchemaHandler) Refresh(w http.ResponseWriter, r *http.Request)
Refresh forces an immediate schema refresh of the ?tenant= from its ClickHouse, then returns its schemas. A tenant with no open pool — one could not be opened for it, such as by the connection ceiling — is a 503 with Retry-After, like the reads it would answer.
type SettingsHandler ¶ added in v0.2.0
SettingsHandler serves the ops surface for the hot-reloadable settings directory. Constructed only when a settings directory is configured — with none there is nothing to reload, so the route is simply absent (the same pattern as the DLQ and policy handlers).
func NewSettingsHandler ¶ added in v0.2.0
func NewSettingsHandler(tenants *settings.Registry) *SettingsHandler
func (*SettingsHandler) Reload ¶ added in v0.2.0
func (h *SettingsHandler) Reload(w http.ResponseWriter, r *http.Request)
Reload handles POST /v1/ops/settings/reload — the API trigger for the same serialized reload path SIGHUP and the directory watcher run. 200 when the directory was adopted (warnings included in the body), 422 when validation rejected it and the previous settings remain in effect.
?tenant= narrows the reload to that tenant's folder of a nested directory (400 malformed, 404 unknown): adopted then speaks for that folder alone, and a 422 means the tenant is no longer served. Without it the whole tree is reloaded, and over a nested directory a 422 can mean adopted in part — the error findings name the folders that were not (settings.Registry.Reload).
type StreamHandler ¶
type StreamHandler struct {
Hub *stream.Hub
Replayer mq.Replayer // gap-fill source; nil disables replay
// Heartbeater yields the request tenant's keepalive wheel — its own, at
// its own stream.keepalive_* settings (#597) — or nil for a tenant no
// longer served, whose streams are ending (Hub.Prune, Served). A nil func
// means no keepalive is wired (tests).
Heartbeater func(*settings.Store) *stream.Heartbeater
Metrics *stream.Metrics
// Closing, when set, is closed as the server begins shutting down, and
// every open stream ends at once — mid-replay too: a stream is a
// connection to close, not in-flight work for the drain to wait on, and
// the client reconnects and gap-fills via Last-Event-ID. A nil channel
// never fires (a harness that serves the handler itself).
Closing <-chan struct{}
// Served, when set, reports whether a tenant is still being served. A
// reload that stops serving one evicts its streams (Hub.Prune), but not a
// stream TenantMW admitted just before the reload and registered with the
// Hub just after it: this check, made after registering, ends that one.
Served func(tenant.ID) bool
}
StreamHandler handles GET /v1/stream
func NewStreamHandler ¶
func NewStreamHandler(hub *stream.Hub, replayer mq.Replayer) *StreamHandler
func (*StreamHandler) Handle ¶
func (h *StreamHandler) Handle(w http.ResponseWriter, r *http.Request)
type StructuredQueryHandler ¶
type StructuredQueryHandler struct {
// CHConn yields the request tenant's connection (chconn.Pools.For in
// production); nil is a tenant on no pool, a 503.
CHConn func(*settings.Store) driver.Conn
Cache cache.Cache
Registry RegistrySource
PolicySource PolicySource
// contains filtered or unexported fields
}
StructuredQueryHandler handles POST /v1/query?table={table}
func (*StructuredQueryHandler) Handle ¶
func (h *StructuredQueryHandler) Handle(w http.ResponseWriter, r *http.Request)
type VersionHandler ¶
type VersionHandler struct {
// contains filtered or unexported fields
}
VersionHandler serves the binary's build metadata at /version. The payload is fully static for the process lifetime (the ldflags-injected build info plus the Go runtime version), so it's marshaled once in NewVersionHandler and the cached bytes are written on each request — no per-request encoding.
func NewVersionHandler ¶
func NewVersionHandler(version, gitCommit, buildTime string) *VersionHandler
NewVersionHandler pre-marshals the build-info JSON from the ldflags-injected values (version, git commit, build time — see the Makefile / .goreleaser.yaml) plus runtime.Version(). Marshaling a struct of strings cannot fail, so the error is discarded.
func (*VersionHandler) Handle ¶
func (h *VersionHandler) Handle(w http.ResponseWriter, _ *http.Request)
Handle writes the pre-marshaled build metadata. No auth gate (wired among the public routes in NewRouter) — the values are non-sensitive and already in the startup logs.