websocket

package
v0.1.0-alpha.5 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 16, 2026 License: MIT Imports: 32 Imported by: 0

README

websocket

The output component that streams the graph to browsers and other WebSocket clients.

Overview

websocket runs an HTTP server and upgrades each request on its path (default /ws) to a WebSocket. It has exactly one input, chosen in configuration, so one instance sends one kind of frame:

  • Applied state (the default): a kv-watch of every ENTITY_STATES key, the bucket graph-ingest writes. A client sees the entities as they are now, then each change.
  • A core NATS subject: every JSON message published on it.

To stream both, run two instances on two ports.

A frame is one WebSocket text message holding one JSON object. A revision is the number NATS gives each write to a bucket.

Configuration

Applied state, which is also what {} gives:

{
  "type": "output",
  "name": "websocket",
  "enabled": true,
  "config": {
    "ports": {
      "inputs": [
        {"name": "entity_watch", "config": {"kind": "kv-watch", "bucket": "ENTITY_STATES"}}
      ],
      "outputs": [
        {"name": "websocket_server", "config": {"kind": "network", "protocol": "http", "host": "0.0.0.0", "port": 8081}}
      ]
    },
    "path": "/ws",
    "ping_interval": "30s"
  }
}

A subject instead streams one subject (a core NATS subject, wildcards as NATS defines them, no queue group):

{"name": "boids_frames", "config": {"kind": "nats", "subject": "boids.frames"}}
Option Type Default Description
ports object the ports above A ports block that is present is used as written; one that is absent takes the default
path string /ws Path the clients upgrade on; must begin with / and be a valid HTTP ServeMux pattern
ping_interval string 30s Time between pings to each client, a Go duration; a value that does not parse or is not above zero is refused

Exactly one input is required: a kv-watch of every ENTITY_STATES key with no history, or a core NATS subject with no queue group (a queue group would split messages between instances). Any other input is refused naming the port; a jetstream input is refused with the replacement. Exactly one output is required, an http network port from 1024 to 65535 (port 0 is refused: the address clients dial must be known before Start). The configuration is decoded strictly: an unknown key is refused at construction, naming it. A missing NATS client is refused too.

Wire format

Every frame is a JSON object with a type. timestamp is Unix milliseconds.

{"type":"data","subject":"boids.frames","timestamp":1731330000123,"payload":{"...":"the message, unchanged"}}
{"type":"put","bucket":"ENTITY_STATES","key":"<entity id>","revision":42,"timestamp":1731330000100,"payload":{"...":"stored"}}
{"type":"delete","bucket":"ENTITY_STATES","key":"<entity id>","revision":43,"timestamp":1731330000200}
{"type":"poisoned","bucket":"ENTITY_STATES","key":"<entity id>","revision":44,"error":"<why the value does not decode>"}
{"type":"synced","bucket":"ENTITY_STATES","revision":44}
Type Instance Meaning
data subject One message. payload is the message's bytes unchanged; timestamp is when the component received it
put applied state The entity key was written. payload is the bytes graph-ingest stored; timestamp is the write time in the bucket
delete applied state The entity key was deleted or purged
poisoned applied state The value stored at key does not decode as an entity state. Drop your copy of key; a later valid write heals it
synced applied state Every frame before it is the state at revision; every frame after it is a change above it

Frames a client sends are read and discarded.

What an applied-state client sees
  1. On connect: one put per current entity, one poisoned per poisoned key, then synced. Entities and poisoned keys are each in key order.
  2. After synced: the changes, as they apply.

Keep a map by key: put sets it, delete and poisoned remove it. The map then equals the bucket's current state once the client holds each key's latest revision. A key's frames carry increasing revisions and its latest write is never omitted, but intermediate revisions can be: the shared view merges writes inside one 250 ms window, and a client that is behind receives only the latest change per key. Do not count frames, and do not expect every revision.

What a subject client sees

Each message that is valid JSON is sent to every connected client as a data frame. A message that is not JSON is not sent; it is counted (messages_dropped_total{reason="not_json"}) and logged. Delivery is at most once, best effort: the component acknowledges nothing, replays nothing, and a client that was not connected misses the message.

Readiness and closing

  • Applied state before it is ready. Until the instance's view of ENTITY_STATES has caught up (a bucket that does not exist yet counts), and while its watch is lost, an upgrade is answered 503 with the reason and Health reports it as degraded: applied-state view catching up or applied-state watch lost, reopening. A watch lost on a running instance closes every client with code 1013 (try again later); reconnect after a 503 clears to get a fresh snapshot.
  • Stop. New upgrades get 503, every client is sent close code 1001 (going away), and Stop returns nil once the server and every client handler have returned; the port is then released. If cleanup fails or Stop's context ends first, Stop returns that error and keeps what it has not released, and a later Stop finishes it.
  • Start. Start binds the listener, loads TLS when enabled, and subscribes or starts the view. A failure returns the error after releasing what it took; if that release fails too, Start reports both errors and keeps the rest for Stop. Ending Start's context afterwards stops nothing; only Stop does (ADR-113).

Slow and silent clients

A client that stops reading never slows another.

  • Subject: each client has a queue of 256 frames; when it is full the client's oldest frame is dropped (messages_dropped_total{reason="client_queue_full"}).
  • Applied state: the client's pending changes merge per key, so it gets the latest change per key when it catches up (changes_coalesced_total).

Each drop is counted at once. It is logged as a warning naming the reason, the listener and why continuing is safe: the first of a reason immediately, after that at most once a minute per reason, with the count since the last line. Merging inside the view's 250 ms window is part of the applied-state contract and is not counted.

Keep-alive: the component pings each client every ping_interval, and a pong extends the client's read deadline to twice that. A client that answers pings but sends nothing stays connected; one that answers none is closed (client_disconnections_total{reason="read_timeout"}). At gorilla/websocket v1.5.3, a pong that does not extend the read deadline lets the read time out with pongs received, so the handler extends it (TestQuietClientStaysConnected). A write to a client that does not complete in 10 seconds closes that client; what gorilla/websocket does when that deadline passes is not promised.

Origins and authentication

Every Origin is accepted, as in the SemStreams original, and the output has no authentication. Anything that can reach the port can read the stream. Bind it to an interface the deployment trusts, or put a proxy in front that checks origin and credentials.

TLS

TLS comes from the deployment's server TLS configuration (component.Dependencies.Security), loaded in Start through internal/tlsutil: certificate and key files, with client certificates (mTLS) applied when that configuration asks. An acme mode is refused at Start naming the key; ACME is not supported. With TLS enabled the server speaks HTTP/1.1 only, which the WebSocket upgrade needs, and clients dial wss://.

Metrics

Collectors register on the injected metric.MetricsRegistry, never on Prometheus' global registry; with no registry they are not registered. Every metric is labelled listener (host:port), so two instances on one registry stay apart.

Metric (semengine_websocket_ prefix) Type Meaning
messages_received_total counter Subject messages received, or applied-state writes the view decoded
messages_sent_total counter Data and state frames written to a client, once per client
bytes_sent_total counter Bytes of those frames
message_size_bytes histogram Size of each such frame
clients_connected gauge Clients connected now
client_connections_total counter Requests upgraded to a WebSocket
client_disconnections_total{reason} counter read_timeout, client_closed, write_failed, stopped, watch_lost
errors_total{type} counter upgrade (a request that could not be upgraded), serve (the server stopped before Stop)
messages_dropped_total{reason} counter not_json, client_queue_full
changes_coalesced_total counter Applied-state changes merged into a later one for a client that was behind

Migration from SemStreams

The wire format is new and the SemStreams MessageEnvelope is gone: its id is dropped (it served the acknowledgement path), and the payload is no longer rewritten with timestamp and subject. These keys are refused at construction with a line saying what replaces them: delivery_mode, ack_timeout, passthrough.

  • semsource: drop the JetStream input (the default is the kv-watch) and delivery_mode. /ws now carries current entity state (put, delete, poisoned, synced), not every graph.ingest.* message: no manifest, status or predicate payloads, and no promise of every revision. Keep a map by key and treat the latest frame as truth. A configuration that still has delivery_mode fails boot naming it.
  • semboids: drop delivery_mode. Read the message from frame.payload (its bytes are unchanged), and the subject and time from frame.subject and frame.timestamp.

Known limits

  • At most once on a subject. A message sent while no client is connected, or dropped from a full queue, is gone.
  • Convergence, not history, on applied state. Intermediate revisions can be merged away.
  • Memory. The instance holds every entity's stored bytes in memory, so its footprint grows with ENTITY_STATES.
  • No authentication, any origin.
  • COMMUNITY_INDEX is not streamed. State frames carry bucket, so another bucket could be added without a wire change; none is present.

Dependencies

  • Upstream: graph-ingest writes ENTITY_STATES; a subject input needs a NATS client.
  • Reads: pkg/graphview (one shared view per instance), graph/kvcatalog, natsclient, internal/tlsutil, internal/httpadmission, gorilla/websocket v1.5.3.

Documentation

Overview

Package websocket is the websocket output component: an HTTP server whose upgrade path streams frames to WebSocket clients. Each instance has exactly one input. A kv-watch of every ENTITY_STATES key streams applied state: a client receives one put frame per current entity, then a synced frame, then the changes after it. A core NATS subject streams its JSON messages as data frames, each delivered at most once, with nothing acknowledged or replayed. Every frame is one JSON text message; README.md is the wire format.

Applied state converges rather than replays: a client that folds the frames it receives holds each entity's current state, but intermediate revisions of one key may be merged away. A client that stops reading never slows another: a subject client's oldest queued frame is dropped, an applied-state client's pending changes merge per key, and each is counted and logged.

Every Origin is accepted and the output has no authentication, so it belongs behind the deployment's own network boundary. TLS comes from the deployment's server TLS configuration (file certificates; ACME is not supported).

Start binds the listener and Stop is the only thing that ends it: ending Start's context afterwards stops nothing (ADR-113). A composition registers the factory with Register and creates the component through the component manager, with a configuration that README.md describes.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Register

func Register(registry *component.Registry) error

Register registers the websocket output factory with the component registry.

Types

This section is empty.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL