service

package
v0.1.0-alpha.5 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 16, 2026 License: MIT Imports: 52 Imported by: 0

README

service

Package service holds what every long-running SemEngine service shares: BaseService, which a service embeds for its name, status and health check, and the Service interface a service manager drives. Manager is that service manager: it builds the services a configuration names, starts them, serves the HTTP routes around them and stops them. ComponentManager is the service that owns the components. Metrics is the service that serves the Prometheus endpoint. MessageLogger is the service that keeps the newest messages seen on chosen NATS subjects for inspection. StorageObservabilityService is the service that reports how full the NATS account's storage is. HeartbeatService is the service that logs a liveness line at a fixed interval, MetricsForwarder is the service that publishes the process's metrics on NATS, and LogForwarder is the service that checks the log-forwarding settings and says it forwards nothing yet. The package also holds the helpers a composition root (the process's main function, which builds and wires everything) calls itself: MaybeStartPProf, WireGraphRuntime and the OpenAPI registry, plus the service-configuration and schema helpers the service manager reads.

Of the built-in services the metrics service and the heartbeat are on by default. The metrics forwarder, the log forwarder, the message logger and storage observability run only when the configuration enables them. Each has a section below: The component manager, The message logger, The storage observability service, The heartbeat service, The metrics forwarder and The log forwarder.

Embedding BaseService

A service embeds the *BaseService that NewBaseServiceWithOptions builds from its name and options:

type ruleConfigService struct {
    *service.BaseService
}

svc := &ruleConfigService{
    BaseService: service.NewBaseServiceWithOptions("rule-config",
        service.WithNATS(client),
        service.WithMetrics(registry),
        service.WithLogger(logger),
    ),
}
Option What it does
WithNATS(client) the health check also fails while client.IsHealthy() is false
WithMetrics(registry) every status change is recorded on registry (CoreMetrics().RecordServiceStatus)
WithLogger(logger) replaces the default logger, slog.Default() with a service attribute; nil keeps the default. BaseService itself logs nothing; the services in this package log through it
Start and Stop

Start(ctx) reports StatusRunning and runs the health check at once and then every 30 seconds. It refuses a nil or ended ctx, a second Start, and a Start after any Stop (errs.ErrAlreadyStarted). A BaseService runs once: build a new one to start again. Once Start has returned, ending its ctx stops nothing; Stop is the only stop (ADR-113).

Stop(ctx) refuses a nil ctx. On a started service any non-nil ctx (live or already ended) initiates teardown: it reports StatusStopping, cancels the health monitor's run context, and bounds only the wait for the health goroutine to exit, returning nil once joined (leaving StatusStopped and IsHealthy() false) or ctx's error if ctx ended first so a later Stop waits again. Pass Stop a fresh bounded context so the goroutine has time to join cleanly instead of returning an already-ended signal context's error:

stopCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if err := svc.Stop(stopCtx); err != nil {
    logger.Warn("rule-config stop", "error", err)
}

A Stop after one that joined everything returns nil for any non-nil ctx. A Stop while another Stop is still waiting is refused as a transient error (errs.IsTransient). Stop before Start returns nil, and the service then refuses to start.

A service that relied on the end of its Start context to stop its health loop, as SemStreams' BaseService allowed, now calls Stop.

Health

The health check fails when the function set with SetHealthCheck returns an error, or when the client from WithNATS is not healthy. IsHealthy() is then false and Health() reports unhealthy; the next passing check makes the service healthy again. While the last check passed, Health() reports a running service as healthy and a stopping one as degraded; a stopped service is unhealthy. GetStatus() returns the counts of checks run and failed.

The service manager

Composing and starting

A composition root builds a Manager from a Registry of service constructors, hands it the services section of the configuration, and starts it:

services := service.NewServiceRegistry()
if err := service.RegisterAll(services); err != nil { // offers every built-in service
    return err
}

manager := service.NewServiceManager(services)
manager.UseHTTPMiddleware(authMiddleware) // optional; before StartAll

deps := &service.Dependencies{
    NATSClient: client, MetricsRegistry: registry, Logger: logger,
    Platform: platform, Manager: configManager,
    ComponentRegistry: components, PayloadRegistry: payloads,
}
if err := manager.ConfigureFromServices(cfg.Services, deps); err != nil {
    return err
}
if err := manager.StartAll(startCtx); err != nil {
    return err
}

// ... run ...

stopCtx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
if err := manager.StopAll(stopCtx); err != nil {
    logger.Warn("stop services", "error", err)
}

ConfigureFromServices resolves the configuration with ResolveServiceConfigs and constructs every enabled service from its registered constructor. Nothing is composed unless every constructor succeeds. component-manager and service-manager may not be disabled, and a composition whose component-manager has no registered constructor is refused. RegisterAll registers one, and the component manager needs the Manager, ComponentRegistry, Platform and PayloadRegistry fields shown above (see Composing it). CreateService and RegisterInstance compose one more service by hand, and GetService finds a composed one.

The service-manager entry configures the Manager itself (ManagerConfig):

Key Default What it does
http_port 8080 the main listener's port, on every interface
swagger_ui false serves /docs when true
server_info SemEngine API, SemEngine framework API, 0.0.0 title, description and version of the API document
http_read_timeout 30s how long a request body may take (a Go duration string)
http_write_timeout 120s how long a response may take

A service's config is decoded strictly. An unknown key is refused, and so is a key that differs from a field's name only in letter case ("Port" for "port"), with both spellings named in the error. This holds for the service-manager, metrics and component-manager entries, which this package decodes. A service defined elsewhere decodes its own config, and the check applies to it only if it makes it.

Starting and stopping

StartAll(ctx) creates component-manager if no service has that name, then seals the composition: CreateService, RegisterInstance and ConfigureFromServices are refused from then on. It binds the main listener on http_port, starts the metrics server it claims, starts every service in registration order, builds the full route set, and starts a publisher that sends each service's health on health.service.<name> every 5 seconds. It runs once: a second StartAll, and a StartAll after StopAll, are refused with errs.ErrAlreadyStarted. If a step fails, StartAll stops what it started and returns both errors; if that stop fails too, StopAll runs it again.

The context given to StartAll bounds StartAll and nothing else. Ending it after StartAll returned stops no service, and no request's context derives from it (ADR-113). StopAll(ctx) is the only stop. It stops the services in reverse registration order, then the health publisher, the listeners and the metrics server. Each listener refuses new requests and StopAll waits for those in flight. If ctx ends first, StopAll closes every connection still open and returns ctx's error, and a later StopAll waits again for the handlers that are still running. So a service that serves a long-lived response, such as a stream, ends it in its own Stop.

StopAll refuses a nil context; any non-nil context (live or already ended) initiates teardown across every started service and listener, using ctx only to bound graceful drain and join waits. Pass a fresh bounded context so services and listeners have time to drain. A service whose Stop returns errs.ErrAlreadyStopped counts as stopped. Any other failure is returned naming the service, and the next StopAll runs the whole stop again. A StopAll after one that joined everything returns nil for any non-nil context; one while another runs is refused as transient.

StartHealthListener(ctx, port) binds a second listener that serves only /health and /healthz, for container probes that want a port apart from http_port. Call it before StartAll and probes work while a slow boot runs. Port 0 does nothing, a second call is refused, and StopAll stops the listener whether or not StartAll ran.

Middleware

UseHTTPMiddleware(mws...) wraps every route the main listener serves, the startup routes as well as the full set. The first middleware passed is the outermost. The framework ships none: authentication, request logging, panic recovery, rate limiting and CORS are the product's. The dedicated health listener takes no middleware. Call it before StartAll; a later call is ignored with a Warn, because the listener's handler is built when it binds. This is phase 1 of SemStreams ADR-030, carried as docs/adr/030-http-middleware-and-identity-pattern.md.

Boot seals the composition

A change to the services section of the configuration after StartAll is desired state for the next boot. It starts, stops and reconfigures nothing, and /services reports it with restart_required set. This is the part of SemStreams ADR-094 that the Manager carries, in docs/adr/094-boot-only-composition-and-observable-rule-activation.md.

The routes

The main listener serves the first five routes below from the moment StartAll binds it, and answers every other path 503 NOT READY until StartAll has started every service and committed the full route set. The last four come with the commit.

Route Answer
/health every service's health and the NATS connection's, aggregated; 503 when unhealthy
/healthz 200 OK while the listener serves
/readyz 200 READY once every service has started and is running and healthy; 503 NOT READY while starting, after a start failed, and from the moment StopAll is entered, even while a service's Start is still running
/services each service's status and health, the startup counts, and restart_required with the pending configuration changes
/services/health each service's health and their aggregate; 503 when unhealthy
/graph/triples triples read from the graph's ENTITY_STATES bucket (below)
each HTTPHandler service's routes under a prefix taken from the service's name: the name with its hyphens removed, except component-manager at /components and message-logger and storage-observability as named
/openapi.json the API document, in OpenAPI 3.0 form: the paths of every HTTPHandler service under its prefix, and the schemas of the types registered with RegisterOpenAPISpec
/docs a Swagger UI page for /openapi.json, only when swagger_ui is true; it loads its script and style sheet from unpkg.com, so the browser needs network access to it

The counts /services reports are also exported as the gauge semengine_startup_units, with an owner label (services or components) and a stage label. It has nine series: admitted, starts_invoked, starts_completed and starts_failed for services, and those four plus lifecycle_participants for components. The gauge is read from the same counts each time Prometheus collects, so it is never older than /services. StartAll registers it on the injected metrics registry, else on the metrics service's, and registers nothing when neither exists.

GET /graph/triples is for an operator or a test, not a query API: it scans the bucket. Its query parameters are all optional:

Parameter Meaning
subject a literal entity ID
predicate a three-part predicate
object compared as text
limit default 100, at most 1000

It answers a JSON array sorted by entity ID, never null. A malformed parameter is 400 naming the parameter, a method other than GET is 405, and a bucket that cannot be read is 500. With no NATS client in the Dependencies, it answers [] for every query: there is no store to read. The Manager only reads the bucket (ADR-110).

The component manager

A component is one unit of a running composition: an input, processor, output or storage unit. A binary makes the component factories available through a component.Registry, and the components section of the configuration names the instances to build. ComponentManager is the service that builds every enabled component, starts them, serves their status over HTTP and stops them. Every composition has one: component-manager may not be disabled.

Composing it

RegisterAll registers the constructor, NewComponentManager, and the component manager's OpenAPI spec (see OpenAPI). The component-manager entry takes no configuration: "config": {} or none at all. Any key is refused, naming it. The components come from the Dependencies the composition root passes to ConfigureFromServices:

Dependencies field What the component manager does with it
Manager (*config.Manager) required; the boot configuration it reads, once
ComponentRegistry required; the factories it builds from, and the registry it seals
PayloadRegistry given to each component
NATSClient given to each component; with none, the manager publishes no component health
Platform the deployment's identity, given to each component
MetricsRegistry, Logger given to each component; the manager's own metrics and log lines

NewComponentManager reads the boot configuration once, and later writes to it change nothing in the running set: they are desired state for the next boot (SemStreams ADR-094 and ADR-096, carried in docs/adr). It builds each enabled component through Registry.CreateComponent, in instance-name order, and skips the disabled ones. It then checks the whole set with composition.Analyze and seals the registry (ADR-100). It returns an error, and no service, when:

  • the Manager or the ComponentRegistry is missing;
  • any enabled component cannot be built. The error names every one that failed, because a boot that quietly runs without a configured component would report ready while doing less than it was told;
  • the composition has an error-severity finding. The error reads composition validation refused boot with N error finding(s): ... and names each finding.
Starting and stopping

Start(ctx) refuses a nil or ended ctx, a second Start, and a Start after any Stop (errs.ErrAlreadyStarted). The start order is fixed when the manager is built: components that provide stores (component.StoreProvider) by instance name, then the rest by instance name. Start runs in two phases, providers first. Each phase starts its components concurrently, and the next phase begins only after every Start in the current one returned, so a provider's stores are in the shared store registry before any consumer starts. A failure in the first phase starts nothing of the second. A component that does not implement component.LifecycleComponent is built and listed but never started or stopped.

Each component's Start receives the context the manager's Start received, the same value. The manager keeps no context of its own for a component and never ends the one it was given, Stop included (ADR-113). A component therefore makes its own run context in Start and ends it in Stop; a component that hangs its continuing work on its Start context stops when the caller ends that context, which for a program that passes its signal context is the start of shutdown, before Stop reaches the component. For the same reason ending the manager's Start context after Start returned stops nothing. The manager's health publisher (below) runs under a run context that Start creates and Stop ends.

A failed Start returns every component failure joined, each naming its component, after one rollback: it stops every component it launched, the failing ones included, in reverse start order. A component whose Start returned nil after ctx ended counts as failed. If the rollback fails too, its error is joined into Start's and the next Stop runs it again.

Stop(ctx) is the only stop. It refuses a nil ctx; any non-nil ctx (live or already ended) initiates teardown and uses ctx only to bound waits, so pass a fresh bounded context on shutdown to give components drain time. Before any Start it returns nil, and a later Start is refused. Otherwise it marks the stop begun (/readyz answers 503 NOT READY from then), stops each started component with ctx in reverse start order (removing each provider's stores from the store registry before its Stop, even when ctx has already ended), then ends the manager's own loops and its base service. It returns nil only once everything is released. A component whose Stop failed (including one cut short by ctx) is stopped again by the next Stop; one whose Stop returned nil is not called again. A Stop after one that joined everything returns nil for any non-nil ctx, and a Stop while another runs is refused as transient. A Stop that arrives while Start is still running waits for every component's Start to return, or returns its own context's error first; if that Start failed, it then stops only what the rollback left. A caller cuts a slow boot short only by ending the context it gave Start.

Reading status during and after Stop

GetComponentHealth, GetComponentStatus, the /components routes and the manager's own health check wait on no lock and not on Start or Stop. Each reads the component's record and asks the component itself, so a request that arrives during or after Stop gets whatever the component answers then. SemStreams kept such requests away from a stopping component; this manager does not. The line a component author holds is in component.LifecycleComponent: Health, DataFlow and DebugStatus may be called at any time and return without waiting for Start or Stop. The tree enforces it for graph-ingest (TestGraphIngestHealthAndDataFlowAnswerDuringAndAfterStop); for a component outside this module it is review only, and its author checks it.

The manager's own health, which BaseService runs and /readyz reads, is unhealthy while any component is in the failed state. It names each failed component in name order with its last error, and reads every component each time.

What it publishes and reports
  • Component health. With a NATS client, the manager publishes {"timestamp","name","health"} for each admitted component on health.component.<name>, once as Start returns and then every 5 seconds. A publish or encode that fails is counted on semengine_errors_total{service="component-manager",type="health_publish"} and logged at Debug with the subject. The next component and the next round are still published, because a missed report is replaced by the next one.
The routes

The routes are mounted under /components and answer GET only (405 otherwise). A component name that is empty, ., .. or decodes to a path separator is 400, and an unknown one is 404. Until StartAll commits the full route set, only health, list and status/{name} are served, and every other path answers 503 NOT READY.

Route Answer
/components/health each admitted component's health and their aggregate; 503 when any component is unhealthy
/components/list each admitted component's name, state, factory, category, enabled and health, in no fixed order
/components/types the registered factories with their schemas and default ports, as an array
/components/types/{id} one factory's entry
/components/status/{name} one component's state, health and last error; ?debug=true adds its DebugStatus when it has one
/components/config/{name} the boot configuration the instance was built from
/components/flowgraph the composition's graph as JSON, or Mermaid text with ?format=mermaid
/components/validate the composition check's result: status, errors and warnings
/components/paths for each input component, every component reachable from it, with counts

All three read the composition result kept when the manager was built, so they cannot disagree with the check that admitted the boot (ADR-100). validate serves that result as is. flowgraph serves its graph, as JSON or as composition.Mermaid. paths serves composition.FlowPaths of its graph, with counts of that answer (the number of inputs, the longest path, the total and the average). GetFlowPaths is the same answer as a call. BuildComponentTypeCatalog(registry, logger) returns what /components/types serves, for a product that serves the catalog elsewhere. GetComponentStatus and ComponentStatus are read inside this package only.

A declared difference you may meet: GET /components/types/{id} for an unregistered type answers 404 with the body {"error":"Component type <id> not found"} sent as text/plain, not JSON, with the request's own path segment in it. SemStreams answers it the same way, and no contract line promises the body, so read the status code and not the text.

Rule packs

A rule processor (a component this repository has not ported yet) can declare contracts for the predicate groups it writes, and replace a whole group with one reconcile call (projection.PredicateReconciler, implemented by projection.MutationClient). It implements ProjectionBinder to receive that client. The component manager does not hand it one; the composition root does, in this order, after ConfigureFromServices and before StartAll:

if err := manager.ConfigureFromServices(cfg.Services, deps); err != nil {
    return err
}
if err := service.ConfigureRulePackMutations(manager); err != nil {
    return err
}
if err := manager.StartAll(startCtx); err != nil {
    return err
}

ConfigureRulePackMutations finds the admitted components that implement ProjectionBinder and runs every binder's PreflightProjectionMutations before changing anything, so one failure injects nothing. It then validates each pack_id and refuses a duplicate, copies and validates each pack's contracts, refuses contracts when there is no NATS client, and builds one projection.MutationClient per pack that has contracts and hands it to that binder's SetPredicateReconciler. If a composition root never makes the call, no binder is given a reconciler. No rule processor is in this tree yet, so today that has no effect; what a rule processor does without one is stated when it is ported. ComponentsImplementing[T](manager) returns the admitted components that implement T, in start order, and nil before a component manager is composed. Manager.ProjectionBinders is that call for ProjectionBinder.

The component manager's OpenAPI spec

/openapi.json takes its schemas, here Graph and Result for the flowgraph and validate responses, only from the process-wide registry that RegisterOpenAPISpec fills. RegisterAll registers the component manager's spec there, because importing the package registers nothing. A composition that registers NewComponentManager by hand instead of calling RegisterAll must register the spec itself, or those two references name schemas the document does not carry:

if svc, ok := manager.GetService("component-manager"); ok {
    service.RegisterOpenAPISpec("component-manager", svc.(service.HTTPHandler).OpenAPISpec())
}

The rule for any service that does this is in the developer contract: every SchemaRef type is listed in its spec's ResponseTypes or RequestBodyTypes, and the composition root registers the spec.

Not ported

Initialize (construction does that work), IsInitialized, IsStarted, GetHealthyComponents, GetUnhealthyComponents, DefaultComponentManagerConfig, the configuration's enabled_components key and its Validate, and the component manager's config subscribers and live-configuration routes, which ADR-094 retired.

The message logger

MessageLogger is a service that subscribes to NATS subjects and keeps the newest messages it sees in memory, so a developer can look at what flows through a running system. It also reads NATS KV buckets (NATS's key-value store) over HTTP. It writes nothing to NATS and nothing to the graph.

Enabling it

The message logger is off unless the configuration turns it on: ResolveServiceConfigs adds component-manager, service-manager and metrics when they are missing, and does not add message-logger. RegisterAll registers its constructor, NewMessageLoggerService, and its OpenAPI spec. Enable it in the services section:

{
  "services": {
    "message-logger": {
      "enabled": true,
      "config": {
        "monitor_subjects": ["agent.>", "tool.>"],
        "max_entries": 10000
      }
    }
  }
}

Its Dependencies fields: NATSClient and PayloadRegistry are required, ComponentRegistry is required when monitor_subjects holds "*", and MetricsRegistry and Logger are optional.

Key Meaning
monitor_subjects NATS subjects to watch. "*" stands for the NATS, NATS-request and JetStream port subjects of every admitted component (never *, > or _INBOX subjects). Empty means ["*"].
max_entries How many messages to keep, 0 to 100000. 0 means 10000. The oldest are overwritten.
sample_rate Store one message in every sample_rate and only count the rest. 0 and 1 store every message.
output_to_stdout Also print one line per stored message.

Construction refuses, naming the problem: an unknown key (or a key in another letter case), a nil NATS client, a nil payload registry, "*" without a component registry, max_entries below 0 or above 100000, and sample_rate below 0.

What it captures

Start works out the subject set once. Each subject is then checked against one grammar, owned by internal/flowgraph: one or more .-separated tokens, none empty; * and > only as a whole token; > only as the last; no space, tab, line feed, carriage return or form feed. This is stricter than NATS in two ways: NATS takes a token such as a* as plain text and accepts a form feed, and the logger refuses both. A subject that fails the check makes Start fail, as invalid data, with an error naming the subject and where it came from (monitor_subjects, or the component and port that declare it). The check runs before the base service starts and before any NATS call, so a refused Start holds nothing, connected or not.

A subject that another subject in the set covers (a.> covers a.b) is not subscribed, and the omission is listed in /stats under subject_overlaps. Two subscribed subjects can still overlap without either covering the other (a.b.* and *.b.c both match a.b.c); a message on such a subject is then stored twice.

Each message is stored as an entry: a sequence number, the time, the subject, the message type and summary, the trace and span IDs when the message carries them, and the payload as raw_data. A message that decodes as a typed envelope (a message.BaseMessage whose type is in the payload registry) is stored under its type name; any other is stored with type raw. A payload that is not JSON is stored as a JSON string, so every entry can be served.

Starting and stopping

Start(ctx) refuses a nil or ended ctx, a second Start, and a Start after any Stop (errs.ErrAlreadyStarted); a refused Start holds nothing. ctx bounds Start only: ending it afterwards stops nothing, and messages keep being stored, because each message's context belongs to the NATS client, not to ctx. Start checks ctx before each subscribe and once more after the last; if ctx has ended at any check, Start returns its error and releases what the logger holds. Ending ctx also ends a running subscribe's wait for the server's answer, but not its waits for the connection itself (see "A write the server is not reading holds the connection" in the transport-client spec), so how long Start can outlast its context is not promised.

A subscribe that fails for any other reason does not fail Start. The logger keeps running and retries that subject every 100 ms until it succeeds. While any subject is unsubscribed:

  • /stats reports subject_reconciliation_degraded as true and subject_reconciliation_error names each subject with its last error;
  • the logger's health check fails, naming the subjects, so IsHealthy turns false and /readyz answers not ready. Health checks run every 30 seconds, so the change shows within one interval of the failure and again within one interval of the retry succeeding. On a normal boot the first check runs before any subscribe and is healthy.

Stop(ctx) is the only stop. It refuses a nil ctx. On any non-nil ctx (live or already ended) it fences new KV watch streams, ends the retry, initiates drain on every subscription (a message being handled when Stop begins is stored), closes every open KV watch stream, and stops the base service, using ctx only to bound waits and returning nil once all are released. If ctx ends first, or a drain fails, Stop returns the error and leaves what is left for the next Stop. A Stop after one that joined everything returns nil for any non-nil ctx; a Stop while another runs is refused as transient. Messages published after Stop are not stored.

What /stats reports
Key Meaning
total_messages Every message received and not dropped, sampled or not.
dropped_messages Messages not stored (and not in total_messages) because their handler context had ended; the NATS client ends it once its Close has begun. Each is logged at Warn with its subject.
sampled_messages Messages stored under sample_rate.
valid_messages Stored messages that decoded as typed envelopes.
invalid_messages Stored messages that did not (type raw).
sample_rate, max_entries The configured values, after defaults.
start_time, last_message_time, uptime_seconds Logger start, last message received, and seconds since start.
monitored_subjects The subjects currently subscribed, sorted; empty after Stop.
subject_overlaps Each omitted covered subject: broader, covered, and resolution (covered subscription omitted).
subject_reconciliation_degraded True while a subject is unsubscribed.
subject_reconciliation_error The unsubscribed subjects and their last errors; empty when none.
The routes

Mounted under /message-logger/ and answering GET and HEAD (405 for any other method). entries, stats and trace answer from memory at any time, including after Stop.

  • entries: the newest entries, newest first. limit defaults to 100 and is capped at 10000; a missing, non-numeric, zero or negative value gives 100. subject is a NATS subject pattern (* one token, > the remaining tokens) applied before the limit; a pattern NATS cannot parse matches nothing. subject=* matches only one-token subjects: write > for all, and agent.> for everything under agent.
  • stats: the object above.
  • subjects: the subscribed subjects as a JSON array; [] after Stop.
  • trace/{traceID}: the entries still held for a trace ID (32 hex characters, else 400), oldest first, as {"trace_id", "count", "entries"}. An entry the buffer has overwritten is gone.
  • kv/{bucket}: the entries of an existing KV bucket, keys sorted before limit applies. pattern is a key glob: one * as prefix, suffix, both ends or in the middle, or none for an exact key; the default matches every key. limit defaults to 100 and is capped at 1000. status keeps only records whose top-level JSON status equals it. Values are answered as stored (a value that is not JSON as a JSON string). A bucket that does not exist is 404 and is never created; a bucket name that is empty, ., .. or holds /, \, * or > is 400; a lookup, listing or read failure is 500, naming the key for a read.
  • kv/{bucket}/watch: a stream of Server-Sent Events (SSE), described next.
The watch stream

kv/{bucket}/watch answers 503 once Stop has begun, 400 for a bad bucket name or pattern (empty or * watches every key; anything else must be a NATS KV key filter, and a final > is accepted), 404 for a missing bucket and 500 for another lookup failure; none of these opens a stream. Otherwise it sends:

  1. event: connected with {"bucket","pattern","message"}, then retry: 5000;
  2. one event: kv_change for each key's current value, then one with operation initial_sync_complete;
  3. one kv_change per later change. operation is what NATS records for the entry: put, delete or purge. value is present for a put only.

The logger does not read Last-Event-ID: a client that reconnects gets a new initial replay. Each stream buffers up to 100 events between its KV watcher and the client. If the client reads too slowly and an event does not fit, the stream ends: the events already buffered are sent, then one event: error with {"error":"watch fell behind: events were dropped"}, and the response ends. No kv_change, and no initial_sync_complete, follows a dropped event, so a client that sees initial_sync_complete has seen every initial value. Each stream runs its own KV watcher.

Not ported

GetMessages, ConfigSchema, DefaultMessageLoggerConfig, the exported NewMessageLogger and SetDecoder, the reconciliation of a changing subject set, and the test-only seams SemStreams carried. service/configurable.go (ConfigSchema, PropertySchema, NewConfigSchema) is removed with them.

The storage observability service

StorageObservabilityService watches how full the NATS account's storage is. Every minute it lists the account's JetStream streams and key-value buckets (JetStream is NATS's persistent storage; each stream or bucket is one "resource"), works out how much each holds, how fast that grows and how close it is to its limit, and publishes the result as a report in the STORAGE_REPORT bucket. It then serves that one report three ways: Prometheus metrics, a health message and an HTTP route. It writes no graph entity.

The service is report-only. It rejects no write, throttles no component, fails no readiness or health check and applies no retention because of what it reports. A pressure state ("normal", "warning", "high" or "critical", set by how much of a limit is still free and how soon growth would use it up) is a fact for a human or an alert rule to act on. The SemStreams pin's storage observability spec records why nothing gates on pressure yet: capacity admission (refusing a write when storage is tight) stays deferred until four conditions hold. The report runs in production; the projection has been checked against what happened on at least three real resources; no critical state has cleared without an operator acting; and the rejection path is proven to classify a refusal as transient and NAK it (ask the sender to retry) instead of acknowledging and dropping the write. That text is SemStreams' and is not ported here. A consumer that wants backpressure today reads the report and decides in its own code. The reasoning behind reporting through logs and metrics, not graph triples, is ADR-098.

Enabling it

The service is off unless the configuration turns it on: ResolveServiceConfigs does not add it. RegisterAll registers its constructor, NewStorageObservabilityService, under storage-observability, and no OpenAPI spec for it (the route's spec names no schema). Enable it in the services section; every key is optional:

{
  "services": {
    "storage-observability": {
      "enabled": true,
      "config": {
        "interval": "1m",
        "timeout": "15s",
        "pressure_thresholds": {"critical_headroom": 0.05}
      }
    }
  }
}

It needs Dependencies.NATSClient and Dependencies.MetricsRegistry; Logger is optional. The service manager passes the registry it holds.

Key Meaning
interval How often the account is listed. Default 1m. Every process that polls multiplies the cost, so increase the interval to poll less often when many instances run.
timeout The bound on one collection, both listings included. Default 15s; it must not exceed interval.
pressure_thresholds When a resource counts as warning, high or critical: warning_headroom (default 0.25), high_headroom (0.15) and critical_headroom (0.05) are the free fraction at or below which the state applies; warning_horizon (72h), high_horizon (24h) and critical_horizon (4h) are the projected time until the critical level is reached.

Construction refuses, as invalid data naming the input: an unknown key (nested keys under pressure_thresholds included, and a key in another letter case), nil dependencies, a nil NATS client, a nil metrics registry, an interval or timeout that does not parse or is not positive, a timeout above the interval, and thresholds that are out of order or out of range. Configuration is read once; to change a value, restart.

Starting and stopping

Start(ctx) refuses a nil or ended ctx, a second Start, and a Start after any Stop (errs.ErrAlreadyStarted); a refused Start holds nothing. It does no I/O and never fails on account or bucket state: it starts two background loops and returns. One collects every interval. The other watches STORAGE_REPORT and feeds the metrics, the health message and the route. The service creates or reconciles STORAGE_REPORT on first use, so a deployment that names this service still boots when the account cannot be read; each loop retries on its own schedule (the watch every 5 seconds). ctx bounds Start only: ending it afterwards stops nothing.

Stop(ctx) is the only stop. It ends both loops, waits for them and stops the base service, returning nil once all three are done. If ctx ends first, Stop returns that error and the next Stop waits again. How soon a collection already inside the account listing notices that Stop began depends on nats.go's listing honouring its context, which nats.go does not promise; Stop stays bounded by its own ctx either way. A Stop after one that returned nil returns nil; a Stop while another runs is refused as transient.

The health message

Health returns the base service's verdict untouched and appends the storage picture to its message, joined with a semicolon. Pressure never changes Healthy, Status or a sub-status, so critical pressure leaves /readyz answering READY and /health answering 200. The picture is one line, for example storage pressure (report-only): critical=1 high=0 warning=0 normal=12 not-evaluated=0; worst: ORDERS critical (headroom), then any over-committed tier, then how long ago the last collection ran. Before the first report is read it says storage report has not been read yet; after a failed collection it says so and serves the last result. After Stop the verdict is the base service's own (Service is unhealthy (failed checks: 0)) with the last picture appended. Both reads do no I/O.

The route

GET /storage-observability/report (mounted by the service manager; HEAD is answered as GET, any other method gets 405) always answers 200, whatever the report says. The body:

  • report_only: always true.
  • synced: whether the watch has read every current value at least once. Until then an empty resources means "not read yet", not "the account holds nothing".
  • updated_at: when the last change reached this process; absent before the first.
  • summary: resources (a count), pressure (rows per evaluated state, {} when none), not_evaluated (rows with no pressure state because their capacity could not be read, or because they are unbounded and their tier has no limit either) and worst_pressure (absent when nothing was evaluated; absent does not mean normal).
  • account: per storage tier (file, memory, unknown), the declared bounds against the account limit; absent until an account row is read.
  • resources: every published row, sorted by name, never null, never filtered. A resource with no bound of its own is evaluated against its tier's account limit and says so in pressure.evaluated_against (account-tier): that state is shared by every unbounded resource in the tier, and changing one resource cannot relieve it.

The health message, the metrics and the route read one consumer of the report, so they cannot disagree.

The metrics

The service registers 19 gauges on the injected registry under the name storage-observability, all named semengine_storage_*. The SemStreams pin published the same series as semstreams_storage_*; nothing renames them for you, so an alert or dashboard still on the old names reads nothing.

  • Per resource, labelled resource, owner, kind and tier: resource_used_bytes, resource_limit_bytes, resource_headroom_bytes, resource_headroom_ratio, resource_growth_bytes_per_second, resource_time_to_threshold_seconds and resource_pressure (0 normal, 1 warning, 2 high, 3 critical).
  • Counts: resources (by tier and capacity_state) and pressure_resources (by tier and pressure_state, including not-evaluated).
  • Per storage tier, labelled tier (and state for the three that carry one): account_limit_bytes, account_used_bytes, account_declared_bytes, account_limit_state, account_overcommitment, account_growth_bytes_per_second, account_headroom_bytes, account_time_to_threshold_seconds and account_pressure_state.
  • report_collected_timestamp_seconds: when the freshest row was collected (not scraped), so time() - semengine_storage_report_collected_timestamp_seconds > 900 catches a collector that stopped. It is zero until a row with a collection time is read.

A value that was not measured has no series; it is never zero, because zero means "measured, and not growing". A series is deleted when its value goes away, when a resource leaves the report, and, for the per-tier series, when the account row is dropped. The count and state families are always emitted in full.

Example Prometheus alert rules are at configs/prometheus/rules/storage-pressure.yml: 12 alerts, each reading only published names, none firing because a report row disappeared, with unbounded and unreadable resources kept visible. Copy the file and tune it; the engine does not load it. service/storage_observability_http_test.go checks it against the metric names.

Not ported

Snapshot and RegisterMetrics (nothing read them), the schema struct tags on StorageObservabilityConfig, and the pin's function fields and nil checks on the two surfaces.

The heartbeat service

HeartbeatService writes one log line at a fixed interval so an operator can see the process is alive and how many of its components are healthy. It is on by default: the configuration loader's defaults turn on a heartbeat service with a 30 second interval, so a composition that names no services still gets it. RegisterAll registers its constructor, NewHeartbeatService, under heartbeat, with no OpenAPI spec because it serves no route. To change the interval, or to turn it off, name it in services:

{
  "services": {
    "heartbeat": {"enabled": true, "config": {"interval": "1m"}}
  }
}
Key Meaning
interval How often a line is written. Default 30s; an empty value takes the default; it must be at least 1s.

It needs Dependencies.ServiceManager; Logger is optional. Construction refuses, as invalid data naming the input: an unknown key or a key in another letter case, nil dependencies, a nil service manager, and an interval that does not parse or is under 1s. A miscased key matters because the loader merges your services block with the defaults: {"Interval":"5s"} beside the default {"interval":"30s"} used to run at 30 seconds without a word; now the composition is refused, naming both spellings. Configuration is read once; to change a value, restart.

The line is written at Info, once when the service starts and then once per interval. Its message is System heartbeat, and it carries uptime (the service's, rounded to the second), goroutines (the process's count), components_healthy and components_total (how many of the component manager's components report healthy, out of how many it holds). The SemStreams pin wrote it at Debug, which shows nothing at the default log level; the owner chose Info (2026-10-23).

Start(ctx) looks up the component manager through the service manager the heartbeat was built with and refuses, naming component-manager, when there is none or it is not the framework's *ComponentManager; the component manager need only be composed, not started. It also refuses a nil or ended ctx, a second Start and a Start after any Stop (errs.ErrAlreadyStarted); a refused Start holds nothing. It starts one background loop and returns; ctx bounds Start only, and ending it afterwards stops nothing. Stop(ctx) is the only stop: it ends the loop, waits for it and stops the base service. If ctx ends first, Stop returns that error and the next Stop waits again. A Stop after one that returned nil returns nil; a Stop while another runs is refused as transient.

Not ported

HeartbeatConfig.Validate, the test-only constructor and the interface SemStreams used to swap the component manager in tests.

The metrics forwarder

MetricsForwarder copies the process's own metrics onto NATS. At Start and then every push_interval it reads every metric from the metrics registry it was given (Dependencies.MetricsRegistry) and publishes each sample as one JSON object through the NATS client's JetStream publish. It is off unless the configuration enables it: ResolveServiceConfigs does not add it. RegisterAll registers its constructor, NewMetricsForwarderService, under metrics-forwarder, with no OpenAPI spec because it serves no route.

{
  "services": {
    "metrics-forwarder": {
      "enabled": true,
      "config": {"push_interval": "5s", "include_go_metrics": false, "include_proc_metrics": false}
    }
  }
}
Key Meaning
push_interval How often a round is published. Default 5s; an empty value takes the default; it must be positive.
include_go_metrics Also forward the go_* runtime families (goroutines, memory, garbage collection). Default false.
include_proc_metrics Also forward the process_* families (CPU, open files, memory). Default false.

It needs Dependencies.NATSClient and Dependencies.MetricsRegistry; Logger is optional. Construction refuses, as invalid data naming the input: an unknown key or a key in another letter case, nil dependencies, a nil NATS client, a nil metrics registry, and a push_interval that does not parse or is not positive. Configuration is read once; to change a value, restart.

Each sample goes to the subject metrics.<component>.<name>, where <component> is the sample's component label (system when it has none) and <name> is the metric's name. The payload is one JSON object with six fields: timestamp (Unix milliseconds; the samples of one round share it), name, component, type (counter, gauge, histogram, summary or untyped), value and labels (an object, {} when the sample has none). value is the counter, gauge or untyped value; for a histogram or summary it is the number of observations, not a distribution. A JetStream stream must capture metrics.> for the samples to be kept. The forwarder creates none: config already declares the framework's METRICS stream over metrics.> (config/streams.go), and the composition provisions it with config.StreamsManager.EnsureStreams, or points at an existing stream that captures metrics.>. The SemStreams pin's metrics carried the prefix semstreams_; these are named semengine_*, and nothing renames the old series, so a reader still on the old names finds none.

A sample that cannot be published is dropped and counted; the loop goes on and the next round publishes the series again. A sample that does not encode (a NaN or infinite value) or does not publish increments semengine_errors_total{service="metrics-forwarder",type="metrics_publish"}. A gather that fails increments type="metrics_gather" and skips the whole round. Each is also logged at Debug. A label value is spliced into the subject unchecked, and a publish that fails for that reason is counted like any other. The forwarder also records its state on semengine_service_status{service="metrics-forwarder"} (0 stopped, 2 running), as the other services do.

Start(ctx) refuses a nil or ended ctx, a second Start and a Start after any Stop (errs.ErrAlreadyStarted); a refused Start holds nothing. It does no I/O: it starts one background loop and returns, and a NATS client that is not connected makes each publish fail and be counted. ctx bounds Start only; ending it afterwards stops nothing (in SemStreams it marked the service stopped while the loop went on publishing). Stop(ctx) is the only stop and behaves as the heartbeat's does.

Not ported

The MetricsGatherer interface, MetricsForwarderConfig.Validate and the test-only constructors.

The log forwarder

LogForwarder forwards nothing yet. Its name comes from SemStreams, where a log handler that boot installed (pkg/logging) sent log records to NATS and this service only checked the handler's settings. This module has no log handler that publishes logs to NATS, and pkg/logging and the boot wiring that would install one are not ported. What the service does today is check its settings, run the base service's lifecycle, and write one record saying so. It is off unless the configuration enables it: ResolveServiceConfigs does not add it. RegisterAll registers its constructor, NewLogForwarderService, under log-forwarder, with no OpenAPI spec because it serves no route.

{
  "services": {
    "log-forwarder": {
      "enabled": true,
      "config": {"min_level": "INFO", "exclude_sources": ["a.b"]}
    }
  }
}
Key Meaning
min_level The lowest level a forwarded record would need: DEBUG, INFO, WARN or ERROR, in any letter case. Default INFO; an empty value takes the default.
exclude_sources Source prefixes, in dotted notation, whose records would not be forwarded: a.b would also exclude a.b.c but not a. Each entry is trimmed, empty entries are dropped and duplicates are removed. Nothing is added to the list.

Both values are checked and reported, not applied: nothing reads them to filter a record. A composition root that wants logs on NATS installs its own slog.Handler that publishes them (semteams does so in its own code; cmd/semteams/logging.go:71-93 at the time the design was written).

Construction needs Dependencies and nothing else: no NATS client, no I/O. Logger and MetricsRegistry are optional. It refuses, as invalid data naming the input: nil dependencies, a document that does not parse or holds more than one JSON value, an unknown key or a key in another letter case (Min_Level is refused, min_level is written), and a min_level other than the four names above. An empty, whitespace-only or null configuration takes the defaults. Configuration is read once; to change a value, restart.

At Start the service writes one record at Warn, through the injected logger (slog.Default() when none), with the message log-forwarder forwards nothing: no log handler in this module publishes logs to NATS and the attributes min_level (the resolved level's name) and exclude_sources (the resolved list, [] when empty). The logger carries service=log-forwarder, so the record has that attribute once. It writes no other record while it runs, and none when Start is refused. The service also records its state on semengine_service_status{service="log-forwarder"} (0 stopped, 2 running), as the other services do.

Start(ctx) refuses a nil or ended ctx, a second Start and a Start after any Stop (errs.ErrAlreadyStarted); a refused Start holds nothing. It starts no goroutine of its own; the one the base service runs is its health check. ctx bounds Start only; ending it afterwards stops nothing. Stop(ctx) is the base service's, and the only stop: it refuses a nil ctx, returns nil before Start, and once started stops the base service on any non-nil ctx (returning ctx.Err() if ctx had ended, or nil otherwise); a later Stop returns nil.

Not ported

NewLogForwarder, LogForwarderConfig.Validate, LogForwarder.Config and the Stop override. The SemStreams service also always excluded the source flow-service.websocket; this one adds nothing to the configured list. service/milestone_service.go, the one file in SemStreams' service package that reached an agentic package, is not ported either; a contract test (TestServiceReachesNoAgenticPackage) fails if service comes to depend on one.

The metrics service

Metrics serves the Prometheus endpoint. Its config keys are port (default 9090) and path (default /metrics). Started by itself, it binds the port between Start and Stop. Composed under a Manager, the Manager claims the server before any service starts, serves it until StopAll, and Metrics reports healthy while the server runs. A Metrics runs once: a second Start, and a Start after any Stop, are refused with errs.ErrAlreadyStarted, and the context given to Start bounds Start only. RegisterAll registers metrics, component-manager, message-logger, storage-observability, heartbeat, metrics-forwarder and log-forwarder.

The profiling server

MaybeStartPProf(debug, port, logger) serves http.DefaultServeMux on :port, on every interface, when debug is true and port is positive, and returns nil otherwise. The pprof handlers are on that mux only if the caller's main has the blank import import _ "net/http/pprof"; importing service does not add them. MaybeStartPProf binds the port before it returns. A bind failure is logged at Warn on logger (slog.Default() when nil) and returns nil: profiling is a debug aid, never a reason to fail the boot.

It is not a Service. The composition root calls it early, before connecting to NATS, so that a slow boot can be profiled (SemStreams ADR-058, carried as docs/adr/058-boot-lifecycle-phases.md). It shuts the server down on its exit path, after its services have stopped:

pprofServer := service.MaybeStartPProf(debug, port, logger)

// ... start the services, run, stop the services ...

stopCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := pprofServer.Shutdown(stopCtx); err != nil {
    logger.Warn("pprof shutdown", "error", err)
}

Shutdown(ctx) stops accepting connections and returns nil once every request in progress has finished and the serving goroutine has exited. If ctx ends first it returns ctx's error, and a later Shutdown waits again. Shutdown on a nil *PProfServer returns nil, so the code above is safe whatever MaybeStartPProf returned. A nil context is refused, on a nil server too.

The graph runtime

WireGraphRuntime(ctx, client, logger, contracts...) builds the graph mutation client a composition root hands to the code that writes graph state through graph-ingest (ADR-110):

mutations, err := service.WireGraphRuntime(ctx, client, logger, contracts...)
if err != nil {
    return err
}

It first runs kvcatalog.EnsureCatalogRetentionClean, which removes a retention limit, such as a TTL, left on a framework catalog bucket whose entries must never expire (SemStreams ADR-068), and returns its error if that fails; a nil context or client is refused there. Then it builds the client with projection.NewMutationClient. It starts no goroutine and owns nothing that needs stopping.

OpenAPI specs

A service that serves HTTP implements HTTPHandler: RegisterHTTPHandlers(prefix, mux) adds its routes, and OpenAPISpec() describes them with OpenAPISpec, PathSpec, OperationSpec, ParameterSpec, RequestBodySpec, ResponseSpec, TagSpec and Schema. OpenAPIDocument and its parts are the whole document a service manager serves.

RegisterOpenAPISpec(name, spec) adds a spec to a registry shared by the whole process, keyed by service name. Call it from the composition root, never from an init function: importing a package registers nothing. GetAllOpenAPISpecs() returns a copy of the registry's map, but its values are the registered pointers themselves, so a change made through one is seen by every later caller.

Configuration helpers

These are read by the service manager.

  • ResolveServiceConfigs(configs) turns the services section of a configuration into the map a service manager starts from. Each entry's config is rewritten in one canonical JSON form, and component-manager, service-manager and metrics are added, enabled, when the configuration leaves them out (an explicit "enabled": false stays). It never reads inside a service's config; the service that decodes it does.
  • PendingServiceChange, CompositionSealedError, DuplicateServiceError and MandatoryServiceDisabledError are what the service manager reports about that map.
  • SchemaFromType and TypeNameFromReflect build the JSON schema of a Go type for an OpenAPI document.

Documentation

Overview

Package service holds what every long-running SemEngine service shares: BaseService, which a service embeds for its name, status and health check, and the Service interface a service manager drives. Manager is that service manager: it builds the services a configuration names, starts them, serves the HTTP routes around them and stops them. ComponentManager is the service that owns the components. Metrics is the service that serves the Prometheus endpoint. MessageLogger is the service that keeps the newest messages seen on chosen NATS subjects for inspection. StorageObservabilityService is the service that reports how full the NATS account's storage is. HeartbeatService is the service that logs a liveness line at a fixed interval, MetricsForwarder is the service that publishes the process's metrics on NATS, and LogForwarder is the service that checks the log-forwarding settings and says it forwards nothing yet. The package also holds the helpers a composition root (the process's main function, which builds and wires everything) calls itself: MaybeStartPProf, WireGraphRuntime and the OpenAPI registry, plus the service-configuration and schema helpers the service manager reads.

Embedding BaseService

A service embeds the *BaseService that NewBaseServiceWithOptions builds from its name and options:

type ruleConfigService struct {
	*service.BaseService
}

svc := &ruleConfigService{
	BaseService: service.NewBaseServiceWithOptions("rule-config",
		service.WithNATS(client), service.WithMetrics(registry), service.WithLogger(logger)),
}

WithNATS adds the client's health to the health check, WithMetrics records every status change on that registry, and WithLogger replaces the default logger (slog.Default() with a "service" attribute). BaseService itself logs nothing; the services in this package log through it.

Start(ctx) reports StatusRunning and runs the health check at once and then every 30 seconds. It refuses a nil or ended ctx, a second Start, and a Start after any Stop (errs.ErrAlreadyStarted): a BaseService runs once, so build a new one to start again. Ending ctx after Start returned stops nothing; Stop is the only stop (ADR-113).

Stop(ctx) refuses a nil ctx. On a started service any non-nil ctx (live or already ended) initiates teardown: it reports StatusStopping, cancels the health monitor's run context, and bounds only the wait for the goroutine to exit, returning nil once joined (leaving StatusStopped and IsHealthy false) or ctx's error if ctx ended first so a later Stop waits again. Pass Stop a fresh bounded context so the goroutine has time to join cleanly:

stopCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if err := svc.Stop(stopCtx); err != nil {
	logger.Warn("rule-config stop", "error", err)
}

A Stop after one that joined everything returns nil for any non-nil ctx. A Stop while another Stop is still waiting is refused as a transient error. Stop before Start returns nil, and the service then refuses to start.

The health check fails when the function set by SetHealthCheck returns an error, or when the NATS client from WithNATS is not healthy. IsHealthy is then false and Health reports unhealthy; the next passing check makes the service healthy again.

The service manager

A composition root builds a Manager from a Registry of service constructors, hands it the services section of the configuration, and starts it:

services := service.NewServiceRegistry()
if err := service.RegisterAll(services); err != nil { // every built-in service
	return err
}

manager := service.NewServiceManager(services)
manager.UseHTTPMiddleware(authMiddleware) // optional; before StartAll

deps := &service.Dependencies{
	NATSClient: client, MetricsRegistry: registry, Logger: logger,
	Platform: platform, Manager: configManager,
	ComponentRegistry: components, PayloadRegistry: payloads,
}
if err := manager.ConfigureFromServices(cfg.Services, deps); err != nil {
	return err
}
if err := manager.StartAll(startCtx); err != nil {
	return err
}

// ... run ...

stopCtx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
if err := manager.StopAll(stopCtx); err != nil {
	logger.Warn("stop services", "error", err)
}

ConfigureFromServices resolves the configuration with ResolveServiceConfigs and constructs every enabled service from its registered constructor. Nothing is composed unless every constructor succeeds. component-manager and service-manager may not be disabled, and a composition whose component-manager has no registered constructor is refused. The service-manager entry configures the Manager itself (ManagerConfig): http_port (default 8080), swagger_ui (default false), server_info (title, description and version of the API document, default "SemEngine API", "SemEngine framework API" and "0.0.0"), http_read_timeout (default 30s) and http_write_timeout (default 120s). The configs of the services this package defines, service-manager, metrics and component-manager, are decoded strictly: an unknown key is refused, and so is a key that differs from a field's name only in letter case ("Port" for "port"), naming both spellings. A service defined elsewhere receives its raw JSON and decodes it itself. CreateService and RegisterInstance compose one more service by hand, and GetService finds a composed one.

StartAll creates component-manager if no service has that name, seals the composition (CreateService, RegisterInstance and ConfigureFromServices are refused from then on), binds the main listener on http_port, starts the metrics server it claims, starts every service in registration order, builds the full route set and starts a publisher that sends each service's health on health.service.<name> every 5 seconds. It runs once: a second StartAll, and a StartAll after StopAll, are refused with errs.ErrAlreadyStarted. If a step fails, StartAll stops what it started and returns both errors; if that stop fails too, StopAll runs it again.

The context given to StartAll bounds StartAll and nothing else. Ending it after StartAll returned stops no service, and no request's context derives from it (ADR-113). StopAll(ctx) is the only stop: it stops the services in reverse registration order, then the health publisher, the listeners and the metrics server. Each listener refuses new requests and StopAll waits for those in flight; if ctx ends first it closes every connection still open and returns ctx's error, and a later StopAll waits again for the handlers that are still running. So a service that serves a long-lived response, such as a stream, must end it in its own Stop. StopAll refuses a nil context; any non-nil context (live or already ended) initiates teardown across every started service and listener, using ctx only to bound waits. A service whose Stop returns errs.ErrAlreadyStopped counts as stopped; any other failure is returned naming the service, and the next StopAll runs the whole stop again.

UseHTTPMiddleware wraps every route the main listener serves, the startup routes as well as the full set, outermost first. It must be called before StartAll; a later call is ignored with a Warn. The dedicated health listener takes no middleware, and the framework supplies none (ADR-030, docs/adr/030-http-middleware-and-identity-pattern.md).

StartHealthListener(ctx, port) binds a second listener that serves only /health and /healthz, for container probes that want a port apart from http_port. It may be called before StartAll, so probes work while a slow boot runs; port 0 does nothing, and a second call is refused. StopAll stops it whether or not StartAll ran.

Boot seals the composition: a change to the services section of the configuration after StartAll is desired state for the next boot and starts, stops or reconfigures nothing, and /services reports it as restart_required (ADR-094, docs/adr/094-boot-only-composition-and-observable-rule-activation.md).

The routes

The main listener serves /health, /healthz, /readyz, /services and /services/health from the moment StartAll binds it, and answers every other path 503 NOT READY until StartAll has started every service and committed the full route set. After the commit it also serves /graph/triples, each HTTPHandler service's routes under a prefix taken from its name (the name with its hyphens removed, except component-manager at /components and message-logger and storage-observability as named), /openapi.json, and /docs when swagger_ui is true.

/health answers every service's health and the NATS connection's, aggregated, 503 when unhealthy. /healthz answers 200 OK while the listener serves. /readyz answers 200 READY once every service has started and is running and healthy, and 503 NOT READY otherwise: while starting, after a start failed, and from the moment StopAll is entered, even while a service's Start is still running. /services lists each service's status and health, the startup counts and whether the configuration has changed since boot. /services/health answers each service's health and their aggregate, 503 when unhealthy.

GET /graph/triples reads triples out of the graph's ENTITY_STATES bucket for an operator or a test; it is a low-volume scan, not a query API. Its query parameters, all optional: subject (a literal entity ID), predicate (a three-part predicate), object (compared as text) and limit (default 100, at most 1000). It answers a JSON array, sorted by entity ID and never null, 400 naming the parameter for a malformed one, 405 for a method other than GET, and 500 when the bucket cannot be read. With no NATS client in the Dependencies it answers [] for every query: there is no store to read. The Manager only reads the bucket (ADR-110).

/openapi.json answers the API document in OpenAPI 3.0 form: the paths of every HTTPHandler service under its prefix, and the schemas of the types registered with RegisterOpenAPISpec. /docs serves a Swagger UI page that loads its script and style sheet from unpkg.com, so a browser needs network access to it.

The component manager

A component is one unit of a running composition: an input, processor, output or storage unit. A binary makes the component factories available through a component.Registry, and the components section of the configuration names the instances to build. ComponentManager builds every enabled component, starts them, serves their status over HTTP and stops them. Every composition has one: component-manager may not be disabled. RegisterAll registers its constructor, NewComponentManager, and its OpenAPI spec. Its config takes no key; any key is refused, naming it. Its Dependencies fields: Manager (the *config.Manager whose boot configuration it reads) and ComponentRegistry are required; PayloadRegistry, NATSClient, Platform, MetricsRegistry and Logger are given to each component, and with no NATSClient the manager publishes no component health.

NewComponentManager reads the boot configuration once; later writes to it are desired state for the next boot and change nothing in the running set (ADR-094 and ADR-096, docs/adr). It builds each enabled component through Registry.CreateComponent in instance-name order, checks the whole set with composition.Analyze, and seals the registry (ADR-100). It returns an error, and no service, when the Manager or ComponentRegistry is missing, when any enabled component cannot be built (the error names every one that failed), or when the composition has an error-severity finding.

Start(ctx) refuses a nil or ended ctx, a second Start, and a Start after any Stop (errs.ErrAlreadyStarted). The start order is fixed at construction: components that provide stores (component.StoreProvider) by instance name, then the rest by instance name. Start runs two phases, providers first; each phase starts its components concurrently and the next begins only after every Start in the current one returned. A failure in the first phase starts nothing of the second. A component that is not a component.LifecycleComponent is built and listed but never started or stopped.

Each component's Start receives the context the manager's Start received. The manager keeps no context of its own for a component and never ends the one it was given, Stop included (ADR-113). A component makes its own run context in Start and ends it in Stop; one that hangs its continuing work on its Start context stops when the caller ends that context, before Stop reaches it. The manager's health publisher runs under a run context that Start creates and Stop ends.

A failed Start returns every component failure joined, each naming its component, after one rollback: it stops every component it launched, the failing ones included, in reverse start order. If the rollback fails too, the next Stop runs it again.

Stop(ctx) is the only stop. It refuses a nil ctx; any non-nil ctx (live or already ended) initiates teardown and uses ctx only to bound waits. Before any Start it returns nil and a later Start is refused. Otherwise it marks the stop begun (/readyz answers 503 NOT READY from then), stops each started component with ctx in reverse start order (removing each provider's stores from the store registry before its Stop, even when ctx has already ended), then ends the manager's own loops and its base service, and returns nil only once all are released. A component whose Stop failed (including one cut short by ctx) is stopped again by the next Stop; one whose Stop returned nil is not called again. A Stop while another runs is refused as transient. A Stop that arrives while Start is running waits for every component's Start to return, or returns its own context's error first; a caller cuts a slow boot short only by ending the context it gave Start.

GetComponentHealth, GetComponentStatus, the /components routes and the manager's health check wait on no lock and not on Start or Stop. Each asks the component itself, so a request during or after Stop gets whatever the component answers then; the manager does not keep such requests away from a stopping component. A component author holds the line in component.LifecycleComponent: Health, DataFlow and DebugStatus may be called at any time and return without waiting for Start or Stop. The tree enforces it for graph-ingest only. The manager's own health is unhealthy while any component is in the failed state, naming each one with its last error.

With a NATS client the manager publishes each admitted component's health on health.component.<name> once as Start returns and then every 5 seconds. A publish that fails is counted on semengine_errors_total{service="component-manager",type="health_publish"} and logged at Debug; the next component and round are still published.

The routes, mounted under /components and answering GET only (405 otherwise), are health, list, types, types/{id}, status/{name} (?debug=true adds a component's DebugStatus), config/{name}, flowgraph (JSON, or Mermaid with ?format=mermaid), validate and paths. A name that is empty, "." or ".." or decodes to a path separator is 400, an unknown one 404. Until StartAll commits the full route set only health, list and status/{name} are served. flowgraph, validate and paths read the composition result kept at construction: validate serves it as is, flowgraph serves its graph, and paths serves composition.FlowPaths of that graph with counts of the answer. BuildComponentTypeCatalog returns what types serves, for a product that serves the catalog elsewhere. One declared quirk: GET /components/types/{id} for an unregistered type answers 404 with the body {"error":"Component type <id> not found"} sent as text/plain, not JSON; read the status code, not the text.

ConfigureRulePackMutations(manager) hands a reconciler to each rule processor. A composition root must call it after ConfigureFromServices and before StartAll; without the call no ProjectionBinder is given one. It finds the admitted components that implement ProjectionBinder (ComponentsImplementing returns the components that implement any T, in start order), runs every binder's PreflightProjectionMutations before changing anything, validates each pack ID and refuses a duplicate, copies and validates each pack's contracts, refuses contracts when there is no NATS client, and gives each pack that has contracts one projection.MutationClient through SetPredicateReconciler. No rule processor is in this tree yet, so today the call has no effect.

/openapi.json takes its schemas only from the registry RegisterOpenAPISpec fills, and the component manager's flowgraph and validate responses refer to the Graph and Result schemas there. RegisterAll registers the component manager's spec; a composition that registers NewComponentManager by hand must register the spec itself, as

if svc, ok := manager.GetService("component-manager"); ok {
	service.RegisterOpenAPISpec("component-manager", svc.(service.HTTPHandler).OpenAPISpec())
}

or those references name schemas the document does not carry.

The message logger

MessageLogger subscribes to NATS subjects and keeps the newest messages in memory so a developer can inspect what flows through a running system. It also reads NATS KV buckets over HTTP. It writes nothing to NATS or the graph. It is off unless the configuration enables it: ResolveServiceConfigs does not add it. RegisterAll registers its constructor, NewMessageLoggerService, and its OpenAPI spec. Its config keys are monitor_subjects ("*" stands for the NATS, NATS-request and JetStream port subjects of every admitted component; empty means ["*"]), max_entries (0 to 100000; 0 means 10000), sample_rate (store one message in every sample_rate; 0 and 1 store all) and output_to_stdout. Construction refuses an unknown key (or one in another letter case), a nil NATS client, a nil payload registry, "*" without a component registry, max_entries outside 0 to 100000 and a negative sample_rate, naming the problem.

Start works out the subject set once and checks every subject against the grammar of internal/flowgraph before it starts the base service or makes any NATS call: tokens separated by ".", none empty; "*" and ">" only as a whole token and ">" only last; no space, tab, line feed, carriage return or form feed. That is stricter than NATS, which takes "a*" as plain text and accepts a form feed. A subject that fails the check refuses Start as invalid data, naming the subject and its source (monitor_subjects, or the declaring component and port), and holds nothing. A subject another subject covers is not subscribed and is listed in /stats under subject_overlaps; two subscribed subjects can still overlap without either covering the other, and a message on both is stored twice.

Start refuses a nil or ended ctx, a second Start, and a Start after any Stop (errs.ErrAlreadyStarted). Start subscribes under ctx and checks it before each subscribe and once more after the last; if ctx has ended, Start returns its error and holds nothing, except that a message delivered to a subscribe ctx cut after its SUB was sent can still be stored. Ending ctx after Start returned stops nothing (ADR-113). A subscribe that fails does not fail Start: the subject is retried every 100 ms until it succeeds. Meanwhile /stats reports subject_reconciliation_degraded true with subject_reconciliation_error, and the logger's health check fails naming the subjects, so IsHealthy turns false and /readyz answers not ready; health checks run every 30 seconds, so each change shows within one interval. Stop is the only stop: it refuses a nil ctx, and on any non-nil ctx (live or already ended) fences new KV watches, ends the retry, initiates every subscription drain, closes open KV watch streams, and stops the base service, using ctx only to bound waits; if ctx ends first the next Stop finishes the rest.

A message that decodes as a typed envelope is stored under its type name and counted in valid_messages; any other is stored with type "raw" and counted in invalid_messages. A payload that is not JSON is stored as a JSON string. A message whose handler context has ended (the NATS client ends it once its Close has begun) is not stored: it is logged at Warn, naming the subject, and counted in dropped_messages, not in total_messages. /stats also reports total_messages, dropped_messages, sampled_messages, sample_rate, max_entries, start_time, last_message_time, uptime_seconds, monitored_subjects (empty after Stop) and subject_overlaps.

The routes, mounted under /message-logger and answering GET and HEAD (405 for any other method), are entries (newest first; limit defaults to 100, capped at 10000; subject is a NATS pattern applied before the limit, and a pattern NATS cannot parse matches nothing), stats, subjects, trace/{traceID} (32 hex characters, else 400; the entries still held, oldest first), kv/{bucket} (an existing bucket's entries, keys sorted before limit, default 100, capped at 1000; pattern is a key glob; status keeps records whose top-level JSON status equals it; 404 for a missing bucket, which is never created) and kv/{bucket}/watch. The watch answers 503 once Stop has begun, 400 for a bad bucket name or key filter, 404 for a missing bucket. It then streams Server-Sent Events: connected, one kv_change per key's current value, a kv_change with operation initial_sync_complete, then each change with operation put, delete or purge. Each stream buffers 100 events for a slow client. If an event does not fit, the stream sends the buffered events, one error event {"error":"watch fell behind: events were dropped"} and ends, so no kv_change and no initial_sync_complete follows a dropped event. The logger does not read Last-Event-ID; a client that reconnects gets a new initial replay.

The storage observability service

StorageObservabilityService reports how full the NATS account's storage is. Each interval it lists the account's JetStream streams and key-value buckets (its resources), derives usage, growth and pressure for each, and publishes the result as a report in the STORAGE_REPORT bucket. A second loop watches that bucket and feeds three surfaces from one view: Prometheus metrics, the health message and GET /storage-observability/report. It writes no graph entity.

It is report-only. It rejects no write, throttles no component, fails no readiness or health check and applies no retention because of pressure. The service does no capacity admission: a consumer that wants backpressure reads the report and decides for itself.

It is off unless the configuration enables it: ResolveServiceConfigs does not add it. RegisterAll registers its constructor, NewStorageObservabilityService, under storage-observability and no OpenAPI spec, because the route's spec names no schema. It needs Dependencies.NATSClient and Dependencies.MetricsRegistry. Its keys are interval (default 1m), timeout (default 15s, at most the interval) and pressure_thresholds (warning_headroom 0.25, high_headroom 0.15, critical_headroom 0.05, warning_horizon 72h, high_horizon 24h, critical_horizon 4h). Construction refuses, as invalid data naming the input, an unknown key (nested keys and another letter case included), nil dependencies, a nil NATS client, a nil metrics registry, an interval or timeout that does not parse or is not positive, a timeout above the interval and thresholds that are out of order or out of range. Configuration is read once; restart to change it.

Start refuses a nil or ended ctx, a second Start and a Start after any Stop (errs.ErrAlreadyStarted), and does no I/O: it starts the collection loop and the watch loop and returns, so a deployment still boots when the account cannot be read; each loop retries on its own schedule (the watch every 5 seconds). ctx bounds Start only (ADR-113). Stop is the only stop: it ends both loops, waits for them and stops the base service, and if its ctx ends first the next Stop waits again. How soon a collection already inside the account listing sees Stop begin depends on nats.go, which does not promise it.

Health returns the base service's verdict untouched and appends the storage picture to its message, so critical pressure leaves /readyz answering READY and /health answering 200. After Stop the verdict is the base service's own. GET /storage-observability/report (HEAD is answered as GET, other methods get 405) always answers 200 with report_only, synced, updated_at (once a change was read), a summary (resources, pressure, not_evaluated, and worst_pressure when anything was evaluated), account (once an account row was read) and every published row in resources, unfiltered.

The 19 metrics are named semengine_storage_*. A value that was not measured has no series, and a series is deleted when its value goes away. configs/prometheus/rules holds example alert rules over these names; the engine does not load them. See service/README.md for the full list.

The heartbeat service

HeartbeatService logs one line, "System heartbeat", at Info when it starts and then every interval, so an operator can see the process is alive. The line carries uptime, goroutines, components_healthy and components_total. It is on by default: the configuration loader's defaults enable a heartbeat service at 30 s, so a composition that names no services still gets it. Its only key is interval (default 30s, at least 1s; an empty value takes the default). Construction refuses an unknown key, a key in another letter case, nil dependencies, a nil service manager and a bad interval, as invalid data naming the input. Start refuses, naming component-manager, when the service manager holds no *ComponentManager; that manager need only be composed. Start bounds Start only and Stop is the only stop. See service/README.md.

The metrics forwarder

MetricsForwarder publishes the metrics registry it is given on NATS, at Start and then every push_interval (default 5s). Each sample is one JSON object {timestamp, name, component, type, value, labels} on metrics.<component>.<name>, where component is the sample's component label, else "system". The go_* and process_* families are left out unless include_go_metrics or include_proc_metrics is set. It is off unless the configuration enables it. A sample that does not encode or publish is counted on semengine_errors_total with type metrics_publish, a failed gather with type metrics_gather, and the loop goes on. Construction refuses an unknown key, a key in another letter case, nil dependencies, a nil NATS client, a nil metrics registry and a bad push_interval. See service/README.md.

The log forwarder

LogForwarder forwards nothing yet: no log handler in this module publishes logs to NATS, and pkg/logging and the boot wiring that would install one are not in this module. It checks its settings, runs the base service's lifecycle and, at Start, writes one record at Warn saying so, with the resolved min_level (DEBUG, INFO, WARN or ERROR, default INFO) and exclude_sources (the configured source prefixes, trimmed and de-duplicated, with nothing added). Both values are checked and reported, not applied. It is off unless the configuration enables it, needs no NATS client and does no I/O. Construction refuses an unknown key, a key in another letter case, nil dependencies and a bad min_level, as invalid data naming the input. Start bounds Start only and Stop is the only stop. See service/README.md.

The metrics service

Metrics serves the Prometheus endpoint. Its config keys are port (default 9090) and path (default "/metrics"). Started by itself, it binds the port between Start and Stop. Composed under a Manager, the Manager claims the server before any service starts, serves it until StopAll, and Metrics reports healthy while the server runs. A Metrics runs once: a second Start, and a Start after any Stop, are refused with errs.ErrAlreadyStarted, and the context given to Start bounds Start only. RegisterAll registers Metrics, the component manager, the message logger, storage observability, the heartbeat, the metrics forwarder and the log forwarder.

The profiling server

MaybeStartPProf(debug, port, logger) serves http.DefaultServeMux on :port, every interface, when debug is true and port is positive, and returns nil otherwise. The pprof handlers are on that mux only if the caller's main has the blank import `import _ "net/http/pprof"`. It binds before it returns; a bind failure is logged at Warn on logger and returns nil, because profiling is a debug aid, never a reason to fail the boot. It is not a Service: the composition root calls it early, before NATS, so a slow boot can be profiled (ADR-058, docs/adr/058-boot-lifecycle-phases.md).

The composition root shuts it down on its exit path, after its services have stopped. Shutdown on a nil *PProfServer returns nil, so this is safe whatever MaybeStartPProf returned:

pprofServer := service.MaybeStartPProf(debug, port, logger)

stopCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := pprofServer.Shutdown(stopCtx); err != nil {
	logger.Warn("pprof shutdown", "error", err)
}

Shutdown stops accepting connections and returns nil once every request in progress has finished and the serving goroutine has exited. If its context ends first it returns that context's error, and a later Shutdown waits again. It refuses a nil context, on a nil server too.

The graph runtime

WireGraphRuntime(ctx, client, logger, contracts...) first runs kvcatalog.EnsureCatalogRetentionClean, which strips a retention limit left on a framework catalog bucket this boot does not use, and returns its error if it fails. Then it builds the graph mutation client for contracts with projection.NewMutationClient. It starts no goroutine.

OpenAPI

A service that serves HTTP implements HTTPHandler, whose OpenAPISpec method describes its endpoints with OpenAPISpec, PathSpec, OperationSpec, ParameterSpec, ResponseSpec, TagSpec and Schema. RegisterOpenAPISpec adds a spec to a process-wide registry, keyed by service name; call it from the composition root, never from an init function, since importing a package registers nothing. GetAllOpenAPISpecs returns a copy of the registry's map whose values are the registered pointers themselves.

Configuration helpers

ResolveServiceConfigs turns the services section of a configuration into the map a service manager starts from: each entry's config canonicalized, and component-manager, service-manager and metrics added, enabled, when the configuration leaves them out. It does not read inside a service's config; the service that decodes it does. PendingServiceChange, CompositionSealedError, DuplicateServiceError and MandatoryServiceDisabledError are what the service manager reports about that map. SchemaFromType and TypeNameFromReflect build the JSON schema of a Go type for an OpenAPI document.

Package service provides OpenAPI specification types for HTTP endpoint documentation

Index

Constants

View Source
const (
	DefaultHTTPReadTimeout  = 30 * time.Second
	DefaultHTTPWriteTimeout = 120 * time.Second
)

HTTP server timeout defaults. Tuned for LLM-backed workloads — the prior 10s/10s defaults killed every long graph query with EOF. See commit message + feedback memory for the case study.

View Source
const StorageObservabilityServiceName = "storage-observability"

StorageObservabilityServiceName is the key this service is configured and registered under.

Variables

This section is empty.

Functions

func BuildComponentTypeCatalog

func BuildComponentTypeCatalog(registry *component.Registry, logger *slog.Logger) []map[string]any

BuildComponentTypeCatalog returns the list of registered component factory types with their schemas and default ports. The GET /components/types handler calls it, and so may a consumer that serves the catalog elsewhere (an agent's list-components tool, for one); the one home for the shape is composition.Catalog, projected here to the flat map array the route serves.

func ComponentsImplementing

func ComponentsImplementing[T any](m *Manager) []T

ComponentsImplementing returns the admitted components that implement T, in the component manager's start order. The component set is fixed when the component manager is constructed, so a composition root can wire a component-facing seam once, after ConfigureFromServices. It returns nil before a component manager is composed, or when the one composed is not the framework's *ComponentManager.

func ConfigureRulePackMutations

func ConfigureRulePackMutations(manager *Manager) error

ConfigureRulePackMutations validates every enabled pack before injecting one canonical mutation client per contract-bearing processor. A composition root calls it after ConfigureFromServices and before StartAll; without the call no binder is handed a reconciler.

func GetAllOpenAPISpecs

func GetAllOpenAPISpecs() map[string]*OpenAPISpec

GetAllOpenAPISpecs returns a copy of the registration map. The *OpenAPISpec values are the registered pointers themselves, so a change through one is seen by every later caller. The service manager's /openapi.json handler reads it.

func RegisterAll

func RegisterAll(registry *Registry) error

RegisterAll registers the framework's built-in services with the registry: metrics, the component manager, the message logger, storage observability, the heartbeat, the metrics forwarder and the log forwarder, which forwards nothing yet: no log handler in this module publishes logs to NATS.

It also registers the component manager's and the message logger's OpenAPI specifications, because /openapi.json takes its schemas only from that registry: without them, the flowgraph and validate responses' Graph and Result references, and the entries response's MessageLogEntry reference, dangle. Storage observability's specification names no schema, and none of the heartbeat, the metrics forwarder and the log forwarder serves a route, so none is registered for any of the four. A composition that registers NewComponentManager or NewMessageLoggerService by hand instead of calling RegisterAll registers the specification itself.

func RegisterOpenAPISpec

func RegisterOpenAPISpec(name string, spec *OpenAPISpec)

RegisterOpenAPISpec registers an OpenAPI specification for a service. Call it from the composition root, never from an init function: importing a package registers nothing.

func ResolveServiceConfigs

func ResolveServiceConfigs(configs types.ServiceConfigs) (types.ServiceConfigs, error)

ResolveServiceConfigs returns the deterministic outer desired-service map. It owns map structure and activation defaults only; service constructors remain the sole interpreters of inner configuration.

func SchemaFromType

func SchemaFromType(t reflect.Type) map[string]any

SchemaFromType generates a JSON Schema from a reflect.Type. It handles primitives, structs, slices, maps, pointers, and time.Time.

func TypeNameFromReflect

func TypeNameFromReflect(t reflect.Type) string

TypeNameFromReflect returns the OpenAPI component name for t. A pointer is named by its element, and a named type by its bare name ("service.RuntimeHealthResponse" -> "RuntimeHealthResponse"). An unnamed type is named by its shape, so a collection's schema never takes its element's name: []X is "list-X", [2]X is "array-2-X", map[K]V is "map-K-V", and any other unnamed type, an anonymous struct among them, is its kind and a hash of its Go spelling ("struct-1a2b3c4d"). Every name matches OpenAPI's component-name grammar, ^[A-Za-z0-9._-]+$, when its named parts do.

func WireGraphRuntime

func WireGraphRuntime(
	ctx context.Context,
	natsClient *natsclient.Client,
	logger *slog.Logger,
	contracts ...projection.Contract,
) (*projection.MutationClient, error)

WireGraphRuntime applies the existing storage-retention guard and constructs the built-in graph mutation client. It creates no ownership substrate or background coordination service.

Types

type BaseService

type BaseService struct {
	// contains filtered or unexported fields
}

BaseService provides common functionality for all services

func NewBaseServiceWithOptions

func NewBaseServiceWithOptions(name string, opts ...Option) *BaseService

NewBaseServiceWithOptions creates a new base service using functional options pattern

func (*BaseService) GetStatus

func (s *BaseService) GetStatus() Info

GetStatus returns the current service information

func (*BaseService) Health

func (s *BaseService) Health() health.Status

Health returns the standard health status for the service

func (*BaseService) IsHealthy

func (s *BaseService) IsHealthy() bool

IsHealthy returns whether the service is healthy

func (*BaseService) Name

func (s *BaseService) Name() string

Name returns the service name

func (*BaseService) SetHealthCheck

func (s *BaseService) SetHealthCheck(fn HealthCheckFunc)

SetHealthCheck sets a custom health check function

func (*BaseService) Start

func (s *BaseService) Start(ctx context.Context) error

Start starts the service: it reports StatusRunning and runs the health check at once and then every 30 seconds. It refuses a nil or ended ctx, a second Start, and a Start after any Stop (errs.ErrAlreadyStarted); a refused Start starts nothing.

Ending ctx after Start returned stops nothing: the health check runs under a run context Start creates and Stop ends (ADR-113). Stop is the only stop.

func (*BaseService) Status

func (s *BaseService) Status() Status

Status returns the current service status

func (*BaseService) Stop

func (s *BaseService) Stop(ctx context.Context) error

Stop stops the service. It refuses a nil ctx. On a started service any non-nil ctx (live or already ended) initiates teardown: it reports StatusStopping, cancels the health monitor's run context, and bounds only the wait for the goroutine to exit, returning nil once joined (leaving StatusStopped) or ctx's error if ctx ends before the join completes so a later Stop waits again. Stop before Start, and a Stop after one that joined everything, return nil for any non-nil ctx.

type ComponentManager

type ComponentManager struct {
	*BaseService
	// contains filtered or unexported fields
}

ComponentManager owns the boot composition's components (inputs, processors, outputs, storage) for the life of the process.

NewComponentManager composes the set once, from the configuration manager's boot configuration: it builds every enabled component, analyses the composition, and seals the component registry. The set is fixed from then on. Start starts it once, store providers first; Stop is the only stop, and stops the components in reverse start order (ADR-113).

func (*ComponentManager) GetComponentHealth

func (cm *ComponentManager) GetComponentHealth() map[string]component.HealthStatus

GetComponentHealth returns each admitted component's own health answer. It asks every component, before, during and after Start and Stop, with nothing held.

func (*ComponentManager) GetComponentStatus

func (cm *ComponentManager) GetComponentStatus() map[string]ComponentStatus

GetComponentStatus returns each admitted component's lifecycle state and last error, read from its record, beside its own health and flow answers, asked with nothing held.

func (*ComponentManager) GetFlowPaths

func (cm *ComponentManager) GetFlowPaths() (map[string][]string, error)

GetFlowPaths returns composition.FlowPaths of the graph the boot composition result keeps (ADR-100): for each origin, every component reachable from it. It does not rebuild a graph from the registry, so it cannot disagree with the result `<components>/validate` and `<components>/flowgraph` serve: one interpreter stands behind every answer about a composition. It derives no severity: it is a projection, not a judgment.

func (*ComponentManager) OpenAPISpec

func (cm *ComponentManager) OpenAPISpec() *OpenAPISpec

OpenAPISpec returns the OpenAPI specification for ComponentManager endpoints

func (*ComponentManager) RegisterHTTPHandlers

func (cm *ComponentManager) RegisterHTTPHandlers(prefix string, mux *http.ServeMux)

RegisterHTTPHandlers registers HTTP endpoints for the ComponentManager service

func (*ComponentManager) Start

func (cm *ComponentManager) Start(ctx context.Context) error

Start starts the boot component set once (ADR-113). It refuses a nil or ended ctx, a second Start and a Start after any Stop (errs.ErrAlreadyStarted); a refused Start starts nothing.

It starts the lifecycle components in two phases, store providers first and then the rest. Each phase starts its components concurrently and ends only once every Start in it returned; a failure in the first phase starts nothing of the second. Each component's Start gets ctx itself, which the manager never ends. Start then starts the base service and the supervisor (the component health publisher), which runs under a run context of its own until Stop: ending ctx after Start returned stops nothing.

A failed Start returns every component failure joined, each naming its component, after rolling back once: every component it launched, the failing ones included, is stopped in reverse start order. A rollback that fails is joined into the error and left for the next Stop.

func (*ComponentManager) Stop

func (cm *ComponentManager) Stop(ctx context.Context) error

Stop is the only stop (ADR-113). It refuses a nil ctx; any non-nil ctx (live or already ended) initiates teardown and bounds only waits. Before any Start it returns nil and a later Start is refused. A Stop during Start waits for every component's Start to return (or returns ctx's error first if Start is still running); if Start failed, it then stops only what Start's rollback left.

Otherwise it marks the stop begun (/readyz answers NOT READY from then), stops each started component with ctx in reverse start order, removing a provider's stores before its Stop even when ctx has already ended, then ends the supervisor and stops the base service. It returns nil only once all are released. A component whose Stop failed (including one whose cleanup was cut short by ctx) is stopped again by the next Stop; one whose Stop returned nil is not called again. A Stop after one that returned nil returns nil for any non-nil ctx and calls nothing; a Stop while another runs is refused as transient.

type ComponentManagerConfig

type ComponentManagerConfig struct{}

ComponentManagerConfig configures the ComponentManager service. It defines no key: the manager composes the one boot component set from the configuration manager's components, so a strict decode refuses every key, naming it, and the retired enabled_components and watch_config fail loudly instead of being ignored.

type ComponentStatus

type ComponentStatus struct {
	Name      string                 `json:"name"`
	State     component.State        `json:"state"`
	Health    component.HealthStatus `json:"health"`
	DataFlow  component.FlowMetrics  `json:"data_flow"`
	LastError error                  `json:"last_error,omitempty"`
}

ComponentStatus combines lifecycle state with health and flow metrics

type ComponentsSpec

type ComponentsSpec struct {
	Schemas map[string]any `json:"schemas,omitempty"`
}

ComponentsSpec holds reusable OpenAPI component definitions

type CompositionSealedError

type CompositionSealedError struct {
	Operation string
	Name      string
}

CompositionSealedError reports an attempted composition write after StartAll fixed the process service identity set.

func (*CompositionSealedError) Error

func (e *CompositionSealedError) Error() string

type Constructor

type Constructor func(rawConfig json.RawMessage, deps *Dependencies) (Service, error)

Constructor defines the standard constructor signature for all services. Every service must have a constructor that follows this pattern. The constructor receives raw JSON config and must handle its own parsing.

type Dependencies

type Dependencies struct {
	NATSClient        *natsclient.Client
	MetricsRegistry   *metric.MetricsRegistry
	Logger            *slog.Logger
	Platform          types.PlatformMeta        // Platform identity
	Manager           *config.Manager           // Centralized configuration management
	ComponentRegistry *component.Registry       // Component registry for ComponentManager
	PayloadRegistry   *payloadregistry.Registry // Shared payload registry plumbed to component deps
	ServiceManager    *Manager                  // The manager that constructs the service
}

Dependencies provides the standard dependencies that all services receive. Services should use HTTP or NATS RPC for inter-service communication.

type DuplicateServiceError

type DuplicateServiceError struct {
	Name string
}

DuplicateServiceError reports two composition writers claiming one service map-key identity.

func (*DuplicateServiceError) Error

func (e *DuplicateServiceError) Error() string

type HTTPHandler

type HTTPHandler interface {
	RegisterHTTPHandlers(prefix string, mux *http.ServeMux)
	OpenAPISpec() *OpenAPISpec // Returns OpenAPI specification for this service
}

HTTPHandler is an optional interface for services that want to expose HTTP endpoints

type HTTPMiddleware

type HTTPMiddleware func(http.Handler) http.Handler

HTTPMiddleware wraps an http.Handler. Products supply middleware via Manager.UseHTTPMiddleware before StartAll; the Manager wraps the framework's HTTP mux with the chain at server boot.

Application order is outermost-first: the slice element at index 0 is the outermost wrapper — it sees the request first and the response last. The element at index len-1 is innermost, executing closest to the registered handler. This matches the convention established in ADR-030 and is the same direction the standard library's reverse-build composition pattern produces (see chainMiddleware below).

The framework ships zero default middleware. Common cross-cutting concerns — auth, request logging, panic recovery, rate limiting, CORS — are product policy and live in product-shell middleware.

type HealthCheckFunc

type HealthCheckFunc func() error

HealthCheckFunc defines a custom health check function

type HeartbeatConfig

type HeartbeatConfig struct {
	// Interval between heartbeat records (e.g. "30s", "1m"), at least 1s. Default: "30s".
	Interval string `json:"interval"`
}

HeartbeatConfig holds configuration for the Heartbeat service.

type HeartbeatService

type HeartbeatService struct {
	*BaseService
	// contains filtered or unexported fields
}

HeartbeatService logs one "System heartbeat" record at Info when it starts and then once per interval: the service's uptime, the process's goroutine count, and how many of the component manager's components report healthy out of how many there are.

func (*HeartbeatService) Start

func (hb *HeartbeatService) Start(ctx context.Context) error

Start finds the component manager through the service manager the heartbeat was built with, starts the base service, then the loop, and returns nil. It refuses, naming component-manager, when no service has that name or the service is not the framework's *ComponentManager; the manager need not be started. It refuses a nil or ended context, a second Start and a Start after any Stop (errs.ErrAlreadyStarted); a refused Start holds nothing. ctx bounds Start only; ending it after Start returned stops nothing.

func (*HeartbeatService) Stop

func (hb *HeartbeatService) Stop(ctx context.Context) error

Stop ends the run context, waits for the loop to return and stops the base service. It returns nil only once both are done. If ctx ends first it returns that context's error and the next Stop waits again. A Stop after one that returned nil returns nil; a Stop while another runs is refused as transient.

type Info

type Info struct {
	Name               string        `json:"name"`
	Status             Status        `json:"status"`
	Uptime             time.Duration `json:"uptime"`
	StartTime          time.Time     `json:"start_time"`
	MessagesProcessed  int64         `json:"messages_processed"`
	LastActivity       time.Time     `json:"last_activity"`
	HealthChecks       int64         `json:"health_checks"`
	FailedHealthChecks int64         `json:"failed_health_checks"`
}

Info holds runtime information for a service

type InfoSpec

type InfoSpec struct {
	Title       string `json:"title"`
	Description string `json:"description"`
	Version     string `json:"version"`
}

InfoSpec contains API metadata

type KVWatchConnectedEvent

type KVWatchConnectedEvent struct {
	Bucket  string `json:"bucket"`
	Pattern string `json:"pattern"`
	Message string `json:"message"`
}

KVWatchConnectedEvent is the data of the connected event that opens the KV watch stream.

type KVWatchEvent

type KVWatchEvent struct {
	Bucket string `json:"bucket"`
	Key    string `json:"key"`
	// Operation is the operation NATS records for the entry: "put", "delete" or "purge"; or
	// "initial_sync_complete" once every key's current value has been sent.
	Operation string          `json:"operation"`
	Value     json.RawMessage `json:"value,omitempty"`
	Revision  uint64          `json:"revision"`
	Timestamp time.Time       `json:"timestamp"`
}

KVWatchEvent is the data of a kv_change event on the KV watch stream.

type LogForwarder

type LogForwarder struct {
	*BaseService
	// contains filtered or unexported fields
}

LogForwarder forwards nothing: no log handler in this module publishes logs to NATS. It checks its configuration, runs the base service's lifecycle, and says so in one record at Start.

func (*LogForwarder) Start

func (lf *LogForwarder) Start(ctx context.Context) error

Start starts the base service, then writes one record at Warn saying the service forwards nothing, with the resolved min_level and exclude_sources, and returns nil. It refuses a nil or ended context, a second Start and a Start after any Stop (errs.ErrAlreadyStarted); a refused Start holds nothing and writes no record. ctx bounds Start only; ending it after Start returned stops nothing.

type LogForwarderConfig

type LogForwarderConfig struct {
	// MinLevel is the minimum level a forwarded record would need (DEBUG, INFO, WARN, ERROR),
	// in any letter case. Default: INFO. Checked and reported, not applied.
	MinLevel string `json:"min_level"`

	// ExcludeSources lists the source prefixes, in dotted notation, whose records would not be
	// forwarded: excluding "a.b" would also exclude "a.b.c" but not "a". Entries are trimmed,
	// empty ones dropped and duplicates removed. Checked and reported, not applied.
	ExcludeSources []string `json:"exclude_sources"`
}

LogForwarderConfig holds configuration for the LogForwarder service. The service forwards nothing yet: its values are checked and reported at Start, not applied. Note: The service is enabled/disabled via types.ServiceConfig.Enabled at the outer level.

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager composes services, starts them in registration order and stops them in reverse, and serves the HTTP surface around them: the main listener, an optional dedicated health listener, and the metrics server it claims from the metrics service. StartAll and StopAll are its lifecycle; the Manager is not itself a Service.

func NewServiceManager

func NewServiceManager(registry *Registry) *Manager

NewServiceManager returns a Manager that constructs services from registry. It logs through slog's default logger until ConfigureFromServices is given a logger.

func (*Manager) ConfigureFromServices

func (m *Manager) ConfigureFromServices(services types.ServiceConfigs, deps *Dependencies) error

ConfigureFromServices configures the manager from the services configuration and constructs every enabled service but service-manager from its registered constructor. The service-manager entry configures the manager itself: http_port defaults to 8080, and the API document's title, description and version to "SemEngine API", "SemEngine framework API" and "0.0.0". component-manager and service-manager may not be disabled, and a composition whose component-manager has no registered constructor is refused. Nothing is composed unless every constructor succeeds. It is refused once StartAll sealed the composition.

func (*Manager) CreateService

func (m *Manager) CreateService(name string, rawConfig json.RawMessage, deps *Dependencies) (Service, error)

CreateService constructs the service registered as name from rawConfig and composes it. It is refused for a name already composed and once StartAll sealed the composition; a refused call runs no constructor.

func (*Manager) GetService

func (m *Manager) GetService(name string) (Service, bool)

GetService returns the service composed under name. After a StopAll that returned nil it finds nothing.

func (*Manager) ProjectionBinders

func (m *Manager) ProjectionBinders() []ProjectionBinder

ProjectionBinders returns enabled components that require projection clients.

func (*Manager) RegisterInstance

func (m *Manager) RegisterInstance(name string, svc Service) error

RegisterInstance composes a prebuilt service under name, as CreateService composes a constructed one. It is refused for a name already composed and once StartAll sealed the composition.

func (*Manager) StartAll

func (m *Manager) StartAll(ctx context.Context) error

StartAll starts the composition. It refuses a nil or ended context, a second StartAll and a StartAll after StopAll (errs.ErrAlreadyStarted); a refused StartAll starts nothing. Otherwise it creates component-manager from its registered constructor if no service has that name, seals the composition, binds the main listener (http_port) serving only the startup routes, starts the metrics server it claims from the metrics service, starts every service in registration order with ctx, builds the full route set, starts the health publisher, and commits the full route set. A failure is returned joined with the rollback's error: the rollback stops what started, and a rollback that fails is run again by StopAll.

ctx bounds StartAll only: ending it after StartAll returned stops nothing, and no request's context derives from it. StopAll is the only stop.

func (*Manager) StartHealthListener

func (m *Manager) StartHealthListener(ctx context.Context, port int) error

StartHealthListener binds a dedicated listener on port serving only /health and /healthz, for container probes that want a port apart from http_port. Port 0 does nothing. A bind failure is returned. ctx bounds the bind only: ending it after StartHealthListener returned stops nothing. A second call, and one after StopAll was called, are refused. StopAll stops the listener, whether or not StartAll ran.

func (*Manager) StopAll

func (m *Manager) StopAll(ctx context.Context) error

StopAll stops the composition. It refuses a nil context; any non-nil context (live or already ended) initiates teardown and uses ctx only to bound waits. Before any StartAll it stops the health listener, if one was started, and returns nil once released; a later StartAll is refused. Otherwise it makes /readyz answer NOT READY at once, stops every started service in reverse registration order with ctx, then the health publisher, the health and main listeners, and the metrics server last. A listener refuses new requests and StopAll waits for those in flight; if ctx ends first, every connection still open is closed, which ends each open request's context, and a later StopAll waits for their handlers to return. StopAll returns nil only once everything is released. A service whose Stop returns errs.ErrAlreadyStopped counts as stopped. Any other failure is returned naming the service or listener, and the next StopAll runs the whole stop again. A StopAll after one that returned nil returns nil for any non-nil context and calls nothing; one while another runs is refused as transient.

func (*Manager) UseHTTPMiddleware

func (m *Manager) UseHTTPMiddleware(mws ...HTTPMiddleware)

UseHTTPMiddleware appends product-supplied middleware to the chain wrapped around every route the main listener serves, the startup routes as well as the full set. Order is outermost-first: the first middleware passed, in this call or across calls, is the outermost wrapper.

This is the framework's only HTTP middleware seam. The framework ships no default middleware; auth, request logging, panic recovery, rate limiting and CORS are product policy. The dedicated health listener takes no middleware.

It must be called before StartAll binds the main listener. A later call is ignored with a Warn: the listener's handler is built at the bind, so the middleware would never run.

type ManagerConfig

type ManagerConfig struct {
	HTTPPort   int      `json:"http_port"`
	SwaggerUI  bool     `json:"swagger_ui"`
	ServerInfo InfoSpec `json:"server_info"`

	// HTTPReadTimeout caps how long the server waits for the request body.
	// Go duration string (e.g. "30s"). Empty falls back to
	// DefaultHTTPReadTimeout. Bump for paths that stream large request
	// bodies; the default is enough for typical JSON GraphQL queries.
	HTTPReadTimeout string `json:"http_read_timeout,omitempty"`

	// HTTPWriteTimeout caps how long the server takes to write the
	// response. Go duration string (e.g. "120s"). Empty falls back to
	// DefaultHTTPWriteTimeout. CRITICAL for LLM-backed paths (globalSearch
	// answer synthesis, classifier, etc.) — the prior hardcoded 10s
	// killed every long-running query mid-write with EOF on the client
	// side. Set to >= 2× the slowest expected handler time.
	HTTPWriteTimeout string `json:"http_write_timeout,omitempty"`
}

ManagerConfig holds configuration for the Manager HTTP server Simple struct - no UnmarshalJSON, no Enabled field

func (ManagerConfig) ResolvedHTTPReadTimeout

func (c ManagerConfig) ResolvedHTTPReadTimeout() time.Duration

ResolvedHTTPReadTimeout returns the configured read timeout or the default when unset/invalid.

func (ManagerConfig) ResolvedHTTPWriteTimeout

func (c ManagerConfig) ResolvedHTTPWriteTimeout() time.Duration

ResolvedHTTPWriteTimeout returns the configured write timeout or the default when unset/invalid.

func (ManagerConfig) Validate

func (c ManagerConfig) Validate() error

Validate checks if the configuration is valid

type MandatoryServiceDisabledError

type MandatoryServiceDisabledError struct {
	Name string
}

MandatoryServiceDisabledError reports desired state that cannot form a valid framework process composition.

func (*MandatoryServiceDisabledError) Error

type MessageLogEntry

type MessageLogEntry struct {
	Sequence    uint64          `json:"sequence"` // Monotonic sequence; owns the entry's buffer slot
	Timestamp   time.Time       `json:"timestamp"`
	Subject     string          `json:"subject"`
	MessageType string          `json:"message_type,omitempty"`
	MessageID   string          `json:"message_id,omitempty"`
	TraceID     string          `json:"trace_id,omitempty"` // W3C trace ID (32 hex chars)
	SpanID      string          `json:"span_id,omitempty"`  // W3C span ID (16 hex chars)
	Summary     string          `json:"summary"`
	RawData     json.RawMessage `json:"raw_data,omitempty"`
	Metadata    map[string]any  `json:"metadata,omitempty"`
}

MessageLogEntry is one stored message. RawData is the payload as published when it is JSON, and the payload as a JSON string when it is not.

type MessageLogger

type MessageLogger struct {
	*BaseService
	// contains filtered or unexported fields
}

MessageLogger observes the messages published on its subjects and keeps the newest in memory for its HTTP routes. Start subscribes; Stop is the only stop (ADR-113).

func (*MessageLogger) GetEntriesByTrace

func (ml *MessageLogger) GetEntriesByTrace(traceID string) []MessageLogEntry

GetEntriesByTrace returns the held entries of the trace traceID, oldest first. It scans the buffer, so an entry the buffer has overwritten is gone.

func (*MessageLogger) GetLogEntries

func (ml *MessageLogger) GetLogEntries(limit int) []MessageLogEntry

GetLogEntries returns the newest stored entries, newest first, at most limit of them; a limit of 0 or less returns every stored entry.

func (*MessageLogger) GetStatistics

func (ml *MessageLogger) GetStatistics() map[string]any

GetStatistics returns the logger's counts and subject state. valid_messages counts the messages decoded as typed envelopes, invalid_messages the rest. dropped_messages counts the messages not stored because their handler context had ended; total_messages does not count them. subject_reconciliation_degraded is true while a subject is not subscribed, and subject_reconciliation_error names each one with its last subscribe error.

func (*MessageLogger) OpenAPISpec

func (ml *MessageLogger) OpenAPISpec() *OpenAPISpec

OpenAPISpec returns the OpenAPI specification for MessageLogger endpoints

func (*MessageLogger) RegisterHTTPHandlers

func (ml *MessageLogger) RegisterHTTPHandlers(prefix string, mux *http.ServeMux)

RegisterHTTPHandlers mounts the logger's routes under prefix. Every route answers GET, and HEAD as ServeMux serves it on a "GET " pattern; the mux answers any other method with 405.

func (*MessageLogger) Start

func (ml *MessageLogger) Start(ctx context.Context) error

Start computes the subjects to monitor, refuses a malformed one, starts the base service and subscribes to each subject in sorted order. It refuses a nil or ended context, a second Start and a Start after any Stop (errs.ErrAlreadyStarted); a refused Start holds nothing.

The subjects are the explicit monitor_subjects and, with "*", the NATS, NATS-request and JetStream port subjects of every admitted component (skipping "*", ">" and _INBOX). Every one is checked with flowgraph.ValidSubject before anything else happens; the first malformed one refuses Start as invalid data, naming the subject and where it came from. A subject another covers is not subscribed and is reported in the statistics' subject_overlaps.

A subscribe that fails leaves the logger running: the subject is retried every 100 ms until it subscribes, and meanwhile the statistics report the logger degraded and its health check fails, naming the subject. Start subscribes under ctx and checks it before each subscribe and once more after the last, before it logs that it started; if ctx has ended at any of these checks, Start returns its error and releases everything the logger holds. A subscribe ctx cuts after its SUB was sent is removed by the NATS client, which returns no subscription; a message delivered to it before that can still be stored after Start failed, and the client's Close waits for its callback. Ending ctx after Start returned stops nothing: each message's context is the NATS client's, not ctx's (natsclient.Client.Subscribe).

func (*MessageLogger) Stop

func (ml *MessageLogger) Stop(ctx context.Context) error

Stop ends the subscribe retry and waits for it, drains every subscription so a message being handled is stored, then ends the run context, ends every open KV watch stream and waits for its watcher to stop, and stops the base service. It returns nil only once all are released. If ctx ends first, or a drain fails, it returns that error and leaves what is not yet released for the next Stop, which runs the same steps. A Stop after one that returned nil returns nil; a Stop while another runs is refused as transient. A watch requested once Stop has begun is refused.

type MessageLoggerConfig

type MessageLoggerConfig struct {
	// MonitorSubjects are the NATS subjects to monitor. "*" adds the NATS, NATS-request and
	// JetStream port subjects of every admitted component. Empty means ["*"].
	// Example: ["*"] or ["*", "debug.>"] or ["raw.udp.messages", "processed.>"]
	MonitorSubjects []string `json:"monitor_subjects"`

	// MaxEntries is how many entries the logger keeps in memory, 0 to 100000; 0 means 10000.
	MaxEntries int `json:"max_entries"`

	// OutputToStdout prints one line per stored entry.
	OutputToStdout bool `json:"output_to_stdout"`

	// SampleRate stores one in every SampleRate messages and counts the rest; 0 and 1 store
	// every message.
	SampleRate int `json:"sample_rate"`
}

MessageLoggerConfig holds configuration for the MessageLogger service.

func (MessageLoggerConfig) Validate

func (c MessageLoggerConfig) Validate() error

Validate checks the configuration's bounds.

type Metrics

type Metrics struct {
	*BaseService
	// contains filtered or unexported fields
}

Metrics is a service that provides the Prometheus metrics endpoint. On its own it serves a metric.Server on its configured port between Start and Stop. Composed by the service manager, the manager claims that server before any service starts, serves it around every service, and reports whether it serves; Metrics is then healthy while it does.

func (*Metrics) Start

func (m *Metrics) Start(ctx context.Context) error

Start starts the metrics HTTP server, unless the manager claimed it, and then the base service. A bind failure is returned and nothing is held. ctx bounds Start only: ending it after Start returned stops nothing. A second Start, and a Start after any Stop, are refused with errs.ErrAlreadyStarted.

func (*Metrics) Stop

func (m *Metrics) Stop(ctx context.Context) error

Stop stops the metrics server Metrics started, then the base service. It returns nil only once both are released; if ctx ends first it returns ctx's error and a later Stop tries again. A Stop while another runs is refused as transient.

type MetricsConfig

type MetricsConfig struct {
	Port int    `json:"port"`
	Path string `json:"path"`
}

MetricsConfig holds configuration for the metrics service Simple struct - no UnmarshalJSON, no Enabled field

func (MetricsConfig) Validate

func (c MetricsConfig) Validate() error

Validate checks if the configuration is valid

type MetricsForwarder

type MetricsForwarder struct {
	*BaseService
	// contains filtered or unexported fields
}

MetricsForwarder gathers the composition's metrics registry when it starts and then once per push interval, and publishes each sample as one JSON object on metrics.<component>.<name> through the JetStream publish of the NATS client it was built with. A sample that does not encode or does not publish, and a gather that fails, are counted on semengine_errors_total and logged at Debug; the loop goes on.

func (*MetricsForwarder) Start

func (mf *MetricsForwarder) Start(ctx context.Context) error

Start starts the base service, then the publish loop, and returns nil; it does no I/O. It refuses a nil or ended context, a second Start and a Start after any Stop (errs.ErrAlreadyStarted); a refused Start holds nothing. ctx bounds Start only; ending it after Start returned stops nothing.

func (*MetricsForwarder) Stop

func (mf *MetricsForwarder) Stop(ctx context.Context) error

Stop ends the run context, waits for the loop to return and stops the base service. It returns nil only once both are done. If ctx ends first it returns that context's error and the next Stop waits again. A Stop after one that returned nil returns nil; a Stop while another runs is refused as transient.

type MetricsForwarderConfig

type MetricsForwarderConfig struct {
	// PushInterval between publishing rounds (e.g. "5s", "1m"). Default: "5s".
	PushInterval string `json:"push_interval"`

	// IncludeGoMetrics forwards the go_* runtime families (goroutines, memory, GC). Default: false.
	IncludeGoMetrics bool `json:"include_go_metrics"`

	// IncludeProcMetrics forwards the process_* families (CPU, open FDs, memory). Default: false.
	IncludeProcMetrics bool `json:"include_proc_metrics"`
}

MetricsForwarderConfig holds configuration for the MetricsForwarder service.

type OpenAPIDocument

type OpenAPIDocument struct {
	OpenAPI    string              `json:"openapi"`
	Info       InfoSpec            `json:"info"`
	Servers    []ServerSpec        `json:"servers"`
	Paths      map[string]PathSpec `json:"paths"`
	Components *ComponentsSpec     `json:"components,omitempty"`
	Tags       []TagSpec           `json:"tags,omitempty"`
}

OpenAPIDocument represents the complete OpenAPI 3.0 specification

func (OpenAPIDocument) MarshalJSON

func (doc OpenAPIDocument) MarshalJSON() ([]byte, error)

MarshalJSON writes the document in OpenAPI 3.0's shape. Each operation's RequestBodySpec becomes requestBody, and its request body and each response with a schema or content type become content[mediaType].schema: a $ref, or an array of $ref items when IsArray is set. A request body, or a response with a schema, that names no content type is application/json; a content type with no schema has a media type with no schema. The service descriptors' own JSON is unchanged.

type OpenAPISpec

type OpenAPISpec struct {
	Paths            map[string]PathSpec `json:"paths"`
	Components       map[string]any      `json:"components,omitempty"`
	Tags             []TagSpec           `json:"tags,omitempty"`
	ResponseTypes    []reflect.Type      `json:"-"` // Response types to generate schemas for (not serialized)
	RequestBodyTypes []reflect.Type      `json:"-"` // Request body types to generate schemas for (not serialized)
}

OpenAPISpec represents a service's OpenAPI specification fragment

func (*OpenAPISpec) MarshalJSON

func (spec *OpenAPISpec) MarshalJSON() ([]byte, error)

MarshalJSON implements json.Marshaler for OpenAPISpec

func (*OpenAPISpec) UnmarshalJSON

func (spec *OpenAPISpec) UnmarshalJSON(data []byte) error

UnmarshalJSON implements json.Unmarshaler for OpenAPISpec

type OperationSpec

type OperationSpec struct {
	Summary     string                  `json:"summary"`
	Description string                  `json:"description,omitempty"`
	Parameters  []ParameterSpec         `json:"parameters,omitempty"`
	RequestBody *RequestBodySpec        `json:"request_body,omitempty"`
	Responses   map[string]ResponseSpec `json:"responses"`
	Tags        []string                `json:"tags,omitempty"`
}

OperationSpec defines a single HTTP operation

type Option

type Option func(*BaseService)

Option is a functional option for configuring BaseService

func WithLogger

func WithLogger(logger *slog.Logger) Option

WithLogger sets a custom logger for the service

func WithMetrics

func WithMetrics(registry *metric.MetricsRegistry) Option

WithMetrics sets the metrics registry for the service

func WithNATS

func WithNATS(client *natsclient.Client) Option

WithNATS sets the NATS client for the service

type PProfServer

type PProfServer struct {
	// contains filtered or unexported fields
}

PProfServer is the profiling server MaybeStartPProf started. Shutdown stops it. A nil *PProfServer is a server that was never started; its Shutdown returns nil.

func MaybeStartPProf

func MaybeStartPProf(debug bool, port int, logger *slog.Logger) *PProfServer

MaybeStartPProf serves http.DefaultServeMux on :port when debug && port > 0, and returns nil otherwise. It binds :port before it returns. A bind failure is logged at Warn on logger (slog.Default() when nil) and returns nil; profiling is a debug aid, never a boot gate.

ADR-058: pprof is deliberately NOT a service.Service. Wrapping it in a StartAll-managed Service would move the start from the composition root (before NATS) to StartAll (after NATS and all wiring), losing the ability to profile a wedged or slow boot, the canonical pprof use case. So it is a plain helper the composition root calls early. It is not fire-and-forget: the composition root calls Shutdown on its exit path, after its services have stopped (background-work spec).

The pprof handlers are registered on http.DefaultServeMux by the CALLER's `import _ "net/http/pprof"` blank import; this helper serves that mux (nil handler). The blank import stays in the caller's main, not here, so importing package service does not arm pprof everywhere.

func (*PProfServer) Shutdown

func (s *PProfServer) Shutdown(ctx context.Context) error

Shutdown stops the server accepting and returns nil only after every in-flight request has finished and the serving goroutine has exited. If ctx ends first it returns ctx's error, and a later Shutdown waits again. A serve error other than http.ErrServerClosed is returned once the goroutine has exited. A nil ctx is refused, on a nil server too; a nil server's Shutdown otherwise returns nil.

type ParameterSpec

type ParameterSpec struct {
	Name        string `json:"name"`
	In          string `json:"in"` // "query", "path", "header"
	Description string `json:"description,omitempty"`
	Required    bool   `json:"required,omitempty"`
	Schema      Schema `json:"schema,omitempty"`
}

ParameterSpec defines an operation parameter

type PathSpec

type PathSpec struct {
	GET    *OperationSpec `json:"get,omitempty"`
	POST   *OperationSpec `json:"post,omitempty"`
	PUT    *OperationSpec `json:"put,omitempty"`
	PATCH  *OperationSpec `json:"patch,omitempty"`
	DELETE *OperationSpec `json:"delete,omitempty"`
}

PathSpec defines HTTP operations for a specific path

type PendingServiceChange

type PendingServiceChange struct {
	Name   string `json:"name"`
	Change string `json:"change"`
}

PendingServiceChange describes one structural desired-service change that a process restart is required to attempt to consume.

type ProjectionBinder

type ProjectionBinder interface {
	ProjectionBindings() (packID string, contracts []projection.Contract)
	PreflightProjectionMutations() error
	SetPredicateReconciler(projection.PredicateReconciler) error
}

ProjectionBinder is implemented by rule processors that use reconciled projection writes. Contracts are validated before any client is injected.

type Registry

type Registry struct {
	// contains filtered or unexported fields
}

Registry manages service constructor registration

func NewServiceRegistry

func NewServiceRegistry() *Registry

NewServiceRegistry creates a new service registry

func (*Registry) Constructor

func (r *Registry) Constructor(name string) (Constructor, bool)

Constructor returns a constructor for the given service name

func (*Registry) Register

func (r *Registry) Register(name string, constructor Constructor) error

Register registers a service constructor

type RequestBodySpec

type RequestBodySpec struct {
	Description string `json:"description,omitempty"`
	ContentType string `json:"content_type,omitempty"` // defaults to "application/json"
	SchemaRef   string `json:"schema_ref,omitempty"`   // e.g. "#/components/schemas/ReviewRequest"
	Required    bool   `json:"required,omitempty"`
}

RequestBodySpec defines an operation request body

type ResponseSpec

type ResponseSpec struct {
	Description string `json:"description"`
	ContentType string `json:"content_type,omitempty"`
	SchemaRef   string `json:"schema_ref,omitempty"` // $ref to schema, e.g., "#/components/schemas/RuntimeHealthResponse"
	IsArray     bool   `json:"is_array,omitempty"`   // If true, response is an array of SchemaRef items
}

ResponseSpec defines an operation response

type Schema

type Schema struct {
	Type   string `json:"type"`
	Format string `json:"format,omitempty"`
}

Schema defines parameter or response schema

type ServerSpec

type ServerSpec struct {
	URL         string `json:"url"`
	Description string `json:"description"`
}

ServerSpec defines an API server

type Service

type Service interface {
	Name() string
	Start(ctx context.Context) error
	// Stop stops the service. It returns nil only once the service has
	// released everything it started; if ctx ends first it returns ctx's
	// error and a later Stop tries again. A Stop after one that returned nil
	// returns nil.
	Stop(ctx context.Context) error
	Status() Status
	IsHealthy() bool       // Keep for compatibility during migration
	GetStatus() Info       // Keep for compatibility during migration
	Health() health.Status // NEW: Standard health reporting
}

Service interface defines the contract for all services

func NewComponentManager

func NewComponentManager(rawConfig json.RawMessage, deps *Dependencies) (Service, error)

NewComponentManager decodes the component manager's configuration, which defines no key, and composes the boot component set. It refuses a configuration key, a missing configuration manager or component registry, a component that cannot be built (naming every one) and a composition with an error-severity finding; a refusal returns a nil Service.

func NewHeartbeatService

func NewHeartbeatService(rawConfig json.RawMessage, deps *Dependencies) (Service, error)

NewHeartbeatService builds the heartbeat service from its JSON configuration (HeartbeatConfig). A key the configuration does not define, or one in another letter case, is refused, naming the key. It refuses, as invalid data and naming the input, nil dependencies, a nil service manager, and an interval that does not parse, is not positive or is under 1s. It looks nothing up and does no I/O: the component manager is found at Start.

func NewLogForwarderService

func NewLogForwarderService(rawConfig json.RawMessage, deps *Dependencies) (Service, error)

NewLogForwarderService builds the log forwarder from its JSON configuration (LogForwarderConfig). A key the configuration does not define, or one in another letter case, is refused, naming the key. It refuses, as invalid data and naming the input, nil dependencies and a min_level other than DEBUG, INFO, WARN or ERROR. It needs no NATS client and does no I/O.

func NewMessageLoggerService

func NewMessageLoggerService(rawConfig json.RawMessage, deps *Dependencies) (Service, error)

NewMessageLoggerService creates the message logger from its JSON configuration (MessageLoggerConfig). A key the configuration does not define, or one in another letter case, is refused, naming the key. It refuses, as invalid data and naming the input, a nil NATS client, a nil payload registry, "*" in monitor_subjects (the default) without a component registry, max_entries below 0 or above 100000, and sample_rate below 0. The logger decodes typed envelopes through deps.PayloadRegistry and, with "*", reads the subjects of the admitted components from deps.ComponentRegistry when it starts.

func NewMetrics

func NewMetrics(rawConfig json.RawMessage, deps *Dependencies) (Service, error)

NewMetrics creates the metrics service from its JSON configuration: port (default 9090) and path (default "/metrics"). A key the configuration does not define, or one in another letter case, is refused. The server uses the platform TLS settings from deps.Manager and logs through deps.Logger.

func NewMetricsForwarderService

func NewMetricsForwarderService(rawConfig json.RawMessage, deps *Dependencies) (Service, error)

NewMetricsForwarderService builds the metrics forwarder from its JSON configuration (MetricsForwarderConfig). A key the configuration does not define, or one in another letter case, is refused, naming the key. It refuses, as invalid data and naming the input, nil dependencies, a nil NATS client, a nil metrics registry, and a push_interval that does not parse or is not positive; an empty or absent push_interval takes 5s. It does no I/O.

func NewStorageObservabilityService

func NewStorageObservabilityService(rawConfig json.RawMessage, deps *Dependencies) (Service, error)

NewStorageObservabilityService builds the service from its JSON configuration (StorageObservabilityConfig). A key the configuration does not define, or one in another letter case, nested ones included, is refused, naming the key. It refuses, as invalid data and naming the input, nil dependencies, a nil NATS client, a nil metrics registry, an interval or timeout that does not parse or is not positive, a timeout above the interval, and pressure thresholds that do not resolve. It does no I/O: the report bucket is acquired by the loops Start runs.

type Status

type Status int

Status represents the current status of a service

const (
	StatusStopped Status = iota
	StatusStarting
	StatusRunning
	StatusStopping
)

Possible service statuses

func (Status) String

func (s Status) String() string

String returns the string representation of Status

type StorageObservabilityConfig

type StorageObservabilityConfig struct {
	// Interval is how often the account is enumerated. Every process polling
	// account-wide multiplies cost by deployment size, so an operator running
	// many instances turns this down rather than losing the view entirely.
	// Default: 1m.
	Interval string `json:"interval,omitempty"`

	// Timeout bounds ONE collection, both account listings included. It must
	// fit inside Interval. Default: 15s.
	Timeout string `json:"timeout,omitempty"`

	// PressureThresholds are the headroom and time-to-threshold bands pressure
	// is derived from. Report-only: nothing is rejected, throttled, or degraded
	// when a band is crossed.
	PressureThresholds storagereport.StoragePressureThresholds `json:"pressure_thresholds,omitempty"`
}

StorageObservabilityConfig is the operator surface for account storage observability.

Durations are strings ("1m", "30s") following the repository's service configuration convention. Every field is optional and takes its documented default; an unusable value is an ERROR at construction rather than a silent fallback, because a silently defaulted knob applies a number the operator did not choose and is indistinguishable from a working edit.

type StorageObservabilityService

type StorageObservabilityService struct {
	*BaseService
	// contains filtered or unexported fields
}

StorageObservabilityService collects the account storage inventory, publishes the report, and serves it as metrics, health status and an HTTP route.

func (*StorageObservabilityService) Health

Health reports the storage picture WITHOUT letting it become a verdict.

The lifecycle status comes from BaseService untouched; pressure, capacity and over-commitment appear in the message. That separation is the report-only guarantee expressed where it would break first: readiness reads Status(), and /health aggregates Health(), so a pressure-derived verdict at either point would turn observability into admission control. Both reads do no I/O.

func (*StorageObservabilityService) OpenAPISpec

func (s *StorageObservabilityService) OpenAPISpec() *OpenAPISpec

OpenAPISpec documents the report route. Its paths are relative to the prefix the service is mounted under, which the runtime document applies. RegisterAll registers no spec for this service, because the spec names no schema.

func (*StorageObservabilityService) RegisterHTTPHandlers

func (s *StorageObservabilityService) RegisterHTTPHandlers(prefix string, mux *http.ServeMux)

RegisterHTTPHandlers mounts GET <prefix>/report, the published report. The prefix may end in a slash or not. The route is a GET pattern, so ServeMux answers HEAD as GET and refuses any other method with 405. It reads the consumer's snapshot, which does no I/O, so it answers before Start, while running and after Stop.

func (*StorageObservabilityService) Start

Start starts the base service, then the inventory collector's and the report consumer's loops, and returns nil. It does no I/O and never fails on account or bucket state: an unreadable account or an unreachable bucket is retried by the loops, each on its own schedule. It refuses a nil or ended context, a second Start and a Start after any Stop (errs.ErrAlreadyStarted); a refused Start holds nothing. ctx bounds Start only; ending it after Start returned stops nothing.

func (*StorageObservabilityService) Stop

Stop ends the run context, waits for both loops to return and stops the base service. It returns nil only once all three are done. If ctx ends first it returns that context's error and the next Stop waits again. A Stop after one that returned nil returns nil; a Stop while another runs is refused as transient. How soon a collection in progress sees the run context end depends on whether the JetStream listing honours its context, which nats.go does not promise; Stop stays bounded by its own context either way.

type StorageReportResponse

type StorageReportResponse struct {
	// ReportOnly is always true and is stated on the response's face. A reader
	// automating against this route must be able to see, without consulting a
	// document, that nothing was rejected, throttled, or degraded because of
	// what it says.
	ReportOnly bool `json:"report_only"`

	// Synced reports that the watch has delivered every current value at least
	// once. Before it, an empty Resources means "not read yet" rather than "the
	// account holds nothing": two facts an operator acts on differently.
	Synced bool `json:"synced"`

	// UpdatedAt is when the last change was applied to the in-process view. A
	// POINTER because `omitempty` is a no-op on a time.Time: a value field would
	// publish a zero timestamp on an unread report and invite a reader to treat
	// it as a real one.
	UpdatedAt *time.Time `json:"updated_at,omitempty"`

	// Summary is a TALLY of what the rows say, carried through from the
	// consumer. It is never a re-evaluation.
	Summary StorageReportSummary `json:"summary"`

	// Account is the per-tier declared-versus-limit comparison. Absent, not
	// zero-valued, when no account row has been read, because an empty
	// comparison reads as "no tiers, nothing over-committed".
	Account *storagereport.AccountReport `json:"account,omitempty"`

	// Resources is every published row, sorted by resource name. Never
	// filtered: see the file comment.
	Resources []storagereport.ResourceReport `json:"resources"`
}

StorageReportResponse is the body of `GET /storage-observability/report`.

Resources are the PUBLISHED rows verbatim, the same storagereport.ResourceReport values the collector wrote and the metrics surface consumed, so the HTTP shape cannot drift away from the bucket's. Nothing here is recomputed.

type StorageReportSummary

type StorageReportSummary struct {
	// Resources is how many rows the report carries.
	Resources int `json:"resources"`

	// Pressure counts the EVALUATED rows by state.
	Pressure map[storagereport.PressureState]int `json:"pressure"`

	// NotEvaluated counts the rows carrying no pressure state at all. Its own
	// bucket rather than folded into normal: otherwise the resources nothing can
	// be said about would be the ones that disappear from the summary.
	//
	// It is not "the unbounded resources". A resource with no bound of its own is
	// evaluated against its storage tier's ceiling and counts under Pressure by
	// that state; what remains here is a resource whose capacity could not be
	// READ, plus an unbounded one whose tier offers no ceiling either. To count
	// the unbounded set, read the rows' capacity state, not this field.
	NotEvaluated int `json:"not_evaluated"`

	// WorstPressure is the worst EVALUATED state, omitted when nothing was
	// evaluated. Omission is NOT normal: an account whose every row declined to
	// evaluate has no pressure verdict, and publishing one would manufacture it.
	WorstPressure storagereport.PressureState `json:"worst_pressure,omitempty"`
}

StorageReportSummary is the account at a glance.

type TagSpec

type TagSpec struct {
	Name        string `json:"name"`
	Description string `json:"description"`
}

TagSpec defines an API tag for grouping operations

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL