hub

package
v0.0.310 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 26, 2026 License: MIT Imports: 19 Imported by: 0

Documentation

Overview

Package hub models a fleet of term-llm web nodes for `term-llm serve hub`.

The core object is a Node: a reachable term-llm serve (web/API endpoint) with an identity, a backend URL + base path, an optional bearer token, and a source describing which resolver produced it. Resolvers (static config, contain workspaces, the local UI-added store) feed a Registry, which is the single lookup surface the hub server routes and proxies from.

Reverse nodes may self-register with the hub when the hub operator enables a registration token; scheduling and mTLS between hub and nodes remain out of scope for v1. See docs-site guide "Hub".

Index

Constants

View Source
const (
	DelegationStatusPending         = "pending"
	DelegationStatusRunning         = "running"
	DelegationStatusCancelRequested = "cancel_requested"
	DelegationStatusSucceeded       = "succeeded"
	DelegationStatusFailed          = "failed"
	DelegationStatusCancelled       = "cancelled"
	DelegationStatusTimedOut        = "timed_out"
	DelegationStatusError           = "error"
)

Delegation statuses. Pending/running/cancel_requested are active; the rest are terminal. They deliberately mirror jobs-v2 run statuses since a delegation is fronted by one jobs-v2 run on the target node, with "error" added for delegations that broke outside the target run itself.

View Source
const (
	SourceConfig  = "config"
	SourceContain = "contain"
	SourceLocal   = "local"
)

Source labels for the built-in resolvers.

View Source
const DefaultDelegationAgent = "developer"

DefaultDelegationAgent is the agent delegated jobs run as when the request names none; with no allowed_agents policy it is also the ONLY agent a target accepts.

View Source
const DefaultDelegationMaxDepth = 3

DefaultDelegationMaxDepth bounds delegation chains (A -> B -> C is depth 3) so two mutually-delegating nodes cannot recurse forever.

View Source
const DefaultDelegationMaxInFlight = 4

DefaultDelegationMaxInFlight bounds concurrently active delegations per target node when the node's policy does not set max_in_flight.

Variables

This section is empty.

Functions

func DelegationStatusTerminal

func DelegationStatusTerminal(status string) bool

DelegationStatusTerminal reports whether a delegation status is final.

func NewDelegationID

func NewDelegationID() (string, error)

NewDelegationID returns a fresh delegation id ("dlg_" + 16 hex chars).

func ParseNodeURL

func ParseNodeURL(raw string) (origin, basePath string, err error)

ParseNodeURL splits a node URL like "http://127.0.0.1:8081/chat" into its origin ("http://127.0.0.1:8081") and normalized base path ("/chat"). A URL without a path yields an empty base path; Normalize rejects that unless an explicit BasePath is supplied, because Hub v1's path proxy needs a prefix.

func SlugID

func SlugID(name string) (string, error)

SlugID derives a valid node ID from a free-form name, for nodes added without an explicit ID. Returns an error when nothing usable remains.

func ValidateID

func ValidateID(id string) error

ValidateID rejects node IDs that are empty or unsafe as a single proxy path segment. The charset deliberately excludes '/', '%', and whitespace so an ID can never smuggle extra path segments into /node/<id>/.

Types

type ContainResolver

type ContainResolver struct {
	// Host is the host the workspaces' serves are published on (loopback in
	// the one-container-per-agent shape).
	Host string
	// List enumerates contain workspace definitions.
	List func() ([]contain.ListEntry, error)
	// Read resolves a workspace's web config from its .env.
	Read func(name string) (contain.WebConfig, error)
	// Workspace resolves a workspace's in-container workspace path (the
	// compose x-term-llm workspace hint). A non-empty result becomes the
	// node's delegation workdir, opting the workspace in to accepting
	// hub-mediated delegations; an empty result leaves the node unable to
	// accept (default deny).
	Workspace func(name string) string
}

ContainResolver discovers nodes from local contain workspaces: each workspace with a provisioned web token becomes a node whose URL/token come from its .env (via contain.ReadWebConfig). List and Read are fields so tests can substitute fakes for the container config directory.

func NewContainResolver

func NewContainResolver() *ContainResolver

NewContainResolver returns a resolver over the local contain workspaces.

func (*ContainResolver) Nodes

func (c *ContainResolver) Nodes() ([]Node, error)

Nodes implements Resolver. Workspaces whose .env cannot be read or that have no provisioned web token are skipped (they have no reachable serve to front), so a half-created workspace never breaks hub discovery.

func (*ContainResolver) Source

func (c *ContainResolver) Source() string

Source implements Resolver.

type Delegation

type Delegation struct {
	ID         string `json:"id"`
	OriginNode string `json:"origin_node"`
	TargetNode string `json:"target_node"`
	AgentName  string `json:"agent_name,omitempty"`
	// Prompt is stored truncated (see delegationPromptLimit) for audit; the
	// full prompt only travels to the target job's instructions.
	Prompt string `json:"prompt,omitempty"`
	Model  string `json:"model,omitempty"`
	Cwd    string `json:"cwd,omitempty"`
	// JobID/RunID identify the jobs-v2 job and run on the TARGET node.
	JobID  string `json:"job_id,omitempty"`
	RunID  string `json:"run_id,omitempty"`
	Status string `json:"status"`
	// Depth is 1 for a direct delegation and parent depth+1 for delegations
	// created with parent_delegation_id. Chain lists the node ids involved
	// from the root origin through this delegation's target, used to refuse
	// delegation loops.
	Depth    int      `json:"depth"`
	Chain    []string `json:"chain,omitempty"`
	ParentID string   `json:"parent_delegation_id,omitempty"`
	// Response holds the target run's final response (truncated) once the
	// delegation reaches a terminal status.
	Response  string    `json:"response,omitempty"`
	Error     string    `json:"error,omitempty"`
	CreatedAt time.Time `json:"created_at"`
	UpdatedAt time.Time `json:"updated_at"`
}

Delegation is one hub-mediated cross-node delegation: origin node asked the hub to run a prompt as a jobs-v2 LLM job on the target node. The record deliberately holds NO tokens — node credentials live only in node sources — so the ledger and every API view built from it are safe to serve.

type DelegationPolicy

type DelegationPolicy struct {
	Enabled     bool     `json:"enabled,omitempty" yaml:"enabled"`
	To          []string `json:"to,omitempty" yaml:"to"`
	AcceptFrom  []string `json:"accept_from,omitempty" yaml:"accept_from"`
	Workdir     string   `json:"workdir,omitempty" yaml:"workdir"`
	MaxInFlight int      `json:"max_in_flight,omitempty" yaml:"max_in_flight"`
	// AllowedAgents lists agent names origins may request on this node.
	// Empty means only DefaultDelegationAgent; "*" allows any plain agent
	// name, but path-like names (containing a separator or "..") always
	// require an exact entry.
	AllowedAgents []string `json:"allowed_agents,omitempty" yaml:"allowed_agents"`
	// AllowedModels lists model overrides origins may request. Empty means no
	// override is accepted (the target's own default is used); "*" allows any.
	AllowedModels []string `json:"allowed_models,omitempty" yaml:"allowed_models"`
}

DelegationPolicy is a node's cross-node delegation policy, configured per node in the hub config:

nodes:
  - name: jarvis
    url: http://127.0.0.1:8081/chat
    token: secret
    delegation:
      to: ["*"]           # node ids this node may delegate to
      accept_from: ["*"]  # node ids this node accepts delegations from
      workdir: /work      # REQUIRED to accept; cwd for delegated jobs
      max_in_flight: 4    # concurrently active delegations targeting this node
      allowed_agents: []  # agents origins may request (default: developer only)
      allowed_models: []  # model overrides origins may request (default: none)

Defaults are off: a node must set delegation.enabled to true before it can originate or accept delegated work. Accepting still also requires a workdir.

type DelegationStore

type DelegationStore struct {
	// contains filtered or unexported fields
}

DelegationStore persists delegation records in a local JSON ledger. The file is written 0600 like the node store: the records hold no tokens, but prompts and responses are still private session content.

func NewDelegationStore

func NewDelegationStore(path string) *DelegationStore

NewDelegationStore returns a store backed by the given JSON file path. The file is created lazily on first Add.

func (*DelegationStore) ActiveCounts

func (s *DelegationStore) ActiveCounts() (total int, byOrigin, byTarget map[string]int, err error)

ActiveCounts tallies non-terminal delegations hub-wide, per origin node, and per target node, for in-flight cap enforcement.

func (*DelegationStore) Add

func (s *DelegationStore) Add(d Delegation) error

Add persists a new delegation record. Prompt and response excerpts are truncated to keep the ledger bounded.

func (*DelegationStore) Get

func (s *DelegationStore) Get(id string) (Delegation, bool, error)

Get returns a delegation by id.

func (*DelegationStore) List

func (s *DelegationStore) List() ([]Delegation, error)

List returns all records, newest first.

func (*DelegationStore) Path

func (s *DelegationStore) Path() string

Path returns the backing file path.

func (*DelegationStore) Update

func (s *DelegationStore) Update(id string, fn func(*Delegation)) (Delegation, error)

Update applies fn to the stored record and persists the result. The record's UpdatedAt is stamped and the response excerpt re-truncated.

type Node

type Node struct {
	// ID uniquely identifies the node within the hub and is used as the
	// proxy path segment (/node/<id>/...). Restricted to a path-safe charset.
	ID string `json:"id" yaml:"id"`
	// Name is the human-facing label shown on the dashboard.
	Name string `json:"name" yaml:"name"`
	// Source is the resolver that produced this node (config/contain/local).
	Source string `json:"source" yaml:"-"`
	// Connection is how the Hub reaches this node. Empty/"direct" means the Hub
	// dials URL. "reverse" means the node must dial the Hub and keep a websocket
	// open; useful for private nodes behind NAT/firewalls.
	Connection string `json:"connection,omitempty" yaml:"connection"`
	// URL is the backend origin, e.g. "http://127.0.0.1:8081". Never sent to
	// hub clients together with Token.
	URL string `json:"url" yaml:"url"`
	// BasePath is the URL prefix the node's serve is mounted under, e.g.
	// "/chat". Always rooted, no trailing slash; root-mounted serves are not
	// supported by Hub v1 because path-based proxying needs a stable prefix.
	BasePath string `json:"base_path,omitempty" yaml:"base_path"`
	// Token is the node's web bearer token. It is injected server-side by the
	// hub proxy and MUST never be marshalled into any client-facing response;
	// API handlers convert Node to a public view first.
	Token string `json:"token,omitempty" yaml:"token"`
	// Delegation is the node's cross-node delegation policy. Nil or disabled
	// means the node cannot originate or accept delegations.
	Delegation *DelegationPolicy `json:"delegation,omitempty" yaml:"delegation"`
}

Node is one reachable term-llm web/API endpoint known to the hub.

func ParseConfig

func ParseConfig(data []byte) ([]Node, error)

ParseConfig parses a static hub config document into normalized nodes.

func (Node) AcceptsDelegationAgent

func (n Node) AcceptsDelegationAgent(agent string) error

AcceptsDelegationAgent returns nil when this node's policy allows delegated work to run as the given agent. With no allowed_agents configured only DefaultDelegationAgent is accepted; "*" accepts any plain name; path-like names always need an exact entry.

func (Node) AcceptsDelegationFrom

func (n Node) AcceptsDelegationFrom(originID string) error

AcceptsDelegationFrom returns nil when this node accepts delegated work from the origin node, or an error describing why not. Delegation is default-off; accepting additionally requires an explicit delegation workdir.

func (Node) AcceptsDelegationModel

func (n Node) AcceptsDelegationModel(model string) error

AcceptsDelegationModel returns nil when this node's policy allows the given model override. An empty model (use the target's default) is always allowed; with no allowed_models configured every override is refused.

func (Node) BaseURL

func (n Node) BaseURL() string

BaseURL returns the node's backend origin joined with its base path, without a trailing slash (e.g. "http://127.0.0.1:8081/chat"). Reverse nodes may not have a URL; callers that dial directly must check UsesReverseConnection first.

func (Node) CanDelegateTo

func (n Node) CanDelegateTo(targetID string) bool

CanDelegateTo reports whether this node's policy allows originating a delegation to the target node. Delegation is default-off; once enabled, an empty "to" list means any target (the target still has to accept).

func (Node) DelegationMaxInFlight

func (n Node) DelegationMaxInFlight() int

DelegationMaxInFlight returns the per-target concurrency cap for this node.

func (Node) DelegationWorkdir

func (n Node) DelegationWorkdir() string

DelegationWorkdir returns the node's configured delegation workdir ("" when delegation is disabled or the node does not accept delegations).

func (*Node) Normalize

func (n *Node) Normalize() error

Normalize validates and canonicalizes a node in place: the URL is split into origin + base path (an explicit BasePath wins over a path embedded in URL), the ID falls back to a slug of the name, and the name falls back to the ID.

func (Node) UsesReverseConnection

func (n Node) UsesReverseConnection() bool

type Prober

type Prober struct {
	Client *http.Client
}

Prober checks node health over HTTP. The client must not use an environment proxy: probes carry node bearer tokens, and routing them through an HTTP_PROXY would leak the tokens (callers pass the same direct-dial transport the hub proxy uses).

func NewProber

func NewProber(transport http.RoundTripper) *Prober

NewProber returns a prober over the given transport with a bounded per-probe timeout so one hung node cannot stall a dashboard refresh.

func (*Prober) Probe

func (p *Prober) Probe(ctx context.Context, n Node) Status

Probe checks one node's {base}/healthz, measuring latency and decoding any extended identity fields the node reports.

func (*Prober) ProbeAll

func (p *Prober) ProbeAll(ctx context.Context, nodes []Node) map[string]Status

ProbeAll probes nodes concurrently and returns statuses keyed by node ID.

type Registry

type Registry struct {
	// contains filtered or unexported fields
}

Registry combines several resolvers into one lookup surface. Resolver order is precedence order: when two sources yield the same node ID, the earlier resolver wins and the later duplicate is dropped.

func NewRegistry

func NewRegistry(resolvers ...Resolver) *Registry

NewRegistry builds a registry over the given resolvers in precedence order.

func (*Registry) Lookup

func (r *Registry) Lookup(id string) (Node, bool)

Lookup resolves a single node by ID, honoring the same precedence as Nodes.

func (*Registry) Nodes

func (r *Registry) Nodes() ([]Node, error)

Nodes returns all known nodes sorted by name (then ID), deduplicated by ID with earlier resolvers winning. A failing resolver does not hide the others' nodes: its error is joined into the returned error while the remaining sources still resolve, so one broken source never blanks the dashboard.

type Resolver

type Resolver interface {
	// Source returns the stable label stamped onto nodes from this resolver.
	Source() string
	// Nodes returns the current nodes for this source.
	Nodes() ([]Node, error)
}

Resolver enumerates nodes from one backing source (static config file, contain workspaces, the local UI-added store, ...). Implementations should be cheap to call: the hub re-resolves on each dashboard/API request so config edits and new workspaces are picked up without a restart.

type StaticResolver

type StaticResolver struct {
	Path string
}

StaticResolver resolves nodes from a config file on every call, so edits are picked up without restarting the hub.

func NewStaticResolver

func NewStaticResolver(path string) *StaticResolver

NewStaticResolver returns a resolver over the given config file path.

func (*StaticResolver) Nodes

func (s *StaticResolver) Nodes() ([]Node, error)

Nodes implements Resolver by re-reading and re-parsing the config file.

func (*StaticResolver) Source

func (s *StaticResolver) Source() string

Source implements Resolver.

type Status

type Status struct {
	// Reachable reports whether the node answered its health endpoint.
	Reachable bool `json:"reachable"`
	// State is "ok" when reachable, otherwise a short failure label
	// ("unreachable", "error <code>").
	State string `json:"state"`
	// LatencyMS is the health round-trip in milliseconds (reachable only).
	LatencyMS int64 `json:"latency_ms"`
	// Version/Agent/Capabilities are best-effort, reported by nodes whose
	// healthz includes them (newer term-llm serves, when the probe carries a
	// valid bearer token).
	Version      string   `json:"version,omitempty"`
	Agent        string   `json:"agent,omitempty"`
	Capabilities []string `json:"capabilities,omitempty"`
	// Details carries small transport-specific facts such as reverse connection
	// timestamps. It is diagnostic only and never carries credentials.
	Details map[string]string `json:"details,omitempty"`
	// Error holds the failure detail for unreachable nodes.
	Error string `json:"error,omitempty"`
}

Status is the probed health of one node, safe to send to hub clients (it never carries the node token).

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store persists UI-added nodes in a local JSON file (0600 — it holds node bearer tokens). It doubles as a Resolver so UI-added nodes resolve through the same Registry as config/contain nodes.

func NewStore

func NewStore(path string) *Store

NewStore returns a store backed by the given JSON file path. The file is created lazily on first Add.

func (*Store) Add

func (s *Store) Add(n Node) (Node, error)

Add normalizes and persists a new node. The node ID must not collide with another stored node.

func (*Store) Nodes

func (s *Store) Nodes() ([]Node, error)

Nodes implements Resolver. A missing file is an empty store, not an error.

func (*Store) Path

func (s *Store) Path() string

Path returns the backing file path.

func (*Store) Remove

func (s *Store) Remove(id string) error

Remove deletes a stored node by ID. Removing an unknown ID is an error so the UI can surface a stale dashboard.

func (*Store) RemoveIfExists added in v0.0.299

func (s *Store) RemoveIfExists(id string) (bool, error)

RemoveIfExists deletes a stored node by ID and reports whether it existed. It is intentionally idempotent for lifecycle hooks that may run more than once.

func (*Store) Source

func (s *Store) Source() string

Source implements Resolver.

func (*Store) Upsert added in v0.0.297

func (s *Store) Upsert(n Node) (Node, bool, error)

Upsert normalizes and creates or replaces a stored node by ID. It only affects the local store; callers that combine the store with higher-priority resolvers should reject shadowed IDs before calling Upsert when that matters.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL