Documentation
¶
Overview ¶
Package hub models a fleet of term-llm web nodes for `term-llm serve hub`.
The core object is a Node: a reachable term-llm serve (web/API endpoint) with an identity, a backend URL + base path, an optional bearer token, and a source describing which resolver produced it. Resolvers (static config, contain workspaces, the local UI-added store) feed a Registry, which is the single lookup surface the hub server routes and proxies from.
Reverse nodes may self-register with the hub when the hub operator enables a registration token; scheduling and mTLS between hub and nodes remain out of scope for v1. See docs-site guide "Hub".
Index ¶
- Constants
- func DelegationStatusTerminal(status string) bool
- func NewDelegationID() (string, error)
- func ParseNodeURL(raw string) (origin, basePath string, err error)
- func SlugID(name string) (string, error)
- func ValidateID(id string) error
- type ContainResolver
- type Delegation
- type DelegationPolicy
- type DelegationStore
- func (s *DelegationStore) ActiveCounts() (total int, byOrigin, byTarget map[string]int, err error)
- func (s *DelegationStore) Add(d Delegation) error
- func (s *DelegationStore) Get(id string) (Delegation, bool, error)
- func (s *DelegationStore) List() ([]Delegation, error)
- func (s *DelegationStore) Path() string
- func (s *DelegationStore) Update(id string, fn func(*Delegation)) (Delegation, error)
- type Node
- func (n Node) AcceptsDelegationAgent(agent string) error
- func (n Node) AcceptsDelegationFrom(originID string) error
- func (n Node) AcceptsDelegationModel(model string) error
- func (n Node) BaseURL() string
- func (n Node) CanDelegateTo(targetID string) bool
- func (n Node) DelegationMaxInFlight() int
- func (n Node) DelegationWorkdir() string
- func (n *Node) Normalize() error
- func (n Node) UsesReverseConnection() bool
- type Prober
- type Registry
- type Resolver
- type StaticResolver
- type Status
- type Store
Constants ¶
const ( DelegationStatusPending = "pending" DelegationStatusRunning = "running" DelegationStatusCancelRequested = "cancel_requested" DelegationStatusSucceeded = "succeeded" DelegationStatusFailed = "failed" DelegationStatusCancelled = "cancelled" DelegationStatusTimedOut = "timed_out" DelegationStatusError = "error" )
Delegation statuses. Pending/running/cancel_requested are active; the rest are terminal. They deliberately mirror jobs-v2 run statuses since a delegation is fronted by one jobs-v2 run on the target node, with "error" added for delegations that broke outside the target run itself.
const ( SourceConfig = "config" SourceContain = "contain" SourceLocal = "local" )
Source labels for the built-in resolvers.
const DefaultDelegationAgent = "developer"
DefaultDelegationAgent is the agent delegated jobs run as when the request names none; with no allowed_agents policy it is also the ONLY agent a target accepts.
const DefaultDelegationMaxDepth = 3
DefaultDelegationMaxDepth bounds delegation chains (A -> B -> C is depth 3) so two mutually-delegating nodes cannot recurse forever.
const DefaultDelegationMaxInFlight = 4
DefaultDelegationMaxInFlight bounds concurrently active delegations per target node when the node's policy does not set max_in_flight.
Variables ¶
This section is empty.
Functions ¶
func DelegationStatusTerminal ¶
DelegationStatusTerminal reports whether a delegation status is final.
func NewDelegationID ¶
NewDelegationID returns a fresh delegation id ("dlg_" + 16 hex chars).
func ParseNodeURL ¶
ParseNodeURL splits a node URL like "http://127.0.0.1:8081/chat" into its origin ("http://127.0.0.1:8081") and normalized base path ("/chat"). A URL without a path yields an empty base path; Normalize rejects that unless an explicit BasePath is supplied, because Hub v1's path proxy needs a prefix.
func SlugID ¶
SlugID derives a valid node ID from a free-form name, for nodes added without an explicit ID. Returns an error when nothing usable remains.
func ValidateID ¶
ValidateID rejects node IDs that are empty or unsafe as a single proxy path segment. The charset deliberately excludes '/', '%', and whitespace so an ID can never smuggle extra path segments into /node/<id>/.
Types ¶
type ContainResolver ¶
type ContainResolver struct {
// Host is the host the workspaces' serves are published on (loopback in
// the one-container-per-agent shape).
Host string
// List enumerates contain workspace definitions.
List func() ([]contain.ListEntry, error)
// Read resolves a workspace's web config from its .env.
Read func(name string) (contain.WebConfig, error)
// Workspace resolves a workspace's in-container workspace path (the
// compose x-term-llm workspace hint). A non-empty result becomes the
// node's delegation workdir, opting the workspace in to accepting
// hub-mediated delegations; an empty result leaves the node unable to
// accept (default deny).
Workspace func(name string) string
}
ContainResolver discovers nodes from local contain workspaces: each workspace with a provisioned web token becomes a node whose URL/token come from its .env (via contain.ReadWebConfig). List and Read are fields so tests can substitute fakes for the container config directory.
func NewContainResolver ¶
func NewContainResolver() *ContainResolver
NewContainResolver returns a resolver over the local contain workspaces.
func (*ContainResolver) Nodes ¶
func (c *ContainResolver) Nodes() ([]Node, error)
Nodes implements Resolver. Workspaces whose .env cannot be read or that have no provisioned web token are skipped (they have no reachable serve to front), so a half-created workspace never breaks hub discovery.
func (*ContainResolver) Source ¶
func (c *ContainResolver) Source() string
Source implements Resolver.
type Delegation ¶
type Delegation struct {
ID string `json:"id"`
OriginNode string `json:"origin_node"`
TargetNode string `json:"target_node"`
AgentName string `json:"agent_name,omitempty"`
// Prompt is stored truncated (see delegationPromptLimit) for audit; the
// full prompt only travels to the target job's instructions.
Prompt string `json:"prompt,omitempty"`
Model string `json:"model,omitempty"`
Cwd string `json:"cwd,omitempty"`
// JobID/RunID identify the jobs-v2 job and run on the TARGET node.
JobID string `json:"job_id,omitempty"`
RunID string `json:"run_id,omitempty"`
Status string `json:"status"`
// Depth is 1 for a direct delegation and parent depth+1 for delegations
// created with parent_delegation_id. Chain lists the node ids involved
// from the root origin through this delegation's target, used to refuse
// delegation loops.
Depth int `json:"depth"`
Chain []string `json:"chain,omitempty"`
ParentID string `json:"parent_delegation_id,omitempty"`
// Response holds the target run's final response (truncated) once the
// delegation reaches a terminal status.
Response string `json:"response,omitempty"`
Error string `json:"error,omitempty"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
Delegation is one hub-mediated cross-node delegation: origin node asked the hub to run a prompt as a jobs-v2 LLM job on the target node. The record deliberately holds NO tokens — node credentials live only in node sources — so the ledger and every API view built from it are safe to serve.
type DelegationPolicy ¶
type DelegationPolicy struct {
Enabled bool `json:"enabled,omitempty" yaml:"enabled"`
To []string `json:"to,omitempty" yaml:"to"`
AcceptFrom []string `json:"accept_from,omitempty" yaml:"accept_from"`
Workdir string `json:"workdir,omitempty" yaml:"workdir"`
MaxInFlight int `json:"max_in_flight,omitempty" yaml:"max_in_flight"`
// AllowedAgents lists agent names origins may request on this node.
// Empty means only DefaultDelegationAgent; "*" allows any plain agent
// name, but path-like names (containing a separator or "..") always
// require an exact entry.
AllowedAgents []string `json:"allowed_agents,omitempty" yaml:"allowed_agents"`
// AllowedModels lists model overrides origins may request. Empty means no
// override is accepted (the target's own default is used); "*" allows any.
AllowedModels []string `json:"allowed_models,omitempty" yaml:"allowed_models"`
}
DelegationPolicy is a node's cross-node delegation policy, configured per node in the hub config:
nodes:
- name: jarvis
url: http://127.0.0.1:8081/chat
token: secret
delegation:
to: ["*"] # node ids this node may delegate to
accept_from: ["*"] # node ids this node accepts delegations from
workdir: /work # REQUIRED to accept; cwd for delegated jobs
max_in_flight: 4 # concurrently active delegations targeting this node
allowed_agents: [] # agents origins may request (default: developer only)
allowed_models: [] # model overrides origins may request (default: none)
Defaults are off: a node must set delegation.enabled to true before it can originate or accept delegated work. Accepting still also requires a workdir.
type DelegationStore ¶
type DelegationStore struct {
// contains filtered or unexported fields
}
DelegationStore persists delegation records in a local JSON ledger. The file is written 0600 like the node store: the records hold no tokens, but prompts and responses are still private session content.
func NewDelegationStore ¶
func NewDelegationStore(path string) *DelegationStore
NewDelegationStore returns a store backed by the given JSON file path. The file is created lazily on first Add.
func (*DelegationStore) ActiveCounts ¶
func (s *DelegationStore) ActiveCounts() (total int, byOrigin, byTarget map[string]int, err error)
ActiveCounts tallies non-terminal delegations hub-wide, per origin node, and per target node, for in-flight cap enforcement.
func (*DelegationStore) Add ¶
func (s *DelegationStore) Add(d Delegation) error
Add persists a new delegation record. Prompt and response excerpts are truncated to keep the ledger bounded.
func (*DelegationStore) Get ¶
func (s *DelegationStore) Get(id string) (Delegation, bool, error)
Get returns a delegation by id.
func (*DelegationStore) List ¶
func (s *DelegationStore) List() ([]Delegation, error)
List returns all records, newest first.
func (*DelegationStore) Path ¶
func (s *DelegationStore) Path() string
Path returns the backing file path.
func (*DelegationStore) Update ¶
func (s *DelegationStore) Update(id string, fn func(*Delegation)) (Delegation, error)
Update applies fn to the stored record and persists the result. The record's UpdatedAt is stamped and the response excerpt re-truncated.
type Node ¶
type Node struct {
// ID uniquely identifies the node within the hub and is used as the
// proxy path segment (/node/<id>/...). Restricted to a path-safe charset.
ID string `json:"id" yaml:"id"`
// Name is the human-facing label shown on the dashboard.
Name string `json:"name" yaml:"name"`
// Source is the resolver that produced this node (config/contain/local).
Source string `json:"source" yaml:"-"`
// Connection is how the Hub reaches this node. Empty/"direct" means the Hub
// dials URL. "reverse" means the node must dial the Hub and keep a websocket
// open; useful for private nodes behind NAT/firewalls.
Connection string `json:"connection,omitempty" yaml:"connection"`
// URL is the backend origin, e.g. "http://127.0.0.1:8081". Never sent to
// hub clients together with Token.
URL string `json:"url" yaml:"url"`
// BasePath is the URL prefix the node's serve is mounted under, e.g.
// "/chat". Always rooted, no trailing slash; root-mounted serves are not
// supported by Hub v1 because path-based proxying needs a stable prefix.
BasePath string `json:"base_path,omitempty" yaml:"base_path"`
// Token is the node's web bearer token. It is injected server-side by the
// hub proxy and MUST never be marshalled into any client-facing response;
// API handlers convert Node to a public view first.
Token string `json:"token,omitempty" yaml:"token"`
// Delegation is the node's cross-node delegation policy. Nil or disabled
// means the node cannot originate or accept delegations.
Delegation *DelegationPolicy `json:"delegation,omitempty" yaml:"delegation"`
}
Node is one reachable term-llm web/API endpoint known to the hub.
func ParseConfig ¶
ParseConfig parses a static hub config document into normalized nodes.
func (Node) AcceptsDelegationAgent ¶
AcceptsDelegationAgent returns nil when this node's policy allows delegated work to run as the given agent. With no allowed_agents configured only DefaultDelegationAgent is accepted; "*" accepts any plain name; path-like names always need an exact entry.
func (Node) AcceptsDelegationFrom ¶
AcceptsDelegationFrom returns nil when this node accepts delegated work from the origin node, or an error describing why not. Delegation is default-off; accepting additionally requires an explicit delegation workdir.
func (Node) AcceptsDelegationModel ¶
AcceptsDelegationModel returns nil when this node's policy allows the given model override. An empty model (use the target's default) is always allowed; with no allowed_models configured every override is refused.
func (Node) BaseURL ¶
BaseURL returns the node's backend origin joined with its base path, without a trailing slash (e.g. "http://127.0.0.1:8081/chat"). Reverse nodes may not have a URL; callers that dial directly must check UsesReverseConnection first.
func (Node) CanDelegateTo ¶
CanDelegateTo reports whether this node's policy allows originating a delegation to the target node. Delegation is default-off; once enabled, an empty "to" list means any target (the target still has to accept).
func (Node) DelegationMaxInFlight ¶
DelegationMaxInFlight returns the per-target concurrency cap for this node.
func (Node) DelegationWorkdir ¶
DelegationWorkdir returns the node's configured delegation workdir ("" when delegation is disabled or the node does not accept delegations).
func (*Node) Normalize ¶
Normalize validates and canonicalizes a node in place: the URL is split into origin + base path (an explicit BasePath wins over a path embedded in URL), the ID falls back to a slug of the name, and the name falls back to the ID.
func (Node) UsesReverseConnection ¶
type Prober ¶
Prober checks node health over HTTP. The client must not use an environment proxy: probes carry node bearer tokens, and routing them through an HTTP_PROXY would leak the tokens (callers pass the same direct-dial transport the hub proxy uses).
func NewProber ¶
func NewProber(transport http.RoundTripper) *Prober
NewProber returns a prober over the given transport with a bounded per-probe timeout so one hung node cannot stall a dashboard refresh.
type Registry ¶
type Registry struct {
// contains filtered or unexported fields
}
Registry combines several resolvers into one lookup surface. Resolver order is precedence order: when two sources yield the same node ID, the earlier resolver wins and the later duplicate is dropped.
func NewRegistry ¶
NewRegistry builds a registry over the given resolvers in precedence order.
func (*Registry) Lookup ¶
Lookup resolves a single node by ID, honoring the same precedence as Nodes.
func (*Registry) Nodes ¶
Nodes returns all known nodes sorted by name (then ID), deduplicated by ID with earlier resolvers winning. A failing resolver does not hide the others' nodes: its error is joined into the returned error while the remaining sources still resolve, so one broken source never blanks the dashboard.
type Resolver ¶
type Resolver interface {
// Source returns the stable label stamped onto nodes from this resolver.
Source() string
// Nodes returns the current nodes for this source.
Nodes() ([]Node, error)
}
Resolver enumerates nodes from one backing source (static config file, contain workspaces, the local UI-added store, ...). Implementations should be cheap to call: the hub re-resolves on each dashboard/API request so config edits and new workspaces are picked up without a restart.
type StaticResolver ¶
type StaticResolver struct {
Path string
}
StaticResolver resolves nodes from a config file on every call, so edits are picked up without restarting the hub.
func NewStaticResolver ¶
func NewStaticResolver(path string) *StaticResolver
NewStaticResolver returns a resolver over the given config file path.
func (*StaticResolver) Nodes ¶
func (s *StaticResolver) Nodes() ([]Node, error)
Nodes implements Resolver by re-reading and re-parsing the config file.
func (*StaticResolver) Source ¶
func (s *StaticResolver) Source() string
Source implements Resolver.
type Status ¶
type Status struct {
// Reachable reports whether the node answered its health endpoint.
Reachable bool `json:"reachable"`
// State is "ok" when reachable, otherwise a short failure label
// ("unreachable", "error <code>").
State string `json:"state"`
// LatencyMS is the health round-trip in milliseconds (reachable only).
LatencyMS int64 `json:"latency_ms"`
// Version/Agent/Capabilities are best-effort, reported by nodes whose
// healthz includes them (newer term-llm serves, when the probe carries a
// valid bearer token).
Version string `json:"version,omitempty"`
Agent string `json:"agent,omitempty"`
Capabilities []string `json:"capabilities,omitempty"`
// Details carries small transport-specific facts such as reverse connection
// timestamps. It is diagnostic only and never carries credentials.
Details map[string]string `json:"details,omitempty"`
// Error holds the failure detail for unreachable nodes.
Error string `json:"error,omitempty"`
}
Status is the probed health of one node, safe to send to hub clients (it never carries the node token).
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store persists UI-added nodes in a local JSON file (0600 — it holds node bearer tokens). It doubles as a Resolver so UI-added nodes resolve through the same Registry as config/contain nodes.
func NewStore ¶
NewStore returns a store backed by the given JSON file path. The file is created lazily on first Add.
func (*Store) Add ¶
Add normalizes and persists a new node. The node ID must not collide with another stored node.
func (*Store) Remove ¶
Remove deletes a stored node by ID. Removing an unknown ID is an error so the UI can surface a stale dashboard.
func (*Store) RemoveIfExists ¶ added in v0.0.299
RemoveIfExists deletes a stored node by ID and reports whether it existed. It is intentionally idempotent for lifecycle hooks that may run more than once.