Documentation
¶
Overview ¶
Package protocol implements the JSON request/response wire format used by `nexus eval --inspect-mode`. External harnesses (AISI Inspect AI, Braintrust, custom shims) drive Nexus headlessly by piping a single JSON request to stdin and parsing a single JSON response from stdout.
The wire format is documented at docs/src/eval/inspect-protocol.md and pinned by the schema-stability snapshot test in this package. PRs that change the wire format MUST update the snapshot deliberately — a downstream shim is pinned to it.
Index ¶
- Constants
- func ErrConfigLoad(err error) error
- func ErrEngineBoot(err error) error
- func ErrInternal(err error) error
- func ErrInvalidRequest(msg string) error
- func ErrRunFailed(err error) error
- func ErrTimeout(msg string) error
- func MapError(err error) (string, string)
- func WriteResponse(w io.Writer, resp *Response) error
- type Error
- type Request
- type Response
- type Tokens
- type ToolCall
Constants ¶
const ( CodeInvalidRequest = "INVALID_REQUEST" CodeConfigLoad = "CONFIG_LOAD" CodeEngineBoot = "ENGINE_BOOT" CodeRunFailed = "RUN_FAILED" CodeTimeout = "TIMEOUT" CodeInternal = "INTERNAL" )
Error codes surfaced in Response.Error.Code. Documented at docs/src/eval/inspect-protocol.md.
const SchemaVersion = 1
SchemaVersion is the wire-format version. Bumping this is a deliberate breaking change; the snapshot test refuses silent migration.
Variables ¶
This section is empty.
Functions ¶
func ErrConfigLoad ¶
ErrConfigLoad constructs a CONFIG_LOAD error wrapping the underlying cause.
func ErrEngineBoot ¶
ErrEngineBoot constructs an ENGINE_BOOT error wrapping the underlying cause.
func ErrInternal ¶
ErrInternal constructs an INTERNAL error wrapping the underlying cause.
func ErrInvalidRequest ¶
ErrInvalidRequest constructs an INVALID_REQUEST error.
func ErrRunFailed ¶
ErrRunFailed constructs a RUN_FAILED error wrapping the underlying cause.
func MapError ¶
MapError recovers the wire-format code for an error. Falls back to INTERNAL when the error has no recognized code. Context-cancellation errors map to TIMEOUT — the only context-driven error this package produces is the deadline-exceeded path.
func WriteResponse ¶
WriteResponse writes resp to w as a single JSON object followed by a newline. The newline keeps the output friendly for shell pipelines and stream-based parsers (e.g. `... | jq .`).
Pretty-printing is intentionally avoided: external harnesses parse machine-to-machine, and a compact encoding is the wire-format choice. Tests that want pretty output can re-marshal locally.
Types ¶
type Error ¶
Error is the structured failure envelope. Codes are documented in errors.go and the protocol reference.
type Request ¶
type Request struct {
// Schema is the wire-format version. Must equal SchemaVersion.
Schema int `json:"schema"`
// ConfigPath is the absolute or tilde-prefixed path to a YAML config.
// Mutually exclusive with ConfigInline.
ConfigPath string `json:"config_path,omitempty"`
// ConfigInline is a YAML config body inline. Mutually exclusive with
// ConfigPath.
ConfigInline string `json:"config_inline,omitempty"`
// UserInput is the single prompt to drive the agent with. Required.
UserInput string `json:"user_input"`
// MaxTurns caps the number of agent.turn.end events the runner will
// observe before halting. 0 (or unset) means "no protocol-level cap"
// — the agent's own iteration gate still bounds the run.
MaxTurns int `json:"max_turns,omitempty"`
// Metadata is opaque pass-through round-tripped on the response.
Metadata map[string]any `json:"metadata,omitempty"`
// ExtraPlugins is the in-process embedder seam, mirroring
// runner.Options.ExtraPlugins: plugin factories registered onto the
// request's engine after allplugins.RegisterAll, keyed by plugin ID, so
// a config whose plugins.active names an embedder-authored plugin can
// boot. Registering an ID does not activate it — the config must still
// list it. An ID colliding with a built-in overrides the built-in (the
// registry's last-Register-wins semantics); that is deliberate, so an
// embedder can shadow a built-in under eval.
//
// It is NOT part of the wire format (json:"-"): factories are Go
// values, so only callers of protocol.Run in the same process can set
// it. `nexus eval --inspect-mode` never does.
ExtraPlugins map[string]engine.PluginFactory `json:"-"`
}
Request is the inspect-mode request envelope. Exactly one of ConfigPath or ConfigInline must be set.
func ParseRequest ¶
ParseRequest reads exactly one JSON object from r, with strict "unknown fields rejected" parsing so a typo in the request surfaces as INVALID_REQUEST rather than a silent default.
The caller is expected to terminate the request stream with EOF after the JSON object — a trailing newline is tolerated, additional content is rejected.
type Response ¶
type Response struct {
// Schema mirrors Request.Schema for symmetry.
Schema int `json:"schema"`
// SessionID is the new session's UUID. Empty when the engine never
// reached session bootstrap (early validation errors).
SessionID string `json:"session_id,omitempty"`
// FinalAssistantMessage is the text of the final assistant llm.response
// (FinishReason "end_turn" or equivalent). Empty when the agent never
// produced one.
FinalAssistantMessage string `json:"final_assistant_message"`
// ToolCalls is the ordered list of tool invocations and their summarized
// results. Order matches dispatch order.
ToolCalls []ToolCall `json:"tool_calls"`
// Tokens aggregates input/output usage across every llm.response in the
// session.
Tokens Tokens `json:"tokens"`
// LatencyMs is the wall-clock between the first and last journaled
// envelope of the session, in milliseconds.
LatencyMs int64 `json:"latency_ms"`
// Metadata is the round-tripped Request.Metadata.
Metadata map[string]any `json:"metadata,omitempty"`
// Error is populated on failure. nil on success.
Error *Error `json:"error"`
}
Response is the inspect-mode response envelope. On success, Error is nil. On failure, Error is populated and the process exits non-zero.
func ResponseFromError ¶
ResponseFromError builds a Response carrying err. Used at the CLI seam for errors raised before Run could itself return a Response.
func Run ¶
Run executes a single inspect-mode request end-to-end:
- Validate the request envelope.
- Resolve the config (path or inline) and overlay nexus.io.test + core.sessions.root + core.storage.root so the run is hermetic.
- Boot the engine, drive the single user input, wait for the agent to idle (no further turns) OR the request's MaxTurns cap OR the ctx deadline.
- Project the live session journal into the wire-format response.
- Tear down the engine cleanly.
Run is the embedder seam for the inspect-mode CLI subcommand and for tests. It returns a populated Response on the success path; on failure it returns either (nil, error) when the engine never reached a useful state, or (response, error) when partial data is worth surfacing.
type ToolCall ¶
type ToolCall struct {
// Tool is the tool name (e.g. "shell").
Tool string `json:"tool"`
// Args is the parsed argument map. Best-effort: when the agent emits
// an unparseable JSON arguments string, Args is nil and the raw text
// lives under Args["_raw"] (kept simple and forward-compatible).
Args map[string]any `json:"args,omitempty"`
// ResultSummary is a truncated (≤2KB) stringification of the tool's
// output. Empty when no matching tool.result envelope was observed.
ResultSummary string `json:"result_summary"`
// DurationMs is result.Ts − invoke.Ts in milliseconds. 0 when no
// matching result was observed.
DurationMs int64 `json:"duration_ms"`
}
ToolCall is one invoke→result pair surfaced in the response.