protocol

package
v0.33.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 20, 2026 License: MIT Imports: 15 Imported by: 0

Documentation

Overview

Package protocol implements the JSON request/response wire format used by `nexus eval --inspect-mode`. External harnesses (AISI Inspect AI, Braintrust, custom shims) drive Nexus headlessly by piping a single JSON request to stdin and parsing a single JSON response from stdout.

The wire format is documented at docs/src/eval/inspect-protocol.md and pinned by the schema-stability snapshot test in this package. PRs that change the wire format MUST update the snapshot deliberately — a downstream shim is pinned to it.

Index

Constants

View Source
const (
	CodeInvalidRequest = "INVALID_REQUEST"
	CodeConfigLoad     = "CONFIG_LOAD"
	CodeEngineBoot     = "ENGINE_BOOT"
	CodeRunFailed      = "RUN_FAILED"
	CodeTimeout        = "TIMEOUT"
	CodeInternal       = "INTERNAL"
)

Error codes surfaced in Response.Error.Code. Documented at docs/src/eval/inspect-protocol.md.

View Source
const SchemaVersion = 1

SchemaVersion is the wire-format version. Bumping this is a deliberate breaking change; the snapshot test refuses silent migration.

Variables

This section is empty.

Functions

func ErrConfigLoad

func ErrConfigLoad(err error) error

ErrConfigLoad constructs a CONFIG_LOAD error wrapping the underlying cause.

func ErrEngineBoot

func ErrEngineBoot(err error) error

ErrEngineBoot constructs an ENGINE_BOOT error wrapping the underlying cause.

func ErrInternal

func ErrInternal(err error) error

ErrInternal constructs an INTERNAL error wrapping the underlying cause.

func ErrInvalidRequest

func ErrInvalidRequest(msg string) error

ErrInvalidRequest constructs an INVALID_REQUEST error.

func ErrRunFailed

func ErrRunFailed(err error) error

ErrRunFailed constructs a RUN_FAILED error wrapping the underlying cause.

func ErrTimeout

func ErrTimeout(msg string) error

ErrTimeout constructs a TIMEOUT error.

func MapError

func MapError(err error) (string, string)

MapError recovers the wire-format code for an error. Falls back to INTERNAL when the error has no recognized code. Context-cancellation errors map to TIMEOUT — the only context-driven error this package produces is the deadline-exceeded path.

func WriteResponse

func WriteResponse(w io.Writer, resp *Response) error

WriteResponse writes resp to w as a single JSON object followed by a newline. The newline keeps the output friendly for shell pipelines and stream-based parsers (e.g. `... | jq .`).

Pretty-printing is intentionally avoided: external harnesses parse machine-to-machine, and a compact encoding is the wire-format choice. Tests that want pretty output can re-marshal locally.

Types

type Error

type Error struct {
	Code    string `json:"code"`
	Message string `json:"message"`
}

Error is the structured failure envelope. Codes are documented in errors.go and the protocol reference.

type Request

type Request struct {
	// Schema is the wire-format version. Must equal SchemaVersion.
	Schema int `json:"schema"`
	// ConfigPath is the absolute or tilde-prefixed path to a YAML config.
	// Mutually exclusive with ConfigInline.
	ConfigPath string `json:"config_path,omitempty"`
	// ConfigInline is a YAML config body inline. Mutually exclusive with
	// ConfigPath.
	ConfigInline string `json:"config_inline,omitempty"`
	// UserInput is the single prompt to drive the agent with. Required.
	UserInput string `json:"user_input"`
	// MaxTurns caps the number of agent.turn.end events the runner will
	// observe before halting. 0 (or unset) means "no protocol-level cap"
	// — the agent's own iteration gate still bounds the run.
	MaxTurns int `json:"max_turns,omitempty"`
	// Metadata is opaque pass-through round-tripped on the response.
	Metadata map[string]any `json:"metadata,omitempty"`

	// ExtraPlugins is the in-process embedder seam, mirroring
	// runner.Options.ExtraPlugins: plugin factories registered onto the
	// request's engine after allplugins.RegisterAll, keyed by plugin ID, so
	// a config whose plugins.active names an embedder-authored plugin can
	// boot. Registering an ID does not activate it — the config must still
	// list it. An ID colliding with a built-in overrides the built-in (the
	// registry's last-Register-wins semantics); that is deliberate, so an
	// embedder can shadow a built-in under eval.
	//
	// It is NOT part of the wire format (json:"-"): factories are Go
	// values, so only callers of protocol.Run in the same process can set
	// it. `nexus eval --inspect-mode` never does.
	ExtraPlugins map[string]engine.PluginFactory `json:"-"`
}

Request is the inspect-mode request envelope. Exactly one of ConfigPath or ConfigInline must be set.

func ParseRequest

func ParseRequest(r io.Reader) (*Request, error)

ParseRequest reads exactly one JSON object from r, with strict "unknown fields rejected" parsing so a typo in the request surfaces as INVALID_REQUEST rather than a silent default.

The caller is expected to terminate the request stream with EOF after the JSON object — a trailing newline is tolerated, additional content is rejected.

func (*Request) Validate

func (r *Request) Validate() error

Validate enforces the request invariants documented on Request. It is invoked by Run; ParseRequest does not call it so that callers can inspect a structurally-valid-but-semantically-wrong request before dispatching.

type Response

type Response struct {
	// Schema mirrors Request.Schema for symmetry.
	Schema int `json:"schema"`
	// SessionID is the new session's UUID. Empty when the engine never
	// reached session bootstrap (early validation errors).
	SessionID string `json:"session_id,omitempty"`
	// FinalAssistantMessage is the text of the final assistant llm.response
	// (FinishReason "end_turn" or equivalent). Empty when the agent never
	// produced one.
	FinalAssistantMessage string `json:"final_assistant_message"`
	// ToolCalls is the ordered list of tool invocations and their summarized
	// results. Order matches dispatch order.
	ToolCalls []ToolCall `json:"tool_calls"`
	// Tokens aggregates input/output usage across every llm.response in the
	// session.
	Tokens Tokens `json:"tokens"`
	// LatencyMs is the wall-clock between the first and last journaled
	// envelope of the session, in milliseconds.
	LatencyMs int64 `json:"latency_ms"`
	// Metadata is the round-tripped Request.Metadata.
	Metadata map[string]any `json:"metadata,omitempty"`
	// Error is populated on failure. nil on success.
	Error *Error `json:"error"`
}

Response is the inspect-mode response envelope. On success, Error is nil. On failure, Error is populated and the process exits non-zero.

func ResponseFromError

func ResponseFromError(err error, metadata map[string]any) *Response

ResponseFromError builds a Response carrying err. Used at the CLI seam for errors raised before Run could itself return a Response.

func Run

func Run(ctx context.Context, req *Request) (*Response, error)

Run executes a single inspect-mode request end-to-end:

  1. Validate the request envelope.
  2. Resolve the config (path or inline) and overlay nexus.io.test + core.sessions.root + core.storage.root so the run is hermetic.
  3. Boot the engine, drive the single user input, wait for the agent to idle (no further turns) OR the request's MaxTurns cap OR the ctx deadline.
  4. Project the live session journal into the wire-format response.
  5. Tear down the engine cleanly.

Run is the embedder seam for the inspect-mode CLI subcommand and for tests. It returns a populated Response on the success path; on failure it returns either (nil, error) when the engine never reached a useful state, or (response, error) when partial data is worth surfacing.

type Tokens

type Tokens struct {
	Input  int `json:"input"`
	Output int `json:"output"`
}

Tokens aggregates per-session token usage.

type ToolCall

type ToolCall struct {
	// Tool is the tool name (e.g. "shell").
	Tool string `json:"tool"`
	// Args is the parsed argument map. Best-effort: when the agent emits
	// an unparseable JSON arguments string, Args is nil and the raw text
	// lives under Args["_raw"] (kept simple and forward-compatible).
	Args map[string]any `json:"args,omitempty"`
	// ResultSummary is a truncated (≤2KB) stringification of the tool's
	// output. Empty when no matching tool.result envelope was observed.
	ResultSummary string `json:"result_summary"`
	// DurationMs is result.Ts − invoke.Ts in milliseconds. 0 when no
	// matching result was observed.
	DurationMs int64 `json:"duration_ms"`
}

ToolCall is one invoke→result pair surfaced in the response.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL