Documentation
¶
Overview ¶
Package conformance is the §7.1 harness adapter conformance suite. Any adapter runs it from a test:
conformance.Run(t, conformance.Options{New: func() harnesses.Adapter { return fake.New() }, ...})
The suite supplies a stub platform tool API (ToolServer) and a freshly built steadmesh-tools binary, so adapters exercise the same tool path they use in a seat Pod.
Index ¶
- Constants
- Variables
- func BuildTools(t testing.TB) string
- func Env(dirs Dirs, ts *ToolServer, toolCmd string) harnesses.Environment
- func HarnessBin(t testing.TB, name string) string
- func Run(t *testing.T, opts Options)
- func RunModel(t *testing.T, opts ModelOptions)
- type Dirs
- type ModelChain
- type ModelOptions
- type Options
- type ProxyCall
- type ToolCall
- type ToolServer
Constants ¶
const UpstreamKey = "upstream-key"
UpstreamKey is the credential the stub platform proxy injects upstream.
Variables ¶
var Tools = []runtimeapi.ToolDescriptor{ {Name: "self", Description: "Return the seat identity.", InputSchema: json.RawMessage(`{"type":"object","properties":{}}`)}, {Name: "memory.write", Description: "Create a memory record.", InputSchema: json.RawMessage(`{"type":"object","properties":{"store":{"type":"string"},"path":{"type":"string"},"text":{"type":"string"}},"required":["store","path","text"]}`)}, {Name: "messages.reply", Description: "Reply to a message.", InputSchema: json.RawMessage(`{"type":"object","properties":{"message_id":{"type":"string"},"binding":{"type":"string"},"body":{"type":"string"}},"required":["body"]}`)}, {Name: "messages.send", Description: "Send a message to a seat.", InputSchema: json.RawMessage(`{"type":"object","properties":{"to":{"type":"string"},"body":{"type":"string"},"correlation_id":{"type":"string"}},"required":["to","body"]}`)}, }
Tools are the descriptors the stub lists.
Functions ¶
func BuildTools ¶
BuildTools builds cmd/steadmesh-tools once per test binary and returns its path.
func Env ¶
func Env(dirs Dirs, ts *ToolServer, toolCmd string) harnesses.Environment
Env builds an Environment for dirs against the tool server.
func HarnessBin ¶ added in v0.2.0
HarnessBin returns the path of a pinned harness CLI for conformance runs: $STEADMESH_<NAME>_BIN, or bin/harnesses/<name> in the repository (fetched by build/harness-bins.sh). The test is skipped when neither exists.
func RunModel ¶ added in v0.2.0
func RunModel(t *testing.T, opts ModelOptions)
RunModel runs the conformance suite for a model-backed harness against a scripted model over opts.API, through the real model forwarder: the full lifecycle suite, plus a platform tool call made by the model, the readiness probe turn, the model and destination actually used, and seat token rotation during an active session.
Types ¶
type Dirs ¶
type Dirs struct{ Root, Workspace, Home, Runner, Tmp, TokenFile string }
Dirs are the seat volume directories for one test seat.
type ModelChain ¶ added in v0.2.0
type ModelChain struct {
Stub *modelstub.Stub
Platform *httptest.Server
Forwarder *modelforward.Forwarder
// contains filtered or unexported fields
}
ModelChain is a seat's model path in a test, as in a seat Pod: the harness calls the local forwarder (pkg/modelforward), which adds the seat token read from the token file; a stub platform proxy checks the token, swaps it for the upstream credential and passes the request to a scripted model (modelstub).
func NewModelChain ¶ added in v0.2.0
func NewModelChain(t testing.TB, tokenFile, seatToken string, generation int64) *ModelChain
NewModelChain starts a chain to the scripted model; its proxy accepts seatToken.
func NewProxyChain ¶ added in v0.2.0
func NewProxyChain(t testing.TB, tokenFile, seatToken string, generation int64, upstream http.Handler) *ModelChain
NewProxyChain starts a chain whose stub platform proxy checks the seat token and passes requests, with the /v1/model/<connection> prefix removed, to upstream: for example a model connector's Proxy, which injects a real credential.
func (*ModelChain) AcceptToken ¶ added in v0.2.0
func (c *ModelChain) AcceptToken(tok string)
AcceptToken makes the proxy accept another seat token.
func (*ModelChain) Calls ¶ added in v0.2.0
func (c *ModelChain) Calls() []ProxyCall
Calls returns the requests the proxy saw.
func (*ModelChain) Endpoint ¶ added in v0.2.0
func (c *ModelChain) Endpoint(conn, id, api string, settings map[string]string) *harnesses.ModelEndpoint
Endpoint is the harness model environment for this chain.
func (*ModelChain) SetExecution ¶ added in v0.2.0
func (c *ModelChain) SetExecution(id string)
SetExecution sets the execution header for following requests, as the seat runner does per delivery.
type ModelOptions ¶ added in v0.2.0
type ModelOptions struct {
// New returns a fresh, unprepared adapter.
New func() harnesses.Adapter
// API is the model API the harness is configured to speak.
API string
// Model is the model ID; Settings are model settings.
Model string
Settings map[string]string
// Configure may adjust the environment further (binary paths, config).
Configure func(t *testing.T, env *harnesses.Environment)
// TurnTimeout bounds a turn (default 2m).
TurnTimeout time.Duration
// CheckToolCall, when set, inspects the model requests of the turn in
// which the model called a platform tool.
CheckToolCall func(t *testing.T, reqs []modelstub.Request)
}
ModelOptions configures RunModel.
type Options ¶
type Options struct {
// New returns a fresh, unprepared adapter.
New func() harnesses.Adapter
// QuickBody is a message body the adapter completes promptly.
QuickBody string
// SlowBody is a message body that keeps the turn running for at least
// several seconds, used to test interruption and quiesce.
SlowBody string
// Configure may adjust the environment (e.g. model proxy URL).
Configure func(t *testing.T, env *harnesses.Environment)
// TurnTimeout bounds a quick turn (default 60s).
TurnTimeout time.Duration
// InterruptBound is how long an interrupted turn may take to unwind (default 20s).
InterruptBound time.Duration
}
Options configures a conformance run.
type ToolCall ¶
type ToolCall struct {
Name string
Arguments json.RawMessage
Generation string
Execution string
}
ToolCall is one recorded call to the stub platform.
type ToolServer ¶
type ToolServer struct {
*httptest.Server
Token string
// Handler optionally overrides a tool's result.
Handler func(name string, args json.RawMessage) runtimeapi.ToolCallResult
// contains filtered or unexported fields
}
ToolServer is a stub of the seat-facing tool API (/v1/tools).
func NewToolServer ¶
func NewToolServer(t testing.TB, token string) *ToolServer
NewToolServer starts a stub tool API that requires token.
func (*ToolServer) Calls ¶
func (ts *ToolServer) Calls() []ToolCall
Calls returns the recorded calls.
func (*ToolServer) SetToken ¶ added in v0.2.0
func (ts *ToolServer) SetToken(tok string)
SetToken changes the accepted seat token, as when it is rotated.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package modelstub is a scripted model endpoint speaking the Anthropic Messages, OpenAI Responses and OpenAI Chat Completions APIs, streamed and not.
|
Package modelstub is a scripted model endpoint speaking the Anthropic Messages, OpenAI Responses and OpenAI Chat Completions APIs, streamed and not. |