Documentation
¶
Overview ¶
Package plugintest helps test pluginapi plugins without GoModel: a fake pluginapi.Host with scripted inference and recorded metrics, builders for prompts, completions, and exchanges, and a stream driver that feeds events to a pluginapi.StreamHook the way the host does, lookbehind, overlap, and chunk coalescing included.
Index ¶
- func Completion(texts ...string) *pluginapi.Completion
- func Event(kind pluginapi.EventKind) *pluginapi.StreamEvent
- func Exchange(prompt *pluginapi.Prompt, resp *pluginapi.Completion) *pluginapi.Exchange
- func Init(t testing.TB, factory func() pluginapi.Plugin, cfg string, host pluginapi.Host) pluginapi.Plugin
- func Prompt(msgs ...pluginapi.Message) *pluginapi.Prompt
- func Text(role pluginapi.Role, id, text string) pluginapi.Message
- func TextDelta(text string) *pluginapi.StreamEvent
- type Host
- func (h *Host) Complete(_ context.Context, req pluginapi.InferenceRequest) (*pluginapi.Completion, error)
- func (h *Host) HTTPClient() *http.Client
- func (h *Host) History(context.Context, pluginapi.Meta) ([]pluginapi.Message, error)
- func (h *Host) Inference() pluginapi.Inference
- func (h *Host) Logger() *slog.Logger
- func (h *Host) Metrics() pluginapi.Metrics
- func (h *Host) Recorded() Metrics
- func (h *Host) Requests() []pluginapi.InferenceRequest
- type Metrics
- type StreamResult
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Completion ¶
func Completion(texts ...string) *pluginapi.Completion
Completion returns a completion with one assistant text choice per text, each finished with "stop".
func Event ¶
func Event(kind pluginapi.EventKind) *pluginapi.StreamEvent
Event returns an event of the given kind for choice 0.
func Exchange ¶
Exchange returns an exchange for a hook call, with a Values bag, a stream state, and a request ID in Meta. Either argument may be nil.
func Init ¶
func Init(t testing.TB, factory func() pluginapi.Plugin, cfg string, host pluginapi.Host) pluginapi.Plugin
Init builds a plugin from factory and initializes it with cfg (JSON, or empty for no configuration) and host; a nil host selects a fresh Host. It fails the test when Init returns an error.
func TextDelta ¶
func TextDelta(text string) *pluginapi.StreamEvent
TextDelta returns a text delta event for choice 0.
Types ¶
type Host ¶
type Host struct {
// Replies are the inference replies, consumed in order; when they run
// out, Complete returns a completion without choices.
Replies []string
// Finish is the finish reason of every reply; "" means "stop".
Finish string
// Err, when set, is returned by every Complete call.
Err error
// Reply, when set, answers every Complete call itself; Replies, Finish,
// and Err are then ignored.
Reply func(req pluginapi.InferenceRequest) (*pluginapi.Completion, error)
// Client is what HTTPClient returns; nil selects http.DefaultClient.
Client *http.Client
// Log is what Logger returns; nil selects a logger that discards.
Log *slog.Logger
// contains filtered or unexported fields
}
Host is a fake pluginapi.Host. Its zero value is usable: inference answers with an empty completion, metrics are recorded, and HTTP goes through http.DefaultClient.
func (*Host) Complete ¶
func (h *Host) Complete(_ context.Context, req pluginapi.InferenceRequest) (*pluginapi.Completion, error)
Complete implements pluginapi.Inference with the scripted replies.
func (*Host) HTTPClient ¶
HTTPClient implements pluginapi.Host.
func (*Host) Requests ¶
func (h *Host) Requests() []pluginapi.InferenceRequest
Requests returns every inference request made so far.
type Metrics ¶
type Metrics struct {
// Counts holds the sum per counter name.
Counts map[string]int
// Values holds every observation per name, in order.
Values map[string][]float64
// Labels holds the labels of the last call per name.
Labels map[string]map[string]string
}
Metrics is what a plugin recorded through Host.Metrics.
type StreamResult ¶
type StreamResult struct {
// Text is the delivered text per choice, after the hook's edits.
Text map[int]string
// ToolArguments is the delivered tool-call arguments per choice and
// call index, after the hook's edits.
ToolArguments map[int]map[int]string
// Events are the delivered events in order, text and reasoning deltas
// carrying the text as delivered.
Events []*pluginapi.StreamEvent
// Terminated is the decision the hook cut the stream with, or nil.
Terminated *pluginapi.Decision
// End is the OnStreamEnd decision (or, under a buffering policy, the
// OnResponse decision on the assembled completion). Zero when the
// stream was terminated.
End pluginapi.Decision
// Response is the completion the ResponseHook saw under a buffering
// policy, after its edits; nil otherwise.
Response *pluginapi.Completion
}
StreamResult is what a client would have received from a stream driven through a hook.
func RunStream ¶
func RunStream(ctx context.Context, hook pluginapi.StreamHook, x *pluginapi.Exchange, events []*pluginapi.StreamEvent) (*StreamResult, error)
RunStream drives hook with events the way GoModel does under its StreamPolicy: in transform mode the text deltas of a choice, and the argument deltas of each of its tool calls, form windows that are coalesced until MinChunkChars runes are pending; the last LookbehindChars runes of a delivered window are withheld and shown again in front of the next delta with Overlap set, and pass, replace, drop, and terminate are applied to the whole window. Reasoning deltas are presented as they arrive and may be replaced or dropped too. A delta of another kind for the same choice flushes that choice's windows of other kinds, an event that is not held flushes every window, and so does the end of the stream. In observe mode only terminate has an effect.
In buffer mode nothing is presented per event: the deltas are assembled into a completion (text and reasoning parts, "stop" as finish reason) and the plugin's ResponseHook decides. Set x.Response beforehand to hand the hook a completion of your own, with tool calls, usage, or another finish reason.
Only choice, kind, and text matter on the input events; Seq and Overlap are set by the driver.