livetest

package
v1.12.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 17, 2026 License: Apache-2.0 Imports: 12 Imported by: 0

Documentation

Overview

Package livetest drives an OpenAI-compatible plugin through the core Genkit generate features against the provider's real API: generation, history, system prompts, streaming, tool calling, structured output, reasoning, vision, and the extra config passthrough.

Each plugin's live test builds a Suite naming the models to spend and the capabilities the provider claims, then calls Run after its API key check. The checklist is identical across providers, so one green run per plugin is the release sanity check; provider-specific behavior stays in the plugin's own subtests.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Run

func Run(t *testing.T, g *genkit.Genkit, s Suite)

Run walks the plugin registered on g through the shared checklist. It defines a tool named "gablorken" on g, so call it once per Genkit instance and keep that name free in the plugin's own subtests.

Types

type Suite

type Suite struct {
	// Model answers the cheap text prompts. Every check that needs no
	// special capability runs against it. Required.
	Model ai.ModelArg
	// ReasoningModel emits thinking, usually a [ai.ModelRef] carrying the
	// plugin's thinking config. Nil skips the reasoning checks.
	ReasoningModel ai.ModelArg
	// ReasoningContent reports whether the provider returns the thinking
	// content itself. Endpoints that accept the knob but keep the content
	// server-side leave this false and still get the no-error checks.
	ReasoningContent bool
	// StreamOnlyReasoning skips the non-streaming reasoning check, for
	// providers that only think on streaming calls.
	StreamOnlyReasoning bool
	// VisionModel accepts inline images. Nil skips the vision check.
	VisionModel ai.ModelArg
	// ToolChoice runs the tool-choice checks, for providers whose catalog
	// claims the capability.
	ToolChoice bool
	// ExtraConfig, when set, is the whole request config for the
	// passthrough check and must route at least one field through the
	// config's extra map.
	ExtraConfig map[string]any
}

Suite describes how to drive one provider through the shared checklist.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL