goai

package module
v0.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 20, 2026 License: MIT Imports: 21 Imported by: 2

README

GoAI

GoAI

AI SDK, the Go way.

Go SDK for building AI applications. One SDK, 25+ providers, MCP support.

Streaming Cold Start Memory

1.1x faster streaming, 24x faster cold start, 3.1x less memory vs Vercel AI SDK (benchmarks)

Website · Docs · Architecture · Providers · Examples · Orchestration (zenflow)

Inspired by the Vercel AI SDK. The same clean abstractions, idiomatically adapted for Go with generics, interfaces, and functional options.

What's New

v0.8.5 - New provider: Requesty (OpenAI-compatible LLM gateway). Plus Anthropic native structured output, per-step Reasoning, and Gemini schema sanitization fixes. Changelog →

v0.8.4 - WithRetryObserver callback for observing retry attempts, split tool-call stream metadata fix, and Anthropic redacted_thinking capture in streaming. Changelog →

v0.8.3 - Native Ollama provider (/api/chat + /api/embed, think mode, no third-party SDK) and duplicate tool name detection. Changelog →

v0.8.0 - MCP OAuth 2.1 + PKCE for remote servers, NewTool[In] typed constructor, and lifecycle hook panics surfaced as *PanicError via WithOnPanic (breaking). Changelog →

Features

  • 7 core functions: GenerateText, StreamText, GenerateObject[T], StreamObject[T], Embed, EmbedMany, GenerateImage
  • 25+ providers: OpenAI, Anthropic, Google, Bedrock, Azure, Vertex, Mistral, xAI, Groq, Cohere, DeepSeek, MiniMax, Fireworks, Together, DeepInfra, OpenRouter, Requesty, Perplexity, Cerebras, Ollama, vLLM, RunPod, Cloudflare Workers AI, FPT Smart Cloud, NVIDIA NIM, llama.cpp, + generic OpenAI-compatible
  • Auto tool loop: Define tools with Execute handlers, set MaxSteps for GenerateText and StreamText
  • Structured output: GenerateObject[T] auto-generates JSON Schema from Go types via reflection
  • Streaming: Real-time text and partial object streaming via channels
  • Dynamic auth: TokenSource interface for OAuth, rotating keys, cloud IAM, with CachedTokenSource for TTL-based caching
  • Prompt caching: Automatic cache control for supported providers (Anthropic, Bedrock)
  • Citations/sources: Grounding and inline citations from xAI, Perplexity, Google, OpenAI
  • Web search: Built-in web search tools for OpenAI, Anthropic, Google, Groq, xAI. Model decides when to search
  • Code execution: Server-side Python sandboxes via OpenAI, Anthropic, Google, xAI. No local setup
  • Computer use: Anthropic computer, bash, text editor tools for autonomous desktop interaction
  • 20 provider-defined tools: Web fetch, file search, image generation, X search, and more - full list
  • MCP client: Connect to any MCP server (stdio, HTTP, SSE), auto-convert tools for use with GoAI
  • Observability: Built-in Langfuse and OpenTelemetry (OTel) integrations for tracing generations, tools, and multi-step loops
  • Multi-agent orchestration: For declarative YAML workflows on top of GoAI, see zenflow
  • 9 lifecycle hooks: Observability (OnRequest, OnResponse, OnToolCallStart, OnToolCall, OnStepFinish, OnFinish) and interceptor (OnBeforeToolExecute, OnAfterToolExecute, OnBeforeStep) hooks for permission gates, secret scanning, output transformation, and loop control
  • Retry/backoff: Automatic retry with exponential backoff on retryable HTTP errors (429/5xx)
  • Minimal dependencies: Core depends on golang.org/x/oauth2 + one indirect (cloud.google.com/go/compute/metadata). Optional observability/otel submodule uses separate go.mod with OTel SDK.

Performance vs Vercel AI SDK

Metric GoAI Vercel AI SDK Improvement
Streaming throughput 1.46ms 1.62ms 1.1x faster
Cold start 569us 13.9ms 24x faster
Memory (1 stream) 220KB 676KB 3.1x less
GenerateText 56us 79us 1.4x faster

Mock HTTP server, identical SSE fixtures, Apple M2. Full report

Install

go get github.com/zendev-sh/goai@latest

Requires Go 1.25+.

Quick Start

Most hosted providers auto-resolve API keys from environment variables. Local/custom providers may require explicit options:

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/zendev-sh/goai"
	"github.com/zendev-sh/goai/provider/openai"
)

func main() {
	// Reads OPENAI_API_KEY from environment automatically.
	model := openai.Chat("gpt-4o")

	result, err := goai.GenerateText(context.Background(), model,
		goai.WithPrompt("What is the capital of France?"),
	)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(result.Text)
}

Streaming

ctx := context.Background()

stream, err := goai.StreamText(ctx, model,
	goai.WithSystem("You are a helpful assistant."),
	goai.WithPrompt("Write a haiku about Go."),
)
if err != nil {
	log.Fatal(err)
}

for text := range stream.TextStream() {
	fmt.Print(text)
}

result := stream.Result()
if err := stream.Err(); err != nil {
	log.Fatal(err)
}
fmt.Printf("\nTokens: %d in, %d out\n",
	result.TotalUsage.InputTokens, result.TotalUsage.OutputTokens)

Streaming with tools:

import "github.com/zendev-sh/goai/provider"

stream, err := goai.StreamText(ctx, model,
	goai.WithPrompt("What's the weather in Tokyo?"),
	goai.WithTools(weatherTool),
	goai.WithMaxSteps(5),
)
for chunk := range stream.Stream() {
	switch chunk.Type {
	case provider.ChunkText:
		fmt.Print(chunk.Text)
	case provider.ChunkStepFinish:
		fmt.Println("\n[step complete]")
	}
}

Structured Output

Auto-generates JSON Schema from Go types. Works with OpenAI, Anthropic, and Google.

type Recipe struct {
	Name        string   `json:"name" jsonschema:"description=Recipe name"`
	Ingredients []string `json:"ingredients"`
	Steps       []string `json:"steps"`
	Difficulty  string   `json:"difficulty" jsonschema:"enum=easy|medium|hard"`
}

result, err := goai.GenerateObject[Recipe](ctx, model,
	goai.WithPrompt("Give me a recipe for chocolate chip cookies"),
)
if err != nil {
	log.Fatal(err)
}
fmt.Printf("Recipe: %s (%s)\n", result.Object.Name, result.Object.Difficulty)

Streaming partial objects:

stream, err := goai.StreamObject[Recipe](ctx, model,
	goai.WithPrompt("Give me a recipe for pancakes"),
)
if err != nil {
	log.Fatal(err)
}
for partial := range stream.PartialObjectStream() {
	fmt.Printf("\r%s (%d ingredients so far)", partial.Name, len(partial.Ingredients))
}
final, err := stream.Result()

Tools

Define a tool from a typed input struct with goai.NewTool - the JSON Schema is generated from the struct and arguments are unmarshaled for you. Set MaxSteps to enable the auto tool loop.

weatherTool := goai.NewTool("get_weather", "Get the current weather for a city.",
	func(ctx context.Context, args struct {
		City string `json:"city" jsonschema:"description=City name"`
	}) (string, error) {
		return fmt.Sprintf("22°C and sunny in %s", args.City), nil
	})

result, err := goai.GenerateText(ctx, model,
	goai.WithPrompt("What's the weather in Tokyo?"),
	goai.WithTools(weatherTool),
	goai.WithMaxSteps(3),
)
if err != nil {
	log.Fatal(err)
}
fmt.Println(result.Text) // "It's 22°C and sunny in Tokyo."

MCP (Model Context Protocol)

Connect to any MCP server and use its tools with GoAI. Supports stdio, Streamable HTTP, and legacy SSE transports.

import "github.com/zendev-sh/goai/mcp"

// Connect to any MCP server
transport := mcp.NewStdioTransport("npx", []string{"-y", "@modelcontextprotocol/server-filesystem", "."})
client := mcp.NewClient("my-app", "1.0", mcp.WithTransport(transport))
_ = client.Connect(ctx)
defer client.Close()

// Use MCP tools with GoAI
tools, _ := client.ListTools(ctx, nil)
goaiTools := mcp.ConvertTools(client, tools.Tools)

result, _ := goai.GenerateText(ctx, model,
    goai.WithTools(goaiTools...),
    goai.WithPrompt("List files in the current directory"),
    goai.WithMaxSteps(5),
)

For OAuth-protected remote servers, GoAI handles authorization-server discovery, dynamic client registration, PKCE, token refresh, and retry-on-401 natively — no external wrapper needed:

ts, _ := mcp.NewOAuthTokenSource(ctx, mcp.OAuthConfig{
    ServerURL:   serverURL,
    RedirectURI: "http://127.0.0.1:8765/callback",
    Authorize:   openBrowserAndAwaitRedirect, // returns the OAuth redirect URL
})
transport := mcp.NewHTTPTransport(serverURL, mcp.WithHTTPClient(mcp.NewOAuthHTTPClient(ts, nil)))
client := mcp.NewClient("my-app", "1.0", mcp.WithTransport(transport))

See examples/mcp-tools, examples/mcp-oauth, and the MCP documentation for more.

Companion Projects

zenflow - multi-agent orchestration engine for Go. Declarative YAML workflows, typed inter-agent mailboxes, single static binary. Built on GoAI for the LLM layer; reach for it when one MaxSteps loop is no longer enough.

Citations / Sources

Providers that support grounding (Google, xAI, Perplexity) or inline citations (OpenAI) return sources:

result, err := goai.GenerateText(ctx, model,
	goai.WithPrompt("What were the major news events today?"),
)
if err != nil {
	log.Fatal(err)
}

if len(result.Sources) > 0 {
	for _, s := range result.Sources {
		fmt.Printf("[%s] %s - %s\n", s.Type, s.Title, s.URL)
	}
}

// Sources are also available per-step in multi-step tool loops.
for _, step := range result.Steps {
	for _, s := range step.Sources {
		fmt.Printf("  Step source: %s\n", s.URL)
	}
}

Computer Use

See Provider-Defined Tools > Computer Use and examples/computer-use for Anthropic computer, bash, and text editor tools. Works with both Anthropic direct API and Bedrock.

Embeddings

ctx := context.Background()
model := openai.Embedding("text-embedding-3-small")

// Single
result, err := goai.Embed(ctx, model, "Hello world")
if err != nil {
	log.Fatal(err)
}
fmt.Printf("Dimensions: %d\n", len(result.Embedding))

// Batch (auto-chunked, parallel)
many, err := goai.EmbedMany(ctx, model, []string{"foo", "bar", "baz"},
	goai.WithMaxParallelCalls(4),
)
if err != nil {
	log.Fatal(err)
}

Image Generation

ctx := context.Background()
model := openai.Image("gpt-image-1")

result, err := goai.GenerateImage(ctx, model,
	goai.WithImagePrompt("A sunset over mountains, oil painting style"),
	goai.WithImageSize("1024x1024"),
)
if err != nil {
	log.Fatal(err)
}
os.WriteFile("sunset.png", result.Images[0].Data, 0644)

Also supported: Google Imagen (google.Image("imagen-4.0-generate-001")) and Vertex AI (vertex.Image(...)).

Observability

Built-in Langfuse and OpenTelemetry integrations. Nine lifecycle hooks cover the full generation pipeline -- observability providers use them to trace LLM calls, tool executions, and multi-step agent loops:

import "github.com/zendev-sh/goai/observability/langfuse"

// Credentials from env: LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_HOST
result, err := goai.GenerateText(ctx, model,
    goai.WithPrompt("Hello"),
    goai.WithTools(weatherTool),
    goai.WithMaxSteps(5),
    langfuse.WithTracing(langfuse.TraceName("my-agent")),
)

Interceptor hooks let you control tool execution without modifying core code:

// Permission gate: block dangerous tools
goai.WithOnBeforeToolExecute(func(info goai.BeforeToolExecuteInfo) goai.BeforeToolExecuteResult {
    if info.ToolName == "delete_file" {
        return goai.BeforeToolExecuteResult{Skip: true, Result: "Permission denied."}
    }
    return goai.BeforeToolExecuteResult{}
}),

// Detect max-steps exhaustion
goai.WithOnFinish(func(info goai.FinishInfo) {
    if info.StepsExhausted {
        log.Printf("Loop exhausted after %d steps", info.TotalSteps)
    }
}),

See examples/hooks, examples/langfuse, examples/otel, and the observability docs for details.

Providers

Many providers auto-resolve credentials from environment variables. Others (for example ollama, vllm, compat) use explicit options:

// Auto-resolved: reads OPENAI_API_KEY from env
model := openai.Chat("gpt-4o")

// Explicit key (overrides env)
model := openai.Chat("gpt-4o", openai.WithAPIKey("sk-..."))

// Cloud IAM auth (Vertex, Bedrock)
model := vertex.Chat("gemini-2.5-pro",
	vertex.WithProject("my-project"),
	vertex.WithLocation("us-central1"),
)

// AWS Bedrock (reads AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION from env)
model := bedrock.Chat("anthropic.claude-sonnet-4-6-v1:0")

// Local (Ollama, vLLM)
model := ollama.Chat("llama3", ollama.WithBaseURL("http://localhost:11434/v1"))

result, err := goai.GenerateText(ctx, model, goai.WithPrompt("Hello"))

Provider Table

Provider Chat Embed Image Auth E2E Import
OpenAI gpt-4o, o3, codex-* text-embedding-3-* gpt-image-1 OPENAI_API_KEY, OPENAI_BASE_URL, TokenSource Full provider/openai
Anthropic claude-* - - ANTHROPIC_API_KEY, ANTHROPIC_BASE_URL, TokenSource Full provider/anthropic
Google gemini-* text-embedding-004 imagen-* GOOGLE_GENERATIVE_AI_API_KEY / GEMINI_API_KEY, TokenSource Full provider/google
Bedrock anthropic.*, meta.* titan-embed-*, cohere.embed-*, nova-2-*, marengo-* - AWS keys, AWS_BEARER_TOKEN_BEDROCK, AWS_BEDROCK_BASE_URL Full provider/bedrock
Vertex gemini-* text-embedding-004 imagen-* TokenSource, ADC, or GOOGLE_API_KEY / GEMINI_API_KEY / GOOGLE_GENERATIVE_AI_API_KEY fallback Unit provider/vertex
Azure gpt-4o, claude-* - via Azure AZURE_OPENAI_API_KEY, TokenSource Full provider/azure
OpenRouter various - - OPENROUTER_API_KEY, TokenSource Unit provider/openrouter
Mistral mistral-large, magistral-* - - MISTRAL_API_KEY, TokenSource Full provider/mistral
Groq mixtral-*, llama-* - - GROQ_API_KEY, TokenSource Full provider/groq
xAI grok-* - - XAI_API_KEY, TokenSource Unit provider/xai
Cohere command-r-* embed-* - COHERE_API_KEY, TokenSource Unit provider/cohere
DeepSeek deepseek-* - - DEEPSEEK_API_KEY, TokenSource Unit provider/deepseek
MiniMax MiniMax-M2.7, MiniMax-M2.5, MiniMax-M2.1, MiniMax-M2 - - MINIMAX_API_KEY, MINIMAX_BASE_URL, TokenSource Full provider/minimax
Fireworks various - - FIREWORKS_API_KEY, TokenSource Unit provider/fireworks
Together various - - TOGETHER_AI_API_KEY (or TOGETHER_API_KEY), TokenSource Unit provider/together
DeepInfra various - - DEEPINFRA_API_KEY, TokenSource Unit provider/deepinfra
Perplexity sonar-* - - PERPLEXITY_API_KEY, TokenSource Unit provider/perplexity
Cerebras llama-* - - CEREBRAS_API_KEY, TokenSource Unit provider/cerebras
Ollama local models local models - none Unit provider/ollama
vLLM local models local models - Optional auth via WithAPIKey / WithTokenSource Unit provider/vllm
RunPod any vLLM model - - RUNPOD_API_KEY, TokenSource Unit provider/runpod
Cloudflare @cf/meta/*, @cf/openai/gpt-oss-*, @cf/qwen/* @cf/baai/bge-* - CLOUDFLARE_API_TOKEN, CLOUDFLARE_ACCOUNT_ID, CLOUDFLARE_BASE_URL, TokenSource Unit provider/cloudflare
FPT Cloud Qwen3-*, Llama-*, gpt-oss-*, GLM-*, gemma-* bge-*, gte-*, multilingual-e5-* - FPT_API_KEY, FPT_REGION (global/jp), FPT_BASE_URL, TokenSource Unit provider/fptcloud
NVIDIA NIM nvidia/llama-*, nvidia/nemotron-* nvidia/nv-embed-* - NVIDIA_API_KEY, NVIDIA_BASE_URL, TokenSource Full provider/nvidia
llama.cpp local models local models - Optional auth via WithAPIKey / WithTokenSource Unit provider/llamacpp
Requesty provider/model (e.g. openai/gpt-4o-mini) - - REQUESTY_API_KEY, REQUESTY_BASE_URL, TokenSource Unit provider/requesty
Compat any OpenAI-compatible any - configurable Unit provider/compat

E2E column: "Full" = tested with real API calls. "Unit" = tested with mock HTTP servers (100% coverage).

Tested Models

E2E tested - 104 models across 8 providers (real API calls, click to expand)

Last run: 2026-03-27. 104 models tested (generate + stream).

Provider Models E2E tested (generate + stream)
Google (9) gemini-2.5-flash, gemini-2.5-flash-lite, gemini-2.5-pro (stream), gemini-3-flash-preview, gemini-3-pro-preview, gemini-3.1-pro-preview, gemini-2.0-flash, gemini-flash-latest, gemini-flash-lite-latest
Azure (21) claude-opus-4-6, claude-sonnet-4-6, DeepSeek-V3.2, gpt-4.1, gpt-4.1-mini, gpt-5, gpt-5-codex, gpt-5-mini, gpt-5-pro, gpt-5.1, gpt-5.1-codex, gpt-5.1-codex-max, gpt-5.1-codex-mini, gpt-5.2, gpt-5.2-codex, gpt-5.3-codex, gpt-5.4, gpt-5.4-pro, Kimi-K2.5, model-router, o3
Bedrock (61) Anthropic: claude-sonnet-4-6, claude-sonnet-4-5, claude-sonnet-4, claude-opus-4-6-v1, claude-opus-4-5, claude-opus-4-1, claude-haiku-4-5, claude-3-5-sonnet, claude-3-5-haiku, claude-3-haiku · Amazon: nova-micro, nova-lite, nova-pro, nova-premier, nova-2-lite · Meta: llama4-scout, llama4-maverick, llama3-3-70b, llama3-2-{90,11,3,1}b, llama3-1-{70,8}b, llama3-{70,8}b · Mistral: mistral-large, mixtral-8x7b, mistral-7b, ministral-3-{14,8}b, voxtral-{mini,small} · Others: deepseek.v3, deepseek.r1, ai21.jamba-1-5-{mini,large}, cohere.command-r{-plus,}, google.gemma-3-{4,12,27}b, minimax.{m2,m2.1}, moonshotai.kimi-k2{-thinking,.5}, nvidia.nemotron-nano-{12,9}b, openai.gpt-oss-{120,20}b{,-safeguard}, qwen.qwen3-{32,235,coder-30,coder-480}b, qwen.qwen3-next-80b, writer.palmyra-{x4,x5}, zai.glm-4.7{,-flash}
Groq (2) llama-3.1-8b-instant, llama-3.3-70b-versatile
Mistral (5) mistral-small-latest, mistral-large-latest, devstral-small-2507, codestral-latest, magistral-medium-latest
Cerebras (1) llama3.1-8b
MiniMax (4) MiniMax-M2.7, MiniMax-M2.5, MiniMax-M2.1, MiniMax-M2 (generate + stream + tools + thinking)
NVIDIA (1) meta/llama-3.3-70b-instruct
Unit tested (mock HTTP server, 100% coverage, click to expand)
Provider Models in unit tests
OpenAI gpt-4o, o3, text-embedding-3-small, dall-e-3, gpt-image-1
Anthropic claude-sonnet-4-20250514, claude-sonnet-4-5-20241022, claude-sonnet-4-6-20260310
Google gemini-2.5-flash, gemini-2.5-flash-image, imagen-4.0-fast-generate-001, text-embedding-004
Bedrock us.anthropic.claude-sonnet-4-6, anthropic.claude-sonnet-4-20250514-v1:0, meta.llama3-70b
Azure gpt-4o, gpt-5.2-chat, dall-e-3, claude-sonnet-4-6
Vertex gemini-2.5-pro, imagen-3.0-generate-002, text-embedding-004
Cohere command-r-plus, command-a-reasoning, embed-v4.0
Mistral mistral-large-latest
Groq llama-3.3-70b-versatile
xAI grok-3
DeepSeek deepseek-chat
DeepInfra meta-llama/Llama-3.3-70B-Instruct
Fireworks accounts/fireworks/models/llama-v3p3-70b-instruct
OpenRouter anthropic/claude-sonnet-4
Perplexity sonar-pro
Together meta-llama/Llama-3.3-70B-Instruct-Turbo
Cerebras llama-3.3-70b
NVIDIA meta/llama-3.3-70b-instruct
Ollama llama3, llama3.2:1b, nomic-embed-text
vLLM meta-llama/Llama-3-8b
RunPod meta-llama/Llama-3.3-70B-Instruct
Cloudflare @cf/meta/llama-3.1-8b-instruct, @cf/baai/bge-base-en-v1.5
FPT Cloud Qwen3-32B

Custom / Self-Hosted

Use the compat provider for any OpenAI-compatible endpoint:

model := compat.Chat("my-model",
	compat.WithBaseURL("https://my-api.example.com/v1"),
	compat.WithAPIKey("..."),
)

Dynamic Auth with TokenSource

For OAuth, rotating keys, or cloud IAM:

ts := provider.CachedTokenSource(func(ctx context.Context) (*provider.Token, error) {
	tok, err := fetchOAuthToken(ctx)
	return &provider.Token{
		Value:     tok.AccessToken,
		ExpiresAt: tok.Expiry,
	}, err
})

model := openai.Chat("gpt-4o", openai.WithTokenSource(ts))

CachedTokenSource handles TTL-based caching (zero ExpiresAt = cache forever), thread-safe refresh without holding locks during network calls, and manual token invalidation via the InvalidatingTokenSource interface.

AWS Bedrock

Native Converse API with SigV4 signing (no AWS SDK dependency). Supports cross-region inference fallback, extended thinking, and image/document input:

model := bedrock.Chat("anthropic.claude-sonnet-4-6-v1:0",
	bedrock.WithRegion("us-west-2"),
	bedrock.WithReasoningConfig(bedrock.ReasoningConfig{
		Type:         bedrock.ReasoningEnabled,
		BudgetTokens: 4096,
	}),
)

Auto-resolves AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION from environment. Cross-region fallback retries with us. prefix on model ID mismatch errors.

Azure OpenAI

Supports both OpenAI models (GPT, o-series) and Claude models (routed to Azure Anthropic endpoint automatically):

// OpenAI models
model := azure.Chat("gpt-4o",
	azure.WithEndpoint("https://my-resource.openai.azure.com"),
)

// Claude models (auto-routed to Anthropic endpoint)
model := azure.Chat("claude-sonnet-4-6",
	azure.WithEndpoint("https://my-resource.openai.azure.com"),
)

Auto-resolves AZURE_OPENAI_API_KEY, AZURE_OPENAI_ENDPOINT (or AZURE_RESOURCE_NAME) from environment.

Response Metadata

Every result includes provider response metadata:

result, _ := goai.GenerateText(ctx, model, goai.WithPrompt("Hello"))
fmt.Printf("Request ID: %s\n", result.Response.ID)
fmt.Printf("Model used: %s\n", result.Response.Model)

Options Reference

Generation Options

Option Description Default
WithSystem(s) System prompt -
WithPrompt(s) Single user message -
WithMessages(...) Conversation history -
WithTools(...) Available tools -
WithMaxOutputTokens(n) Response length limit provider default
WithTemperature(t) Randomness (0.0-2.0) provider default
WithTopP(p) Nucleus sampling provider default
WithTopK(k) Top-K sampling provider default
WithFrequencyPenalty(p) Frequency penalty provider default
WithPresencePenalty(p) Presence penalty provider default
WithSeed(s) Deterministic generation -
WithStopSequences(...) Stop triggers -
WithMaxSteps(n) Tool loop iterations 1 (no loop)
WithSequentialToolExecution() Execute tools one at a time parallel
WithMaxRetries(n) Retries on 429/5xx 2
WithTimeout(d) Overall timeout none
WithHeaders(h) Per-request HTTP headers -
WithProviderOptions(m) Provider-specific params -
WithPromptCaching(b) Enable prompt caching false
WithToolChoice(tc) "auto", "none", "required", or tool name -

Lifecycle Hooks

Option Description
WithOnRequest(fn) Called before each API call
WithOnResponse(fn) Called after each API call
WithOnToolCallStart(fn) Called before each tool execution begins
WithOnToolCall(fn) Called after each tool execution
WithOnStepFinish(fn) Called after each tool loop step
WithOnFinish(fn) Called once after all steps complete (carries StepsExhausted)
WithOnBeforeToolExecute(fn) Intercept before tool Execute -- can skip, override ctx/input
WithOnAfterToolExecute(fn) Intercept after tool Execute -- can modify output/error
WithOnBeforeStep(fn) Intercept before step 2+ -- can inject messages or stop loop

Structured Output Options

Option Description
WithExplicitSchema(s) Override auto-generated JSON Schema
WithSchemaName(n) Schema name for provider (default "response")

Embedding Options

Option Description Default
WithMaxParallelCalls(n) Batch parallelism 4
WithEmbeddingProviderOptions(m) Embedding provider params -

Image Options

Option Description
WithImagePrompt(s) Text description
WithImageCount(n) Number of images
WithImageSize(s) Dimensions (e.g., "1024x1024")
WithAspectRatio(s) Aspect ratio (e.g., "16:9")
WithImageMaxRetries(n) Retries on 429/5xx
WithImageTimeout(d) Overall timeout
WithImageProviderOptions(m) Image provider params

Error Handling

GoAI generation and image APIs return typed errors for actionable failure modes (MCP client APIs return *mcp.MCPError):

result, err := goai.GenerateText(ctx, model, goai.WithPrompt("..."))
if err != nil {
	var overflow *goai.ContextOverflowError
	var apiErr *goai.APIError
	switch {
	case errors.As(err, &overflow):
		// Prompt too long - truncate and retry
	case errors.As(err, &apiErr):
		if apiErr.IsRetryable {
			// 429 rate limit, 503 - already retried MaxRetries times
		}
		fmt.Printf("API error %d: %s\n", apiErr.StatusCode, apiErr.Message)
		// HTTP API errors include ResponseBody and ResponseHeaders for debugging
	default:
		// Network error, context cancelled, etc.
	}
}

Error types:

Type Fields When
APIError StatusCode, Message, IsRetryable, ResponseBody, ResponseHeaders Non-2xx API responses
ContextOverflowError Message, ResponseBody Prompt exceeds model context window

Retry behavior: automatic exponential backoff on retryable HTTP errors (429/5xx, plus OpenAI 404 propagation). retry-after-ms and numeric Retry-After (seconds) are respected. Retries apply to request-level failures (including initial stream connection), not mid-stream error events.

Provider-Defined Tools

Providers expose built-in tools that the model can invoke server-side. GoAI supports 20 provider-defined tools across 5 providers:

Provider Tools Import
Anthropic Computer, Computer_20251124, Bash, TextEditor, TextEditor_20250728, WebSearch, WebSearch_20260209, WebFetch, CodeExecution, CodeExecution_20250825 provider/anthropic
OpenAI WebSearch, CodeInterpreter, FileSearch, ImageGeneration provider/openai
Google GoogleSearch, URLContext, CodeExecution provider/google
xAI WebSearch, XSearch provider/xai
Groq BrowserSearch provider/groq

All tools follow the same pattern: create a definition with <provider>.Tools.ToolName() (e.g., openai.Tools, anthropic.Tools), then pass it as a goai.Tool:

// Example: def := openai.Tools.WebSearch(openai.WithSearchContextSize("medium"))
def := <provider>.Tools.ToolName(options...)
result, _ := goai.GenerateText(ctx, model,
    goai.WithTools(goai.Tool{
        Name:                   def.Name,
        ProviderDefinedType:    def.ProviderDefinedType,
        ProviderDefinedOptions: def.ProviderDefinedOptions,
    }),
)

The model searches the web and returns grounded responses. Available from OpenAI, Anthropic, Google, and Groq.

// OpenAI (via Responses API) - also works via Azure
def := openai.Tools.WebSearch(openai.WithSearchContextSize("medium"))

// Anthropic (via Messages API) - also works via Bedrock
def := anthropic.Tools.WebSearch(anthropic.WithMaxUses(5))

// Google (grounding with Google Search) - returns Sources
def := google.Tools.GoogleSearch()
// result.Sources contains grounding URLs from Google Search

// Groq (interactive browser search)
def := groq.Tools.BrowserSearch()

Code Execution

The model writes and runs code in a sandboxed environment. Server-side, no local setup needed.

// OpenAI Code Interpreter - Python sandbox via Responses API
def := openai.Tools.CodeInterpreter()

// Anthropic Code Execution - Python sandbox via Messages API
def := anthropic.Tools.CodeExecution() // v20260120, GA, no beta needed

// Google Code Execution - Python sandbox via Gemini API
def := google.Tools.CodeExecution()

Web Fetch

Claude fetches and processes content from specific URLs directly.

def := anthropic.Tools.WebFetch(
    anthropic.WithWebFetchMaxUses(3),
    anthropic.WithCitations(true),
)

Semantic search over uploaded files in vector stores (OpenAI Responses API).

def := openai.Tools.FileSearch(
    openai.WithVectorStoreIDs("vs_abc123"),
    openai.WithMaxNumResults(5),
)

Image Generation

LLM generates images inline during conversation (different from goai.GenerateImage() which calls the Images API directly).

def := openai.Tools.ImageGeneration(
    openai.WithImageQuality("low"),
    openai.WithImageSize("1024x1024"),
)
// On Azure, also set: azure.WithHeaders(map[string]string{
//     "x-ms-oai-image-generation-deployment": "gpt-image-1.5",
// })

Computer Use

Anthropic computer, bash, and text editor tools for autonomous desktop interaction. Client-side execution required.

computerDef := anthropic.Tools.Computer(anthropic.ComputerToolOptions{
    DisplayWidthPx: 1920, DisplayHeightPx: 1080,
})
bashDef := anthropic.Tools.Bash()
textEditorDef := anthropic.Tools.TextEditor()
// Wrap each with an Execute handler for client-side execution

URL Context

Gemini fetches and processes web content from URLs in the prompt.

def := google.Tools.URLContext()

See examples/ for complete runnable examples of each tool.

Examples

See the examples/ directory:

Project Structure

goai/                       # Core SDK
├── provider/               # Provider interface + shared types
│   ├── provider.go         # LanguageModel, EmbeddingModel, ImageModel interfaces
│   ├── types.go            # Message, Part, Usage, StreamChunk, etc.
│   ├── token.go            # TokenSource, CachedTokenSource
│   ├── openai/             # OpenAI (Chat Completions + Responses API)
│   ├── anthropic/          # Anthropic (Messages API)
│   ├── google/             # Google Gemini (REST API)
│   ├── bedrock/            # AWS Bedrock (Converse API + SigV4 + EventStream)
│   ├── vertex/             # Google Vertex AI (OpenAI-compat)
│   ├── azure/              # Azure OpenAI
│   ├── cohere/             # Cohere (Chat v2 + Embed)
│   ├── minimax/            # MiniMax (Anthropic-compatible API)
│   ├── compat/             # Generic OpenAI-compatible
│   	└── ...                 # 13 more OpenAI-compatible providers
├── internal/
│   ├── openaicompat/       # Shared codec for 13 OpenAI-compat providers
│   ├── gemini/             # Schema sanitization (Vertex, Google)
│   ├── sse/                # SSE line parser
│   └── httpc/              # HTTP utilities
├── examples/               # Usage examples
└── bench/                  # Performance benchmarks (GoAI vs Vercel AI SDK)
    ├── fixtures/           # Shared SSE test fixtures
    ├── go/                 # Go benchmarks (go test -bench)
    ├── ts/                 # TypeScript benchmarks (Bun + Tinybench)
    ├── collect.sh          # Result aggregation → report
    └── Makefile            # make bench-all

Contributing

See CONTRIBUTING.md.

License

MIT

Documentation

Overview

Package goai is a Go SDK for building AI applications. One SDK, 24+ providers.

Inspired by the Vercel AI SDK. The same clean abstractions, idiomatically adapted for Go with generics, interfaces, and functional options.

Core Functions

Providers

24+ providers: OpenAI, Anthropic, Google, Bedrock, Azure, Vertex, Mistral, xAI, Groq, Cohere, DeepSeek, Fireworks, Together, DeepInfra, OpenRouter, Perplexity, Cerebras, MiniMax, Cloudflare Workers AI, FPT Smart Cloud, Ollama, vLLM, RunPod, and any OpenAI-compatible endpoint.

Providers auto-resolve API keys from environment variables: 

model := openai.Chat("gpt-4o") // reads OPENAI_API_KEY

Quick Start 

result, err := goai.GenerateText(ctx, model,
    goai.WithPrompt("What is the capital of France?"),
)
fmt.Println(result.Text)

Streaming 

stream, err := goai.StreamText(ctx, model, goai.WithPrompt("Hello"))
for text := range stream.TextStream() {
    fmt.Print(text)
}

Structured Output 

type Recipe struct {
    Name       string   `json:"name"`
    Ingredients []string `json:"ingredients"`
}
result, err := goai.GenerateObject[Recipe](ctx, model,
    goai.WithPrompt("Give me a recipe for cookies"),
)

Tools

Define tools with JSON Schema and an Execute handler. Set WithMaxSteps to enable the auto tool loop: 

result, err := goai.GenerateText(ctx, model,
    goai.WithPrompt("What's the weather in Tokyo?"),
    goai.WithTools(weatherTool),
    goai.WithMaxSteps(3),
)

See https://goai.sh for full documentation.

Index

Constants

View Source 
const (
	// ToolChoiceAuto lets the model decide whether to call a tool (default behavior).
	ToolChoiceAuto = "auto"

	// ToolChoiceNone prevents the model from calling any tools.
	ToolChoiceNone = "none"

	// ToolChoiceRequired forces the model to call at least one tool.
	ToolChoiceRequired = "required"
)

Tool choice constants for use with WithToolChoice.

Variables

View Source 
var ErrUnknownTool = errors.New("goai: unknown tool")

ErrUnknownTool is returned when a tool call references a tool not in the tool map.

Functions

func AssistantMessage 

func AssistantMessage(text string) provider.Message

AssistantMessage creates an assistant message with text content.

func ClassifyStreamError 

func ClassifyStreamError(body []byte) error

ClassifyStreamError parses a stream error event and returns the appropriate typed error (*ContextOverflowError or *APIError), or nil if the data is not a recognized error event.

func IsOverflow 

func IsOverflow(message string) bool

IsOverflow checks if an error message indicates a context overflow.

func ParseHTTPError 

func ParseHTTPError(providerID string, statusCode int, body []byte) error

ParseHTTPError classifies an HTTP error response.

func ParseHTTPErrorWithHeaders 

func ParseHTTPErrorWithHeaders(providerID string, statusCode int, body []byte, headers http.Header) error

ParseHTTPErrorWithHeaders parses an HTTP error response, preserving retry-related headers.

func SchemaFrom 

func SchemaFrom[T any]() json.RawMessage

SchemaFrom generates a JSON Schema from the Go type T. It panics if the generated schema cannot be marshaled to JSON, which indicates a bug in typeToSchema rather than a runtime error. In normal usage this cannot occur.

The schema is compatible with OpenAI strict mode:

  • All properties are required (pointer types become nullable)
  • additionalProperties: false on all objects

Supports struct tags:

  • json:"name" for field naming, json:"-" to skip
  • jsonschema:"description=...,enum=a|b|c" for descriptions and enums

Results are cached: repeated calls with the same T return a cached value. The cache is bypassed when schemaMarshalFunc has been replaced (test-only).

func SystemMessage 

func SystemMessage(text string) provider.Message

SystemMessage creates a system message with text content.

func ToolCallIDFromContext added in v0.5.7 

func ToolCallIDFromContext(ctx context.Context) string

ToolCallIDFromContext returns the tool call ID from the context. This is available inside a Tool's Execute function to identify which tool call is being executed.

func ToolMessage 

func ToolMessage(toolCallID, toolName, output string) provider.Message

ToolMessage creates a tool result message.

func UserMessage 

func UserMessage(text string) provider.Message

UserMessage creates a user message with text content.

Types

type APIError 

type APIError struct {
	Message         string
	StatusCode      int
	IsRetryable     bool
	ResponseBody    string
	ResponseHeaders map[string]string
}

APIError represents a non-overflow API error.

func (*APIError) Error 

func (e *APIError) Error() string

type AfterToolExecuteInfo added in v0.6.3 

type AfterToolExecuteInfo struct {
	// Ctx is the tool execution context (with tool call ID injected).
	Ctx context.Context

	// ToolCallID is the provider-assigned identifier for this tool call.
	ToolCallID string

	// ToolName is the name of the tool that executed.
	ToolName string

	// Step is the 1-based index of the generation step.
	Step int

	// Input is the raw JSON arguments passed to the tool.
	Input json.RawMessage

	// Output is the string result returned by the tool's Execute function.
	Output string

	// Error is non-nil if the tool's Execute function returned an error.
	Error error
}

AfterToolExecuteInfo is passed to the OnAfterToolExecute hook after a tool's Execute function completes, before the result is sent to the LLM.

type AfterToolExecuteResult added in v0.6.3 

type AfterToolExecuteResult struct {
	// Output replaces the tool's original output. If empty and the original
	// output was non-empty, the original is preserved (use a space to force empty).
	Output string

	// Error, when non-nil, replaces the tool's original error.
	// A nil value preserves the original error (cannot clear an error to nil).
	Error error

	// Metadata is opaque consumer data attached to the tool call result.
	// Passed through to ToolCallInfo.Metadata in the OnToolCall hook.
	// Use for tool titles, display tags, scanner flags, or any consumer-specific
	// data that should travel alongside the tool result.
	// Nil means no metadata.
	Metadata map[string]any
}

AfterToolExecuteResult allows the hook to modify the tool output before it is sent to the LLM.

type AgentState added in v0.7.2 

type AgentState struct {
	// contains filtered or unexported fields
}

AgentState is a race-free observable handle into the tool-loop's lifecycle state. Zero value is usable and reports (StepStarting, 0) until goai begins mutating it.

Concurrency: goai writes atomically from the tool-loop goroutine; consumers may poll Observe concurrently from any number of goroutines.

Layout: we pack (step:uint32, kind:uint32) into a single atomic.Uint64 so that both fields advance together without tearing.

func (*AgentState) Observe added in v0.7.2 

func (s *AgentState) Observe() (kind StepKind, step int)

Observe returns the current (kind, step) of the agent.

Step semantics:

  • Before first LLM call: step == 0.
  • During StepLLMInFlight / StepStepFinished / StepToolExecuting: step == the 1-indexed current step number.
  • At StepIdle: step == the highest step announced via StepLLMInFlight (FIX 47 monotonicity invariant). NOTE: at StepIdle, `step` may EXCEED `len(TextResult.Steps)` when an in-flight step errored before being appended to the step list - the counter was advanced for StepLLMInFlight but the StepResult was never recorded because the error short-circuited the step. Consumers tracking "how many steps completed" should use `len(result.Steps)`, not `state.Observe()`'s step return.

func (*AgentState) SetTerminal added in v0.7.2 

func (s *AgentState) SetTerminal(kind StepKind) bool

SetTerminal transitions the state to one of {StepDone, StepCancelled, StepError} via compare-and-swap. The transition succeeds only if the current state is non-terminal (i.e. !IsTerminal); a second SetTerminal call on an already-terminal state is a no-op and returns false. The step counter is preserved across the transition so pollers reading (kind, step) on a terminal state still see the highest step number reached. Returns true if this call performed the transition, false otherwise.

Intended caller: the consumer that owns the AgentState lifetime (e.g. zenflow's AgentRunner.Run on exit, mapping ctx.Err() / panic-recover to Cancelled / Error and the natural return path to Done). goai's own hooks NEVER call SetTerminal - the consumer is the sole writer of terminal states, satisfying the plan §4.2 #4 single-writer rule.

Passing a non-terminal kind panics: the API is intentionally narrow to prevent misuse (e.g. SetTerminal(StepLLMInFlight) would corrupt the lifecycle).

type BeforeStepInfo added in v0.6.3 

type BeforeStepInfo struct {
	// Ctx is the generation context. Use for cancellation checks or
	// passing context to external calls (e.g., inbox drain, context fetch).
	Ctx context.Context

	// Step is the 1-based step number about to execute.
	Step int

	// Messages is the current conversation history that will be sent to the LLM.
	// This is a shallow clone of the internal messages slice --the slice header is
	// independent but Message fields containing reference types (e.g., Content []Part)
	// share the underlying arrays with the live conversation. Do not mutate message
	// Content directly; use ExtraMessages to inject new messages.
	Messages []provider.Message
}

BeforeStepInfo is passed to the OnBeforeStep hook before each LLM call in a multi-step tool loop (step 2+). It is NOT called before step 1.

type BeforeStepResult added in v0.6.3 

type BeforeStepResult struct {
	// ExtraMessages are appended to the conversation before the LLM call.
	// Use this to inject inter-agent messages, context updates, etc.
	// Ignored when Stop is true.
	ExtraMessages []provider.Message

	// Stop, when true, terminates the tool loop before the next LLM call.
	// The current accumulated result is returned as-is.
	// When Stop is true, ExtraMessages are ignored (Stop takes precedence).
	Stop bool
}

BeforeStepResult controls what happens before the next LLM call.

type BeforeToolExecuteInfo added in v0.6.3 

type BeforeToolExecuteInfo struct {
	// Ctx is the tool execution context (with tool call ID injected).
	// Use this for passing cancellation, timeouts, or tracing context
	// to any work done inside the hook (e.g., spawning child agents).
	Ctx context.Context

	// ToolCallID is the provider-assigned identifier for this tool call.
	ToolCallID string

	// ToolName is the name of the tool about to execute.
	ToolName string

	// Step is the 1-based index of the generation step.
	Step int

	// Input is the raw JSON arguments that will be passed to the tool.
	Input json.RawMessage
}

BeforeToolExecuteInfo is passed to the OnBeforeToolExecute hook before a tool's Execute function runs. The hook can inspect the call and optionally skip execution.

type BeforeToolExecuteResult added in v0.6.3 

type BeforeToolExecuteResult struct {
	// Skip, when true, prevents the tool's Execute function from running.
	// The Result string is used as the tool output sent back to the LLM.
	// If Error is also set, it is reported as a tool error instead.
	Skip bool

	// Result is the synthetic tool output when Skip is true.
	// Ignored when Skip is false.
	// When Error is also non-nil, Result is ignored for the LLM message --only the
	// error is sent (as "error: <message>"). However, OnToolCall still receives Result
	// in its Output field for observability.
	// When Skip is true and both Result and Error are empty/nil, an empty string
	// is sent as the tool output to the LLM. Set Result to a meaningful string
	// to signal the skip reason to the model.
	Result string

	// Error, when non-nil and Skip is true, is reported as the tool error.
	// Result is ignored when Error is set --the LLM receives "error: <message>".
	// Ignored when Skip is false.
	Error error

	// Ctx, when non-nil, replaces the context passed to tool.Execute.
	// Use this to inject per-tool values (auth tokens, session IDs, timeouts)
	// or to wrap the context with context.WithTimeout for per-tool deadlines.
	// Nil means use the default toolCtx (parent ctx + tool call ID).
	// Ignored when Skip is true.
	Ctx context.Context

	// Input, when non-nil, replaces the tool arguments passed to Execute.
	// Use for input normalization, PII redaction, or secret injection.
	// Nil means use the original input from the LLM.
	// Ignored when Skip is true.
	Input json.RawMessage
}

BeforeToolExecuteResult controls what happens after OnBeforeToolExecute runs.

type ContextOverflowError 

type ContextOverflowError struct {
	Message      string
	ResponseBody string
}

ContextOverflowError indicates the prompt exceeded the model's context window.

func (*ContextOverflowError) Error 

func (e *ContextOverflowError) Error() string

type EmbedManyResult 

type EmbedManyResult struct {
	// Embeddings contains the generated vectors (one per input value).
	Embeddings [][]float64

	// Usage is the aggregated token consumption.
	Usage provider.Usage

	// ProviderMetadata contains provider-specific response data.
	ProviderMetadata map[string]map[string]any

	// Response contains provider-specific response metadata from the first chunk.
	// When multiple chunks are used, only the first chunk's response is included.
	Response provider.ResponseMetadata
}

EmbedManyResult is the result of multiple embedding generations.

func EmbedMany 

func EmbedMany(ctx context.Context, model provider.EmbeddingModel, values []string, opts ...Option) (*EmbedManyResult, error)

EmbedMany generates embedding vectors for multiple values. Auto-chunks when values exceed the model's MaxValuesPerCall limit and processes chunks in parallel (controlled by WithMaxParallelCalls).

type EmbedResult 

type EmbedResult struct {
	// Embedding is the generated vector.
	Embedding []float64

	// Usage tracks token consumption.
	Usage provider.Usage

	// ProviderMetadata contains provider-specific response data.
	ProviderMetadata map[string]map[string]any

	// Response contains provider-specific response metadata (ID, model, headers).
	Response provider.ResponseMetadata
}

EmbedResult is the result of a single embedding generation.

func Embed 

func Embed(ctx context.Context, model provider.EmbeddingModel, value string, opts ...Option) (*EmbedResult, error)

Embed generates an embedding vector for a single value.

type FinishInfo added in v0.6.4 

type FinishInfo struct {
	// StepsExhausted is true when MaxSteps was reached while the model still
	// requested tool calls. This is the authoritative signal for "max_steps"
	// termination - it is not available from any per-step hook.
	StepsExhausted bool

	// TotalSteps is the number of generation steps executed.
	TotalSteps int

	// TotalUsage is the aggregated token usage across all steps.
	TotalUsage provider.Usage

	// FinishReason from the last step.
	FinishReason provider.FinishReason

	// StoppedBy classifies how the loop terminated: natural model exit,
	// MaxSteps exhaustion, WithStopWhen predicate, OnBeforeStep.Stop,
	// an error-driven abort, an empty provider stream, or a model
	// requesting tool calls with no executable tools configured.
	// Possible values: "natural", "max-steps", "predicate", "before-step",
	// "abort", "empty" (streaming only), "no-executable-tools" (sync only).
	// Useful when FinishReason alone is ambiguous (e.g. a StopWhen break
	// on a tool-calls step reports FinishToolCalls but
	// StoppedBy=="predicate"). May be empty on pre-loop error paths.
	StoppedBy provider.StopCause
}

FinishInfo is passed to the OnFinish hook after all generation steps complete. It fires exactly once per GenerateText/StreamText/GenerateObject/StreamObject call, after the last OnStepFinish. For StreamText/StreamObject, it fires in the stream goroutine.

type ImageOption 

type ImageOption func(*imageOptions)

ImageOption configures a GenerateImage call.

GenerateImage uses ImageOption (not the general-purpose Option type) because image generation has a distinct parameter set that does not overlap with text generation options. Common cross-cutting options like timeout and retries have dedicated WithImage* equivalents: WithImageTimeout, WithImageMaxRetries.

func WithAspectRatio 

func WithAspectRatio(ratio string) ImageOption

WithAspectRatio sets the aspect ratio (e.g. "16:9", "1:1").

func WithImageCount 

func WithImageCount(n int) ImageOption

WithImageCount sets the number of images to generate. Values below 1 are clamped to 1 (minimum one image).

func WithImageMaxRetries added in v0.4.4 

func WithImageMaxRetries(n int) ImageOption

WithImageMaxRetries sets the maximum number of retries for transient errors. Values below 0 are clamped to 0 (no retries).

func WithImagePrompt 

func WithImagePrompt(prompt string) ImageOption

WithImagePrompt sets the text prompt for image generation.

func WithImageProviderOptions 

func WithImageProviderOptions(opts map[string]any) ImageOption

WithImageProviderOptions sets provider-specific options. Values must be JSON-serializable (no channels, functions, or unsafe pointers).

func WithImageSize 

func WithImageSize(size string) ImageOption

WithImageSize sets the image size (e.g. "1024x1024", "512x512").

func WithImageTimeout added in v0.4.4 

func WithImageTimeout(d time.Duration) ImageOption

WithImageTimeout sets a timeout for the image generation request.

type ImageResult 

type ImageResult struct {
	Images []provider.ImageData

	// ProviderMetadata contains provider-specific response data.
	ProviderMetadata map[string]map[string]any

	// Usage tracks token or operation consumption (if reported by the provider).
	Usage provider.Usage

	// Response contains provider-specific response metadata (ID, model, headers).
	Response provider.ResponseMetadata
}

ImageResult contains the generated images.

func GenerateImage 

func GenerateImage(ctx context.Context, model provider.ImageModel, opts ...ImageOption) (*ImageResult, error)

GenerateImage generates images from a text prompt using the given model.

Note: GenerateImage uses ImageOption, not the general-purpose Option type. Use WithImagePrompt, WithImageTimeout, WithImageMaxRetries, etc. These cannot be mixed with Option values from GenerateText/Embed calls.

type ObjectResult 

type ObjectResult[T any] struct {
	// Object is the parsed structured output.
	Object T

	// Usage tracks total token consumption across all steps.
	Usage provider.Usage

	// FinishReason indicates why generation stopped.
	FinishReason provider.FinishReason

	// Response contains provider metadata (ID, Model).
	Response provider.ResponseMetadata

	// ProviderMetadata contains provider-specific response data.
	ProviderMetadata map[string]map[string]any

	// ResponseMessages contains the assistant and tool messages from all generation steps.
	// For multi-turn conversations, append these to your message history:
	//   messages = append(messages, result.ResponseMessages...)
	//
	// Nil when the response has no content.
	// Reasoning parts are not included (GenerateObject uses non-streaming DoGenerate,
	// and StreamObject does not capture reasoning chunks).
	ResponseMessages []provider.Message

	// Steps contains results from each generation step (for multi-step tool loops).
	// The final step is always the structured output step.
	Steps []StepResult
}

ObjectResult is the final result of a structured output generation.

func GenerateObject 

func GenerateObject[T any](ctx context.Context, model provider.LanguageModel, opts ...Option) (_ *ObjectResult[T], err error)

GenerateObject performs a non-streaming structured output generation. The schema is auto-generated from T, or can be overridden with WithExplicitSchema.

When tools with Execute functions are provided and MaxSteps > 1, GenerateObject runs a tool loop (identical to GenerateText) with ResponseFormat set on every step. The model decides when to call tools vs produce the final JSON output. Structured output is parsed from whichever step returns finishReason "stop".

If MaxSteps is exhausted before a "stop" step occurs, an error is returned. This differs slightly from Vercel AI SDK, which returns a partial result with a nil output field - in Go, returning an error is the idiomatic equivalent.

type ObjectStream 

type ObjectStream[T any] struct {
	// contains filtered or unexported fields
}

ObjectStream is a streaming structured output response.

func StreamObject 

func StreamObject[T any](ctx context.Context, model provider.LanguageModel, opts ...Option) (_ *ObjectStream[T], err error)

StreamObject performs a streaming structured output generation. Returns an ObjectStream that emits progressively populated partial objects.

Unlike GenerateObject, StreamObject is intentionally single-step: it initiates one streaming request and returns immediately. Tool loops are not supported because the caller consumes the stream asynchronously; use GenerateObject when you need tools and multi-step behaviour.

func (*ObjectStream[T]) Err added in v0.4.4 

func (os *ObjectStream[T]) Err() error

Err returns the first stream error encountered, or nil. Must be called after the stream is fully consumed (after Result(), or after the PartialObjectStream() channel is drained). Follows the bufio.Scanner.Err() pattern.

func (*ObjectStream[T]) PartialObjectStream 

func (os *ObjectStream[T]) PartialObjectStream() <-chan *T

PartialObjectStream returns a channel that emits partial objects as JSON accumulates. Each emitted value has progressively more fields populated. Mutually exclusive with Result() - only call one consumption method first.

func (*ObjectStream[T]) Result 

func (os *ObjectStream[T]) Result() (*ObjectResult[T], error)

Result blocks until the stream completes and returns the final validated object. Returns an error if JSON parsing of the accumulated text fails.

type Option 

type Option func(*options)

Option configures a generation call.

func WithEmbeddingProviderOptions 

func WithEmbeddingProviderOptions(opts map[string]any) Option

WithEmbeddingProviderOptions sets provider-specific parameters for embedding requests. Values must be JSON-serializable (no channels, functions, or unsafe pointers).

func WithExplicitSchema 

func WithExplicitSchema(schema json.RawMessage) Option

WithExplicitSchema overrides auto-generated JSON Schema for GenerateObject/StreamObject.

func WithFrequencyPenalty 

func WithFrequencyPenalty(p float64) Option

WithFrequencyPenalty sets the frequency penalty.

func WithHeaders 

func WithHeaders(h map[string]string) Option

WithHeaders sets additional HTTP headers.

func WithMaxOutputTokens 

func WithMaxOutputTokens(n int) Option

WithMaxOutputTokens limits the response length.

func WithMaxParallelCalls 

func WithMaxParallelCalls(n int) Option

WithMaxParallelCalls sets batch parallelism for EmbedMany.

func WithMaxRetries 

func WithMaxRetries(n int) Option

WithMaxRetries sets the retry count for transient errors. Pass -1 for unlimited retries (useful when the application manages its own timeout/cancellation via context). Values below -1 are clamped to 0.

func WithMaxSteps 

func WithMaxSteps(n int) Option

WithMaxSteps sets the maximum auto tool loop iterations.

func WithMessages 

func WithMessages(msgs ...provider.Message) Option

WithMessages sets the conversation history.

func WithOnAfterToolExecute added in v0.6.3 

func WithOnAfterToolExecute(fn func(AfterToolExecuteInfo) AfterToolExecuteResult) Option

WithOnAfterToolExecute adds a callback invoked after each tool's Execute function, before the result is sent to the LLM. The callback can modify the output (e.g., for secret scanning, truncation, or output transformation). Not called when OnBeforeToolExecute skips execution or for unknown tools. Use OnToolCall with Skipped=true to observe skipped tool results. Only one callback is supported; setting a second replaces the first. The callback is panic-recovered (and reported to any OnPanic hook); a panic preserves the original tool result.

func WithOnBeforeStep added in v0.6.3 

func WithOnBeforeStep(fn func(BeforeStepInfo) BeforeStepResult) Option

WithOnBeforeStep adds a callback invoked before each LLM call in a multi-step tool loop (step 2+, after tool execution). The callback can inject additional messages or stop the loop early. Only one callback is supported; setting a second replaces the first.

A panic in the callback is reported to any OnPanic hook and then surfaced as a *PanicError: returned by GenerateText/GenerateObject, or via stream.Err() for StreamText. The loop stops at the panicking step.

func WithOnBeforeToolExecute added in v0.6.3 

func WithOnBeforeToolExecute(fn func(BeforeToolExecuteInfo) BeforeToolExecuteResult) Option

WithOnBeforeToolExecute adds a callback invoked before each known tool's Execute function. Not called for unknown tools (which always fail with ErrUnknownTool). The callback can inspect the tool call and return a BeforeToolExecuteResult to skip execution (e.g., for permission checks, rate limiting, or doom-loop detection). Only one callback is supported; setting a second replaces the first. The callback is panic-recovered (and reported to any OnPanic hook); a panic skips the tool with an error result.

func WithOnFinish added in v0.6.4 

func WithOnFinish(fn func(FinishInfo)) Option

WithOnFinish adds a callback invoked once after all generation steps complete. Multiple callbacks are called in registration order.

A panic in a callback is reported to any OnPanic hook and then surfaced as a *PanicError: returned by GenerateText/GenerateObject, or via stream.Err() for StreamText/StreamObject. Subsequent callbacks do not run.

This hook fires in all code paths:

  • GenerateText: after the tool loop exits (natural, max_steps, or hook_stopped)
  • StreamText: in the stream goroutine, after the tool loop exits
  • GenerateObject: after the tool loop exits (including max_steps error path)
  • StreamObject: in the consume goroutine, after the stream completes

It does NOT fire when DoGenerate/DoStream returns a provider error (API failure, context cancelled). In those cases, the OnResponse hook receives the error. Note: GenerateObject's max_steps error IS a goai-level error, not a provider error, so OnFinish fires before that error is returned. For StreamText/StreamObject, check stream.Err() for definitive error status.

func WithOnPanic added in v0.8.0 

func WithOnPanic(fn func(PanicInfo)) Option

WithOnPanic adds a callback invoked whenever a user callback or the StopWhen predicate panics. Multiple callbacks are called in registration order.

It fires for every panicking callback, both the propagate-fatal lifecycle hooks (OnRequest, OnResponse, OnStepFinish, OnFinish, OnBeforeStep, StopWhen) and the resilient tool-path callbacks (tool Execute, OnToolCallStart, OnToolCall, OnBeforeToolExecute, OnAfterToolExecute). Use it for observability (otel spans, metrics, logging) regardless of how the panic is ultimately handled.

OnPanic fires before the panic is surfaced or recovered. A panic inside an OnPanic callback itself is recovered and discarded.

func WithOnRequest 

func WithOnRequest(fn func(RequestInfo)) Option

WithOnRequest adds a callback invoked before each model call. Multiple callbacks are called in registration order.

A panic in a callback is reported to any OnPanic hook and then surfaced as a *PanicError: returned by GenerateText/GenerateObject, or via stream.Err() for StreamText/StreamObject (step 2+ panics, fired in the stream goroutine, also surface through stream.Err()). Subsequent callbacks do not run.

func WithOnResponse 

func WithOnResponse(fn func(ResponseInfo)) Option

WithOnResponse adds a callback invoked after each model call completes. Multiple callbacks are called in registration order.

A panic in a callback is reported to any OnPanic hook and then surfaced as a *PanicError: returned by GenerateText/GenerateObject, or via stream.Err() for StreamText/StreamObject. Subsequent callbacks do not run.

func WithOnStepFinish 

func WithOnStepFinish(fn func(StepResult)) Option

WithOnStepFinish adds a callback invoked after each generation step completes. Multiple callbacks are called in registration order.

A panic in a callback is reported to any OnPanic hook and then surfaced as a *PanicError: returned by GenerateText/GenerateObject, or via stream.Err() for StreamText/StreamObject. Subsequent callbacks do not run.

Timing: OnStepFinish fires after the LLM call for the step returns and BEFORE any tool executions for that step run. The StepResult passed to the hook therefore always carries an EMPTY ToolResults slice - even in the sync (GenerateText) path. To observe tool results as they complete, use OnToolCall or OnAfterToolExecute; to see the full populated StepResult.ToolResults after tools ran, inspect the TextResult.Steps slice returned by GenerateText or stream.Result().

func WithOnToolCall 

func WithOnToolCall(fn func(ToolCallInfo)) Option

WithOnToolCall adds a callback invoked after each tool execution. Multiple callbacks are called in registration order. Each callback is individually panic-recovered (and reported to any OnPanic hook) so a panic in one does not prevent others from firing or abort the agent loop.

func WithOnToolCallStart added in v0.5.4 

func WithOnToolCallStart(fn func(ToolCallStartInfo)) Option

WithOnToolCallStart adds a callback invoked before each tool execution. Multiple callbacks are called in registration order. Each callback is individually panic-recovered (and reported to any OnPanic hook) so a panic in one does not prevent others from firing. If any callback panics, the tool does not execute.

func WithOptions added in v0.5.6 

func WithOptions(opts ...Option) Option

WithOptions combines multiple Options into a single Option. This is useful for libraries that need to return one Option encapsulating several.

func WithPresencePenalty 

func WithPresencePenalty(p float64) Option

WithPresencePenalty sets the presence penalty.

func WithPrompt 

func WithPrompt(s string) Option

WithPrompt sets a shorthand user message prompt.

func WithPromptCaching 

func WithPromptCaching(b bool) Option

WithPromptCaching enables provider-specific prompt caching. Currently supported by: Anthropic, Bedrock (Anthropic-family models only), and MiniMax (which delegates to Anthropic). Other providers log a warning to stderr when this option is set.

func WithProviderOptions 

func WithProviderOptions(opts map[string]any) Option

WithProviderOptions sets provider-specific request parameters. Values must be JSON-serializable (no channels, functions, or unsafe pointers).

func WithRetryObserver added in v0.8.4 

func WithRetryObserver(fn RetryObserver) Option

WithRetryObserver sets a callback that is invoked before each retry attempt. The callback receives the 0-based retry attempt number, the error that triggered the retry, and the delay before the next attempt. It is called before the sleep, allowing callers to log or observe retry progress.

func WithSchemaName 

func WithSchemaName(name string) Option

WithSchemaName sets the schema name sent to providers (default "response").

func WithSeed 

func WithSeed(s int) Option

WithSeed sets the seed for deterministic generation.

func WithSequentialToolExecution added in v0.6.4 

func WithSequentialToolExecution() Option

WithSequentialToolExecution forces tool calls to execute one at a time instead of in parallel. Useful when tools share non-thread-safe resources.

func WithStateRef added in v0.7.2 

func WithStateRef(ref *AgentState) Option

WithStateRef exposes goai's tool-loop lifecycle state via an externally-owned AgentState. The caller allocates the value and retains a pointer; goai mutates it atomically as the loop progresses.

Typical use (zenflow poller): 

var state goai.AgentState
go pollLoop(&state)
_, err := goai.GenerateText(ctx, model, goai.WithStateRef(&state), ...)

ref must be non-nil; nil refs are ignored.

Supported entry points: GenerateText and StreamText (both single-shot and multi-step tool-loop paths). WithStateRef is NOT supported by GenerateObject or StreamObject - those functions do not run a multi-step loop worth polling, so the AgentState would remain at its zero value (StepStarting, 0) indefinitely. Passing WithStateRef to either function emits a one-shot stderr warning per process per entry point and is otherwise a no-op (FIX 35).

Warning semantics (FIX 48). The stderr warning fires ONCE per process per entry point: GenerateObject and StreamObject each own an independent atomic.Bool latch. A long-lived process that calls GenerateObject with WithStateRef a thousand times sees exactly ONE warning for GenerateObject; if that same process later calls StreamObject with WithStateRef it sees one additional warning for StreamObject. Tests may reset these latches in a hermetic way (see object.go FIX 33 caveat) - production code must not.

Observation semantics (FIX 43). StateRef provides an atomic read of the (kind, step) pair, but reading StepIdle does NOT establish happens-before on other TextStream / TextResult fields (streamErr, ResponseMessages, Steps, final Usage, etc.). A poller that observes StepIdle and immediately reads stream.Err() or the TextResult return value without additional synchronization races against goai's last writes.

Pollers that observe StepIdle MUST still wait on conventional sync before reading result data. The defer-chain ordering of the raw Stream() channel close relative to the StepIdle publish differs between the single-shot and multi-step StreamText paths, so users MUST NOT rely on raw channel close as a StepIdle sync point:

  • StreamText, single-shot path (MaxSteps==1 OR no tools bound): the consume goroutine's defer chain closes the raw Stream() channel BEFORE publishing the StepIdle atomic store. A poller that waits only on raw channel close and then reads state.Observe() may still see StepLLMInFlight / StepToolExecuting.
  • StreamText, multi-step path (MaxSteps>1 with tools): StepIdle is published by the tool-loop goroutine's defer, while the raw output channel is closed by the same goroutine in a separate defer. Callers should not depend on inter-defer ordering here.
  • StreamText, BOTH paths: call stream.Err() or stream.Result() to reliably observe StepIdle. These block on the internal doneCh which closes AFTER the StepIdle atomic store - so a caller that sees Err()/Result() return is guaranteed to observe StepIdle on a subsequent state.Observe().
  • GenerateText: use the TextResult returned by GenerateText - by the time GenerateText returns, StepIdle and result fields are synchronized via the function-return happens-before edge.

Use StateRef for "should I wake up / inject work?" polling decisions; use conventional sync (channel close, function return) for reading result data.

func WithStopSequences 

func WithStopSequences(seqs ...string) Option

WithStopSequences sets stop sequences.

func WithStopWhen added in v0.7.2 

func WithStopWhen(predicate StopCondition) Option

WithStopWhen registers a stop predicate. It is evaluated after the current step's LLM call AND its tool executions complete (tool-result messages already folded into the running message list), and BEFORE the next DoGenerate/DoStream is issued. This matches Vercel AI SDK's placement (packages/ai/src/generate-text/generate-text.ts).

Consequences of this placement:

  • If the predicate returns true on a step whose LLM response contained tool calls, those tool calls ARE executed before the break. Both the assistant message (with tool_use parts) and the paired tool-result message for the last step are included in ResponseMessages, making the transcript safe to replay against strict providers (Anthropic, OpenAI) that reject dangling tool_use.
  • The loop's FinishReason is the last completed step's natural reason (typically FinishToolCalls when stopping mid-loop).
  • StepsExhausted is NOT set by a predicate-driven break even when the break coincides with step == MaxSteps.

If predicate is nil, the option is a no-op. If called multiple times the last value wins. Panics in the predicate are recovered and logged; they are treated as "do not stop".

Aliasing contract (FIX 30 / FIX 36). The predicate receives a SHALLOW clone of goai's internal steps slice: the top-level slice header is safe to reslice / append / zero (those mutations stay local). Nested slices (StepResult.ToolCalls, StepResult.ToolResults, StepResult.Content) are ALIASED into goai's internal state - writing to an element of those nested slices WILL corrupt goai's internal record and may cause incorrect ResponseMessages or undefined downstream behavior. Predicates MUST treat StepResult contents as read-only. goai does NOT enforce this via deep-clone (prohibitive per-step cost for a feature that is not a supported use case). TestStopSafe_AliasingContract_ShallowCloneIsolatesTopLevel locks in both halves of the contract.

WithStopWhen is ignored by GenerateObject and StreamObject (those functions use their own fixed exit conditions tied to structured output parsing). A one-shot stderr warning is emitted the first time WithStopWhen is passed to those paths.

func WithSystem 

func WithSystem(s string) Option

WithSystem sets the system prompt.

func WithTemperature 

func WithTemperature(t float64) Option

WithTemperature controls randomness.

func WithTimeout 

func WithTimeout(d time.Duration) Option

WithTimeout sets the timeout for the entire generation call.

func WithToolChoice 

func WithToolChoice(tc string) Option

WithToolChoice controls tool selection.

func WithTools 

func WithTools(tools ...Tool) Option

WithTools sets the tools available to the model.

func WithTopK 

func WithTopK(k int) Option

WithTopK limits sampling to the top K tokens.

func WithTopP 

func WithTopP(p float64) Option

WithTopP controls nucleus sampling.

func WrapOnAfterToolExecute added in v0.6.4 

func WrapOnAfterToolExecute(wrapper func(existing func(AfterToolExecuteInfo) AfterToolExecuteResult) func(AfterToolExecuteInfo) AfterToolExecuteResult) Option

WrapOnAfterToolExecute returns an Option that wraps the existing OnAfterToolExecute hook. The wrapper function receives the current hook (may be nil) and returns a replacement. The wrapper MUST return the user hook's result verbatim - an empty AfterToolExecuteResult has special semantics (empty Output preserves original, nil Error preserves original). When the existing hook is nil, return a zero AfterToolExecuteResult.

Ordering hazard: same as WrapOnBeforeToolExecute - apply after user hooks.

func WrapOnBeforeStep added in v0.6.4 

func WrapOnBeforeStep(wrapper func(existing func(BeforeStepInfo) BeforeStepResult) func(BeforeStepInfo) BeforeStepResult) Option

WrapOnBeforeStep returns an Option that wraps the existing OnBeforeStep hook. The wrapper function receives the current hook (may be nil) and returns a replacement. The wrapper MUST forward the user hook's Stop and ExtraMessages fields verbatim. When the existing hook is nil, return a zero BeforeStepResult (no stop, no messages).

Ordering hazard: same as WrapOnBeforeToolExecute - apply after user hooks.

func WrapOnBeforeToolExecute added in v0.6.4 

func WrapOnBeforeToolExecute(wrapper func(existing func(BeforeToolExecuteInfo) BeforeToolExecuteResult) func(BeforeToolExecuteInfo) BeforeToolExecuteResult) Option

WrapOnBeforeToolExecute returns an Option that wraps the existing OnBeforeToolExecute hook. The wrapper function receives the current hook (may be nil if none is registered) and returns a replacement. This enables observability packages to add side-effects without replacing the user's hook.

The wrapper MUST return the user hook's result verbatim when delegating - accidentally zeroing the struct suppresses tool executions or input overrides. When the existing hook is nil, return a zero BeforeToolExecuteResult (no skip, no override).

Ordering hazard: the wrapper captures the existing hook at option-apply time. If a user calls WithOnBeforeToolExecute AFTER this wrapper is applied, the user's hook overwrites the wrapper entirely. Callers using this for observability should apply it after user hooks (e.g., WithTracing should be the last option).

type PanicError added in v0.8.0 

type PanicError struct {
	// Phase identifies the callback that panicked, e.g. "OnStepFinish",
	// "OnFinish", "OnResponse", "OnRequest", "OnBeforeStep", "StopWhen".
	Phase string

	// Value is the value passed to panic(). May contain sensitive data; see the
	// type doc. Not included in Error().
	Value any

	// Stack is the goroutine stack captured at the recovery point. May contain
	// sensitive data; see the type doc. Not included in Error().
	Stack []byte
}

PanicError wraps a value recovered from a panic in a user-provided callback (a lifecycle hook or the StopWhen predicate). It is returned by GenerateText and GenerateObject, and surfaced through stream.Err() for StreamText and StreamObject, so callers can intercept callback panics programmatically instead of having them swallowed.

Panics inside the tool path (tool Execute, OnToolCallStart, OnToolCall, OnBeforeToolExecute, OnAfterToolExecute) are NOT wrapped in a PanicError: they keep their resilient behavior (converted to a tool error, the agent loop continues). They are still reported to any OnPanic hook.

Value and Stack may contain sensitive data (panic arguments, captured variables, credentials). They are deliberately NOT included in Error() so a logged error string cannot leak them; access the fields explicitly and sanitize before logging or exporting.

func (*PanicError) Error added in v0.8.0 

func (e *PanicError) Error() string

Error returns a message identifying the phase only. The panic value is intentionally omitted to avoid leaking sensitive data into logged error strings; read Value/Unwrap for the underlying cause.

func (*PanicError) Unwrap added in v0.8.0 

func (e *PanicError) Unwrap() error

Unwrap returns the panic value if it is itself an error, enabling errors.Is/errors.As against the original cause; otherwise it returns nil.

type PanicInfo added in v0.8.0 

type PanicInfo struct {
	// Phase identifies the callback that panicked, e.g. "OnStepFinish",
	// "OnFinish", "OnResponse", "OnRequest", "OnBeforeStep", "StopWhen",
	// "OnToolCallStart", "OnToolCall", "OnBeforeToolExecute",
	// "OnAfterToolExecute", or "tool:<name>" for a tool's Execute function.
	Phase string

	// Value is the value passed to panic(). May contain sensitive data.
	Value any

	// Stack is the goroutine stack captured at the recovery point.
	// May contain sensitive data.
	Stack []byte
}

PanicInfo is passed to the OnPanic hook when a user callback or the StopWhen predicate panics. Value and Stack may contain sensitive data (panic arguments, captured variables, credentials); sanitize before logging or exporting.

type ParsedStreamError 

type ParsedStreamError struct {
	Type         StreamErrorType
	Message      string
	IsRetryable  bool
	ResponseBody string
}

ParsedStreamError represents a parsed error from an SSE stream.

func ParseStreamError 

func ParseStreamError(body []byte) *ParsedStreamError

ParseStreamError parses a stream error event (used by Anthropic/OpenAI error events).

type RequestInfo 

type RequestInfo struct {
	// Ctx is the caller's context for this generation call.
	// Observability hooks can use this for span parenting (e.g., creating
	// child spans under the caller's existing trace context).
	Ctx context.Context

	// Model is the model ID.
	Model string

	// MessageCount is the number of messages in the request.
	MessageCount int

	// ToolCount is the number of tools available.
	ToolCount int

	// Timestamp is when the request was initiated.
	Timestamp time.Time

	// Messages is the full conversation history sent to the model for this call.
	// If a system prompt is set, it is prepended as the first message with role "system".
	// For step-1 this is [system, user]. For step-2+ it also includes assistant
	// tool-call messages and tool result messages from prior steps.
	Messages []provider.Message
}

RequestInfo is passed to the OnRequest hook before a generation call.

type ResponseInfo 

type ResponseInfo struct {
	// Latency is the time from request to response.
	Latency time.Duration

	// Usage is the token consumption for this call.
	Usage provider.Usage

	// FinishReason indicates why generation stopped.
	FinishReason provider.FinishReason

	// Error is non-nil if the API call failed (DoGenerate/DoStream returned error)
	// or if a mid-stream ChunkError occurred (in StreamText/StreamObject consume goroutine).
	// For streaming success paths, check stream.Err() for the definitive error status.
	Error error

	// StatusCode is the HTTP status code (0 if not applicable).
	StatusCode int
}

ResponseInfo is passed to the OnResponse hook after a generation call completes.

type RetryObserver added in v0.8.4 

type RetryObserver func(attempt int, err error, delay time.Duration)

RetryObserver is called before each retry attempt, giving the caller a chance to log or observe the retry step.

type StepKind added in v0.7.2 

type StepKind int

StepKind identifies the lifecycle phase of the tool loop at a given moment.

AgentState (observed via Observe) transitions through these kinds as the goai tool loop progresses. Consumers poll from a separate goroutine to decide whether it is safe to inject work / send wake signals without interrupting an in-flight LLM call or tool execution. 

const (
	// StepStarting is the initial state before the first LLM call begins.
	StepStarting StepKind = iota

	// StepLLMInFlight indicates an LLM request is in flight (sync or stream).
	// For StreamText this covers the period from DoStream dispatch until the
	// last chunk of the step has been drained.
	StepLLMInFlight

	// StepStepFinished indicates the LLM call for the current step has fully
	// returned (DoGenerate returned / the stream's ChunkStepFinish has been
	// emitted) and the step result has been recorded, but tool execution has
	// not yet begun and loop-termination predicates (WithStopWhen,
	// OnBeforeStep) have not yet decided whether to continue. This state
	// exists so that pollers observing between phases cannot mistake the
	// post-LLM / pre-tool window for "LLM still in flight".
	StepStepFinished

	// StepToolExecuting indicates one or more tool Execute functions are
	// running. It is entered after the LLM's stepN response is fully drained
	// and exited when all parallel tool calls return.
	StepToolExecuting

	// StepIdle indicates the tool loop has terminated INTERNALLY (either
	// naturally, by MaxSteps exhaustion, by OnBeforeStep.Stop, or by
	// WithStopWhen returning true). StepIdle is the post-tool-loop "wake
	// eligible" parking state - consumers may inject work and re-enter
	// GenerateText. It is NOT a terminal state: a consumer may transition
	// to one of the three terminal kinds below via SetTerminal once the
	// owning goroutine has decided no further work will run.
	StepIdle

	// StepDone is a terminal state set by the consumer when the runner
	// completed naturally (no error, no ctx cancel, no panic). After
	// StepDone, the state never transitions again. SetTerminal enforces
	// this via CAS.
	StepDone

	// StepCancelled is a terminal state set by the consumer when the
	// runner exited due to context cancellation (ctx.Err() != nil) or a
	// caller-requested cancel signal. Terminal; sticky.
	StepCancelled

	// StepError is a terminal state set by the consumer when the runner
	// returned a non-nil error (or recovered a panic). Terminal; sticky.
	StepError
)

func (StepKind) IsTerminal added in v0.7.2 

func (k StepKind) IsTerminal() bool

IsTerminal reports whether k is one of the three terminal states (StepDone / StepCancelled / StepError). Pollers use this to decide whether further state mutation is possible.

func (StepKind) String added in v0.7.2 

func (k StepKind) String() string

String returns a human-readable name for the kind.

type StepResult 

type StepResult struct {
	// Number is the 1-based step index.
	Number int

	// Text generated in this step (excludes reasoning tokens).
	// For StreamText, reasoning is included in TextResult.Text but excluded here.
	Text string

	// Reasoning is the consolidated thinking/reasoning text for this
	// step (PartReasoning, signature stripped). Populated for both
	// GenerateText and StreamText when the provider returns reasoning.
	Reasoning string

	// ToolCalls requested in this step.
	ToolCalls []provider.ToolCall

	// ToolResults contains one entry per completed ToolCall in this step,
	// populated AFTER executeToolsParallel returns and BEFORE WithStopWhen
	// is evaluated. Ordering matches ToolCalls element-for-element.
	//
	// Empty when the step had no tool calls or when the loop exits before
	// executing tools (e.g. MaxSteps reached with pending tool calls that
	// never ran, or StopCauseNoExecutableTools).
	//
	// Mirrors Vercel AI SDK's DefaultStepResult.toolResults so predicates
	// passed to WithStopWhen can inspect tool outputs (matching the
	// placement documented on StopCondition).
	//
	// Streaming visibility: consumers using StreamText who read raw
	// chunks via stream.Stream() cannot observe per-step ToolResults in
	// real time. The ChunkStepFinish chunk with stepSource="goai" is
	// emitted BEFORE tools execute (so ToolResults is empty at that
	// point); the subsequent goai-internal "goai-tool-results" chunk
	// that backfills ToolResults is consumed by the stream reducer and
	// NOT re-emitted to the raw chunk channel. To observe ToolResults
	// per step from a streaming call, use one of:
	//   - stream.Result() after the stream closes (Steps[].ToolResults
	//     is fully populated).
	//   - OnToolCall hook (fires synchronously after each tool Execute
	//     returns with per-call detail).
	//   - OnAfterToolExecute hook (same timing as OnToolCall, richer
	//     metadata).
	// The OnStepFinish hook always receives a StepResult with an EMPTY
	// ToolResults slice because tools execute AFTER the hook fires.
	ToolResults []provider.ToolResult

	// FinishReason for this step.
	FinishReason provider.FinishReason

	// Usage for this step.
	Usage provider.Usage

	// Response contains provider metadata for this step (ID, Model).
	Response provider.ResponseMetadata

	// ProviderMetadata contains provider-specific response data for this step.
	ProviderMetadata map[string]map[string]any

	// Sources contains citations/references from this step.
	Sources []provider.Source
}

StepResult is the result of a single generation step in a tool loop.

type StopCondition added in v0.7.2 

type StopCondition func(steps []StepResult) bool

StopCondition is a composable predicate evaluated AFTER each step of the tool loop fully completes - including tool execution for that step -- and BEFORE deciding whether to issue the next LLM call. Returning true causes the loop to exit cleanly using the last completed step's natural FinishReason (no synthetic reason is emitted).

The predicate receives the full list of completed steps. Each StepResult exposes both ToolCalls (requests the model made this step) and ToolResults (outputs produced by executeToolsParallel for those calls); ToolResults are populated in element-for-element order with ToolCalls and are visible to the predicate because the predicate fires AFTER tool execution. Predicates may branch on either - e.g. "stop after the first step where any tool result is empty": 

goai.WithStopWhen(func(steps []goai.StepResult) bool {
    last := steps[len(steps)-1]
    for _, r := range last.ToolResults {
        if r.Output == "" && !r.IsError {
            return true
        }
    }
    return false
})

This mirrors Vercel AI SDK's StopCondition placement (packages/ai/src/generate-text/generate-text.ts and stop-condition.ts): the predicate gates the NEXT iteration, not the tool exec of the current iteration. Consumers can compose multiple predicates with a standard OR: 

goai.WithStopWhen(func(steps []goai.StepResult) bool {
    return pred1(steps) || pred2(steps)
})

type StreamErrorType 

type StreamErrorType string

StreamErrorType classifies parsed stream errors. 

const (
	StreamErrorContextOverflow StreamErrorType = "context_overflow"
	StreamErrorAPI             StreamErrorType = "api_error"
)

type TextResult 

type TextResult struct {
	// Text is the accumulated generated text across all steps.
	// For StreamText, this includes reasoning tokens (ChunkReasoning) for backward
	// compatibility. Use Steps[n].Text for text-only content excluding reasoning.
	Text string

	// Reasoning is the model's accumulated thinking/reasoning text
	// across all steps (PartReasoning). Populated for both GenerateText
	// and StreamText when the provider returns reasoning content (e.g.
	// Anthropic extended thinking on Bedrock). Empty when reasoning is
	// disabled or unsupported.
	Reasoning string

	// ToolCalls requested by the model in the final step.
	ToolCalls []provider.ToolCall

	// Steps contains results from each generation step (for multi-step tool loops).
	Steps []StepResult

	// TotalUsage is the aggregated token usage across all steps.
	TotalUsage provider.Usage

	// FinishReason indicates why generation stopped.
	FinishReason provider.FinishReason

	// Response contains provider metadata from the last step (ID, Model).
	Response provider.ResponseMetadata

	// ProviderMetadata contains provider-specific response data from the last step
	// (e.g. logprobs, prediction tokens).
	ProviderMetadata map[string]map[string]any

	// Sources contains citations/references extracted from the response.
	Sources []provider.Source

	// StepsExhausted is true when the tool loop terminated because MaxSteps was reached
	// while the model still requested tool calls. This distinguishes "model finished
	// naturally" (StepsExhausted=false) from "loop was cut short" (StepsExhausted=true).
	StepsExhausted bool

	// ResponseMessages contains the assistant and tool messages from all generation steps.
	// For multi-turn conversations, append these to your message history:
	//   messages = append(messages, result.ResponseMessages...)
	//
	// Nil when the response has no content (empty text and no tool calls).
	// For StreamText, check Err() before using , on stream errors, ResponseMessages
	// may be partial (intermediate tool round-trips lost) or reflect only completed
	// steps. Do not use ResponseMessages for conversation continuation when Err() != nil.
	// Reasoning parts (PartReasoning) are included for StreamText (both single-step
	// and multi-step) but not for GenerateText (which does not expose reasoning).
	// Reasoning chunks are consolidated into a single PartReasoning part with merged
	// metadata (e.g. Anthropic/Bedrock signatures).
	ResponseMessages []provider.Message
}

TextResult is the final result of a text generation call.

func GenerateText 

func GenerateText(ctx context.Context, model provider.LanguageModel, opts ...Option) (_ *TextResult, err error)

GenerateText performs a non-streaming text generation. When tools with Execute functions are provided and MaxSteps > 1, it automatically runs a tool loop: generate → execute tools → re-generate.

type TextStream 

type TextStream struct {
	// contains filtered or unexported fields
}

TextStream is a streaming text generation response.

Callers must consume the stream (via Stream, TextStream, or Result) or cancel the context. Discarding a TextStream without consuming leaks goroutines.

It provides three consumption modes (Stream, TextStream, Result). Stream() and TextStream() are mutually exclusive - only call one. Result() can always be called, including after Stream() or TextStream(), to get the accumulated final result.

func StreamText 

func StreamText(ctx context.Context, model provider.LanguageModel, opts ...Option) (_ *TextStream, err error)

StreamText performs a streaming text generation. When MaxSteps > 1 and executable tools are provided, StreamText runs an automatic tool loop. The initial DoStream failure still returns (nil, error). Subsequent step errors flow through the stream as ChunkError chunks; check stream.Err() after consuming.

func (*TextStream) Err added in v0.4.4 

func (ts *TextStream) Err() error

Err returns the first stream error encountered, or nil. Must be called after the stream is fully consumed (after Result(), or after the Stream()/TextStream() channel is drained). Follows the bufio.Scanner.Err() pattern.

func (*TextStream) Result 

func (ts *TextStream) Result() *TextResult

Result blocks until the stream completes and returns the accumulated result. Check Err() after Result() to detect stream errors - Result does not surface errors directly (use Err or check result.Steps for partial data). Can be called after Stream() or TextStream() to get accumulated data. Note: unlike ObjectStream.Result(), this method does not return an error. Call Err() after Result() to check for stream errors.

func (*TextStream) Stream 

func (ts *TextStream) Stream() <-chan provider.StreamChunk

Stream returns a channel that emits raw StreamChunks from the provider. Mutually exclusive with TextStream() - only call one streaming method.

func (*TextStream) TextStream 

func (ts *TextStream) TextStream() <-chan string

TextStream returns the underlying channel of text chunks. Note: this method has the same name as the containing type (TextStream); call it as stream.TextStream() to receive the channel. Mutually exclusive with Stream() - only call one streaming method.

type Tool 

type Tool struct {
	// Name is the tool's identifier.
	Name string

	// Description explains what the tool does.
	Description string

	// InputSchema is the JSON Schema for the tool's input parameters.
	InputSchema json.RawMessage

	// ProviderDefinedType, when non-empty, marks this as a provider-defined tool
	// (e.g. "computer_20250124", "bash_20250124"). Providers emit the correct
	// API type instead of "custom".
	ProviderDefinedType string

	// ProviderDefinedOptions holds provider-specific tool configuration
	// (e.g. displayWidthPx for computer use).
	ProviderDefinedOptions map[string]any

	// Execute runs the tool with the given JSON input and returns the result text.
	// Both the return value and error string are forwarded to the model as a tool
	// result message. Do not include sensitive data (credentials, internal paths)
	// in error messages as they will be sent to the LLM provider's API.
	Execute func(ctx context.Context, input json.RawMessage) (string, error)
}

Tool defines a tool that can be called by the model during generation. Unlike provider.ToolDefinition (wire-level schema), Tool includes an Execute function that GoAI's auto tool loop invokes.

func NewTool added in v0.8.0 

func NewTool[In any](name, description string, execute func(ctx context.Context, input In) (string, error)) Tool

NewTool builds a Tool from a typed input struct and a typed execute function. The JSON Schema is generated from In via SchemaFrom, and the raw JSON arguments from the model are unmarshaled into In before execute runs - so callers neither hand-write JSON Schema nor unmarshal input themselves.

In is typically a struct whose exported fields carry json and jsonschema tags (see SchemaFrom for the supported tags). Use struct{} for a tool that takes no parameters.

If the model sends arguments that do not unmarshal into In, execute is not called and the error is returned to the model as the tool result. Empty arguments leave In at its zero value. 

weather := goai.NewTool("get_weather", "Get the weather for a city",
	func(ctx context.Context, in struct {
		City string `json:"city" jsonschema:"description=City name"`
	}) (string, error) {
		return forecast(in.City), nil
	})

type ToolCallInfo 

type ToolCallInfo struct {
	// ToolCallID is the provider-assigned identifier for this tool call.
	ToolCallID string

	// ToolName is the name of the tool that was called.
	ToolName string

	// Step is the 1-based index of the generation step in which this tool was called.
	Step int

	// Input is the raw JSON arguments passed to the tool.
	// Replaces the former InputSize int field --use len(Input) for byte size.
	Input json.RawMessage

	// Output is the string result returned by the tool.
	// Typically empty when Error is non-nil, but tools may return both output and error
	// (e.g., partial output on timeout). Empty for unknown tools (ErrUnknownTool).
	// Note: when OnBeforeToolExecute skips with both Result and Error set,
	// Output contains the Result string but the LLM receives "error: <message>"
	// (Error takes precedence in the message sent to the model).
	Output string

	// OutputObject is the parsed JSON value of Output when the tool returned valid JSON.
	// Nil if the output is not valid JSON or the tool returned an error.
	// The dynamic type follows json.Unmarshal rules: JSON objects become map[string]any,
	// arrays become []any, numbers become float64, booleans become bool, strings become string.
	OutputObject any

	// StartTime is when the tool execution began.
	// Zero for unknown tools (ErrUnknownTool). For skipped tools (Skipped=true),
	// reflects when the skip decision was made, not execution start.
	StartTime time.Time

	// Duration measures time from before Execute to after OnAfterToolExecute
	// (if registered). Includes both tool execution and after-hook overhead.
	Duration time.Duration

	// Skipped is true when the tool execution was skipped by OnBeforeToolExecute.
	// When Skipped is true, StartTime reflects when the skip decision was made
	// (not when execution started) and Duration is zero.
	Skipped bool

	// Error is non-nil if the tool execution failed.
	Error error

	// Metadata is opaque consumer data set by OnAfterToolExecute.
	// Nil if no OnAfterToolExecute hook is registered or if the hook
	// returned nil Metadata.
	Metadata map[string]any
}

ToolCallInfo is passed to the OnToolCall hook after a tool executes. It contains the full tool Input and Output, which may include sensitive data. Consumers that log or export hook data should sanitize accordingly.

type ToolCallStartInfo added in v0.5.4 

type ToolCallStartInfo struct {
	// ToolCallID is the provider-assigned identifier for this tool call.
	ToolCallID string

	// ToolName is the name of the tool about to execute.
	ToolName string

	// Step is the 1-based index of the generation step in which this tool was called.
	Step int

	// Input is the raw JSON arguments that will be passed to the tool.
	Input json.RawMessage
}

ToolCallStartInfo is passed to the OnToolCallStart hook before a tool executes.

Source Files

View all Source files

Directories

Path Synopsis
examples
nvidia-chat command
internal
gemini
Package gemini provides shared utilities for Google Gemini API providers.
 Package gemini provides shared utilities for Google Gemini API providers.
httpc
Package httpc provides HTTP helper functions for provider implementations.
 Package httpc provides HTTP helper functions for provider implementations.
openaicompat
Package openaicompat provides shared request building and response parsing for OpenAI-compatible API providers (OpenAI, OpenRouter, Groq, DeepInfra, etc.).
 Package openaicompat provides shared request building and response parsing for OpenAI-compatible API providers (OpenAI, OpenRouter, Groq, DeepInfra, etc.).
sse
Package sse provides a scanner for Server-Sent Events (SSE) streams.
 Package sse provides a scanner for Server-Sent Events (SSE) streams.
Package mcp implements a Model Context Protocol (MCP) client for GoAI.
 Package mcp implements a Model Context Protocol (MCP) client for GoAI.
observability
langfuse module
otel module
Package provider defines the interfaces and types that AI providers implement.
 Package provider defines the interfaces and types that AI providers implement.
anthropic
Package anthropic provides an Anthropic language model implementation for GoAI.
 Package anthropic provides an Anthropic language model implementation for GoAI.
azure
Package azure provides an Azure OpenAI language model implementation for GoAI.
 Package azure provides an Azure OpenAI language model implementation for GoAI.
bedrock
Package bedrock provides an AWS Bedrock language model implementation for GoAI.
 Package bedrock provides an AWS Bedrock language model implementation for GoAI.
cerebras
Package cerebras provides a cerebras language model implementation for GoAI.
 Package cerebras provides a cerebras language model implementation for GoAI.
cloudflare
Package cloudflare provides a Cloudflare Workers AI language and embedding model implementation for GoAI, using the OpenAI-compatible endpoints.
 Package cloudflare provides a Cloudflare Workers AI language and embedding model implementation for GoAI, using the OpenAI-compatible endpoints.
cohere
Package cohere provides a Cohere language model and embedding implementation for GoAI.
 Package cohere provides a Cohere language model and embedding implementation for GoAI.
compat
Package compat provides a generic OpenAI-compatible language model for GoAI.
 Package compat provides a generic OpenAI-compatible language model for GoAI.
deepinfra
Package deepinfra provides a DeepInfra language model implementation for GoAI.
 Package deepinfra provides a DeepInfra language model implementation for GoAI.
deepseek
Package deepseek provides a deepseek language model implementation for GoAI.
 Package deepseek provides a deepseek language model implementation for GoAI.
fireworks
Package fireworks provides a fireworks language model implementation for GoAI.
 Package fireworks provides a fireworks language model implementation for GoAI.
fptcloud
Package fptcloud provides an FPT Smart Cloud AI Marketplace language and embedding model implementation for GoAI, using the OpenAI-compatible endpoints.
 Package fptcloud provides an FPT Smart Cloud AI Marketplace language and embedding model implementation for GoAI, using the OpenAI-compatible endpoints.
google
Package google provides a Google Gemini language model implementation for GoAI.
 Package google provides a Google Gemini language model implementation for GoAI.
groq
Package groq provides a groq language model implementation for GoAI.
 Package groq provides a groq language model implementation for GoAI.
llamacpp
Package llamacpp provides a llama.cpp server language model implementation for GoAI.
 Package llamacpp provides a llama.cpp server language model implementation for GoAI.
minimax
Package minimax provides a MiniMax language model implementation for GoAI.
 Package minimax provides a MiniMax language model implementation for GoAI.
mistral
Package mistral provides a mistral language model implementation for GoAI.
 Package mistral provides a mistral language model implementation for GoAI.
nvidia
Package nvidia provides an NVIDIA NIM language and embedding model implementation for GoAI, using the OpenAI-compatible endpoints.
 Package nvidia provides an NVIDIA NIM language and embedding model implementation for GoAI, using the OpenAI-compatible endpoints.
ollama
Package ollama provides a native Ollama language model implementation for GoAI.
 Package ollama provides a native Ollama language model implementation for GoAI.
openai
Package openai provides an OpenAI language model implementation for GoAI.
 Package openai provides an OpenAI language model implementation for GoAI.
openrouter
Package openrouter provides an OpenRouter language model implementation for GoAI.
 Package openrouter provides an OpenRouter language model implementation for GoAI.
perplexity
Package perplexity provides a perplexity language model implementation for GoAI.
 Package perplexity provides a perplexity language model implementation for GoAI.
requesty
Package requesty provides a Requesty language model implementation for GoAI.
 Package requesty provides a Requesty language model implementation for GoAI.
runpod
Package runpod provides a RunPod language model implementation for GoAI.
 Package runpod provides a RunPod language model implementation for GoAI.
together
Package together provides a together language model implementation for GoAI.
 Package together provides a together language model implementation for GoAI.
vertex
Package vertex provides a Google Cloud Vertex AI language model implementation for GoAI.
 Package vertex provides a Google Cloud Vertex AI language model implementation for GoAI.
vllm
Package vllm provides a vLLM language model implementation for GoAI.
 Package vllm provides a vLLM language model implementation for GoAI.
xai
Package xai provides a xai language model implementation for GoAI.
 Package xai provides a xai language model implementation for GoAI.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.