ai

package
v1.12.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 17, 2026 License: Apache-2.0 Imports: 34 Imported by: 154

Documentation

Overview

Package ai defines Genkit's AI primitives: models, prompts, tools, embedders, retrievers, and evaluators, and the options that configure a request to them.

Applications reach these through the genkit package, whose Define* and Generate* functions register with a github.com/firebase/genkit/go/genkit.Genkit and forward here. Plugins use this package directly: New*Action builds an unregistered primitive to return from a plugin's Init.

Options

Options follow the standard Go functional-options pattern: pass as many as you like, in any order, and they merge left to right. Two rules govern how repeats combine, so composing a request from several helpers is predictable:

A zero value does not fill a slot: WithMaxTurns(0), WithToolChoice(""), or WithConfig(nil) is a no-op, so an earlier non-zero value cannot be un-set by a later zero one. The rules apply within a single options list; APIs that layer two lists (a prompt's define-time options against Execute-time options) document their own precedence.

One combination is refused rather than merged. WithMessagesTemplate lays out the whole conversation, down to where {{history}} puts the caller's, so separately supplied messages have no position relative to it. Passing it to DefinePrompt alongside WithMessages or WithMessagesFn panics. Repeating the template alone is an ordinary slot.

Applying options therefore never fails on a "set more than once" conflict. What does fail is a genuinely invalid argument (a type that WithInputType cannot turn into a schema) or the refused combination above, and both panic at the call site where the mistake is rather than deferring to the request.

Errors

Failures are classified with the sentinels in github.com/firebase/genkit/go/core/status, so callers branch with errors.Is rather than on message text:

if errors.Is(err, ai.ErrMaxTurnsExceeded) { ... } // specific
if errors.Is(err, status.ErrAborted) { ... }      // broad

Writing a plugin

A provider plugin builds primitives with the New*Action constructors and returns them from its Init. ModelOptions and its siblings describe what a model can do; the constructor copies what it is given, so a table of models may share one ModelSupports value. ModelOptions.Overlay is how a plugin lets an application correct that description without restating it: a zero-value field in the override keeps what the plugin already knows.

Index

Examples

Constants

View Source
const (
	// OutputFormatText is the default format.
	OutputFormatText string = "text"
	// OutputFormatJSON is the legacy format for JSON content.
	// For streaming, each chunk represents the full object received up to that point.
	OutputFormatJSON string = "json"
	// OutputFormatJSONL is the format for JSONL content.
	//
	// For streaming, each chunk carries the objects whose line finished since
	// the last chunk. A line still being written is held back until it parses,
	// so no object is ever handed over incomplete or handed over twice.
	OutputFormatJSONL string = "jsonl"
	// OutputFormatMedia is the format for media content.
	OutputFormatMedia string = "media"
	// OutputFormatArray is the format for array content.
	//
	// For streaming, each chunk carries the elements that finished since the
	// last chunk. An element still being written is held back until it parses,
	// so no element is ever handed over incomplete or handed over twice.
	OutputFormatArray string = "array"
	// OutputFormatEnum is the format for enum content.
	// The value must be a string.
	OutputFormatEnum string = "enum"
)

Variables

View Source
var (
	// ErrModelNotFound means the named model is not registered. Usually the
	// providing plugin is missing from genkit.Init.
	ErrModelNotFound = status.ErrNotFound.Subtype("model not found")

	// ErrToolNotFound means the named tool is not registered, either on the
	// request or in the registry the model's tool call resolved against.
	ErrToolNotFound = status.ErrNotFound.Subtype("tool not found")

	// ErrMaxTurnsExceeded means the tool-calling loop hit its turn limit before
	// the model produced a final response. Raise the limit with WithMaxTurns, or
	// look for a tool the model keeps retrying.
	ErrMaxTurnsExceeded = status.ErrAborted.Subtype("max turns exceeded")

	// ErrToolFailed means a tool returned an error or produced output that does
	// not match its declared schema. The tool's own error is wrapped, so
	// errors.Is and errors.As still reach it; the status is INTERNAL because a
	// tool's failure is not a failure of the caller's request.
	ErrToolFailed = status.ErrInternal.Subtype("tool failed")

	// ErrUnsupportedByModel means the request used a capability the model does
	// not advertise (media, tools, tool choice, a system role, ...).
	ErrUnsupportedByModel = status.ErrInvalidArgument.Subtype("unsupported by model")

	// ErrInvalidPart means a Part is malformed for the operation at hand: the
	// wrong kind, missing a required field, or carrying a field its kind does
	// not allow.
	ErrInvalidPart = status.ErrInvalidArgument.Subtype("invalid part")

	// ErrInputTypeMismatch means a prompt's input could not be interpreted as
	// the type a content function declared, so the function was never called.
	// The input may have come from [WithInput], the default recorded by
	// [WithInputType], or the wire.
	ErrInputTypeMismatch = status.ErrInvalidArgument.Subtype("input type mismatch")

	// ErrUnresolvedToolRequest means a resumed generation left an interrupted
	// tool request without a Respond or Restart directive.
	ErrUnresolvedToolRequest = status.ErrInvalidArgument.Subtype("unresolved tool request")
)

Failure modes generation reports. Match them with errors.Is rather than by inspecting message text:

if errors.Is(err, ai.ErrMaxTurnsExceeded) { ... }

Each also matches the base sentinel it derives from, so errors.Is(err, status.ErrNotFound) still catches a missing model or tool.

View Source
var DEFAULT_FORMATS = []Formatter{
	textFormatter{},
	jsonFormatter{},
	jsonlFormatter{},
	arrayFormatter{},
	enumFormatter{},
}

Default formats get automatically registered on registry init

Functions

func CalculateInputOutputUsage added in v1.3.0

func CalculateInputOutputUsage(req *ModelRequest, resp *ModelResponse)

CalculateInputOutputUsage fills in the character, image, and video counts on resp.Usage, for providers whose API reports token counts only.

func ConfigureFormats added in v0.5.0

func ConfigureFormats(reg api.Registry)

ConfigureFormats registers default formats in the registry

func DefineFormats added in v1.12.0

func DefineFormats(r api.Registry, formatters ...Formatter)

DefineFormats registers each Formatter under the name returned by its Name method. It panics if a format with that name is already registered, which includes the built-in formats registered by ConfigureFormats, so a format cannot be replaced once defined.

func DefineGenerateAction added in v0.3.0

func DefineGenerateAction(ctx context.Context, r api.Registry) *generateAction

DefineGenerateAction defines a utility generate action.

func FindMatchingResource added in v0.7.0

func FindMatchingResource(r api.Registry, uri string) (Resource, *ResourceInput, error)

FindMatchingResource finds a resource that matches the given URI.

func GenerateDataStream added in v1.3.0

func GenerateDataStream[Out any](ctx context.Context, r api.Registry, opts ...GenerateOption) iter.Seq2[*StreamValue[Out, Out], error]

GenerateDataStream generates a model response with streaming and returns strongly-typed output. It returns an iterator that yields streaming results.

If the yield function is passed a non-nil error, generation has failed with that error; the yield function will not be called again.

If the yield function's StreamValue argument has Done == true, the value's Output and Response fields contain the final typed output and response; the yield function will not be called again.

Otherwise the Chunk field of the passed StreamValue holds a streamed chunk.

Like GenerateData, the output format is JSON with a schema inferred from Out; overriding the format with a non-JSON WithOutputFormat or WithOutputEnums breaks typed extraction.

func GenerateStream added in v1.3.0

func GenerateStream(ctx context.Context, r api.Registry, opts ...GenerateOption) iter.Seq2[*ModelStreamValue, error]

GenerateStream generates a model response and streams the output. It returns an iterator that yields streaming results.

If the yield function is passed a non-nil error, generation has failed with that error; the yield function will not be called again.

If the yield function's ModelStreamValue argument has Done == true, the value's Response field contains the final response; the yield function will not be called again.

Otherwise the Chunk field of the passed ModelStreamValue holds a streamed chunk.

func GenerateText added in v0.1.0

func GenerateText(ctx context.Context, r api.Registry, opts ...GenerateOption) (string, error)

GenerateText run generate request for this model. Returns generated text only.

func InterruptAs added in v1.4.0

func InterruptAs[T any](p *Part) (T, bool)

InterruptAs extracts strongly-typed metadata from an interrupted tool request Part. Returns the zero value and false if the part is not an interrupt or the type doesn't match.

func InterruptWith added in v1.4.0

func InterruptWith[T any](tc *ToolContext, meta T) error

InterruptWith is a convenience function to interrupt a tool with a strongly-typed metadata value. The metadata is converted to map[string]any via JSON marshaling.

func IsAudioContentType added in v1.0.0

func IsAudioContentType(contentType string) bool

IsAudioContentType checks if the content type represents an audio file.

func IsImageContentType added in v1.0.0

func IsImageContentType(contentType string) bool

IsImageContentType checks if the content type represents an image.

func IsToolInterruptError added in v1.1.0

func IsToolInterruptError(err error) (bool, map[string]any)

IsToolInterruptError determines whether the error is an interrupt error returned by the tool.

func IsToolResumed added in v1.7.0

func IsToolResumed(ctx context.Context) bool

IsToolResumed reports whether the current context is a resumed tool execution. This is intended for use in middleware that needs to distinguish between first-time and restarted tool calls.

func IsVideoContentType added in v1.0.0

func IsVideoContentType(contentType string) bool

IsVideoContentType checks if the content type represents a video.

func LoadPromptDir added in v0.3.0

func LoadPromptDir(r api.Registry, dir string, namespace string)

LoadPromptDir loads prompts and partials from a directory on the local filesystem.

func LoadPromptDirFromFS added in v1.3.0

func LoadPromptDirFromFS(r api.Registry, fsys fs.FS, dir, namespace string)

LoadPromptDirFromFS loads prompts and partials from a filesystem for the given namespace. The fsys parameter should be an fs.FS implementation (e.g., embed.FS or os.DirFS). The dir parameter specifies the directory within the filesystem where prompts are located.

func NewHistoryContext added in v1.12.0

func NewHistoryContext(ctx context.Context, messages []*Message) context.Context

NewHistoryContext returns ctx carrying a conversation for the next Prompt.Render call to place, and is what HistoryFromContext reads back.

Prompt.Execute does this itself, so its callers never need it. It is for code that drives a prompt by hand, pairing Render with GenerateWithRequest, and still wants the prompt to decide where the conversation goes. The agent runtime is the main example.

Scope the returned context to the Render call. Generation should run on the original, so the conversation does not ride along into tool handlers and prompts executed inside the generate loop.

The messages are not copied; Render clones what it places into the request. Nil entries are dropped.

func NewToolInterruptError added in v1.7.0

func NewToolInterruptError(metadata map[string]any) error

NewToolInterruptError creates a tool interrupt error with the given metadata. This is intended for use in middleware that needs to interrupt tool execution without calling the tool itself.

func OriginalInputAs added in v1.4.0

func OriginalInputAs[T any](tc *ToolContext) (T, bool)

OriginalInputAs returns the original input typed appropriately. Returns the zero value and false if not resumed or type doesn't match.

func OutputFrom added in v1.3.0

func OutputFrom[Out any](src outputer) Out

OutputFrom is a convenience function that parses structured output from a ModelResponse or ModelResponseChunk and returns it as a typed value. This is equivalent to calling Output() but returns the value directly instead of requiring a pointer argument. If you need to handle the error, use Output() instead.

func ResumedValue added in v1.4.0

func ResumedValue[T any](ctx context.Context, key string) (T, bool)

ResumedValue retrieves a typed value from the resumed metadata on ctx. Returns the zero value and false if the key doesn't exist or the type doesn't match. Accepts either a plain context.Context (useful in middleware) or a *ToolContext, which embeds context.Context.

Types

type ActionMetadata added in v1.5.1

type ActionMetadata struct {
	ActionType  string `json:"actionType,omitempty"`
	Description string `json:"description,omitempty"`
	// A JSON Schema Draft 7 (http://json-schema.org/draft-07/schema) object.
	InputJsonSchema any            `json:"inputJsonSchema,omitempty"`
	InputSchema     any            `json:"inputSchema,omitempty"`
	Key             string         `json:"key,omitempty"`
	Metadata        map[string]any `json:"metadata,omitempty"`
	Name            string         `json:"name,omitempty"`
	// A JSON Schema Draft 7 (http://json-schema.org/draft-07/schema) object.
	OutputJsonSchema any `json:"outputJsonSchema,omitempty"`
	OutputSchema     any `json:"outputSchema,omitempty"`
	StreamSchema     any `json:"streamSchema,omitempty"`
}

type AugmentWithContextOptions added in v0.3.0

type AugmentWithContextOptions struct {
	Preface      *string                                                                // Preceding text to place before the rendered context documents.
	ItemTemplate func(d Document, index int, options *AugmentWithContextOptions) string // A function to render a document into a text part to be included in the message.
	CitationKey  *string                                                                // Metadata key to use for citation reference. Pass `nil` to provide no citations.
}

AugmentWithContextOptions configures how a request is augmented with context.

type BackgroundModel added in v1.3.0

type BackgroundModel interface {
	// Name returns the registry name of the background model.
	Name() string
	// Register registers the model with the given registry.
	Register(r api.Registry)
	// Start starts a background operation.
	Start(ctx context.Context, req *ModelRequest) (*ModelOperation, error)
	// Check checks the status of a background operation.
	Check(ctx context.Context, op *ModelOperation) (*ModelOperation, error)
	// Cancel cancels a background operation.
	Cancel(ctx context.Context, op *ModelOperation) (*ModelOperation, error)
	// SupportsCancel returns whether the background action supports cancellation.
	SupportsCancel() bool
}

BackgroundModel represents a model that can run operations in the background. It is the type to accept as an argument and to look up by name; implementations are created with NewBackgroundModelAction, or [genkit.DefineBackgroundModelAction] in an application.

func LookupBackgroundModel added in v1.3.0

func LookupBackgroundModel(r api.Registry, name string) BackgroundModel

LookupBackgroundModel looks up a registered BackgroundModel by name. It returns nil if the background model was not found.

func NewBackgroundModel deprecated added in v1.3.0

func NewBackgroundModel(name string, opts *BackgroundModelOptions, startFn StartModelOpFunc, checkFn CheckModelOpFunc) BackgroundModel

NewBackgroundModel defines a new model that runs in the background.

Deprecated: Use NewBackgroundModelAction, which passes the request's config to startFn as a typed value instead of leaving it type-erased on the request.

type BackgroundModelAction added in v1.12.0

type BackgroundModelAction struct {
	// contains filtered or unexported fields
}

BackgroundModelAction is a background model backed by registry actions. It is the concrete type returned by NewBackgroundModelAction; return it from a plugin's Init for the framework to register.

It implements BackgroundModel and api.Action, so it can be passed anywhere either is accepted. The api.Action side is what lets a plugin resolver return the whole model as the resolved action (googlegenai's resolveAction does this), and the three component actions register together through BackgroundModelAction.Register.

func NewBackgroundModelAction added in v1.12.0

func NewBackgroundModelAction[Config any](
	name string,
	opts *BackgroundModelOptions,
	startFn BackgroundModelActionFunc[Config],
	checkFn CheckModelOpFunc,
) *BackgroundModelAction

NewBackgroundModelAction creates an unregistered BackgroundModelAction: return it from a plugin's Init for the framework to register, or call BackgroundModelAction.Register directly. Applications should define background models with [genkit.DefineBackgroundModelAction].

Config is the model's typed configuration; it is usually inferred from startFn's signature. See NewModelAction for how the request's config is deserialized.

func (*BackgroundModelAction) Cancel added in v1.12.0

Cancel cancels a running background operation. It fails with UNAVAILABLE if the model does not support cancellation; see BackgroundModelAction.SupportsCancel.

func (*BackgroundModelAction) Check added in v1.12.0

Check returns the current state of a background operation.

func (*BackgroundModelAction) Desc added in v1.12.0

Desc returns the start action's descriptor: its name, schemas, and metadata.

func (*BackgroundModelAction) Name added in v1.12.0

func (b *BackgroundModelAction) Name() string

Name returns the registry name of the background model.

func (*BackgroundModelAction) Register added in v1.12.0

func (b *BackgroundModelAction) Register(r api.Registry)

Register registers the model's start, check, and cancel actions with r. The cancel action is registered only if the model supports cancellation. A plugin that returns the model from its Init does not need to call this.

func (*BackgroundModelAction) RunJSON added in v1.12.0

RunJSON starts an operation from a JSON-encoded ModelRequest and returns the JSON-encoded operation. The framework uses it to serve reflection and registry-driven calls; prefer BackgroundModelAction.Start.

func (*BackgroundModelAction) RunJSONWithTelemetry added in v1.12.0

RunJSONWithTelemetry is BackgroundModelAction.RunJSON with the run's telemetry returned alongside the output.

func (*BackgroundModelAction) Start added in v1.12.0

Start starts a background operation and returns it without waiting for completion. Poll it with BackgroundModelAction.Check.

func (*BackgroundModelAction) SupportsCancel added in v1.12.0

func (b *BackgroundModelAction) SupportsCancel() bool

SupportsCancel reports whether the model was defined with a cancel function.

type BackgroundModelActionFunc added in v1.12.0

type BackgroundModelActionFunc[Config any] = func(ctx context.Context, req *ModelRequest, config Config) (*ModelOperation, error)

BackgroundModelActionFunc is a StartModelOpFunc that additionally receives the request's typed Config: the framework deserializes the request's raw config into it before calling the function (see NewBackgroundModelAction).

type BackgroundModelOptions added in v1.3.0

type BackgroundModelOptions struct {
	ModelOptions

	// Cancel cancels a running operation. Optional: nil means the model does
	// not support canceling operations.
	Cancel CancelModelOpFunc

	// Metadata is arbitrary key-value data attached to the action descriptor.
	// It is merged over [ModelOptions.Metadata]; this field wins on key
	// conflicts.
	Metadata map[string]any
}

BackgroundModelOptions configures a background model created with NewBackgroundModelAction. It extends ModelOptions with the operation lifecycle hooks a background model needs; the required start and check functions are constructor arguments.

type BatchEvaluatorActionFunc added in v1.12.0

type BatchEvaluatorActionFunc[Config any] = func(context.Context, *EvaluatorRequest, Config) (*EvaluatorResponse, error)

BatchEvaluatorActionFunc is a BatchEvaluatorFunc that additionally receives the request's typed Config: the framework deserializes the request's raw options into it before calling the function (see NewBatchEvaluatorAction).

type BatchEvaluatorFunc added in v0.7.0

type BatchEvaluatorFunc = func(context.Context, *EvaluatorRequest) (*EvaluatorResponse, error)

BatchEvaluatorFunc is the function type for batch evaluator implementations.

type CancelModelOpFunc added in v1.3.0

type CancelModelOpFunc = func(ctx context.Context, op *ModelOperation) (*ModelOperation, error)

CancelModelOpFunc cancels a background model operation.

type CheckModelOpFunc added in v1.3.0

type CheckModelOpFunc = func(ctx context.Context, op *ModelOperation) (*ModelOperation, error)

CheckModelOpFunc checks the status of a background model operation.

type CommonGenOption added in v0.5.0

type CommonGenOption interface {
	PromptOption
	GenerateOption
	PromptExecuteOption
	// contains filtered or unexported methods
}

CommonGenOption is an option common to model generation, prompt definition, and prompt execution.

func WithMaxTurns added in v0.3.0

func WithMaxTurns(maxTurns int) CommonGenOption

WithMaxTurns sets the maximum number of tool call iterations before erroring. A tool call happens when tools are provided in the request and a model decides to call one or more as a response. Each round trip, including multiple tools in parallel, counts as one turn.

func WithMessages added in v0.1.0

func WithMessages(messages ...*Message) CommonGenOption

WithMessages adds messages to the request, placed between the system and user prompts. Repeating this option, or mixing it with WithMessagesFn, appends: messages accumulate in the order the options are passed.

Message text is used verbatim, never compiled as a dotprompt template, so history containing literal braces passes through untouched. To build message text from a prompt's input, use WithMessagesFn, which receives the typed input, or WithMessagesTemplate for a multi-turn template.

Declaring the conversation makes the prompt responsible for the messages passed to Prompt.Execute: they are not spliced in automatically, because only the prompt knows where its examples end and a real conversation begins. Place them with {{history}} in a WithMessagesTemplate or HistoryFromContext in a WithMessagesFn.

func WithMessagesFn added in v0.3.0

func WithMessagesFn[In any](fn func(context.Context, In) ([]*Message, error)) CommonGenOption

WithMessagesFn adds messages produced by fn at request time, placed between the system and user prompts. Like WithMessages, repeating this option (or mixing the two) appends the produced messages in call order.

fn receives the prompt's input converted to In, or the zero value of In when there is none, as at Generate. Its messages are used verbatim, as with WithMessages.

Those messages are the conversation the prompt declares, so messages supplied at execution time are not appended on top of them. fn reads them from HistoryFromContext instead, and can summarize or truncate them rather than only prepend to them.

func WithMiddleware deprecated added in v0.3.0

func WithMiddleware(middleware ...ModelMiddleware) CommonGenOption

WithMiddleware adds middleware to apply to the model request. Repeating this option appends to the chain.

Deprecated: Use WithUse instead, which supports Generate, Model, and Tool hooks.

func WithModel added in v0.3.0

func WithModel(model ModelArg) CommonGenOption

WithModel sets either a Model or a ModelRef that may contain a config. Passing WithConfig will take precedence over the config in WithModel. Repeating this option, or mixing it with WithModelName, takes the last model set.

func WithModelName added in v0.3.0

func WithModelName(name string) CommonGenOption

WithModelName sets the model name to call for generation. The model name will be resolved to a Model and may error if the reference is invalid. Repeating this option, or mixing it with WithModel, takes the last model set.

func WithResources added in v0.7.0

func WithResources(resources ...Resource) CommonGenOption

WithResources specifies resources to be temporarily available during generation. Repeating this option appends. Resources are unregistered resources that get attached to a temporary registry during the generation request and cleaned up afterward.

func WithReturnToolRequests added in v0.3.0

func WithReturnToolRequests(returnReqs bool) CommonGenOption

WithReturnToolRequests configures whether to return tool requests instead of making the tool calls and continuing the generation.

func WithToolChoice added in v0.3.0

func WithToolChoice(toolChoice ToolChoice) CommonGenOption

WithToolChoice configures whether by default tool calls are required, disabled, or optional for the prompt.

func WithTools added in v0.1.0

func WithTools(tools ...ToolRef) CommonGenOption

WithTools adds tools to use for the generate request. Repeating this option appends; duplicate tools (by name) are rejected when the request runs.

func WithUse added in v1.7.0

func WithUse(middleware ...Middleware) CommonGenOption

WithUse adds middleware to apply to generation. Middleware hooks wrap the generate loop, model calls, and tool executions. Repeating this option appends to the chain.

Accepts either a middleware config struct (produced by a plugin) or an inline adapter via MiddlewareFunc. The chain applies outer-to-inner, so WithUse(A, B) expands to A { B { ... } }.

type ConfigOption added in v0.5.0

type ConfigOption interface {
	CommonGenOption
	EmbedderOption
	RetrieverOption
	EvaluatorOption
	// contains filtered or unexported methods
}

ConfigOption is an option for model configuration. It is accepted anywhere a primitive takes a config: generation, prompt definition and execution, embedding, retrieval, and evaluation.

func WithConfig added in v0.1.0

func WithConfig(config any) ConfigOption

WithConfig sets the configuration. Repeating this option takes the last config set.

type ConstrainedSupport added in v0.3.0

type ConstrainedSupport string

ConstrainedSupport indicates the level of constrained generation support.

const (
	ConstrainedSupportNone    ConstrainedSupport = "none"
	ConstrainedSupportAll     ConstrainedSupport = "all"
	ConstrainedSupportNoTools ConstrainedSupport = "no-tools"
)

type DataPrompt added in v1.3.0

type DataPrompt[In, Out any] struct {
	// contains filtered or unexported fields
}

DataPrompt is a prompt with strongly-typed input and output. It wraps an underlying Prompt and provides type-safe Execute and Render methods. The Out type parameter can be string for text outputs or any struct type for JSON outputs.

func AsDataPrompt added in v1.3.0

func AsDataPrompt[In, Out any](p Prompt) *DataPrompt[In, Out]

AsDataPrompt wraps an existing Prompt with type information, returning a DataPrompt. This is useful for adding strong typing to a dynamically obtained prompt.

func DefineDataPrompt added in v1.3.0

func DefineDataPrompt[In, Out any](r api.Registry, name string, opts ...PromptOption) *DataPrompt[In, Out]

DefineDataPrompt creates a new data prompt and registers it. It automatically infers input schema from the In type parameter and configures output schema and JSON format from the Out type parameter (unless Out is string).

func LookupDataPrompt added in v1.3.0

func LookupDataPrompt[In, Out any](r api.Registry, name string) *DataPrompt[In, Out]

LookupDataPrompt looks up a prompt by name and wraps it with type information. This is useful for wrapping prompts loaded from .prompt files with strong types. It returns nil if the prompt was not found.

func (*DataPrompt) Desc added in v1.3.0

func (p *DataPrompt) Desc() api.ActionDesc

Desc returns a descriptor of the prompt with resolved schema references.

func (*DataPrompt[In, Out]) Execute added in v1.3.0

func (dp *DataPrompt[In, Out]) Execute(ctx context.Context, input In, opts ...PromptExecuteOption) (Out, *ModelResponse, error)

Execute executes the typed prompt and returns the strongly-typed output along with the full model response. For structured output types (non-string Out), the prompt must be configured with the appropriate output schema, either through DefineDataPrompt or by using WithOutputType when defining the prompt. The typed input argument fills the input slot last, so it wins over any WithInput passed in opts.

func (*DataPrompt[In, Out]) ExecuteStream added in v1.3.0

func (dp *DataPrompt[In, Out]) ExecuteStream(ctx context.Context, input In, opts ...PromptExecuteOption) iter.Seq2[*StreamValue[Out, Out], error]

ExecuteStream executes the typed prompt with streaming and returns an iterator.

If the yield function is passed a non-nil error, execution has failed with that error; the yield function will not be called again.

If the yield function's StreamValue argument has Done == true, the value's Output and Response fields contain the final typed output and response; the yield function will not be called again.

Otherwise the Chunk field of the passed StreamValue holds a streamed chunk.

For structured output types (non-string Out), the prompt must be configured with the appropriate output schema, either through DefineDataPrompt or by using WithOutputType when defining the prompt. The typed input argument fills the input slot last, so it wins over any WithInput passed in opts.

func (*DataPrompt[In, Out]) Render added in v1.3.0

func (dp *DataPrompt[In, Out]) Render(ctx context.Context, input In) (*GenerateActionOptions, error)

Render renders the typed prompt template with the given input.

type DocsFn added in v1.12.0

type DocsFn = func(context.Context, any) ([]*Document, error)

DocsFn is a function that generates context documents from a prompt's input.

The input is untyped. WithDocsFn takes a function with a concrete input type and converts for you.

type Document

type Document struct {
	// The data that is part of this document.
	Content []*Part `json:"content,omitempty"`
	// The metadata for this document.
	Metadata map[string]any `json:"metadata,omitempty"`
}

A Document is a piece of data that can be embedded, indexed, or retrieved. It includes metadata. It can contain multiple parts.

Example

This example demonstrates the Document type used in RAG applications.

package main

import (
	"fmt"

	"github.com/firebase/genkit/go/ai"
)

func main() {
	// Create a document with text content
	doc := &ai.Document{
		Content: []*ai.Part{
			ai.NewTextPart("This is the document content."),
		},
		Metadata: map[string]any{
			"source": "knowledge-base",
			"page":   42,
		},
	}

	fmt.Println("Content:", doc.Content[0].Text)
	fmt.Println("Source:", doc.Metadata["source"])
}
Output:
Content: This is the document content.
Source: knowledge-base

func DocumentFromText

func DocumentFromText(text string, metadata map[string]any) *Document

DocumentFromText returns a Document containing a single plain text part. This takes ownership of the metadata map.

type DocumentOption added in v0.5.0

type DocumentOption interface {
	PromptOption
	GenerateOption
	PromptExecuteOption
	EmbedderOption
	RetrieverOption
	// contains filtered or unexported methods
}

DocumentOption is an option for providing context or input documents. It applies to DefinePrompt, Generate, Prompt.Execute, Embed, and Retrieve.

func WithDocs added in v0.3.0

func WithDocs(docs ...*Document) DocumentOption

WithDocs adds documents as context for generation or as input to an embedder. Repeating this option (or mixing it with WithTextDocs) appends.

func WithTextDocs added in v0.5.0

func WithTextDocs(text ...string) DocumentOption

WithTextDocs adds text as context documents for generation or as input to an embedder. Repeating this option (or mixing it with WithDocs) appends.

type DownloadMediaOptions added in v0.3.0

type DownloadMediaOptions struct {
	MaxBytes int64                 // Maximum number of bytes to download.
	Filter   func(part *Part) bool // Filter to apply to parts that are media URLs.
}

DownloadMediaOptions configures how media is downloaded in the DownloadRequestMedia middleware.

type EmbedRequest

type EmbedRequest struct {
	// Input is the array of documents to generate embeddings for.
	Input []*Document `json:"input,omitempty"`
	// Options contains embedder-specific configuration parameters.
	Options any `json:"options,omitempty"`
}

EmbedRequest represents a request to generate embeddings for documents.

type EmbedResponse added in v0.0.2

type EmbedResponse struct {
	// Embeddings is the array of generated embedding vectors with metadata.
	Embeddings []*Embedding `json:"embeddings,omitempty"`
}

EmbedResponse contains the generated embeddings from an embed request.

func Embed added in v0.1.0

func Embed(ctx context.Context, r api.Registry, opts ...EmbedderOption) (*EmbedResponse, error)

Embed invokes the embedder with provided options.

type Embedder

type Embedder interface {
	// Name returns the registry name of the embedder.
	Name() string
	// Embed embeds to content as part of the [EmbedRequest].
	Embed(ctx context.Context, req *EmbedRequest) (*EmbedResponse, error)
	// Register registers the embedder with the given registry.
	Register(r api.Registry)
}

Embedder represents an embedder that can perform content embedding. It is the type to accept as an argument and to look up by name; implementations are created with NewEmbedderAction, or [genkit.DefineEmbedderAction] in an application.

func LookupEmbedder

func LookupEmbedder(r api.Registry, name string) Embedder

LookupEmbedder looks up a registered Embedder by name. It will try to resolve the embedder dynamically if the embedder is not found. It returns nil if the embedder was not resolved.

func NewEmbedder deprecated added in v0.7.0

func NewEmbedder(name string, opts *EmbedderOptions, fn EmbedderFunc) Embedder

NewEmbedder creates a new Embedder.

Deprecated: Use NewEmbedderAction, which passes the request's options to fn as a typed value instead of leaving them type-erased on the request.

type EmbedderAction added in v1.12.0

type EmbedderAction struct {
	// contains filtered or unexported fields
}

EmbedderAction is an embedder backed by a registry action. It is the concrete type returned by NewEmbedderAction; pass it to WithEmbedder to use it for embedding, or return it from a plugin's Init for the framework to register.

It implements Embedder and api.Action, so it can be passed anywhere either is accepted. It also promotes core.Action.Run, the typed equivalent of EmbedderAction.Embed.

func NewEmbedderAction added in v1.12.0

func NewEmbedderAction[Config any](
	name string,
	opts *EmbedderOptions,
	fn EmbedderActionFunc[Config],
) *EmbedderAction

NewEmbedderAction creates an unregistered EmbedderAction: return it from a plugin's Init for the framework to register, or call EmbedderAction.Register directly. Applications should define embedders with [genkit.DefineEmbedderAction].

Config is the embedder's typed configuration; it is usually inferred from fn's signature. The framework deserializes the request's raw options into Config before calling fn: the exact Config type (or a pointer to it) and map[string]any (from the Dev UI and other JSON callers) are accepted, and mismatched types are rejected. The request's EmbedRequest.Options is normalized to the converted value, so it always matches the typed parameter. The config's JSON schema is inferred from Config unless EmbedderOptions.ConfigSchema overrides it.

func (*EmbedderAction) Desc added in v1.12.0

func (e *EmbedderAction) Desc() api.ActionDesc

Desc returns the embedder's action descriptor: its name, schemas, and metadata.

func (*EmbedderAction) Embed added in v1.12.0

Embed runs the given Embedder.

func (*EmbedderAction) Name added in v1.12.0

func (e *EmbedderAction) Name() string

Name returns the registry name of the embedder.

func (*EmbedderAction) Register added in v1.12.0

func (e *EmbedderAction) Register(r api.Registry)

Register registers the embedder with r, making it available to lookups and to the Dev UI. A plugin that returns the embedder from its Init does not need to call this.

func (*EmbedderAction) RunJSON added in v1.12.0

RunJSON runs the embedder on a JSON-encoded EmbedRequest and returns a JSON-encoded EmbedResponse. The framework uses it to serve reflection and registry-driven calls; prefer EmbedderAction.Embed.

func (*EmbedderAction) RunJSONWithTelemetry added in v1.12.0

RunJSONWithTelemetry is EmbedderAction.RunJSON with the run's telemetry returned alongside the output.

type EmbedderActionFunc added in v1.12.0

type EmbedderActionFunc[Config any] = func(context.Context, *EmbedRequest, Config) (*EmbedResponse, error)

EmbedderActionFunc is an EmbedderFunc that additionally receives the request's typed Config: the framework deserializes the request's raw options into it before calling the function (see NewEmbedderAction).

type EmbedderArg added in v0.7.0

type EmbedderArg interface {
	Name() string
}

EmbedderArg is the interface for embedder arguments. It can either be the embedder action itself or a reference to be looked up.

type EmbedderFunc added in v0.7.0

type EmbedderFunc = func(context.Context, *EmbedRequest) (*EmbedResponse, error)

EmbedderFunc is the function type for embedding documents.

type EmbedderOption added in v0.5.0

type EmbedderOption interface {
	// contains filtered or unexported methods
}

EmbedderOption is an option for configuring an embedder request. It applies only to Embed.

func WithEmbedder added in v0.7.0

func WithEmbedder(embedder EmbedderArg) EmbedderOption

WithEmbedder sets either a Embedder or a EmbedderRef that may contain a config. Passing WithConfig will take precedence over the config in WithEmbedder.

func WithEmbedderName added in v0.7.0

func WithEmbedderName(name string) EmbedderOption

WithEmbedderName sets the embedder name to call for document embedding. The embedder name will be resolved to a Embedder and may error if the reference is invalid.

type EmbedderOptions added in v0.7.0

type EmbedderOptions struct {
	// ConfigSchema is the JSON schema for the embedder's config.
	ConfigSchema map[string]any `json:"configSchema,omitempty"`
	// Label is a user-friendly name for the embedder model (e.g., "Google AI - Gemini Pro").
	Label string `json:"label,omitempty"`
	// Supports defines the capabilities of the embedder, such as input types and multilingual support.
	Supports *EmbedderSupports `json:"supports,omitempty"`
	// Dimensions specifies the number of dimensions in the embedding vector.
	Dimensions int `json:"dimensions,omitempty"`
	// Metadata is arbitrary key-value data attached to the action descriptor.
	Metadata map[string]any `json:"-"`
}

EmbedderOptions represents the configuration options for an embedder.

func (EmbedderOptions) Overlay added in v1.12.0

func (o EmbedderOptions) Overlay(override EmbedderOptions) EmbedderOptions

Overlay returns o with every field set in override replacing o's. A field left at its zero value in override keeps o's.

type EmbedderRef added in v0.7.0

type EmbedderRef struct {
	// contains filtered or unexported fields
}

EmbedderRef is a struct to hold embedder name and configuration.

func NewEmbedderRef added in v0.7.0

func NewEmbedderRef(name string, config any) EmbedderRef

NewEmbedderRef creates a new EmbedderRef with the given name and configuration.

func (EmbedderRef) Config added in v0.7.0

func (e EmbedderRef) Config() any

Config returns the configuration to use by default for this embedder.

func (EmbedderRef) Name added in v0.7.0

func (e EmbedderRef) Name() string

Name returns the name of the embedder.

type EmbedderSupports added in v0.7.0

type EmbedderSupports struct {
	// Input lists the types of data the model can process (e.g., "text", "image", "video").
	Input []string `json:"input,omitempty"`
	// Multilingual indicates whether the model supports multiple languages.
	Multilingual bool `json:"multilingual,omitempty"`
}

EmbedderSupports represents the supported capabilities of the embedder model.

type Embedding added in v0.3.0

type Embedding struct {
	// Embedding is the vector representation of the input.
	Embedding []float32 `json:"embedding,omitempty"`
	// Metadata identifies which part of a document this embedding corresponds to.
	Metadata map[string]any `json:"metadata,omitempty"`
}

Embedding represents a vector embedding with associated metadata.

Example

This example demonstrates creating an Embedding for vector search.

package main

import (
	"fmt"

	"github.com/firebase/genkit/go/ai"
)

func main() {
	// Create an embedding (typically returned by an embedder)
	embedding := &ai.Embedding{
		Embedding: []float32{0.1, 0.2, 0.3, 0.4, 0.5},
		Metadata: map[string]any{
			"source": "document-1",
		},
	}

	fmt.Printf("Embedding dimensions: %d\n", len(embedding.Embedding))
	fmt.Printf("First value: %.1f\n", embedding.Embedding[0])
}
Output:
Embedding dimensions: 5
First value: 0.1

type EvaluationResult added in v0.3.0

type EvaluationResult struct {
	TestCaseId string  `json:"testCaseId"`
	TraceID    string  `json:"traceId,omitempty"`
	SpanID     string  `json:"spanId,omitempty"`
	Evaluation []Score `json:"evaluation"`
}

EvaluationResult is the result of running the evaluator on a single Example. An evaluator may provide multiple scores simultaneously (e.g. if they are using an API to score on multiple criteria)

type Evaluator added in v0.3.0

type Evaluator interface {
	// Name returns the name of the evaluator.
	Name() string
	// Evaluates a dataset.
	Evaluate(ctx context.Context, req *EvaluatorRequest) (*EvaluatorResponse, error)
	// Register registers the evaluator with the given registry.
	Register(r api.Registry)
}

Evaluator represents an evaluator. It is the type to accept as an argument and to look up by name; implementations are created with NewEvaluatorAction or NewBatchEvaluatorAction, or their [genkit.DefineEvaluatorAction] and [genkit.DefineBatchEvaluatorAction] counterparts in an application.

func LookupEvaluator added in v0.3.0

func LookupEvaluator(r api.Registry, name string) Evaluator

LookupEvaluator looks up a registered Evaluator by name. It returns nil if the evaluator was not defined.

func NewBatchEvaluator deprecated added in v0.7.0

func NewBatchEvaluator(name string, opts *EvaluatorOptions, fn BatchEvaluatorFunc) Evaluator

NewBatchEvaluator creates a new Evaluator. This method provides the full EvaluatorRequest to the callback function, giving more flexibility to the user for processing the data, such as batching or parallelization.

Deprecated: Use NewBatchEvaluatorAction, which passes the request's options to fn as a typed value instead of leaving them type-erased on the request.

func NewEvaluator deprecated added in v0.7.0

func NewEvaluator(name string, opts *EvaluatorOptions, fn EvaluatorFunc) Evaluator

NewEvaluator creates a new Evaluator. This method processes the input dataset one-by-one.

Deprecated: Use NewEvaluatorAction, which passes the request's options to fn as a typed value instead of leaving them type-erased on the request.

type EvaluatorAction added in v1.12.0

type EvaluatorAction struct {
	// contains filtered or unexported fields
}

EvaluatorAction is an evaluator backed by a registry action. It is the concrete type returned by NewEvaluatorAction and NewBatchEvaluatorAction; pass it to WithEvaluator, or return it from a plugin's Init for the framework to register.

It implements Evaluator and api.Action, so it can be passed anywhere either is accepted. It also promotes core.Action.Run, the typed equivalent of EvaluatorAction.Evaluate.

func NewBatchEvaluatorAction added in v1.12.0

func NewBatchEvaluatorAction[Config any](
	name string,
	opts *EvaluatorOptions,
	fn BatchEvaluatorActionFunc[Config],
) *EvaluatorAction

NewBatchEvaluatorAction creates an unregistered EvaluatorAction: return it from a plugin's Init for the framework to register, or call EvaluatorAction.Register directly. Applications should define batch evaluators with [genkit.DefineBatchEvaluatorAction]. This method provides the full EvaluatorRequest to the callback function, giving more flexibility to the user for processing the data, such as batching or parallelization.

Config is the evaluator's typed configuration; it is usually inferred from fn's signature. See NewEvaluatorAction for how the request's options are deserialized.

EvaluatorOptions.ConfigSchema is enforced: it becomes the options slot of the action's input schema and every request is validated against it, so a schema narrower than what callers actually send now fails at the action boundary. Batch evaluators did not validate options before; leave ConfigSchema unset to accept anything.

func NewEvaluatorAction added in v1.12.0

func NewEvaluatorAction[Config any](
	name string,
	opts *EvaluatorOptions,
	fn EvaluatorActionFunc[Config],
) *EvaluatorAction

NewEvaluatorAction creates an unregistered EvaluatorAction: return it from a plugin's Init for the framework to register, or call EvaluatorAction.Register directly. Applications should define evaluators with [genkit.DefineEvaluatorAction]. This method processes the input dataset one-by-one.

Config is the evaluator's typed configuration; it is usually inferred from fn's signature. The framework deserializes the request's raw options into Config before calling fn: the exact Config type (or a pointer to it) and map[string]any (from the Dev UI and other JSON callers) are accepted, and mismatched types are rejected. The config's JSON schema is inferred from Config unless EvaluatorOptions.ConfigSchema overrides it.

func (*EvaluatorAction) Desc added in v1.12.0

func (e *EvaluatorAction) Desc() api.ActionDesc

Desc returns the evaluator's action descriptor: its name, schemas, and metadata.

func (*EvaluatorAction) Evaluate added in v1.12.0

Evaluate runs the given Evaluator.

func (*EvaluatorAction) Name added in v1.12.0

func (e *EvaluatorAction) Name() string

Name returns the registry name of the evaluator.

func (*EvaluatorAction) Register added in v1.12.0

func (e *EvaluatorAction) Register(r api.Registry)

Register registers the evaluator with r, making it available to lookups and to the Dev UI. A plugin that returns the evaluator from its Init does not need to call this.

func (*EvaluatorAction) RunJSON added in v1.12.0

RunJSON runs the evaluator on a JSON-encoded EvaluatorRequest and returns a JSON-encoded EvaluatorResponse. The framework uses it to serve reflection and registry-driven calls; prefer EvaluatorAction.Evaluate.

func (*EvaluatorAction) RunJSONWithTelemetry added in v1.12.0

RunJSONWithTelemetry is EvaluatorAction.RunJSON with the run's telemetry returned alongside the output.

type EvaluatorActionFunc added in v1.12.0

type EvaluatorActionFunc[Config any] = func(context.Context, *EvaluatorCallbackRequest, Config) (*EvaluatorCallbackResponse, error)

EvaluatorActionFunc is an EvaluatorFunc that additionally receives the request's typed Config: the framework deserializes the request's raw options into it before calling the function (see NewEvaluatorAction).

type EvaluatorArg added in v0.7.0

type EvaluatorArg interface {
	Name() string
}

EvaluatorArg is the interface for evaluator arguments. It can either be the evaluator action itself or a reference to be looked up.

type EvaluatorCallbackRequest added in v0.3.0

type EvaluatorCallbackRequest struct {
	Input   Example `json:"input"`
	Options any     `json:"options,omitempty"`
}

EvaluatorCallbackRequest is the data we pass to the callback function provided in defineEvaluator. The Options field is specific to the actual evaluator implementation.

type EvaluatorCallbackResponse added in v0.3.0

type EvaluatorCallbackResponse = EvaluationResult

EvaluatorCallbackResponse is the result on evaluating a single Example

type EvaluatorFunc added in v0.7.0

EvaluatorFunc is the function type for evaluator implementations.

type EvaluatorOption added in v0.5.0

type EvaluatorOption interface {
	// contains filtered or unexported methods
}

EvaluatorOption is an option for providing a dataset to evaluate. It applies only to Evaluator.Evaluate.

func WithDataset added in v0.5.0

func WithDataset(examples ...*Example) EvaluatorOption

WithDataset adds examples to the dataset to evaluate. Repeating this option appends.

func WithEvaluator added in v0.7.0

func WithEvaluator(evaluator EvaluatorArg) EvaluatorOption

WithEvaluator sets either a Evaluator or a EvaluatorRef that may contain a config. Passing WithConfig will take precedence over the config in WithEvaluator.

func WithEvaluatorName added in v0.7.0

func WithEvaluatorName(name string) EvaluatorOption

WithEvaluatorName sets the evaluator name to call for document evaluation. The evaluator name will be resolved to a Evaluator and may error if the reference is invalid.

func WithID added in v0.5.0

func WithID(ID string) EvaluatorOption

WithID sets the ID of the evaluation to uniquely identify it. Repeating this option takes the last ID set.

type EvaluatorOptions added in v0.3.0

type EvaluatorOptions struct {
	// ConfigSchema is the JSON schema for the evaluator's config.
	ConfigSchema map[string]any `json:"configSchema,omitempty"`
	// Metadata is arbitrary key-value data attached to the action descriptor.
	Metadata map[string]any `json:"-"`
	// DisplayName is the name of the evaluator as it appears in the UI.
	DisplayName string `json:"displayName"`
	// Definition is the definition of the evaluator.
	Definition string `json:"definition"`
	// IsBilled is a flag indicating if the evaluator is billed.
	IsBilled bool `json:"isBilled,omitempty"`
}

EvaluatorOptions configures an evaluator created with NewEvaluatorAction or NewBatchEvaluatorAction.

type EvaluatorRef added in v0.7.0

type EvaluatorRef struct {
	// contains filtered or unexported fields
}

EvaluatorRef is a struct to hold evaluator name and configuration.

func NewEvaluatorRef added in v0.7.0

func NewEvaluatorRef(name string, config any) EvaluatorRef

NewEvaluatorRef creates a new EvaluatorRef with the given name and configuration.

func (EvaluatorRef) Config added in v0.7.0

func (e EvaluatorRef) Config() any

Config returns the configuration to use by default for this evaluator.

func (EvaluatorRef) Name added in v0.7.0

func (e EvaluatorRef) Name() string

Name returns the name of the evaluator.

type EvaluatorRequest added in v0.3.0

type EvaluatorRequest struct {
	Dataset      []*Example `json:"dataset"`
	EvaluationId string     `json:"evalRunId"`
	Options      any        `json:"options,omitempty"`
}

EvaluatorRequest is the data we pass to evaluate a dataset. The Options field is specific to the actual evaluator implementation.

type EvaluatorResponse added in v0.3.0

type EvaluatorResponse = []EvaluationResult

EvaluatorResponse is a collection of EvaluationResult structs, it represents the result on the entire input dataset.

func Evaluate added in v0.3.0

func Evaluate(ctx context.Context, r api.Registry, opts ...EvaluatorOption) (*EvaluatorResponse, error)

Evaluate calls the retrivers with provided options.

type Example added in v0.3.0

type Example struct {
	TestCaseId string   `json:"testCaseId,omitempty"`
	Input      any      `json:"input"`
	Output     any      `json:"output,omitempty"`
	Context    []any    `json:"context,omitempty"`
	Reference  any      `json:"reference,omitempty"`
	TraceIds   []string `json:"traceIds,omitempty"`
}

Example is a single example that requires evaluation

type ExecutionOption added in v0.3.0

type ExecutionOption interface {
	GenerateOption
	PromptExecuteOption
	// contains filtered or unexported methods
}

ExecutionOption is an option for the execution of a prompt or generate request. It applies only to Generate() and prompt.Execute().

func WithStreaming added in v0.1.0

func WithStreaming(callback ModelStreamCallback) ExecutionOption

WithStreaming sets the stream callback for the generate request. A callback is a function that is called with each chunk of the generated response before the final response is returned. Repeating this option takes the last callback set. The stream-returning APIs (GenerateStream, Prompt.ExecuteStream, and their typed variants) attach their own iterator callback without displacing one set here; both receive every chunk.

type FinishReason

type FinishReason string

FinishReason indicates why generation stopped.

const (
	FinishReasonStop        FinishReason = "stop"
	FinishReasonLength      FinishReason = "length"
	FinishReasonBlocked     FinishReason = "blocked"
	FinishReasonAborted     FinishReason = "aborted"
	FinishReasonInterrupted FinishReason = "interrupted"
	FinishReasonOther       FinishReason = "other"
	FinishReasonUnknown     FinishReason = "unknown"
)

type FormatHandler added in v0.5.0

type FormatHandler interface {
	// ParseMessage parses the message and returns a new formatted message.
	//
	// Legacy: New format handlers should implement this as a no-op passthrough and implement [StreamingFormatHandler] instead.
	ParseMessage(message *Message) (*Message, error)
	// Instructions returns the formatter instructions to embed in the prompt.
	Instructions() string
	// Config returns the output config for the model request.
	Config() ModelOutputConfig
}

FormatHandler is a handler for formatting messages. A new instance is created via Formatter.Handler for each request.

type Formatter added in v0.5.0

type Formatter interface {
	// Name returns the name of the formatter.
	Name() string
	// Handler returns the handler for the formatter.
	Handler(schema map[string]any) (FormatHandler, error)
}

Formatter represents the Formatter interface.

type GenerateActionOptions added in v0.3.0

type GenerateActionOptions struct {
	// Config contains configuration parameters for the generation request.
	Config any `json:"config,omitempty"`
	// Docs provides retrieved documents to be used as context for this generation.
	Docs []*Document `json:"docs,omitempty"`
	// MaxTurns is the maximum number of tool call iterations that can be performed
	// in a single generate call. Defaults to 5.
	MaxTurns int `json:"maxTurns,omitempty"`
	// Messages contains the conversation history for multi-turn prompting when supported.
	Messages []*Message `json:"messages,omitempty"`
	// Model is a model name (e.g., "vertexai/gemini-flash-latest").
	Model string `json:"model,omitempty"`
	// Output specifies the desired output format. Defaults to the model's default if unspecified.
	Output    *GenerateActionOutputConfig `json:"output,omitempty"`
	Resources []string                    `json:"resources,omitempty"`
	// Resume provides options for resuming an interrupted generation.
	Resume *GenerateActionResume `json:"resume,omitempty"`
	// ReturnToolRequests, when true, returns tool calls for manual processing instead of
	// automatically resolving them.
	ReturnToolRequests bool `json:"returnToolRequests,omitempty"`
	// StepName is a custom step name for this generate call to display in trace views.
	// Defaults to "generate".
	StepName string `json:"stepName,omitempty"`
	// ToolChoice controls tool calling mode. Auto lets the model decide, required forces
	// the model to choose a tool, and none forces the model not to use any tools. Defaults to auto.
	ToolChoice ToolChoice `json:"toolChoice,omitempty"`
	// Tools is a list of registered tool names for this generation if supported.
	Tools []string `json:"tools,omitempty"`
	// Use is middleware to apply to this generation, referenced by name with optional config.
	Use []*MiddlewareRef `json:"use,omitempty"`
}

GenerateActionOptions holds configuration for a generate action request.

type GenerateActionOutputConfig added in v0.3.0

type GenerateActionOutputConfig struct {
	// Constrained indicates whether to enforce strict adherence to the schema.
	Constrained bool `json:"constrained,omitempty"`
	// ContentType specifies the MIME type of the output content.
	ContentType string `json:"contentType,omitempty"`
	// Format specifies the desired output format (e.g., "json", "text").
	Format string `json:"format,omitempty"`
	// Instructions provides additional guidance for the output format.
	Instructions *string `json:"instructions,omitempty"`
	// JsonSchema is a JSON Schema describing the desired structure of JSON output.
	JsonSchema map[string]any `json:"jsonSchema,omitempty"`
}

GenerateActionOutputConfig specifies the desired output format for a generate action.

type GenerateActionResume added in v0.3.0

type GenerateActionResume struct {
	// Metadata contains additional context for resuming the generation.
	Metadata map[string]any `json:"metadata,omitempty"`
	// Respond contains tool response parts to send to the model when resuming.
	Respond []*Part `json:"respond,omitempty"`
	// Restart contains tool request parts to restart when resuming.
	Restart []*Part `json:"restart,omitempty"`
}

GenerateActionResume holds options for resuming an interrupted generation.

func NewResume added in v1.10.0

func NewResume(restarts, responds []*Part) *GenerateActionResume

NewResume constructs a GenerateActionResume from Part slices. This is useful when building GenerateActionOptions directly (e.g., from a rendered prompt) and need to set the Resume field from *Part values produced by ToolAction.RestartWith or ToolAction.RespondWith.

type GenerateNext added in v1.7.0

type GenerateNext = func(ctx context.Context, params *GenerateParams) (*ModelResponse, error)

GenerateNext is the next function in the WrapGenerate hook chain.

type GenerateOption added in v0.1.0

type GenerateOption interface {
	// contains filtered or unexported methods
}

GenerateOption is an option for generating a model response. It applies only to Generate().

func WithStepName added in v1.8.0

func WithStepName(name string) GenerateOption

WithStepName sets a custom name for the generation step in traces.

func WithToolResponses added in v0.6.0

func WithToolResponses(parts ...*Part) GenerateOption

WithToolResponses provides resolved responses for interrupted tool calls. Use this when you already have the result and want to skip re-executing the tool. Repeating this option appends.

func WithToolRestarts added in v0.6.0

func WithToolRestarts(parts ...*Part) GenerateOption

WithToolRestarts re-executes interrupted tool calls with additional metadata. Use this when the original call lacked required context (e.g., auth, user confirmation) that should now allow the tool to complete successfully. Repeating this option appends.

type GenerateParams added in v1.7.0

type GenerateParams struct {
	// Options is the original options passed to [Generate].
	Options *GenerateActionOptions
	// Request is the current model request for this iteration, with accumulated messages.
	Request *ModelRequest
	// Iteration is the current tool-loop iteration (0-indexed).
	Iteration int
	// MessageIndex is the index of the next message in the streamed response sequence.
	// Middleware that streams extra messages (e.g. injected user content) should emit
	// chunks at this index and advance it so downstream middleware and the model
	// receive the shifted value.
	MessageIndex int
	// Callback is the streaming callback supplied to [Generate], or nil if not streaming.
	// Middleware may invoke it to emit chunks, setting [ModelResponseChunk.Role] and
	// [ModelResponseChunk.Index] explicitly.
	Callback ModelStreamCallback
}

GenerateParams holds params for the WrapGenerate hook.

type GenerationCommonConfig

type GenerationCommonConfig struct {
	// API Key to use for the model call, overrides API key provided in plugin config.
	APIKey string `json:"apiKey,omitempty"`
	// MaxOutputTokens limits the maximum number of tokens generated in the response.
	MaxOutputTokens int `json:"maxOutputTokens,omitempty"`
	// StopSequences specifies sequences that will cause generation to stop when encountered.
	StopSequences []string `json:"stopSequences,omitempty"`
	// Temperature controls randomness in generation. Higher values (e.g., 0.9) make output more random,
	// while lower values (e.g., 0.1) make it more deterministic. Typical range is 0.0 to 1.0.
	Temperature float64 `json:"temperature,omitempty"`
	// TopK limits sampling to the K most likely tokens at each step.
	TopK int `json:"topK,omitempty"`
	// TopP (nucleus sampling) limits sampling to tokens whose cumulative probability exceeds P.
	TopP float64 `json:"topP,omitempty"`
	// Version specifies a particular version of a model family,
	// e.g., "gemini-3.5-flash-001" for the "gemini-3.5-flash" family.
	Version string `json:"version,omitempty"`
}

GenerationCommonConfig holds configuration parameters for model generation requests.

type GenerationUsage

type GenerationUsage struct {
	// CachedContentTokens counts tokens that were served from cache.
	CachedContentTokens int `json:"cachedContentTokens,omitempty"`
	// Custom contains additional usage metrics specific to the model provider.
	Custom map[string]float64 `json:"custom,omitempty"`
	// InputAudioFiles is the number of audio files in the input.
	InputAudioFiles int `json:"inputAudioFiles,omitempty"`
	// InputCharacters is the number of characters in the input.
	InputCharacters int `json:"inputCharacters,omitempty"`
	// InputImages is the number of images in the input.
	InputImages int `json:"inputImages,omitempty"`
	// InputTokens is the number of tokens in the input prompt.
	InputTokens int `json:"inputTokens,omitempty"`
	// InputVideos is the number of videos in the input.
	InputVideos int `json:"inputVideos,omitempty"`
	// OutputAudioFiles is the number of audio files generated in the output.
	OutputAudioFiles int `json:"outputAudioFiles,omitempty"`
	// OutputCharacters is the number of characters generated in the output.
	OutputCharacters int `json:"outputCharacters,omitempty"`
	// OutputImages is the number of images generated in the output.
	OutputImages int `json:"outputImages,omitempty"`
	// OutputTokens is the number of tokens generated in the response.
	OutputTokens int `json:"outputTokens,omitempty"`
	// OutputVideos is the number of videos generated in the output.
	OutputVideos int `json:"outputVideos,omitempty"`
	// ThoughtsTokens counts tokens used in reasoning or thinking processes.
	ThoughtsTokens int `json:"thoughtsTokens,omitempty"`
	// TotalTokens is the sum of input and output tokens.
	TotalTokens int `json:"totalTokens,omitempty"`
}

GenerationUsage provides information about resource consumption during generation.

type Hooks added in v1.7.0

type Hooks struct {
	// Tools are additional tools to register during the generation this
	// middleware is attached to. They are available to the model alongside
	// any user-supplied tools.
	Tools []Tool
	// WrapGenerate wraps each iteration of the tool loop. It sees the
	// accumulated request, the iteration index, and the streaming callback.
	// A single Generate() with N tool-call turns invokes this hook N+1 times.
	WrapGenerate func(ctx context.Context, params *GenerateParams, next GenerateNext) (*ModelResponse, error)
	// WrapModel wraps each model API call. Retry, fallback, and caching
	// middleware typically hook here.
	WrapModel func(ctx context.Context, params *ModelParams, next ModelNext) (*ModelResponse, error)
	// WrapTool wraps each tool execution. It may be called concurrently when
	// multiple tools execute in parallel for the same Generate() call; any
	// state closed over from the enclosing scope that this hook mutates must
	// be guarded with sync primitives.
	WrapTool func(ctx context.Context, params *ToolParams, next ToolNext) (*MultipartToolResponse, error)
}

Hooks is the per-call bundle of hook functions produced by a Middleware's New method. Each field is optional; a nil hook is treated as a pass-through.

type InputOption added in v1.3.0

type InputOption interface {
	PromptOption
	// contains filtered or unexported methods
}

InputOption is an option for the input of a prompt. It applies only to DefinePrompt().

type InputSchemaOption added in v1.12.0

type InputSchemaOption interface {
	InputOption
	ToolOption
}

InputSchemaOption is an InputOption that also applies to the tool constructors, where the input schema it supplies stands in for an In type parameter of 'any'. A tool whose input shape a Go type can express should use the type parameter instead.

func WithInputSchema added in v1.3.0

func WithInputSchema(schema map[string]any) InputSchemaOption

WithInputSchema manually provides a schema map for the input.

func WithInputSchemaName added in v1.3.0

func WithInputSchemaName(name string) InputSchemaOption

WithInputSchemaName sets a pre-registered schema by name for the input. The schema is resolved from the registry at execution time; register it with github.com/firebase/genkit/go/genkit.DefineSchema.

func WithInputType added in v0.3.0

func WithInputType(input any) InputSchemaOption

WithInputType uses the type provided to derive the input schema. The inputted value may serve as the default input if no input is given at generation time depending on the action. Only supports structs and map[string]any.

It applies to the tool constructors too, where the derived schema stands in for an In type parameter of 'any'. Prefer the type parameter there: it is the same schema plus a typed function signature.

type InterruptOptions added in v0.3.0

type InterruptOptions struct {
	Metadata map[string]any
}

InterruptOptions provides configuration for tool interruption.

type Media added in v0.3.0

type Media struct {
	// ContentType specifies the MIME type of the media. Inferred from the data URI if not provided.
	ContentType string `json:"contentType,omitempty"`
	// Url is a "data:" or "https:" URI containing the media content.
	Url string `json:"url,omitempty"`
}

Media represents media content with a URL and content type.

type Message

type Message struct {
	// Content holds the message parts (text, media, tool calls, etc.).
	Content []*Part `json:"content,omitempty"`
	// Metadata contains arbitrary key-value data associated with this message.
	Metadata map[string]any `json:"metadata,omitempty"`
	// Role indicates which entity (system, user, model, or tool) generated this message.
	Role Role `json:"role,omitempty"`
}

Message represents the contents of a model message in a conversation.

func HistoryFromContext added in v1.12.0

func HistoryFromContext(ctx context.Context) []*Message

HistoryFromContext returns the conversation history passed to Prompt.Execute via WithMessages or WithMessagesFn, or nil if there is none.

Call it from a prompt's own content functions, where it is the function-form counterpart of {{history}}. A prompt that declares a conversation owns the history: the caller's messages are not appended on top, so a function that wants them must read them here and return them, which is what makes summarizing or truncating possible. A prompt that declares no messages of its own has the caller's history used as the conversation directly.

The returned slice is the caller's; treat it as read-only.

func NewMessage

func NewMessage(role Role, metadata map[string]any, parts ...*Part) *Message

NewMessage creates a new Message with the provided role, metadata and parts. Use NewTextMessage if you have a text-only message.

func NewModelMessage

func NewModelMessage(parts ...*Part) *Message

NewModelMessage creates a new Message with role "model" and provided parts. Use NewModelTextMessage if you have a text-only message.

func NewModelTextMessage

func NewModelTextMessage(text string) *Message

NewModelTextMessage creates a new Message with role "model" and content with a single text part with the content of provided text.

func NewSystemMessage

func NewSystemMessage(parts ...*Part) *Message

NewSystemMessage creates a new Message with role "system" and provided parts. Use NewSystemTextMessage if you have a text-only message.

func NewSystemTextMessage

func NewSystemTextMessage(text string) *Message

NewSystemTextMessage creates a new Message with role "system" and content with a single text part with the content of provided text.

Example

This example demonstrates creating system and model messages.

package main

import (
	"fmt"

	"github.com/firebase/genkit/go/ai"
)

func main() {
	// Create a system message
	sysMsg := ai.NewSystemTextMessage("You are a helpful assistant.")
	fmt.Println("System role:", sysMsg.Role)

	// Create a model response message
	modelMsg := ai.NewModelTextMessage("I'm here to help!")
	fmt.Println("Model role:", modelMsg.Role)
}
Output:
System role: system
Model role: model

func NewTextMessage

func NewTextMessage(role Role, text string) *Message

NewTextMessage creates a new Message with the provided role and content with a single part containint provided text.

func NewUserMessage

func NewUserMessage(parts ...*Part) *Message

NewUserMessage creates a new Message with role "user" and provided parts. Use NewUserTextMessage if you have a text-only message.

Example

This example demonstrates building a multi-turn conversation.

package main

import (
	"fmt"

	"github.com/firebase/genkit/go/ai"
)

func main() {
	// Build a conversation with multiple parts
	userMsg := ai.NewUserMessage(
		ai.NewTextPart("What's in this image?"),
		ai.NewMediaPart("image/jpeg", "base64data..."),
	)

	fmt.Println("Role:", userMsg.Role)
	fmt.Println("Parts:", len(userMsg.Content))
}
Output:
Role: user
Parts: 2

func NewUserMessageWithMetadata added in v0.3.0

func NewUserMessageWithMetadata(metadata map[string]any, parts ...*Part) *Message

NewUserMessageWithMetadata creates a new Message with role "user" with provided metadata and parts. Use NewUserTextMessage if you have a text-only message.

func NewUserTextMessage

func NewUserTextMessage(text string) *Message

NewUserTextMessage creates a new Message with role "user" and content with a single text part with the content of provided text.

Example

This example demonstrates creating a message with text content.

package main

import (
	"fmt"

	"github.com/firebase/genkit/go/ai"
)

func main() {
	// Create a user message with text
	msg := ai.NewUserTextMessage("What is the capital of France?")
	fmt.Println("Role:", msg.Role)
	fmt.Println("Text:", msg.Content[0].Text)
}
Output:
Role: user
Text: What is the capital of France?

func (*Message) Clone added in v1.7.0

func (m *Message) Clone() *Message

Clone returns a shallow copy of the Message with its own Content slice and Metadata map. Callers can replace parts or add metadata keys without mutating the original.

func (*Message) MediaParts added in v1.12.0

func (m *Message) MediaParts() []*Part

MediaParts returns every media part of a Message, each carrying its content type alongside its data. It returns nil if the message has none.

func (*Message) Text added in v0.3.0

func (m *Message) Text() string

Text returns the textual contents of a Message as a string, concatenating its text parts and skipping every other kind. It returns an empty string if the message has none, which is what a message carrying only an image, only raw data, or only a tool request comes back as; read those with Message.MediaParts and ToolRequests instead.

A data part is deliberately not text: it holds a blob, which the plugins send as bytes and plugins/internal/uri.Data decodes as a data: URI, so concatenating one here would splice base64 into prose. If you want to get reasoning from the message, use Reasoning() instead.

func (*Message) WithCacheName added in v0.3.0

func (m *Message) WithCacheName(n string) *Message

WithCacheName adds cache name to use in the generate request

func (*Message) WithCacheTTL added in v0.3.0

func (m *Message) WithCacheTTL(ttlSeconds int) *Message

WithCacheTTL adds cache TTL configuration for the desired message

type MessagesFn added in v0.5.0

type MessagesFn = func(context.Context, any) ([]*Message, error)

MessagesFn is a function that generates messages from a prompt's input.

The input is untyped. WithMessagesFn takes a function with a concrete input type and converts for you.

type Middleware added in v1.7.0

type Middleware interface {
	// Name returns the registered middleware's unique identifier. Must be a
	// stable constant, since it is read from a zero value of the config type
	// during descriptor creation.
	Name() string
	// New produces a fresh [Hooks] bundle for one Generate() call. It is
	// invoked per-Generate, so any state the bundle's hooks need to share
	// (counters, caches) may be allocated in this method and closed over by
	// the returned hooks.
	New(ctx context.Context) (*Hooks, error)
}

Middleware is the contract every value passed to WithUse satisfies. The config struct both identifies the middleware (via [Name]) and produces a per-call Hooks bundle (via [New]).

Plugin-level state belongs on unexported fields of the config type. A plugin's MiddlewarePlugin.Middlewares sets those fields on a prototype that is preserved across JSON dispatch by value-copy inside the descriptor.

type MiddlewareDesc added in v1.5.0

type MiddlewareDesc struct {
	// ConfigSchema is a JSON Schema describing the middleware's configuration.
	ConfigSchema map[string]any `json:"configSchema,omitempty"`
	// Description explains what the middleware does.
	Description string `json:"description,omitempty"`
	// Metadata contains additional context for the middleware.
	Metadata map[string]any `json:"metadata,omitempty"`
	// Name is the middleware's unique identifier.
	Name string `json:"name,omitempty"`
	// contains filtered or unexported fields
}

MiddlewareDesc is the registered descriptor for a middleware.

func LookupMiddleware added in v1.7.0

func LookupMiddleware(r api.Registry, name string) *MiddlewareDesc

LookupMiddleware returns the registered middleware descriptor with the given name, or nil if no such descriptor exists in the registry or any ancestor. Primarily useful for inspection and for the reflection API; callers dispatching middleware should do so through WithUse.

func NewMiddleware added in v1.7.0

func NewMiddleware[M Middleware](description string, prototype M) *MiddlewareDesc

NewMiddleware constructs a descriptor without registering it. Useful for MiddlewarePlugin.Middlewares implementations that defer registration to [genkit.Init]. The prototype argument supplies both the registered name (via its Middleware.Name method) and any plugin-level state that should flow into JSON-dispatched invocations via unexported fields preserved by value-copy.

func (*MiddlewareDesc) Register added in v1.7.0

func (d *MiddlewareDesc) Register(r api.Registry)

Register records this descriptor in the registry under its name so it can be resolved during JSON dispatch and surfaced to the Dev UI.

type MiddlewareFunc added in v1.7.0

type MiddlewareFunc func(ctx context.Context) (*Hooks, error)

MiddlewareFunc adapts a per-call factory closure to the Middleware interface for ad-hoc inline use, without a registered descriptor or plugin wiring. The adapted middleware does not appear in the Dev UI.

Example:

ai.WithUse(ai.MiddlewareFunc(func(ctx context.Context) (*ai.Hooks, error) {
    return &ai.Hooks{WrapModel: ...}, nil
}))

func (MiddlewareFunc) Name added in v1.7.0

func (MiddlewareFunc) Name() string

Name returns the placeholder name shared by all MiddlewareFunc values. Uniqueness is unnecessary: inline middleware is resolved via the fast path in [resolveRefs] and never goes through a name-keyed registry lookup.

func (MiddlewareFunc) New added in v1.7.0

func (f MiddlewareFunc) New(ctx context.Context) (*Hooks, error)

New implements Middleware by calling f.

type MiddlewarePlugin added in v1.7.0

type MiddlewarePlugin interface {
	Middlewares(ctx context.Context) ([]*MiddlewareDesc, error)
}

MiddlewarePlugin is implemented by plugins that provide middleware. The returned descriptors are registered in the registry during [genkit.Init], with any plugin-level state captured by the descriptor's build closure via the prototype passed to NewMiddleware.

type MiddlewareRef added in v1.5.0

type MiddlewareRef struct {
	// Config contains the middleware configuration.
	Config any `json:"config,omitempty"`
	// Name is the name of the registered middleware.
	Name string `json:"name,omitempty"`
}

MiddlewareRef is a serializable reference to a registered middleware with config.

type Model

type Model interface {
	// Name returns the registry name of the model.
	Name() string
	// Generate applies the [Model] to provided request, handling tool requests and handles streaming.
	Generate(ctx context.Context, req *ModelRequest, cb ModelStreamCallback) (*ModelResponse, error)
	// Register registers the model with the given registry.
	Register(r api.Registry)
}

Model represents a model that can generate content based on a request. It is the type to accept as an argument and to look up by name; implementations are created with NewModelAction, or [genkit.DefineModelAction] in an application.

func LookupModel

func LookupModel(r api.Registry, name string) Model

LookupModel looks up a registered Model by name. It will try to resolve the model dynamically if the model is not found. It returns nil if the model was not resolved.

func NewModel deprecated added in v0.7.0

func NewModel(name string, opts *ModelOptions, fn ModelFunc) Model

NewModel creates a new Model.

Deprecated: Use NewModelAction, which passes the request's config to fn as a typed value instead of leaving it type-erased on the request.

type ModelAction added in v0.3.0

type ModelAction struct {
	// contains filtered or unexported fields
}

ModelAction is a generative model backed by a registry action. It is the concrete type returned by NewModelAction; pass it to WithModel to use it for generation, or return it from a plugin's Init for the framework to register.

It implements Model and api.Action, so it can be passed anywhere either is accepted. It also promotes core.Action.Run, the typed equivalent of ModelAction.Generate.

func NewModelAction added in v1.12.0

func NewModelAction[Config any](
	name string,
	opts *ModelOptions,
	fn ModelActionFunc[Config],
) *ModelAction

NewModelAction creates an unregistered ModelAction: return it from a plugin's Init for the framework to register, or call ModelAction.Register directly. Applications should define models with [genkit.DefineModelAction].

Config is the model's typed configuration; it is usually inferred from fn's signature. The framework deserializes the request's raw config into Config before calling fn: the exact Config type (or a pointer to it) and map[string]any (from the Dev UI and other JSON callers) are accepted, and mismatched types are rejected. The request's ModelRequest.Config is normalized to the converted value, so it always matches the typed parameter. The config's JSON schema is inferred from Config unless ModelOptions.ConfigSchema overrides it.

The config schema is enforced by input validation on every call, so if Config's JSON marshaling diverges from its reflected schema (e.g. SDK wrapper types like Opt[float64] that marshal to primitives but reflect as objects), set ModelOptions.ConfigSchema explicitly or requests will be rejected at the action boundary.

func (*ModelAction) Desc added in v1.12.0

func (m *ModelAction) Desc() api.ActionDesc

Desc returns the model's action descriptor: its name, schemas, and metadata.

func (*ModelAction) Generate added in v1.12.0

Generate applies the [Action] to provided request.

func (*ModelAction) Name added in v1.12.0

func (m *ModelAction) Name() string

Name returns the registry name of the model.

func (*ModelAction) Register added in v1.12.0

func (m *ModelAction) Register(r api.Registry)

Register registers the model with r, making it available to lookups and to the Dev UI. A plugin that returns the model from its Init does not need to call this.

func (*ModelAction) RunJSON added in v1.12.0

RunJSON runs the model on a JSON-encoded ModelRequest and returns a JSON-encoded ModelResponse. The framework uses it to serve reflection and registry-driven calls; prefer ModelAction.Generate.

func (*ModelAction) RunJSONWithTelemetry added in v1.12.0

RunJSONWithTelemetry is ModelAction.RunJSON with the run's telemetry returned alongside the output.

type ModelActionFunc added in v1.12.0

type ModelActionFunc[Config any] = func(context.Context, *ModelRequest, Config, ModelStreamCallback) (*ModelResponse, error)

ModelActionFunc is a ModelFunc that additionally receives the request's typed Config: the framework deserializes the request's raw config into it before calling the function (see NewModelAction).

type ModelArg added in v0.5.0

type ModelArg interface {
	Name() string
}

ModelArg is the interface for model arguments. It can either be the retriever action itself or a reference to be looked up.

type ModelFunc added in v0.3.0

ModelFunc is a streaming function that takes in a ModelRequest and generates a ModelResponse, optionally streaming ModelResponseChunks.

type ModelInfo

type ModelInfo struct {
	// ConfigSchema defines the model-specific configuration schema.
	ConfigSchema map[string]any `json:"configSchema,omitempty"`
	// Label is a friendly display name for this model (e.g., "Google AI - Gemini Pro").
	Label string `json:"label,omitempty"`
	// Stage indicates the development stage of this model.
	// Featured models are recommended for general use, stable models are well-tested,
	// unstable models are experimental, legacy models are not recommended for new projects,
	// and deprecated models may be removed in future versions.
	Stage ModelStage `json:"stage,omitempty"`
	// Supports describes the capabilities that this model supports.
	Supports *ModelSupports `json:"supports,omitempty"`
	// Versions lists acceptable names for this model (e.g., different versions).
	Versions []string `json:"versions,omitempty"`
}

ModelInfo contains metadata about a model's capabilities and characteristics.

type ModelMiddleware deprecated added in v0.3.0

ModelMiddleware is middleware for model generate requests that takes in a ModelFunc, does something, then returns another ModelFunc.

Deprecated: Use Middleware interface with WithUse instead, which supports Generate, Model, and Tool hooks.

func DownloadRequestMedia added in v0.3.0

func DownloadRequestMedia(opts *DownloadMediaOptions) ModelMiddleware

DownloadRequestMedia downloads media from a URL and replaces the media part with a base64 encoded string.

type ModelNext added in v1.7.0

type ModelNext = func(ctx context.Context, params *ModelParams) (*ModelResponse, error)

ModelNext is the next function in the WrapModel hook chain.

type ModelOperation added in v1.3.0

type ModelOperation = core.Operation[*ModelResponse]

ModelOperation is a background operation for a model.

func CheckModelOperation added in v1.3.0

func CheckModelOperation(ctx context.Context, r api.Registry, op *ModelOperation) (*ModelOperation, error)

CheckModelOperation checks the status of a background model operation by looking up the model and calling its Check method.

func GenerateOperation added in v1.3.0

func GenerateOperation(ctx context.Context, r *registry.Registry, opts ...GenerateOption) (*ModelOperation, error)

GenerateOperation generates a model response as a long-running operation based on the provided options.

type ModelOptions added in v0.7.0

type ModelOptions struct {
	// ConfigSchema is the JSON schema for the model's config. Inferred from the
	// constructor's Config type parameter when nil.
	ConfigSchema map[string]any
	// Label is a user-friendly name for the model. Defaults to its name.
	Label string
	// Stage indicates the maturity stage of the model.
	Stage ModelStage
	// Supports describes what the model can do. A nil value claims nothing.
	Supports *ModelSupports
	// Versions lists the model versions a request may pin through its config.
	Versions []string
	// Metadata is arbitrary key-value data attached to the action descriptor.
	Metadata map[string]any
}

ModelOptions represents the configuration options for a model.

func (ModelOptions) Overlay added in v1.12.0

func (o ModelOptions) Overlay(override ModelOptions) ModelOptions

Overlay returns o with every field set in override replacing o's. A field left at its zero value in override keeps o's.

type ModelOutputConfig added in v0.3.0

type ModelOutputConfig struct {
	// Constrained indicates whether to enforce strict adherence to the schema.
	Constrained bool `json:"constrained,omitempty"`
	// ContentType specifies the MIME type of the output content.
	ContentType string `json:"contentType,omitempty"`
	// Format specifies the desired output format (e.g., "json", "text").
	Format string `json:"format,omitempty"`
	// Schema is a JSON Schema describing the desired structure of the output.
	Schema map[string]any `json:"schema,omitempty"`
}

OutputConfig describes the structure that the model's output should conform to. If Format is OutputFormatJSON, then Schema can describe the desired form of the generated JSON.

type ModelParams added in v1.7.0

type ModelParams struct {
	// Request is the model request about to be sent.
	Request *ModelRequest
	// Callback is the streaming callback, or nil if not streaming.
	Callback ModelStreamCallback
}

ModelParams holds params for the WrapModel hook.

type ModelRef added in v0.5.0

type ModelRef struct {
	// contains filtered or unexported fields
}

ModelRef is a struct to hold model name and configuration.

ModelRef supports JSON marshaling: it serializes as a plain string when only a name is present, or as {"name": "...", "config": ...} when configuration is also set.

func NewModelRef added in v0.5.0

func NewModelRef(name string, config any) ModelRef

NewModelRef creates a new ModelRef with the given name and configuration.

Example

This example demonstrates creating a model reference with configuration.

package main

import (
	"fmt"

	"github.com/firebase/genkit/go/ai"
)

func main() {
	// Create a reference to a model with custom configuration
	// The config type depends on the model provider
	modelRef := ai.NewModelRef("googleai/gemini-flash-latest", map[string]any{
		"temperature": 0.7,
	})

	fmt.Println("Model name:", modelRef.Name())
}
Output:
Model name: googleai/gemini-flash-latest

func (ModelRef) Config added in v0.5.0

func (m ModelRef) Config() any

Config returns the configuration to use by default for this model.

func (ModelRef) JSONSchema added in v1.7.0

func (ModelRef) JSONSchema() *jsonschema.Schema

JSONSchema implements the invopop/jsonschema customSchemaImpl interface so that schema reflection produces the correct object schema instead of an empty object (ModelRef has only unexported fields).

func (ModelRef) MarshalJSON added in v1.7.0

func (m ModelRef) MarshalJSON() ([]byte, error)

MarshalJSON implements json.Marshaler. ModelRef always marshals as a JSON object with "name" and optional "config" fields.

func (ModelRef) Name added in v0.5.0

func (m ModelRef) Name() string

Name returns the name of the model.

func (*ModelRef) UnmarshalJSON added in v1.7.0

func (m *ModelRef) UnmarshalJSON(data []byte) error

UnmarshalJSON implements json.Unmarshaler. It accepts either a JSON object with "name" and optional "config" fields, or a plain string (interpreted as the model name).

type ModelReference added in v1.5.0

type ModelReference struct {
	Config any    `json:"config,omitempty"`
	Name   string `json:"name,omitempty"`
}

type ModelRequest added in v0.3.0

type ModelRequest struct {
	// Config holds model-specific configuration parameters.
	Config any `json:"config,omitempty"`
	// Docs provides retrieved documents to be used as context for this generation.
	Docs []*Document `json:"docs,omitempty"`
	// Messages contains the conversation history for the model.
	Messages []*Message `json:"messages,omitempty"`
	// Output describes the desired response format.
	Output *ModelOutputConfig `json:"output,omitempty"`
	// ToolChoice controls how the model uses tools (auto, required, or none).
	ToolChoice ToolChoice `json:"toolChoice,omitempty"`
	// Tools lists the available tools that the model can ask the client to run.
	Tools []*ToolDefinition `json:"tools,omitempty"`
}

A ModelRequest is a request to generate completions from a model.

func NewModelRequest added in v0.3.0

func NewModelRequest(config any, messages ...*Message) *ModelRequest

NewModelRequest create a new ModelRequest with provided config and messages.

type ModelResponse added in v0.3.0

type ModelResponse struct {
	// Custom contains model-specific extra information. Deprecated: use Raw instead.
	Custom any `json:"custom,omitempty"`
	// FinishMessage provides additional details about why generation finished.
	FinishMessage string `json:"finishMessage,omitempty"`
	// FinishReason indicates why generation stopped (e.g., stop, length, blocked).
	FinishReason FinishReason `json:"finishReason,omitempty"`
	// LatencyMs is the time the request took in milliseconds.
	LatencyMs float64 `json:"latencyMs,omitempty"`
	// Message contains the generated response content.
	Message *Message `json:"message,omitempty"`
	// Operation provides information about a long-running background task if applicable.
	Operation *Operation `json:"operation,omitempty"`
	// Raw contains the unprocessed model-specific response data.
	Raw any `json:"raw,omitempty"`
	// Request is the ModelRequest struct used to trigger this response.
	Request *ModelRequest `json:"request,omitempty"`
	// Usage describes how many resources were used by this generation request.
	Usage *GenerationUsage `json:"usage,omitempty"`
	// contains filtered or unexported fields
}

A ModelResponse is a model's response to a ModelRequest.

func Generate added in v0.1.0

func Generate(ctx context.Context, r api.Registry, opts ...GenerateOption) (*ModelResponse, error)

Generate generates a model response based on the provided options.

func GenerateData added in v0.1.0

func GenerateData[Out any](ctx context.Context, r api.Registry, opts ...GenerateOption) (*Out, *ModelResponse, error)

GenerateData runs a generate request and returns strongly-typed output. If the response doesn't contain text output (e.g., contains tool requests or interrupts instead), the output will be nil and no error is returned. Check resp.Interrupts() or resp.ToolRequests() to handle these cases.

The output format is JSON with a schema inferred from Out; an explicit WithOutputSchema or WithOutputSchemaName overrides the schema while extraction into Out keeps working. Overriding the format itself with a non-JSON WithOutputFormat or WithOutputEnums breaks that extraction: the response text will not parse into Out.

func GenerateWithRequest added in v0.3.0

func GenerateWithRequest(ctx context.Context, r api.Registry, opts *GenerateActionOptions, mmws []ModelMiddleware, cb ModelStreamCallback) (*ModelResponse, error)

GenerateWithRequest is the central generation implementation for ai.Generate(), prompt.Execute(), and the GenerateAction direct call.

func (*ModelResponse) History added in v0.3.0

func (mr *ModelResponse) History() []*Message

History returns messages from the request combined with the response message to represent the conversation history. The result is always freshly allocated, so callers may retain or append to it without disturbing Request.Messages.

func (*ModelResponse) Interrupts added in v1.0.3

func (mr *ModelResponse) Interrupts() []*Part

Interrupts returns the interrupted tool request parts from the response.

func (*ModelResponse) Media added in v1.1.0

func (mr *ModelResponse) Media() string

Media returns the media content of the ModelResponse as a string.

Only the first media part is returned, and its content type is left behind, so a response carrying more than one image, or one whose content type is needed to render it, is better read with ModelResponse.MediaParts.

func (*ModelResponse) MediaParts added in v1.12.0

func (mr *ModelResponse) MediaParts() []*Part

MediaParts returns every media part of the ModelResponse, each carrying its content type alongside its data. It returns nil if the response has none.

A model that may answer with media often answers with media alone, so this pairs with ModelResponse.Text rather than replacing it: the two read disjoint halves of a response and either may come back empty.

func (*ModelResponse) Output added in v0.3.0

func (mr *ModelResponse) Output(v any) error

Output parses the structured output from the response and unmarshals it into v. If a format handler is set, it uses the handler's ParseOutput method. Otherwise, it falls back to parsing the response text as JSON.

func (*ModelResponse) Reasoning added in v0.6.0

func (mr *ModelResponse) Reasoning() string

Reasoning concatenates all reasoning parts present in the message

func (*ModelResponse) Text added in v0.3.0

func (mr *ModelResponse) Text() string

Text returns the contents of the first candidate in a ModelResponse as a string. It returns an empty string if there are no candidates or if the candidate has no message.

func (*ModelResponse) ToolRequests added in v0.5.0

func (mr *ModelResponse) ToolRequests() []*Part

ToolRequests returns the tool requests from the response.

type ModelResponseChunk added in v0.3.0

type ModelResponseChunk struct {
	// Aggregated indicates whether the chunk includes all data from previous chunks.
	// If false, the chunk is considered incremental.
	Aggregated bool `json:"aggregated,omitempty"`
	// Content is the chunk of message parts to stream right now.
	Content []*Part `json:"content,omitempty"`
	// Custom contains model-specific extra information attached to this chunk.
	Custom any `json:"custom,omitempty"`
	// Index of the message this chunk belongs to.
	Index int `json:"index"`
	// Role indicates the entity that generated this chunk.
	Role Role `json:"role,omitempty"`
	// contains filtered or unexported fields
}

A ModelResponseChunk is the portion of the ModelResponse that is passed to a streaming callback.

func (*ModelResponseChunk) Interrupts added in v1.10.0

func (c *ModelResponseChunk) Interrupts() []*Part

Interrupts returns the interrupted tool request parts from the chunk.

func (*ModelResponseChunk) Output added in v1.3.0

func (c *ModelResponseChunk) Output(v any) error

Output parses the chunk using the format handler and unmarshals the result into v. Returns an error if the format handler is not set or does not support parsing chunks.

func (*ModelResponseChunk) Reasoning added in v1.3.0

func (c *ModelResponseChunk) Reasoning() string

Reasoning returns the reasoning content of the ModelResponseChunk as a string. It returns an empty string if there is no Content in the response chunk.

func (*ModelResponseChunk) Text added in v0.3.0

func (c *ModelResponseChunk) Text() string

Text returns the text content of the ModelResponseChunk as a string, concatenating its text parts and skipping every other kind, as Message.Text does. It returns an empty string if the chunk has none. For the parsed structured output, use ModelResponseChunk.Output instead.

func (*ModelResponseChunk) ToolResponses added in v1.10.0

func (c *ModelResponseChunk) ToolResponses() []*Part

ToolResponses returns the tool response parts from the chunk. Use Part.IsPartial to distinguish streaming progress updates from final tool results.

type ModelStage added in v0.3.0

type ModelStage string

ModelStage indicates the development stage of a model.

const (
	ModelStageFeatured   ModelStage = "featured"
	ModelStageStable     ModelStage = "stable"
	ModelStageUnstable   ModelStage = "unstable"
	ModelStageLegacy     ModelStage = "legacy"
	ModelStageDeprecated ModelStage = "deprecated"
)

type ModelStreamCallback added in v0.3.0

type ModelStreamCallback = func(context.Context, *ModelResponseChunk) error

ModelStreamCallback is a stream callback of a ModelAction.

type ModelStreamValue added in v1.3.0

type ModelStreamValue = StreamValue[struct{}, *ModelResponseChunk]

ModelStreamValue is a stream value for a model response. Out is never set because the output is already available in the Response field.

type ModelSupports added in v0.3.0

type ModelSupports struct {
	// Constrained indicates the level of constrained generation support (none, all, or no-tools).
	Constrained ConstrainedSupport `json:"constrained,omitempty"`
	// ContentType lists the content types the model supports for output.
	ContentType []string `json:"contentType,omitempty"`
	// Context indicates whether the model can natively support document-based context grounding.
	Context bool `json:"context,omitempty"`
	// LongRunning indicates whether the model supports long-running operations.
	LongRunning bool `json:"longRunning,omitempty"`
	// Media indicates whether the model can process media as part of the prompt (multimodal input).
	Media bool `json:"media,omitempty"`
	// Multiturn indicates whether the model can process historical messages passed with a prompt.
	Multiturn bool `json:"multiturn,omitempty"`
	// Output lists the types of data the model can generate.
	Output []string `json:"output,omitempty"`
	// SystemRole indicates whether the model can accept messages with role "system".
	SystemRole bool `json:"systemRole,omitempty"`
	// ToolChoice indicates whether the model supports controlling tool choice (e.g., forced tool calling).
	ToolChoice bool `json:"toolChoice,omitempty"`
	// Tools indicates whether the model can perform tool calls.
	Tools bool `json:"tools,omitempty"`
}

ModelSupports describes the capabilities that a model supports.

type MultipartToolFunc added in v1.3.0

type MultipartToolFunc[In any] = func(ctx *ToolContext, input In) (*MultipartToolResponse, error)

MultipartToolFunc is the function type for multipart tool implementations. Unlike regular tools that return just an output value, multipart tools can return both an output value and additional content parts (like media).

type MultipartToolResponse added in v1.3.0

type MultipartToolResponse struct {
	// Content holds additional message parts providing context or details.
	Content  []*Part        `json:"content,omitempty"`
	Metadata map[string]any `json:"metadata,omitempty"`
	// Output contains the structured output data from the tool.
	Output any `json:"output,omitempty"`
}

MultipartToolResponse represents a tool response with both structured output and content parts.

type Operation added in v1.3.0

type Operation struct {
	// Action is the name of the action being performed by this operation.
	Action string `json:"action"`
	// Done indicates whether the operation has completed.
	Done bool `json:"done"`
	// Error contains error information if the operation failed.
	Error *OperationError `json:"error,omitempty"`
	// Id is the unique identifier for this operation.
	Id string `json:"id"`
	// Metadata contains additional information about the operation.
	Metadata map[string]any `json:"metadata,omitempty"`
	// Output contains the result of the operation if it has completed successfully.
	Output any `json:"output,omitempty"`
}

Operation represents a long-running background task.

type OperationError added in v1.3.0

type OperationError struct {
	// Message describes the error that occurred.
	Message string `json:"message,omitempty"`
}

OperationError contains error information for a failed operation.

type OutputOption added in v0.3.0

type OutputOption interface {
	PromptOption
	GenerateOption
	// contains filtered or unexported methods
}

OutputOption is an option for the output of a prompt or generate request. It applies only to DefinePrompt() and Generate().

func WithCustomConstrainedOutput added in v0.5.0

func WithCustomConstrainedOutput() OutputOption

WithCustomConstrainedOutput opts out of using the model's native constrained output generation.

By default, the system will use the model's native constrained output capabilities when available. When this option is set, or when the model doesn't support native constraints, the system will use custom implementation to guide the model toward producing properly formatted output.

func WithOutputEnums added in v1.3.0

func WithOutputEnums[T ~string](values ...T) OutputOption

WithOutputEnums sets the output format to enum and the schema based on the given values. Accepts any string-based type (e.g. type MyEnum string).

func WithOutputFormat added in v0.1.0

func WithOutputFormat(format string) OutputOption

WithOutputFormat sets the format of the output.

func WithOutputInstructions added in v0.5.0

func WithOutputInstructions(instructions string) OutputOption

WithOutputInstructions sets custom instructions for constraining output format in the prompt.

When WithOutputType is used without this option, default instructions will be automatically set. If you provide empty instructions, no instructions will be added to the prompt.

This will automatically set WithCustomConstrainedOutput.

func WithOutputType added in v0.3.0

func WithOutputType(output any) OutputOption

WithOutputType sets the output format to JSON and the schema derived from the given value.

type OutputSchemaOption added in v1.12.0

type OutputSchemaOption interface {
	OutputOption
	ToolOption
}

OutputSchemaOption is an OutputOption that provides an explicit output schema, inline or by name. Unlike the schema-inference and generation-steering options, it also applies to the tool constructors: a tool's output shape is decided by its function, so the only output options that make sense there are explicit schemas standing in for an Out type parameter of 'any'; a tool whose output a Go type can express should use the type parameter instead.

func WithOutputSchema added in v0.1.0

func WithOutputSchema(schema map[string]any) OutputSchemaOption

WithOutputSchema manually provides a schema map for the output.

func WithOutputSchemaName added in v1.3.0

func WithOutputSchemaName(name string) OutputSchemaOption

WithOutputSchemaName sets the schema name that will be resolved at execution time. Register the schema with github.com/firebase/genkit/go/genkit.DefineSchema.

type Part

type Part struct {
	Kind         PartKind       `json:"kind,omitempty"`
	ContentType  string         `json:"contentType,omitempty"`  // valid for kind==blob
	Text         string         `json:"text,omitempty"`         // valid for kind∈{text,blob}
	ToolRequest  *ToolRequest   `json:"toolRequest,omitempty"`  // valid for kind==partToolRequest
	ToolResponse *ToolResponse  `json:"toolResponse,omitempty"` // valid for kind==partToolResponse
	Resource     *ResourcePart  `json:"resource,omitempty"`     // valid for kind==partResource
	Custom       map[string]any `json:"custom,omitempty"`       // valid for plugin-specific custom parts
	Metadata     map[string]any `json:"metadata,omitempty"`     // valid for all kinds
}

A Part is one part of a Document. This may be plain text or it may be a URL (possibly a "data:" URL with embedded data).

func NewCustomPart added in v0.5.1

func NewCustomPart(customData map[string]any) *Part

NewCustomPart returns a Part containing custom plugin-specific data.

func NewDataPart

func NewDataPart(contents string) *Part

NewDataPart returns a Part containing raw string data.

Example

This example demonstrates creating a data part for raw string content.

package main

import (
	"fmt"

	"github.com/firebase/genkit/go/ai"
)

func main() {
	// Create a data part with raw string content
	part := ai.NewDataPart(`{"name": "Alice", "age": 30}`)
	fmt.Println("Is data part:", part.IsData())
	fmt.Println("Content:", part.Text)
}
Output:
Is data part: true
Content: {"name": "Alice", "age": 30}

func NewJSONPart

func NewJSONPart(text string) *Part

NewJSONPart returns a Part containing JSON.

func NewMediaPart

func NewMediaPart(mimeType, contents string) *Part

NewMediaPart returns a Part containing structured data described by the given mimeType.

Example

This example demonstrates creating a media part for images or other media.

package main

import (
	"fmt"

	"github.com/firebase/genkit/go/ai"
)

func main() {
	// Create a media part with base64-encoded image data
	// In practice, you would encode actual image bytes
	imageData := "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJ..."
	part := ai.NewMediaPart("image/png", imageData)

	fmt.Println("Is media:", part.IsMedia())
	fmt.Println("Content type:", part.ContentType)
}
Output:
Is media: true
Content type: image/png

func NewPartialToolResponsePart added in v1.10.0

func NewPartialToolResponsePart(r *ToolResponse) *Part

NewPartialToolResponsePart returns a Part containing a partial tool response. Partial tool responses are streamed during tool execution for client-side display (e.g., progress indicators) and are not included in conversation history.

func NewReasoningPart added in v0.6.0

func NewReasoningPart(text string, signature []byte) *Part

NewReasoningPart returns a Part containing reasoning text

func NewResourcePart added in v0.7.0

func NewResourcePart(uri string) *Part

NewResourcePart returns a Part containing a resource reference.

func NewResponseForToolRequest added in v0.6.0

func NewResponseForToolRequest(p *Part, output any) *Part

NewResponseForToolRequest returns a Part containing the results of executing the tool request part.

func NewTextPart

func NewTextPart(text string) *Part

NewTextPart returns a Part containing text.

Example

This example demonstrates creating different types of message parts.

package main

import (
	"fmt"

	"github.com/firebase/genkit/go/ai"
)

func main() {
	// Create a text part
	part := ai.NewTextPart("Hello, world!")
	fmt.Println(part.Text)
}
Output:
Hello, world!

func NewToolRequestPart

func NewToolRequestPart(r *ToolRequest) *Part

NewToolRequestPart returns a Part containing a request from the model to the client to run a Tool. (Only genkit plugins should need to use this function.)

func NewToolResponsePart

func NewToolResponsePart(r *ToolResponse) *Part

NewToolResponsePart returns a Part containing the results of applying a Tool that the model requested.

func (*Part) Clone added in v1.7.0

func (p *Part) Clone() *Part

Clone returns a shallow copy of the Part with its own Metadata and Custom maps. Callers can add or remove map keys without mutating the original.

func (*Part) IsAudio added in v1.0.0

func (p *Part) IsAudio() bool

IsAudio reports whether the Part contains an audio file.

func (*Part) IsCustom added in v0.5.1

func (p *Part) IsCustom() bool

IsCustom reports whether the Part contains custom plugin-specific data.

func (*Part) IsData

func (p *Part) IsData() bool

IsData reports whether the Part contains unstructured data.

func (*Part) IsImage added in v1.0.0

func (p *Part) IsImage() bool

IsImage reports whether the Part contains an image.

func (*Part) IsInterrupt added in v1.0.3

func (p *Part) IsInterrupt() bool

IsInterrupt reports whether the Part contains a tool request that was interrupted.

func (*Part) IsMedia

func (p *Part) IsMedia() bool

IsMedia reports whether the Part contains structured media data.

func (*Part) IsPartial added in v1.10.0

func (p *Part) IsPartial() bool

IsPartial reports whether the Part contains a partial tool response streamed during tool execution (e.g., a progress update).

func (*Part) IsReasoning added in v0.6.0

func (p *Part) IsReasoning() bool

IsReasoning reports whether the Part contains a reasoning text

func (*Part) IsResource added in v0.7.0

func (p *Part) IsResource() bool

IsResource reports whether the Part contains a resource reference.

func (*Part) IsText

func (p *Part) IsText() bool

IsText reports whether the Part contains plain text.

func (*Part) IsToolRequest

func (p *Part) IsToolRequest() bool

IsToolRequest reports whether the Part contains a request to run a tool.

func (*Part) IsToolResponse

func (p *Part) IsToolResponse() bool

IsToolResponse reports whether the Part contains the result of running a tool.

func (*Part) IsVideo added in v1.0.0

func (p *Part) IsVideo() bool

IsVideo reports whether the Part contains a video.

func (Part) JSONSchemaAlias

func (Part) JSONSchemaAlias() any

JSONSchemaAlias tells the JSON schema reflection code to use a different type for the schema for this type. This is needed because the JSON marshaling of Part uses a schema that matches the TypeScript code, rather than the natural JSON marshaling. This matters because the current JSON validation code works by marshaling the JSON.

func (*Part) MarshalJSON

func (p *Part) MarshalJSON() ([]byte, error)

MarshalJSON is called by the JSON marshaler to write out a Part.

func (*Part) UnmarshalJSON

func (p *Part) UnmarshalJSON(b []byte) error

UnmarshalJSON is called by the JSON unmarshaler to read a Part.

func (*Part) UnmarshalYAML added in v0.3.0

func (p *Part) UnmarshalYAML(unmarshal func(any) error) error

UnmarshalYAML implements goccy/go-yaml library's InterfaceUnmarshaler interface.

type PartKind

type PartKind int8

PartKind is what a Part carries: text, media, a tool request, and so on.

const (
	PartText PartKind = iota
	PartMedia
	PartData
	PartToolRequest
	PartToolResponse
	PartCustom
	PartReasoning
	PartResource
)

type PartsFn added in v1.12.0

type PartsFn = func(context.Context, any) ([]*Part, error)

PartsFn is a function that generates message content from a prompt's input.

The input is untyped. WithPromptPartsFn and WithSystemPartsFn take a function with a concrete input type and convert for you.

type PathMetadata added in v0.3.0

type PathMetadata struct {
	// Error contains error information if the path failed.
	Error string `json:"error,omitempty"`
	// Latency is the execution time for this path in milliseconds.
	Latency float64 `json:"latency,omitempty"`
	// Path is the identifier for this execution path.
	Path string `json:"path,omitempty"`
	// Status indicates the outcome of this path.
	Status string `json:"status,omitempty"`
}

PathMetadata contains metadata about a single execution path in a trace.

type Prompt

type Prompt interface {
	// Name returns the name of the prompt.
	Name() string
	// Execute executes the prompt with the given options and returns a [ModelResponse].
	//
	// # Options
	//
	// Input:
	//
	//   - [WithInput]: Supply the prompt's input, overriding the default from [WithInputType]
	//
	// Conversation:
	//
	//   - [WithMessages]: Supply the conversation this execution continues
	//   - [WithMessagesFn]: As above, computed from the input
	//
	// A prompt that declares a conversation of its own decides where these go,
	// with {{history}} or [HistoryFromContext]. One that declares none uses
	// them directly, between the system message and the user prompt.
	//
	// Overrides, each replacing what the prompt was defined with:
	//
	//   - [WithModel], [WithModelName]: Call a different model
	//   - [WithConfig]: Replace the generation config
	//   - [WithDocs], [WithTextDocs]: Replace the context documents, skipping any [WithDocsFn]
	//   - [WithTools], [WithToolChoice], [WithMaxTurns], [WithReturnToolRequests]: Change tool behavior
	//   - [WithMiddleware], [WithUse]: Add middleware for this execution
	//   - [WithStreaming]: Receive streamed chunks
	Execute(ctx context.Context, opts ...PromptExecuteOption) (*ModelResponse, error)
	// ExecuteStream executes the prompt with streaming and returns an iterator.
	// It accepts the same options as Execute.
	ExecuteStream(ctx context.Context, opts ...PromptExecuteOption) iter.Seq2[*ModelStreamValue, error]
	// Render renders the prompt with the given input and returns a [GenerateActionOptions] to be used with [GenerateWithRequest].
	Render(ctx context.Context, input any) (*GenerateActionOptions, error)
}

Prompt is the interface for a prompt that can be executed and rendered.

func DefinePrompt

func DefinePrompt(r api.Registry, name string, opts ...PromptOption) Prompt

DefinePrompt creates a new Prompt and registers it.

func LoadPrompt added in v0.3.0

func LoadPrompt(r api.Registry, dir, filename, namespace string) Prompt

LoadPrompt loads a single prompt from a directory on the local filesystem into the registry.

func LoadPromptFromFS added in v1.3.0

func LoadPromptFromFS(r api.Registry, fsys fs.FS, dir, filename, namespace string) Prompt

LoadPromptFromFS loads a single prompt from a filesystem into the registry. The fsys parameter should be an fs.FS implementation (e.g., embed.FS or os.DirFS). The dir parameter specifies the directory within the filesystem where the prompt is located.

func LoadPromptFromSource added in v1.3.0

func LoadPromptFromSource(r api.Registry, source, name, namespace string) (Prompt, error)

LoadPromptFromSource loads a prompt from raw .prompt file content. The source parameter should contain the complete .prompt file text (frontmatter + template). The name parameter is the prompt name (may include variant suffix like "myPrompt.variant").

func LookupPrompt

func LookupPrompt(r api.Registry, name string) Prompt

LookupPrompt looks up a Prompt registered by DefinePrompt. It returns nil if the prompt was not defined.

type PromptExecuteOption added in v0.5.0

type PromptExecuteOption interface {
	// contains filtered or unexported methods
}

PromptExecuteOption is an option for executing a prompt. It applies only to [prompt.Execute].

func WithInput added in v0.3.0

func WithInput(input any) PromptExecuteOption

WithInput sets the input for the prompt request. Input must conform to the prompt's input schema and can either be a map[string]any or a struct of the same api. Repeating this option takes the last input set. APIs that take the input as a typed argument (DataPrompt.Execute, DataPrompt.ExecuteStream) apply that argument after these options, so the typed argument wins.

type PromptFn added in v0.5.0

type PromptFn = func(context.Context, any) (string, error)

PromptFn is a function that generates a prompt from a prompt's input.

The input is untyped. WithPromptFn and WithSystemFn take a function with a concrete input type and convert for you.

type PromptOption added in v0.3.0

type PromptOption interface {
	// contains filtered or unexported methods
}

PromptOption is an option for defining a prompt. It applies only to DefinePrompt().

func WithDescription added in v0.3.0

func WithDescription(description string) PromptOption

WithDescription sets the description of the prompt. Repeating this option takes the last description set.

func WithDocsFn added in v1.12.0

func WithDocsFn[In any](fn func(context.Context, In) ([]*Document, error)) PromptOption

WithDocsFn sets the function that selects the context documents for a prompt, such as by querying a retriever. It applies only to DefinePrompt, since only a prompt has input to give it.

fn receives the prompt's input converted to In.

Documents accumulate: repeating this option, or combining it with WithDocs or WithTextDocs, adds to the set rather than replacing it. The fixed documents resolve first, then the computed ones, each in call order.

func WithMessagesTemplate added in v1.12.0

func WithMessagesTemplate(text string, args ...any) PromptOption

WithMessagesTemplate sets the conversation to a dotprompt template. Optional args are applied to text with fmt.Sprintf.

This is the multi-turn form: each {{role "..."}} block starts a new message, so one template can express a system preamble, few-shot examples, and the user turn. It is what a .prompt file's body compiles to.

Messages passed to Prompt.Execute reach the template at {{history}}, or, without an explicit {{history}}, are inserted before its final user message.

Unlike WithMessages, the text here is compiled, so literal braces in it are template syntax. Content from a user or a remote source belongs in WithMessages or WithMessagesFn, which never compile it.

The template is the whole conversation, so nothing else can contribute to it: passing this in the same DefinePrompt call as WithMessages or WithMessagesFn panics. Write those messages as {{role}} blocks instead. Repeating this option alone fills one slot, so the last one set wins.

Compiling needs a prompt, so this is a PromptOption: passing it to Generate or Prompt.Execute does not compile. Use WithMessages there.

func WithMetadata added in v0.3.0

func WithMetadata(metadata map[string]any) PromptOption

WithMetadata sets arbitrary metadata for the prompt. Repeating this option replaces the metadata rather than merging it.

type PromptingOption added in v0.3.0

type PromptingOption interface {
	PromptOption
	GenerateOption
	// contains filtered or unexported methods
}

PromptingOption is an option for the system and user prompts of a prompt or generate request. It applies only to DefinePrompt() and Generate().

func WithPrompt added in v0.5.0

func WithPrompt(text string, args ...any) PromptingOption

WithPrompt sets the user prompt message. The user prompt is always the last message in the list.

With DefinePrompt, the text is compiled as a dotprompt template against the prompt's input, so it may reference input fields such as {{name}}. args, if given, are applied with fmt.Sprintf first.

A {{role}} marker in the text is an error: this slot is one user message. WithMessagesTemplate is where turns with their own roles belong.

func WithPromptFn added in v0.3.0

func WithPromptFn[In any](fn func(context.Context, In) (string, error)) PromptingOption

WithPromptFn sets the function that generates the user prompt message. The user prompt is always the last message in the list.

fn receives the prompt's input converted to In, or the zero value of In when there is none, as at Generate. Its string is used verbatim, never compiled as a template, so it may safely hold user content and literal braces. Use WithPromptPartsFn to return non-text content.

It shares one slot with WithPrompt, WithPromptParts, and WithPromptPartsFn: the last one set wins.

func WithPromptParts added in v1.12.0

func WithPromptParts(parts ...*Part) PromptingOption

WithPromptParts sets the content of the user prompt message. The user prompt is always the last message in the list.

It is the multi-part form of WithPrompt, for prompts that mix text with media or other non-text parts. The parts are used verbatim, never compiled as a template. Use WithPromptPartsFn when the content depends on the prompt's input.

genkit.Generate(ctx, g,
	ai.WithModelName("googleai/gemini-flash-latest"),
	ai.WithPromptParts(
		ai.NewTextPart("What is in this image?"),
		ai.NewMediaPart("image/png", imageURL),
	),
)

It shares one slot with WithPrompt, WithPromptFn, and WithPromptPartsFn: the last one set wins.

func WithPromptPartsFn added in v1.12.0

func WithPromptPartsFn[In any](fn func(context.Context, In) ([]*Part, error)) PromptingOption

WithPromptPartsFn sets the function that generates the content of the user prompt message. The user prompt is always the last message in the list.

It is the multi-part form of WithPromptFn. The returned parts are used verbatim.

It shares one slot with WithPrompt, WithPromptParts, and WithPromptFn: the last one set wins.

func WithSystem added in v0.5.0

func WithSystem(text string, args ...any) PromptingOption

WithSystem sets the system prompt message. The system prompt is always the first message in the list.

With DefinePrompt, the text is compiled as a dotprompt template against the prompt's input, so it may reference input fields such as {{name}}. args, if given, are applied with fmt.Sprintf first.

A {{role}} marker in the text is an error: this slot is one system message. WithMessagesTemplate is where turns with their own roles belong.

func WithSystemFn added in v0.3.0

func WithSystemFn[In any](fn func(context.Context, In) (string, error)) PromptingOption

WithSystemFn sets the function that generates the system prompt message. The system prompt is always the first message in the list.

fn receives the prompt's input converted to In, or the zero value of In when there is none, as at Generate. Its string is used verbatim, never compiled as a template, so it may safely hold user content and literal braces. Use WithSystemPartsFn to return non-text content.

It shares one slot with WithSystem, WithSystemParts, and WithSystemPartsFn: the last one set wins.

func WithSystemParts added in v1.12.0

func WithSystemParts(parts ...*Part) PromptingOption

WithSystemParts sets the content of the system prompt message. The system prompt is always the first message in the list.

It is the multi-part form of WithSystem, for instructions that mix text with media or other non-text parts. The parts are used verbatim, never compiled as a template. Use WithSystemPartsFn when the content depends on the prompt's input.

It shares one slot with WithSystem, WithSystemFn, and WithSystemPartsFn: the last one set wins.

func WithSystemPartsFn added in v1.12.0

func WithSystemPartsFn[In any](fn func(context.Context, In) ([]*Part, error)) PromptingOption

WithSystemPartsFn sets the function that generates the content of the system prompt message. The system prompt is always the first message in the list.

It is the multi-part form of WithSystemFn. The returned parts are used verbatim.

It shares one slot with WithSystem, WithSystemParts, and WithSystemFn: the last one set wins.

type RankedDocumentData added in v0.3.0

type RankedDocumentData struct {
	// Content holds the document's parts (text and media).
	Content []*Part `json:"content,omitempty"`
	// Metadata contains the reranking score and other arbitrary key-value data.
	Metadata *RankedDocumentMetadata `json:"metadata,omitempty"`
}

RankedDocumentData represents a document with a relevance score from reranking.

type RankedDocumentMetadata added in v0.3.0

type RankedDocumentMetadata struct {
	// Score is the relevance score assigned by the reranker.
	Score float64 `json:"score,omitempty"`
}

RankedDocumentMetadata contains the relevance score and other metadata for a reranked document.

type RerankerRequest added in v0.3.0

type RerankerRequest struct {
	// Documents is the array of documents to rerank.
	Documents []*Document `json:"documents,omitempty"`
	// Options contains reranker-specific configuration parameters.
	Options any `json:"options,omitempty"`
	// Query is the document to use for reranking.
	Query *Document `json:"query,omitempty"`
}

RerankerRequest represents a request to rerank documents based on relevance.

type RerankerResponse added in v0.3.0

type RerankerResponse struct {
	// Documents is the array of reranked documents with scores.
	Documents []*RankedDocumentData `json:"documents,omitempty"`
}

RerankerResponse contains the reranked documents with relevance scores.

type Resource added in v0.7.0

type Resource interface {
	// Name returns the name of the resource.
	Name() string
	// Matches reports whether this resource matches the given URI.
	Matches(uri string) bool
	// ExtractVariables extracts variables from a URI using this resource's template.
	ExtractVariables(uri string) (map[string]string, error)
	// Execute runs the resource with the given input.
	Execute(ctx context.Context, input *ResourceInput) (*ResourceOutput, error)
	// Register registers the resource with the given registry.
	Register(r api.Registry)
}

Resource represents an instance of a resource.

func LookupResource added in v0.7.0

func LookupResource(r api.Registry, name string) Resource

LookupResource looks up the resource in the registry by provided name and returns it.

func NewResource added in v0.7.0

func NewResource(name string, opts *ResourceOptions, fn ResourceFunc) Resource

NewResource creates a resource but does not register it in the registry. It can be registered later via the Register method.

type ResourceFunc added in v0.7.0

type ResourceFunc = func(context.Context, *ResourceInput) (*ResourceOutput, error)

ResourceFunc is a function that loads content for a resource.

type ResourceInput added in v0.7.0

type ResourceInput struct {
	URI       string            `json:"uri"`       // The resource URI
	Variables map[string]string `json:"variables"` // Extracted variables from URI template matching
}

ResourceInput represents the input to a resource function.

type ResourceOptions added in v0.7.0

type ResourceOptions struct {
	// URI is the resource's static URI. Mutually exclusive with Template.
	URI string
	// Template is a URI template the resource matches. Mutually exclusive with URI.
	Template string
	// Description is the resource's human-readable description.
	Description string
	// Metadata is arbitrary key-value data attached to the action descriptor.
	Metadata map[string]any
}

ResourceOptions configures a resource definition.

type ResourceOutput added in v0.7.0

type ResourceOutput struct {
	Content []*Part `json:"content"` // The content parts returned by the resource
}

ResourceOutput represents the output from a resource function.

type ResourcePart added in v0.7.0

type ResourcePart struct {
	// Uri is the URI of the external resource.
	Uri string `json:"uri,omitempty"`
}

type RespondOptions added in v0.6.0

type RespondOptions struct {
	// Metadata is additional metadata to include in the response.
	Metadata map[string]any
}

RespondOptions provides configuration options for responding to a tool request.

type RespondWithOption added in v1.4.0

type RespondWithOption[Out any] interface {
	// contains filtered or unexported methods
}

RespondWithOption is a functional option for ToolAction.RespondWith.

func WithResponseMetadata added in v1.4.0

func WithResponseMetadata[Out any](meta map[string]any) RespondWithOption[Out]

WithResponseMetadata sets metadata for the response. Repeating this option replaces the metadata rather than merging it.

type RestartOptions added in v0.6.0

type RestartOptions struct {
	// ReplaceInput allows replacing the existing input arguments to the tool with different ones,
	// for example if the user revised an action before confirming. When input is replaced,
	// the existing tool request will be amended in the message history.
	ReplaceInput any
	// ResumedMetadata is the metadata you want to provide to the tool to aide in reprocessing.
	// Defaults to true if none is supplied.
	ResumedMetadata any
}

RestartOptions provides configuration options for restarting a tool.

type RestartWithOption added in v1.4.0

type RestartWithOption[In any] interface {
	// contains filtered or unexported methods
}

RestartWithOption is a functional option for ToolAction.RestartWith.

func WithNewInput added in v1.4.0

func WithNewInput[In any](input In) RestartWithOption[In]

WithNewInput sets a new input value to replace the original tool request input. Repeating this option takes the last input set.

func WithResumedMetadata added in v1.4.0

func WithResumedMetadata[In any](meta map[string]any) RestartWithOption[In]

WithResumedMetadata sets metadata to pass to the resumed tool execution. The metadata will be available in the tool's ToolContext.Resumed field. Repeating this option replaces the metadata rather than merging it.

type Retriever

type Retriever interface {
	// Name returns the name of the retriever.
	Name() string
	// Retrieve retrieves the documents.
	Retrieve(ctx context.Context, req *RetrieverRequest) (*RetrieverResponse, error)
	// Register registers the retriever with the given registry.
	Register(r api.Registry)
}

Retriever represents a document retriever. It is the type to accept as an argument and to look up by name; implementations are created with NewRetrieverAction, or [genkit.DefineRetrieverAction] in an application.

func LookupRetriever

func LookupRetriever(r api.Registry, name string) Retriever

LookupRetriever looks up a registered Retriever by name. It will try to resolve the retriever dynamically if the retriever is not found. It returns nil if the retriever was not resolved.

func NewRetriever deprecated added in v0.7.0

func NewRetriever(name string, opts *RetrieverOptions, fn RetrieverFunc) Retriever

NewRetriever creates a new Retriever.

Deprecated: Use NewRetrieverAction, which passes the request's options to fn as a typed value instead of leaving them type-erased on the request.

type RetrieverAction added in v1.12.0

type RetrieverAction struct {
	// contains filtered or unexported fields
}

RetrieverAction is a retriever backed by a registry action. It is the concrete type returned by NewRetrieverAction; pass it to WithRetriever to retrieve with it, or return it from a plugin's Init for the framework to register.

It implements Retriever and api.Action, so it can be passed anywhere either is accepted. It also promotes core.Action.Run, the typed equivalent of RetrieverAction.Retrieve.

func NewRetrieverAction added in v1.12.0

func NewRetrieverAction[Config any](
	name string,
	opts *RetrieverOptions,
	fn RetrieverActionFunc[Config],
) *RetrieverAction

NewRetrieverAction creates an unregistered RetrieverAction: return it from a plugin's Init for the framework to register, or call RetrieverAction.Register directly. Applications should define retrievers with [genkit.DefineRetrieverAction].

Config is the retriever's typed configuration; it is usually inferred from fn's signature. The framework deserializes the request's raw options into Config before calling fn: the exact Config type (or a pointer to it) and map[string]any (from the Dev UI and other JSON callers) are accepted, and mismatched types are rejected. The request's RetrieverRequest.Options is normalized to the converted value, so it always matches the typed parameter. The config's JSON schema is inferred from Config unless RetrieverOptions.ConfigSchema overrides it.

func (*RetrieverAction) Desc added in v1.12.0

func (r *RetrieverAction) Desc() api.ActionDesc

Desc returns the retriever's action descriptor: its name, schemas, and metadata.

func (*RetrieverAction) Name added in v1.12.0

func (r *RetrieverAction) Name() string

Name returns the registry name of the retriever.

func (*RetrieverAction) Register added in v1.12.0

func (r *RetrieverAction) Register(reg api.Registry)

Register registers the retriever with reg, making it available to lookups and to the Dev UI. A plugin that returns the retriever from its Init does not need to call this.

func (*RetrieverAction) Retrieve added in v1.12.0

Retrieve runs the given Retriever.

func (*RetrieverAction) RunJSON added in v1.12.0

RunJSON runs the retriever on a JSON-encoded RetrieverRequest and returns a JSON-encoded RetrieverResponse. The framework uses it to serve reflection and registry-driven calls; prefer RetrieverAction.Retrieve.

func (*RetrieverAction) RunJSONWithTelemetry added in v1.12.0

RunJSONWithTelemetry is RetrieverAction.RunJSON with the run's telemetry returned alongside the output.

type RetrieverActionFunc added in v1.12.0

type RetrieverActionFunc[Config any] = func(context.Context, *RetrieverRequest, Config) (*RetrieverResponse, error)

RetrieverActionFunc is a RetrieverFunc that additionally receives the request's typed Config: the framework deserializes the request's raw options into it before calling the function (see NewRetrieverAction).

type RetrieverArg added in v0.7.0

type RetrieverArg interface {
	Name() string
}

RetrieverArg is the interface for retriever arguments. It can either be the retriever action itself or a reference to be looked up.

type RetrieverFunc added in v0.5.0

type RetrieverFunc = func(context.Context, *RetrieverRequest) (*RetrieverResponse, error)

RetrieverFunc is the function type for retriever implementations.

type RetrieverOption added in v0.5.0

type RetrieverOption interface {
	// contains filtered or unexported methods
}

RetrieverOption is an option for configuring a retriever request. It applies only to Retriever.Retrieve.

func WithRetriever added in v0.7.0

func WithRetriever(retriever RetrieverArg) RetrieverOption

WithRetriever sets either a Retriever or a RetrieverRef that may contain a config. Passing WithConfig will take precedence over the config in WithRetriever.

func WithRetrieverName added in v0.7.0

func WithRetrieverName(name string) RetrieverOption

WithRetrieverName sets the retriever name to call for document retrieval. The retriever name will be resolved to a Retriever and may error if the reference is invalid.

type RetrieverOptions added in v0.7.0

type RetrieverOptions struct {
	// ConfigSchema is the JSON schema for the retriever's config.
	ConfigSchema map[string]any `json:"configSchema,omitempty"`
	// Label is a user-friendly name for the retriever.
	Label string `json:"label,omitempty"`
	// Supports defines the capabilities of the retriever, such as media support.
	Supports *RetrieverSupports `json:"supports,omitempty"`
	// Metadata is arbitrary key-value data attached to the action descriptor.
	Metadata map[string]any `json:"-"`
}

RetrieverOptions represents the configuration options for a retriever.

type RetrieverRef added in v0.7.0

type RetrieverRef struct {
	// contains filtered or unexported fields
}

RetrieverRef is a struct to hold retriever name and configuration.

func NewRetrieverRef added in v0.7.0

func NewRetrieverRef(name string, config any) RetrieverRef

NewRetrieverRef creates a new RetrieverRef with the given name and configuration.

func (RetrieverRef) Config added in v0.7.0

func (r RetrieverRef) Config() any

Config returns the configuration to use by default for this retriever.

func (RetrieverRef) Name added in v0.7.0

func (r RetrieverRef) Name() string

Name returns the name of the retriever.

type RetrieverRequest

type RetrieverRequest struct {
	// Options contains retriever-specific configuration parameters.
	Options any `json:"options,omitempty"`
	// Query is the document to use for retrieval.
	Query *Document `json:"query,omitempty"`
}

RetrieverRequest represents a request to retrieve relevant documents.

type RetrieverResponse

type RetrieverResponse struct {
	// Documents is the array of retrieved documents.
	Documents []*Document `json:"documents,omitempty"`
}

RetrieverResponse contains the retrieved documents from a retriever request.

func Retrieve added in v0.1.0

func Retrieve(ctx context.Context, r api.Registry, opts ...RetrieverOption) (*RetrieverResponse, error)

Retrieve calls the retriever with the provided options.

type RetrieverSupports added in v0.7.0

type RetrieverSupports struct {
	// Media indicates whether the retriever supports media content.
	Media bool `json:"media,omitempty"`
}

RetrieverSupports defines the supported capabilities of the retriever.

type Role

type Role string

Role indicates which entity is responsible for the content of a message.

const (
	// RoleSystem indicates this message is user-independent context.
	RoleSystem Role = "system"
	// RoleUser indicates this message was generated by the client.
	RoleUser Role = "user"
	// RoleModel indicates this message was generated by the model during a previous interaction.
	RoleModel Role = "model"
	// RoleTool indicates this message was generated by a local tool, likely triggered by a request
	// from the model in one of its previous responses.
	RoleTool Role = "tool"
)

type Score added in v0.3.0

type Score struct {
	Id      string         `json:"id,omitempty"`
	Score   any            `json:"score,omitempty"`
	Status  string         `json:"status,omitempty" jsonschema:"enum=UNKNOWN,enum=FAIL,enum=PASS"`
	Error   string         `json:"error,omitempty"`
	Details map[string]any `json:"details,omitempty"`
}

Score is the evaluation score that represents the result of an evaluator. This struct includes information such as the score (numeric, string or other types), the reasoning provided for this score (if any), the score status (if any) and other details.

type ScoreDetails added in v0.3.0

type ScoreDetails struct {
	// Reasoning explains the rationale behind the score.
	Reasoning string `json:"reasoning,omitempty"`
}

ScoreDetails provides additional context and explanation for an evaluation score.

type ScoreStatus added in v0.3.0

type ScoreStatus int

ScoreStatus is an enum used to indicate if a Score has passed or failed. This drives additional features in tooling / the Dev UI.

const (
	ScoreStatusUnknown ScoreStatus = iota
	ScoreStatusFail
	ScoreStatusPass
)

func (ScoreStatus) String added in v0.3.0

func (ss ScoreStatus) String() string

String returns the wire name of the score status ("PASS", "FAIL", or "UNKNOWN").

type StartModelOpFunc added in v1.3.0

type StartModelOpFunc = func(ctx context.Context, req *ModelRequest) (*ModelOperation, error)

StartModelOpFunc starts a background model operation.

type StreamValue added in v1.3.0

type StreamValue[Out, Stream any] struct {
	Done     bool
	Chunk    Stream         // valid if Done is false
	Output   Out            // valid if Done is true
	Response *ModelResponse // valid if Done is true
}

StreamValue is either a streamed chunk or the final response of a generate request.

type StreamingFormatHandler added in v1.3.0

type StreamingFormatHandler interface {
	// ParseOutput parses the final output and returns parsed output.
	ParseOutput(message *Message) (any, error)
	// ParseChunk processes a streaming chunk and returns parsed output.
	// The handler maintains its own internal state. When the chunk's index changes, the state is reset for the new turn.
	// Returns parsed output, or nil if nothing can be parsed yet.
	ParseChunk(chunk *ModelResponseChunk) (any, error)
}

StreamingFormatHandler is a handler for formatting messages that supports streaming. This interface must be implemented to be able to use ModelResponse.Output and ModelResponseChunk.Output.

type Tool

type Tool interface {
	// Name returns the name of the tool.
	Name() string
	// Definition returns the definition for this tool to be passed to models.
	Definition() *ToolDefinition
	// RunRaw runs this tool using the provided raw input and returns just the output.
	RunRaw(ctx context.Context, input any) (any, error)
	// RunRawMultipart runs this tool and returns the full [MultipartToolResponse].
	RunRawMultipart(ctx context.Context, input any) (*MultipartToolResponse, error)
	// Respond constructs a [Part] with a [ToolResponse] for a given interrupted tool request.
	Respond(toolReq *Part, outputData any, opts *RespondOptions) *Part
	// Restart constructs a [Part] with a new [ToolRequest] to re-trigger a tool,
	// potentially with new input and metadata.
	Restart(toolReq *Part, opts *RestartOptions) *Part
	// Register registers the tool with the given registry.
	Register(r api.Registry)
}

Tool represents a tool that can be called by a model. It is the type to accept as an argument and to look up by name; implementations are created with NewTool or NewMultipartTool, or their [genkit.DefineTool] and [genkit.DefineMultipartTool] counterparts in an application.

func LookupTool added in v0.1.0

func LookupTool(r api.Registry, name string) Tool

LookupTool looks up the tool in the registry by provided name and returns it. It checks for "tool.v2" first, then falls back to "tool" for legacy compatibility. Since the types are not known at lookup time, it returns a type-erased tool.

type ToolAction added in v1.12.0

type ToolAction[In, Out any] struct {
	// contains filtered or unexported fields
}

ToolAction is a tool backed by a registry action. It is the concrete type returned by NewTool and NewMultipartTool. Internally, all tools use the v2 format (returning MultipartToolResponse). For regular tools, RunRaw unwraps the Output field for backward compatibility.

It implements Tool and api.Action, so it can be passed anywhere either is accepted, including the action slice a plugin returns from Init. Unlike the other primitives it holds its action in a named field rather than embedding it, so its documented methods are its whole surface.

func NewMultipartTool added in v1.3.0

func NewMultipartTool[In any](name, description string, fn MultipartToolFunc[In], opts ...ToolOption) *ToolAction[In, *MultipartToolResponse]

NewMultipartTool creates a new multipart ToolAction. It can be passed directly to Generate. Multipart tools can return both output data and additional content parts (like media). Use WithInputSchema to provide a custom JSON schema instead of inferring from the type parameter. Use WithOutputSchema or WithOutputSchemaName to advertise the logical output the tool produces (the envelope's output field); the wire format stays the multipart response envelope.

func NewTool added in v0.6.0

func NewTool[In, Out any](name, description string, fn ToolFunc[In, Out], opts ...ToolOption) *ToolAction[In, Out]

NewTool creates a new ToolAction. It can be passed directly to Generate. Use WithInputSchema or WithOutputSchema to provide custom JSON schemas instead of inferring them from the type parameters.

func NewToolWithInputSchema deprecated added in v1.0.0

func NewToolWithInputSchema[Out any](name, description string, inputSchema map[string]any, fn ToolFunc[any, Out]) *ToolAction[any, Out]

NewToolWithInputSchema creates a new ToolAction with a custom input schema. It can be passed directly to Generate.

Deprecated: Use NewTool with WithInputSchema instead.

func (*ToolAction[In, Out]) Definition added in v1.12.0

func (t *ToolAction[In, Out]) Definition() *ToolDefinition

Definition returns ToolDefinition for for this tool.

func (*ToolAction[In, Out]) Desc added in v1.12.0

func (t *ToolAction[In, Out]) Desc() api.ActionDesc

Desc returns the tool's action descriptor: its name, schemas, and metadata.

func (*ToolAction[In, Out]) IsMultipart added in v1.12.0

func (t *ToolAction[In, Out]) IsMultipart() bool

IsMultipart returns true if the tool is a multipart tool (tool.v2 only).

func (*ToolAction[In, Out]) Name added in v1.12.0

func (t *ToolAction[In, Out]) Name() string

Name returns the name of the tool.

func (*ToolAction[In, Out]) Register added in v1.12.0

func (t *ToolAction[In, Out]) Register(r api.Registry)

Register registers the tool with the given registry.

func (*ToolAction[In, Out]) Respond deprecated added in v1.12.0

func (t *ToolAction[In, Out]) Respond(toolReq *Part, output any, opts *RespondOptions) *Part

Respond creates a part for WithToolResponses to provide a resolved response for an interrupted tool call. Returns nil if the part is not a tool request.

Deprecated: Use ToolAction.RespondWith instead for strongly-typed options.

func (*ToolAction[In, Out]) RespondWith added in v1.12.0

func (t *ToolAction[In, Out]) RespondWith(toolReq *Part, output Out, opts ...RespondWithOption[Out]) (*Part, error)

RespondWith creates a part for WithToolResponses to provide a resolved response for an interrupted tool call.

Example:

part, err := myTool.RespondWith(toolReq, output, WithResponseMetadata[MyOutput](meta))

func (*ToolAction[In, Out]) Restart deprecated added in v1.12.0

func (t *ToolAction[In, Out]) Restart(p *Part, opts *RestartOptions) *Part

Restart creates a part for WithToolRestarts to re-execute an interrupted tool call with additional context. Returns nil if the part is not a tool request.

Deprecated: Use ToolAction.RestartWith instead for strongly-typed options.

func (*ToolAction[In, Out]) RestartWith added in v1.12.0

func (t *ToolAction[In, Out]) RestartWith(toolReq *Part, opts ...RestartWithOption[In]) (*Part, error)

RestartWith creates a part for WithToolRestarts to re-execute an interrupted tool call with additional context.

Example:

part, err := myTool.RestartWith(toolReq, WithNewInput(newInput), WithResumedMetadata[MyInput](meta))

func (*ToolAction[In, Out]) RunJSON added in v1.12.0

func (t *ToolAction[In, Out]) RunJSON(ctx context.Context, input json.RawMessage, cb core.StreamCallback[json.RawMessage]) (json.RawMessage, error)

RunJSON runs the tool on JSON-encoded input and returns the JSON-encoded multipart response envelope, which is what the registry serves for this tool. Prefer ToolAction.RunRaw, which unwraps the envelope's output for a regular tool.

func (*ToolAction[In, Out]) RunJSONWithTelemetry added in v1.12.0

func (t *ToolAction[In, Out]) RunJSONWithTelemetry(ctx context.Context, input json.RawMessage, cb core.StreamCallback[json.RawMessage]) (*api.ActionRunResult[json.RawMessage], error)

RunJSONWithTelemetry is ToolAction.RunJSON with the run's telemetry returned alongside the output.

func (*ToolAction[In, Out]) RunRaw added in v1.12.0

func (t *ToolAction[In, Out]) RunRaw(ctx context.Context, input any) (any, error)

RunRaw runs this tool using the provided raw map format data (JSON parsed as map[string]any).

func (*ToolAction[In, Out]) RunRawMultipart added in v1.12.0

func (t *ToolAction[In, Out]) RunRawMultipart(ctx context.Context, input any) (*MultipartToolResponse, error)

RunRawMultipart runs this tool using the provided raw map format data (JSON parsed as map[string]any). It returns the full multipart response.

type ToolChoice added in v0.3.0

type ToolChoice string

ToolChoice controls how the model uses tools.

const (
	ToolChoiceAuto     ToolChoice = "auto"
	ToolChoiceRequired ToolChoice = "required"
	ToolChoiceNone     ToolChoice = "none"
)

type ToolConfig added in v0.3.0

type ToolConfig struct {
	MaxTurns           int  // Maximum number of tool call iterations before erroring.
	ReturnToolRequests bool // Whether to return tool requests instead of making the tool calls and continuing the generation.
}

ToolConfig handles configuration around tool calls during generation.

type ToolContext added in v0.3.0

type ToolContext struct {
	context.Context
	// Resumed is optional metadata that can be used to resume the tool execution.
	// Map is not nil only if the tool was interrupted.
	Resumed map[string]any
	// OriginalInput is the original input to the tool if the tool was interrupted, otherwise nil.
	OriginalInput any
}

ToolContext provides context and utility functions for tool execution.

func (*ToolContext) Interrupt added in v0.3.0

func (tc *ToolContext) Interrupt(opts *InterruptOptions) error

Interrupt interrupts the tool execution and returns control to the caller with the total model response so far. The provided metadata is preserved and passed back via ToolContext.Resumed when the tool is restarted.

func (*ToolContext) IsResumed added in v1.4.0

func (tc *ToolContext) IsResumed() bool

IsResumed returns true if this tool execution is a resumption after an interrupt.

type ToolDef deprecated added in v0.1.0

type ToolDef[In, Out any] = ToolAction[In, Out]

ToolDef is the previous name for ToolAction. It was renamed because it read as a sibling of ToolDefinition, the wire type a tool advertises to the model, which it is not.

Deprecated: use ToolAction.

type ToolDefinition

type ToolDefinition struct {
	// Description explains what the tool does and when to use it.
	Description string `json:"description,omitempty"`
	// InputSchema is a valid JSON Schema representing the input parameters of the tool.
	InputSchema map[string]any `json:"inputSchema,omitempty"`
	Key         string         `json:"key,omitempty"`
	// Metadata contains additional information about this tool definition.
	Metadata map[string]any `json:"metadata,omitempty"`
	// Name is the unique identifier for this tool.
	Name string `json:"name,omitempty"`
	// OutputSchema is a valid JSON Schema describing the output of the tool.
	OutputSchema map[string]any `json:"outputSchema,omitempty"`
}

A ToolDefinition describes a tool.

type ToolFunc added in v0.7.0

type ToolFunc[In, Out any] = func(ctx *ToolContext, input In) (Out, error)

ToolFunc is the function type for tool implementations.

type ToolName added in v0.3.0

type ToolName string

ToolName is a distinct type for a tool name. It is meant to be passed where a ToolRef is expected but no Tool is had.

func (ToolName) Name added in v0.3.0

func (t ToolName) Name() string

Name returns the name of the tool.

type ToolNext added in v1.7.0

type ToolNext = func(ctx context.Context, params *ToolParams) (*MultipartToolResponse, error)

ToolNext is the next function in the WrapTool hook chain.

type ToolOption added in v1.3.0

type ToolOption interface {
	// contains filtered or unexported methods
}

ToolOption is an option for defining a tool.

func WithStrictSchema added in v1.9.0

func WithStrictSchema(strict bool) ToolOption

WithStrictSchema controls whether the provider enforces strict JSON schema validation on this tool's input. Strict mode requires recursive additionalProperties: false and may reject some JSON Schema keywords (e.g. minItems/maxItems on Anthropic).

When unset, the provider's default applies. Providers without strict-tool support ignore this option.

type ToolParams added in v1.7.0

type ToolParams struct {
	// Request is the tool request about to be executed.
	Request *ToolRequest
	// Tool is the resolved tool being called.
	Tool Tool
}

ToolParams holds params for the WrapTool hook.

type ToolRef added in v0.3.0

type ToolRef interface {
	Name() string
}

ToolRef is a reference to a tool.

type ToolRequest

type ToolRequest struct {
	// Input is a JSON object containing the input parameters for the tool.
	// For example: map[string]any{"country":"USA", "president":3}.
	Input any `json:"input,omitempty"`
	// Name is the name of the tool to call.
	Name string `json:"name,omitempty"`
	// Partial indicates whether this is a partial streaming chunk.
	Partial bool `json:"partial,omitempty"`
	// Ref is the call ID or reference for this specific request.
	Ref string `json:"ref,omitempty"`
}

A ToolRequest is a message from the model to the client that it should run a specific tool and pass a ToolResponse to the model on the next chat request it makes. Any ToolRequest will correspond to some ToolDefinition previously sent by the client.

type ToolResponse

type ToolResponse struct {
	// Content holds additional message parts that provide context or details about the tool response.
	Content []*Part `json:"content,omitempty"`
	// Name is the name of the tool that was executed.
	Name string `json:"name,omitempty"`
	// Output is a JSON object describing the results of running the tool.
	// For example: map[string]any{"name":"Thomas Jefferson", "born":1743}.
	Output any `json:"output,omitempty"`
	// Ref is the call ID or reference matching the original request.
	Ref string `json:"ref,omitempty"`
}

A ToolResponse is a message from the client to the model containing the results of running a specific tool on the arguments passed to the client by the model in a ToolRequest.

type TraceMetadata added in v0.3.0

type TraceMetadata struct {
	// FeatureName identifies the feature being traced.
	FeatureName string `json:"featureName,omitempty"`
	// Paths contains metadata for each path executed during the trace.
	Paths []*PathMetadata `json:"paths,omitempty"`
	// Timestamp is when the trace was created.
	Timestamp float64 `json:"timestamp,omitempty"`
}

TraceMetadata contains metadata about a trace execution.

Directories

Path Synopsis
exp
Package exp provides experimental AI primitives for Genkit.
Package exp provides experimental AI primitives for Genkit.
localstore
Package localstore provides single-process exp.SessionStore implementations suitable for local development, tests, and single-instance apps (CLI tools, desktop apps, local web services).
Package localstore provides single-process exp.SessionStore implementations suitable for local development, tests, and single-instance apps (CLI tools, desktop apps, local web services).
tool
Package tool provides runtime helpers for use inside tool functions.
Package tool provides runtime helpers for use inside tool functions.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL