Documentation
¶
Overview ¶
Package ai defines Genkit's AI primitives: models, prompts, tools, embedders, retrievers, and evaluators, and the options that configure a request to them.
Applications reach these through the genkit package, whose Define* and Generate* functions register with a github.com/firebase/genkit/go/genkit.Genkit and forward here. Plugins use this package directly: New*Action builds an unregistered primitive to return from a plugin's Init.
Options ¶
Options follow the standard Go functional-options pattern: pass as many as you like, in any order, and they merge left to right. Two rules govern how repeats combine, so composing a request from several helpers is predictable:
- Collection options accumulate. Repeating one, or mixing its variants, appends in call order: WithMessages(a), WithMessages(b) sends [a, b], and WithTools, WithResources, WithUse, WithDocs, and WithDataset behave the same.
- Single-value options take the last one set. WithConfig, WithModel, WithSystem, WithPrompt, the output-schema options, and the like each fill one slot, so the final call wins and earlier ones are overwritten rather than rejected. Every option that fills a slot shares it: WithSystem, WithSystemParts, WithSystemFn, and WithSystemPartsFn are four ways to write one system message, so they do not merge.
A zero value does not fill a slot: WithMaxTurns(0), WithToolChoice(""), or WithConfig(nil) is a no-op, so an earlier non-zero value cannot be un-set by a later zero one. The rules apply within a single options list; APIs that layer two lists (a prompt's define-time options against Execute-time options) document their own precedence.
One combination is refused rather than merged. WithMessagesTemplate lays out the whole conversation, down to where {{history}} puts the caller's, so separately supplied messages have no position relative to it. Passing it to DefinePrompt alongside WithMessages or WithMessagesFn panics. Repeating the template alone is an ordinary slot.
Applying options therefore never fails on a "set more than once" conflict. What does fail is a genuinely invalid argument (a type that WithInputType cannot turn into a schema) or the refused combination above, and both panic at the call site where the mistake is rather than deferring to the request.
Errors ¶
Failures are classified with the sentinels in github.com/firebase/genkit/go/core/status, so callers branch with errors.Is rather than on message text:
if errors.Is(err, ai.ErrMaxTurnsExceeded) { ... } // specific
if errors.Is(err, status.ErrAborted) { ... } // broad
Writing a plugin ¶
A provider plugin builds primitives with the New*Action constructors and returns them from its Init. ModelOptions and its siblings describe what a model can do; the constructor copies what it is given, so a table of models may share one ModelSupports value. ModelOptions.Overlay is how a plugin lets an application correct that description without restating it: a zero-value field in the override keeps what the plugin already knows.
Index ¶
- Constants
- Variables
- func CalculateInputOutputUsage(req *ModelRequest, resp *ModelResponse)
- func ConfigureFormats(reg api.Registry)
- func DefineFormats(r api.Registry, formatters ...Formatter)
- func DefineGenerateAction(ctx context.Context, r api.Registry) *generateAction
- func FindMatchingResource(r api.Registry, uri string) (Resource, *ResourceInput, error)
- func GenerateDataStream[Out any](ctx context.Context, r api.Registry, opts ...GenerateOption) iter.Seq2[*StreamValue[Out, Out], error]
- func GenerateStream(ctx context.Context, r api.Registry, opts ...GenerateOption) iter.Seq2[*ModelStreamValue, error]
- func GenerateText(ctx context.Context, r api.Registry, opts ...GenerateOption) (string, error)
- func InterruptAs[T any](p *Part) (T, bool)
- func InterruptWith[T any](tc *ToolContext, meta T) error
- func IsAudioContentType(contentType string) bool
- func IsImageContentType(contentType string) bool
- func IsToolInterruptError(err error) (bool, map[string]any)
- func IsToolResumed(ctx context.Context) bool
- func IsVideoContentType(contentType string) bool
- func LoadPromptDir(r api.Registry, dir string, namespace string)
- func LoadPromptDirFromFS(r api.Registry, fsys fs.FS, dir, namespace string)
- func NewHistoryContext(ctx context.Context, messages []*Message) context.Context
- func NewToolInterruptError(metadata map[string]any) error
- func OriginalInputAs[T any](tc *ToolContext) (T, bool)
- func OutputFrom[Out any](src outputer) Out
- func ResumedValue[T any](ctx context.Context, key string) (T, bool)
- type ActionMetadata
- type AugmentWithContextOptions
- type BackgroundModel
- type BackgroundModelAction
- func (b *BackgroundModelAction) Cancel(ctx context.Context, op *ModelOperation) (*ModelOperation, error)
- func (b *BackgroundModelAction) Check(ctx context.Context, op *ModelOperation) (*ModelOperation, error)
- func (b *BackgroundModelAction) Desc() api.ActionDesc
- func (b *BackgroundModelAction) Name() string
- func (b *BackgroundModelAction) Register(r api.Registry)
- func (b *BackgroundModelAction) RunJSON(ctx context.Context, input json.RawMessage, ...) (json.RawMessage, error)
- func (b *BackgroundModelAction) RunJSONWithTelemetry(ctx context.Context, input json.RawMessage, ...) (*api.ActionRunResult[json.RawMessage], error)
- func (b *BackgroundModelAction) Start(ctx context.Context, req *ModelRequest) (*ModelOperation, error)
- func (b *BackgroundModelAction) SupportsCancel() bool
- type BackgroundModelActionFunc
- type BackgroundModelOptions
- type BatchEvaluatorActionFunc
- type BatchEvaluatorFunc
- type CancelModelOpFunc
- type CheckModelOpFunc
- type CommonGenOption
- func WithMaxTurns(maxTurns int) CommonGenOption
- func WithMessages(messages ...*Message) CommonGenOption
- func WithMessagesFn[In any](fn func(context.Context, In) ([]*Message, error)) CommonGenOption
- func WithMiddleware(middleware ...ModelMiddleware) CommonGenOptiondeprecated
- func WithModel(model ModelArg) CommonGenOption
- func WithModelName(name string) CommonGenOption
- func WithResources(resources ...Resource) CommonGenOption
- func WithReturnToolRequests(returnReqs bool) CommonGenOption
- func WithToolChoice(toolChoice ToolChoice) CommonGenOption
- func WithTools(tools ...ToolRef) CommonGenOption
- func WithUse(middleware ...Middleware) CommonGenOption
- type ConfigOption
- type ConstrainedSupport
- type DataPrompt
- func (p *DataPrompt) Desc() api.ActionDesc
- func (dp *DataPrompt[In, Out]) Execute(ctx context.Context, input In, opts ...PromptExecuteOption) (Out, *ModelResponse, error)
- func (dp *DataPrompt[In, Out]) ExecuteStream(ctx context.Context, input In, opts ...PromptExecuteOption) iter.Seq2[*StreamValue[Out, Out], error]
- func (dp *DataPrompt[In, Out]) Render(ctx context.Context, input In) (*GenerateActionOptions, error)
- type DocsFn
- type Document
- type DocumentOption
- type DownloadMediaOptions
- type EmbedRequest
- type EmbedResponse
- type Embedder
- type EmbedderAction
- func (e *EmbedderAction) Desc() api.ActionDesc
- func (e *EmbedderAction) Embed(ctx context.Context, req *EmbedRequest) (*EmbedResponse, error)
- func (e *EmbedderAction) Name() string
- func (e *EmbedderAction) Register(r api.Registry)
- func (e *EmbedderAction) RunJSON(ctx context.Context, input json.RawMessage, ...) (json.RawMessage, error)
- func (e *EmbedderAction) RunJSONWithTelemetry(ctx context.Context, input json.RawMessage, ...) (*api.ActionRunResult[json.RawMessage], error)
- type EmbedderActionFunc
- type EmbedderArg
- type EmbedderFunc
- type EmbedderOption
- type EmbedderOptions
- type EmbedderRef
- type EmbedderSupports
- type Embedding
- type EvaluationResult
- type Evaluator
- type EvaluatorAction
- func (e *EvaluatorAction) Desc() api.ActionDesc
- func (e *EvaluatorAction) Evaluate(ctx context.Context, req *EvaluatorRequest) (*EvaluatorResponse, error)
- func (e *EvaluatorAction) Name() string
- func (e *EvaluatorAction) Register(r api.Registry)
- func (e *EvaluatorAction) RunJSON(ctx context.Context, input json.RawMessage, ...) (json.RawMessage, error)
- func (e *EvaluatorAction) RunJSONWithTelemetry(ctx context.Context, input json.RawMessage, ...) (*api.ActionRunResult[json.RawMessage], error)
- type EvaluatorActionFunc
- type EvaluatorArg
- type EvaluatorCallbackRequest
- type EvaluatorCallbackResponse
- type EvaluatorFunc
- type EvaluatorOption
- type EvaluatorOptions
- type EvaluatorRef
- type EvaluatorRequest
- type EvaluatorResponse
- type Example
- type ExecutionOption
- type FinishReason
- type FormatHandler
- type Formatter
- type GenerateActionOptions
- type GenerateActionOutputConfig
- type GenerateActionResume
- type GenerateNext
- type GenerateOption
- type GenerateParams
- type GenerationCommonConfig
- type GenerationUsage
- type Hooks
- type InputOption
- type InputSchemaOption
- type InterruptOptions
- type Media
- type Message
- func HistoryFromContext(ctx context.Context) []*Message
- func NewMessage(role Role, metadata map[string]any, parts ...*Part) *Message
- func NewModelMessage(parts ...*Part) *Message
- func NewModelTextMessage(text string) *Message
- func NewSystemMessage(parts ...*Part) *Message
- func NewSystemTextMessage(text string) *Message
- func NewTextMessage(role Role, text string) *Message
- func NewUserMessage(parts ...*Part) *Message
- func NewUserMessageWithMetadata(metadata map[string]any, parts ...*Part) *Message
- func NewUserTextMessage(text string) *Message
- type MessagesFn
- type Middleware
- type MiddlewareDesc
- type MiddlewareFunc
- type MiddlewarePlugin
- type MiddlewareRef
- type Model
- type ModelAction
- func (m *ModelAction) Desc() api.ActionDesc
- func (m *ModelAction) Generate(ctx context.Context, req *ModelRequest, cb ModelStreamCallback) (*ModelResponse, error)
- func (m *ModelAction) Name() string
- func (m *ModelAction) Register(r api.Registry)
- func (m *ModelAction) RunJSON(ctx context.Context, input json.RawMessage, ...) (json.RawMessage, error)
- func (m *ModelAction) RunJSONWithTelemetry(ctx context.Context, input json.RawMessage, ...) (*api.ActionRunResult[json.RawMessage], error)
- type ModelActionFunc
- type ModelArg
- type ModelFunc
- type ModelInfo
- type ModelMiddlewaredeprecated
- type ModelNext
- type ModelOperation
- type ModelOptions
- type ModelOutputConfig
- type ModelParams
- type ModelRef
- type ModelReference
- type ModelRequest
- type ModelResponse
- func Generate(ctx context.Context, r api.Registry, opts ...GenerateOption) (*ModelResponse, error)
- func GenerateData[Out any](ctx context.Context, r api.Registry, opts ...GenerateOption) (*Out, *ModelResponse, error)
- func GenerateWithRequest(ctx context.Context, r api.Registry, opts *GenerateActionOptions, ...) (*ModelResponse, error)
- func (mr *ModelResponse) History() []*Message
- func (mr *ModelResponse) Interrupts() []*Part
- func (mr *ModelResponse) Media() string
- func (mr *ModelResponse) MediaParts() []*Part
- func (mr *ModelResponse) Output(v any) error
- func (mr *ModelResponse) Reasoning() string
- func (mr *ModelResponse) Text() string
- func (mr *ModelResponse) ToolRequests() []*Part
- type ModelResponseChunk
- type ModelStage
- type ModelStreamCallback
- type ModelStreamValue
- type ModelSupports
- type MultipartToolFunc
- type MultipartToolResponse
- type Operation
- type OperationError
- type OutputOption
- type OutputSchemaOption
- type Part
- func NewCustomPart(customData map[string]any) *Part
- func NewDataPart(contents string) *Part
- func NewJSONPart(text string) *Part
- func NewMediaPart(mimeType, contents string) *Part
- func NewPartialToolResponsePart(r *ToolResponse) *Part
- func NewReasoningPart(text string, signature []byte) *Part
- func NewResourcePart(uri string) *Part
- func NewResponseForToolRequest(p *Part, output any) *Part
- func NewTextPart(text string) *Part
- func NewToolRequestPart(r *ToolRequest) *Part
- func NewToolResponsePart(r *ToolResponse) *Part
- func (p *Part) Clone() *Part
- func (p *Part) IsAudio() bool
- func (p *Part) IsCustom() bool
- func (p *Part) IsData() bool
- func (p *Part) IsImage() bool
- func (p *Part) IsInterrupt() bool
- func (p *Part) IsMedia() bool
- func (p *Part) IsPartial() bool
- func (p *Part) IsReasoning() bool
- func (p *Part) IsResource() bool
- func (p *Part) IsText() bool
- func (p *Part) IsToolRequest() bool
- func (p *Part) IsToolResponse() bool
- func (p *Part) IsVideo() bool
- func (Part) JSONSchemaAlias() any
- func (p *Part) MarshalJSON() ([]byte, error)
- func (p *Part) UnmarshalJSON(b []byte) error
- func (p *Part) UnmarshalYAML(unmarshal func(any) error) error
- type PartKind
- type PartsFn
- type PathMetadata
- type Prompt
- func DefinePrompt(r api.Registry, name string, opts ...PromptOption) Prompt
- func LoadPrompt(r api.Registry, dir, filename, namespace string) Prompt
- func LoadPromptFromFS(r api.Registry, fsys fs.FS, dir, filename, namespace string) Prompt
- func LoadPromptFromSource(r api.Registry, source, name, namespace string) (Prompt, error)
- func LookupPrompt(r api.Registry, name string) Prompt
- type PromptExecuteOption
- type PromptFn
- type PromptOption
- type PromptingOption
- func WithPrompt(text string, args ...any) PromptingOption
- func WithPromptFn[In any](fn func(context.Context, In) (string, error)) PromptingOption
- func WithPromptParts(parts ...*Part) PromptingOption
- func WithPromptPartsFn[In any](fn func(context.Context, In) ([]*Part, error)) PromptingOption
- func WithSystem(text string, args ...any) PromptingOption
- func WithSystemFn[In any](fn func(context.Context, In) (string, error)) PromptingOption
- func WithSystemParts(parts ...*Part) PromptingOption
- func WithSystemPartsFn[In any](fn func(context.Context, In) ([]*Part, error)) PromptingOption
- type RankedDocumentData
- type RankedDocumentMetadata
- type RerankerRequest
- type RerankerResponse
- type Resource
- type ResourceFunc
- type ResourceInput
- type ResourceOptions
- type ResourceOutput
- type ResourcePart
- type RespondOptions
- type RespondWithOption
- type RestartOptions
- type RestartWithOption
- type Retriever
- type RetrieverAction
- func (r *RetrieverAction) Desc() api.ActionDesc
- func (r *RetrieverAction) Name() string
- func (r *RetrieverAction) Register(reg api.Registry)
- func (r *RetrieverAction) Retrieve(ctx context.Context, req *RetrieverRequest) (*RetrieverResponse, error)
- func (r *RetrieverAction) RunJSON(ctx context.Context, input json.RawMessage, ...) (json.RawMessage, error)
- func (r *RetrieverAction) RunJSONWithTelemetry(ctx context.Context, input json.RawMessage, ...) (*api.ActionRunResult[json.RawMessage], error)
- type RetrieverActionFunc
- type RetrieverArg
- type RetrieverFunc
- type RetrieverOption
- type RetrieverOptions
- type RetrieverRef
- type RetrieverRequest
- type RetrieverResponse
- type RetrieverSupports
- type Role
- type Score
- type ScoreDetails
- type ScoreStatus
- type StartModelOpFunc
- type StreamValue
- type StreamingFormatHandler
- type Tool
- type ToolAction
- func NewMultipartTool[In any](name, description string, fn MultipartToolFunc[In], opts ...ToolOption) *ToolAction[In, *MultipartToolResponse]
- func NewTool[In, Out any](name, description string, fn ToolFunc[In, Out], opts ...ToolOption) *ToolAction[In, Out]
- func NewToolWithInputSchema[Out any](name, description string, inputSchema map[string]any, fn ToolFunc[any, Out]) *ToolAction[any, Out]deprecated
- func (t *ToolAction[In, Out]) Definition() *ToolDefinition
- func (t *ToolAction[In, Out]) Desc() api.ActionDesc
- func (t *ToolAction[In, Out]) IsMultipart() bool
- func (t *ToolAction[In, Out]) Name() string
- func (t *ToolAction[In, Out]) Register(r api.Registry)
- func (t *ToolAction[In, Out]) Respond(toolReq *Part, output any, opts *RespondOptions) *Partdeprecated
- func (t *ToolAction[In, Out]) RespondWith(toolReq *Part, output Out, opts ...RespondWithOption[Out]) (*Part, error)
- func (t *ToolAction[In, Out]) Restart(p *Part, opts *RestartOptions) *Partdeprecated
- func (t *ToolAction[In, Out]) RestartWith(toolReq *Part, opts ...RestartWithOption[In]) (*Part, error)
- func (t *ToolAction[In, Out]) RunJSON(ctx context.Context, input json.RawMessage, ...) (json.RawMessage, error)
- func (t *ToolAction[In, Out]) RunJSONWithTelemetry(ctx context.Context, input json.RawMessage, ...) (*api.ActionRunResult[json.RawMessage], error)
- func (t *ToolAction[In, Out]) RunRaw(ctx context.Context, input any) (any, error)
- func (t *ToolAction[In, Out]) RunRawMultipart(ctx context.Context, input any) (*MultipartToolResponse, error)
- type ToolChoice
- type ToolConfig
- type ToolContext
- type ToolDefdeprecated
- type ToolDefinition
- type ToolFunc
- type ToolName
- type ToolNext
- type ToolOption
- type ToolParams
- type ToolRef
- type ToolRequest
- type ToolResponse
- type TraceMetadata
Examples ¶
Constants ¶
const ( // OutputFormatText is the default format. OutputFormatText string = "text" // OutputFormatJSON is the legacy format for JSON content. // For streaming, each chunk represents the full object received up to that point. OutputFormatJSON string = "json" // OutputFormatJSONL is the format for JSONL content. // // For streaming, each chunk carries the objects whose line finished since // the last chunk. A line still being written is held back until it parses, // so no object is ever handed over incomplete or handed over twice. OutputFormatJSONL string = "jsonl" // OutputFormatMedia is the format for media content. OutputFormatMedia string = "media" // OutputFormatArray is the format for array content. // // For streaming, each chunk carries the elements that finished since the // last chunk. An element still being written is held back until it parses, // so no element is ever handed over incomplete or handed over twice. OutputFormatArray string = "array" // OutputFormatEnum is the format for enum content. // The value must be a string. OutputFormatEnum string = "enum" )
Variables ¶
var ( // ErrModelNotFound means the named model is not registered. Usually the // providing plugin is missing from genkit.Init. ErrModelNotFound = status.ErrNotFound.Subtype("model not found") // ErrToolNotFound means the named tool is not registered, either on the // request or in the registry the model's tool call resolved against. ErrToolNotFound = status.ErrNotFound.Subtype("tool not found") // ErrMaxTurnsExceeded means the tool-calling loop hit its turn limit before // the model produced a final response. Raise the limit with WithMaxTurns, or // look for a tool the model keeps retrying. ErrMaxTurnsExceeded = status.ErrAborted.Subtype("max turns exceeded") // ErrToolFailed means a tool returned an error or produced output that does // not match its declared schema. The tool's own error is wrapped, so // errors.Is and errors.As still reach it; the status is INTERNAL because a // tool's failure is not a failure of the caller's request. ErrToolFailed = status.ErrInternal.Subtype("tool failed") // ErrUnsupportedByModel means the request used a capability the model does // not advertise (media, tools, tool choice, a system role, ...). ErrUnsupportedByModel = status.ErrInvalidArgument.Subtype("unsupported by model") // ErrInvalidPart means a Part is malformed for the operation at hand: the // wrong kind, missing a required field, or carrying a field its kind does // not allow. ErrInvalidPart = status.ErrInvalidArgument.Subtype("invalid part") // ErrInputTypeMismatch means a prompt's input could not be interpreted as // the type a content function declared, so the function was never called. // The input may have come from [WithInput], the default recorded by // [WithInputType], or the wire. ErrInputTypeMismatch = status.ErrInvalidArgument.Subtype("input type mismatch") // ErrUnresolvedToolRequest means a resumed generation left an interrupted // tool request without a Respond or Restart directive. ErrUnresolvedToolRequest = status.ErrInvalidArgument.Subtype("unresolved tool request") )
Failure modes generation reports. Match them with errors.Is rather than by inspecting message text:
if errors.Is(err, ai.ErrMaxTurnsExceeded) { ... }
Each also matches the base sentinel it derives from, so errors.Is(err, status.ErrNotFound) still catches a missing model or tool.
var DEFAULT_FORMATS = []Formatter{
textFormatter{},
jsonFormatter{},
jsonlFormatter{},
arrayFormatter{},
enumFormatter{},
}
Default formats get automatically registered on registry init
Functions ¶
func CalculateInputOutputUsage ¶ added in v1.3.0
func CalculateInputOutputUsage(req *ModelRequest, resp *ModelResponse)
CalculateInputOutputUsage fills in the character, image, and video counts on resp.Usage, for providers whose API reports token counts only.
func ConfigureFormats ¶ added in v0.5.0
ConfigureFormats registers default formats in the registry
func DefineFormats ¶ added in v1.12.0
DefineFormats registers each Formatter under the name returned by its Name method. It panics if a format with that name is already registered, which includes the built-in formats registered by ConfigureFormats, so a format cannot be replaced once defined.
func DefineGenerateAction ¶ added in v0.3.0
DefineGenerateAction defines a utility generate action.
func FindMatchingResource ¶ added in v0.7.0
FindMatchingResource finds a resource that matches the given URI.
func GenerateDataStream ¶ added in v1.3.0
func GenerateDataStream[Out any](ctx context.Context, r api.Registry, opts ...GenerateOption) iter.Seq2[*StreamValue[Out, Out], error]
GenerateDataStream generates a model response with streaming and returns strongly-typed output. It returns an iterator that yields streaming results.
If the yield function is passed a non-nil error, generation has failed with that error; the yield function will not be called again.
If the yield function's StreamValue argument has Done == true, the value's Output and Response fields contain the final typed output and response; the yield function will not be called again.
Otherwise the Chunk field of the passed StreamValue holds a streamed chunk.
Like GenerateData, the output format is JSON with a schema inferred from Out; overriding the format with a non-JSON WithOutputFormat or WithOutputEnums breaks typed extraction.
func GenerateStream ¶ added in v1.3.0
func GenerateStream(ctx context.Context, r api.Registry, opts ...GenerateOption) iter.Seq2[*ModelStreamValue, error]
GenerateStream generates a model response and streams the output. It returns an iterator that yields streaming results.
If the yield function is passed a non-nil error, generation has failed with that error; the yield function will not be called again.
If the yield function's ModelStreamValue argument has Done == true, the value's Response field contains the final response; the yield function will not be called again.
Otherwise the Chunk field of the passed ModelStreamValue holds a streamed chunk.
func GenerateText ¶ added in v0.1.0
GenerateText run generate request for this model. Returns generated text only.
func InterruptAs ¶ added in v1.4.0
InterruptAs extracts strongly-typed metadata from an interrupted tool request Part. Returns the zero value and false if the part is not an interrupt or the type doesn't match.
func InterruptWith ¶ added in v1.4.0
func InterruptWith[T any](tc *ToolContext, meta T) error
InterruptWith is a convenience function to interrupt a tool with a strongly-typed metadata value. The metadata is converted to map[string]any via JSON marshaling.
func IsAudioContentType ¶ added in v1.0.0
IsAudioContentType checks if the content type represents an audio file.
func IsImageContentType ¶ added in v1.0.0
IsImageContentType checks if the content type represents an image.
func IsToolInterruptError ¶ added in v1.1.0
IsToolInterruptError determines whether the error is an interrupt error returned by the tool.
func IsToolResumed ¶ added in v1.7.0
IsToolResumed reports whether the current context is a resumed tool execution. This is intended for use in middleware that needs to distinguish between first-time and restarted tool calls.
func IsVideoContentType ¶ added in v1.0.0
IsVideoContentType checks if the content type represents a video.
func LoadPromptDir ¶ added in v0.3.0
LoadPromptDir loads prompts and partials from a directory on the local filesystem.
func LoadPromptDirFromFS ¶ added in v1.3.0
LoadPromptDirFromFS loads prompts and partials from a filesystem for the given namespace. The fsys parameter should be an fs.FS implementation (e.g., embed.FS or os.DirFS). The dir parameter specifies the directory within the filesystem where prompts are located.
func NewHistoryContext ¶ added in v1.12.0
NewHistoryContext returns ctx carrying a conversation for the next Prompt.Render call to place, and is what HistoryFromContext reads back.
Prompt.Execute does this itself, so its callers never need it. It is for code that drives a prompt by hand, pairing Render with GenerateWithRequest, and still wants the prompt to decide where the conversation goes. The agent runtime is the main example.
Scope the returned context to the Render call. Generation should run on the original, so the conversation does not ride along into tool handlers and prompts executed inside the generate loop.
The messages are not copied; Render clones what it places into the request. Nil entries are dropped.
func NewToolInterruptError ¶ added in v1.7.0
NewToolInterruptError creates a tool interrupt error with the given metadata. This is intended for use in middleware that needs to interrupt tool execution without calling the tool itself.
func OriginalInputAs ¶ added in v1.4.0
func OriginalInputAs[T any](tc *ToolContext) (T, bool)
OriginalInputAs returns the original input typed appropriately. Returns the zero value and false if not resumed or type doesn't match.
func OutputFrom ¶ added in v1.3.0
func OutputFrom[Out any](src outputer) Out
OutputFrom is a convenience function that parses structured output from a ModelResponse or ModelResponseChunk and returns it as a typed value. This is equivalent to calling Output() but returns the value directly instead of requiring a pointer argument. If you need to handle the error, use Output() instead.
func ResumedValue ¶ added in v1.4.0
ResumedValue retrieves a typed value from the resumed metadata on ctx. Returns the zero value and false if the key doesn't exist or the type doesn't match. Accepts either a plain context.Context (useful in middleware) or a *ToolContext, which embeds context.Context.
Types ¶
type ActionMetadata ¶ added in v1.5.1
type ActionMetadata struct {
ActionType string `json:"actionType,omitempty"`
Description string `json:"description,omitempty"`
// A JSON Schema Draft 7 (http://json-schema.org/draft-07/schema) object.
InputJsonSchema any `json:"inputJsonSchema,omitempty"`
InputSchema any `json:"inputSchema,omitempty"`
Key string `json:"key,omitempty"`
Metadata map[string]any `json:"metadata,omitempty"`
Name string `json:"name,omitempty"`
// A JSON Schema Draft 7 (http://json-schema.org/draft-07/schema) object.
OutputJsonSchema any `json:"outputJsonSchema,omitempty"`
OutputSchema any `json:"outputSchema,omitempty"`
StreamSchema any `json:"streamSchema,omitempty"`
}
type AugmentWithContextOptions ¶ added in v0.3.0
type AugmentWithContextOptions struct {
Preface *string // Preceding text to place before the rendered context documents.
ItemTemplate func(d Document, index int, options *AugmentWithContextOptions) string // A function to render a document into a text part to be included in the message.
CitationKey *string // Metadata key to use for citation reference. Pass `nil` to provide no citations.
}
AugmentWithContextOptions configures how a request is augmented with context.
type BackgroundModel ¶ added in v1.3.0
type BackgroundModel interface {
// Name returns the registry name of the background model.
Name() string
// Register registers the model with the given registry.
Register(r api.Registry)
// Start starts a background operation.
Start(ctx context.Context, req *ModelRequest) (*ModelOperation, error)
// Check checks the status of a background operation.
Check(ctx context.Context, op *ModelOperation) (*ModelOperation, error)
// Cancel cancels a background operation.
Cancel(ctx context.Context, op *ModelOperation) (*ModelOperation, error)
// SupportsCancel returns whether the background action supports cancellation.
SupportsCancel() bool
}
BackgroundModel represents a model that can run operations in the background. It is the type to accept as an argument and to look up by name; implementations are created with NewBackgroundModelAction, or [genkit.DefineBackgroundModelAction] in an application.
func LookupBackgroundModel ¶ added in v1.3.0
func LookupBackgroundModel(r api.Registry, name string) BackgroundModel
LookupBackgroundModel looks up a registered BackgroundModel by name. It returns nil if the background model was not found.
func NewBackgroundModel
deprecated
added in
v1.3.0
func NewBackgroundModel(name string, opts *BackgroundModelOptions, startFn StartModelOpFunc, checkFn CheckModelOpFunc) BackgroundModel
NewBackgroundModel defines a new model that runs in the background.
Deprecated: Use NewBackgroundModelAction, which passes the request's config to startFn as a typed value instead of leaving it type-erased on the request.
type BackgroundModelAction ¶ added in v1.12.0
type BackgroundModelAction struct {
// contains filtered or unexported fields
}
BackgroundModelAction is a background model backed by registry actions. It is the concrete type returned by NewBackgroundModelAction; return it from a plugin's Init for the framework to register.
It implements BackgroundModel and api.Action, so it can be passed anywhere either is accepted. The api.Action side is what lets a plugin resolver return the whole model as the resolved action (googlegenai's resolveAction does this), and the three component actions register together through BackgroundModelAction.Register.
func NewBackgroundModelAction ¶ added in v1.12.0
func NewBackgroundModelAction[Config any]( name string, opts *BackgroundModelOptions, startFn BackgroundModelActionFunc[Config], checkFn CheckModelOpFunc, ) *BackgroundModelAction
NewBackgroundModelAction creates an unregistered BackgroundModelAction: return it from a plugin's Init for the framework to register, or call BackgroundModelAction.Register directly. Applications should define background models with [genkit.DefineBackgroundModelAction].
Config is the model's typed configuration; it is usually inferred from startFn's signature. See NewModelAction for how the request's config is deserialized.
func (*BackgroundModelAction) Cancel ¶ added in v1.12.0
func (b *BackgroundModelAction) Cancel(ctx context.Context, op *ModelOperation) (*ModelOperation, error)
Cancel cancels a running background operation. It fails with UNAVAILABLE if the model does not support cancellation; see BackgroundModelAction.SupportsCancel.
func (*BackgroundModelAction) Check ¶ added in v1.12.0
func (b *BackgroundModelAction) Check(ctx context.Context, op *ModelOperation) (*ModelOperation, error)
Check returns the current state of a background operation.
func (*BackgroundModelAction) Desc ¶ added in v1.12.0
func (b *BackgroundModelAction) Desc() api.ActionDesc
Desc returns the start action's descriptor: its name, schemas, and metadata.
func (*BackgroundModelAction) Name ¶ added in v1.12.0
func (b *BackgroundModelAction) Name() string
Name returns the registry name of the background model.
func (*BackgroundModelAction) Register ¶ added in v1.12.0
func (b *BackgroundModelAction) Register(r api.Registry)
Register registers the model's start, check, and cancel actions with r. The cancel action is registered only if the model supports cancellation. A plugin that returns the model from its Init does not need to call this.
func (*BackgroundModelAction) RunJSON ¶ added in v1.12.0
func (b *BackgroundModelAction) RunJSON(ctx context.Context, input json.RawMessage, cb core.StreamCallback[json.RawMessage]) (json.RawMessage, error)
RunJSON starts an operation from a JSON-encoded ModelRequest and returns the JSON-encoded operation. The framework uses it to serve reflection and registry-driven calls; prefer BackgroundModelAction.Start.
func (*BackgroundModelAction) RunJSONWithTelemetry ¶ added in v1.12.0
func (b *BackgroundModelAction) RunJSONWithTelemetry(ctx context.Context, input json.RawMessage, cb core.StreamCallback[json.RawMessage]) (*api.ActionRunResult[json.RawMessage], error)
RunJSONWithTelemetry is BackgroundModelAction.RunJSON with the run's telemetry returned alongside the output.
func (*BackgroundModelAction) Start ¶ added in v1.12.0
func (b *BackgroundModelAction) Start(ctx context.Context, req *ModelRequest) (*ModelOperation, error)
Start starts a background operation and returns it without waiting for completion. Poll it with BackgroundModelAction.Check.
func (*BackgroundModelAction) SupportsCancel ¶ added in v1.12.0
func (b *BackgroundModelAction) SupportsCancel() bool
SupportsCancel reports whether the model was defined with a cancel function.
type BackgroundModelActionFunc ¶ added in v1.12.0
type BackgroundModelActionFunc[Config any] = func(ctx context.Context, req *ModelRequest, config Config) (*ModelOperation, error)
BackgroundModelActionFunc is a StartModelOpFunc that additionally receives the request's typed Config: the framework deserializes the request's raw config into it before calling the function (see NewBackgroundModelAction).
type BackgroundModelOptions ¶ added in v1.3.0
type BackgroundModelOptions struct {
ModelOptions
// Cancel cancels a running operation. Optional: nil means the model does
// not support canceling operations.
Cancel CancelModelOpFunc
// Metadata is arbitrary key-value data attached to the action descriptor.
// It is merged over [ModelOptions.Metadata]; this field wins on key
// conflicts.
Metadata map[string]any
}
BackgroundModelOptions configures a background model created with NewBackgroundModelAction. It extends ModelOptions with the operation lifecycle hooks a background model needs; the required start and check functions are constructor arguments.
type BatchEvaluatorActionFunc ¶ added in v1.12.0
type BatchEvaluatorActionFunc[Config any] = func(context.Context, *EvaluatorRequest, Config) (*EvaluatorResponse, error)
BatchEvaluatorActionFunc is a BatchEvaluatorFunc that additionally receives the request's typed Config: the framework deserializes the request's raw options into it before calling the function (see NewBatchEvaluatorAction).
type BatchEvaluatorFunc ¶ added in v0.7.0
type BatchEvaluatorFunc = func(context.Context, *EvaluatorRequest) (*EvaluatorResponse, error)
BatchEvaluatorFunc is the function type for batch evaluator implementations.
type CancelModelOpFunc ¶ added in v1.3.0
type CancelModelOpFunc = func(ctx context.Context, op *ModelOperation) (*ModelOperation, error)
CancelModelOpFunc cancels a background model operation.
type CheckModelOpFunc ¶ added in v1.3.0
type CheckModelOpFunc = func(ctx context.Context, op *ModelOperation) (*ModelOperation, error)
CheckModelOpFunc checks the status of a background model operation.
type CommonGenOption ¶ added in v0.5.0
type CommonGenOption interface {
PromptOption
GenerateOption
PromptExecuteOption
// contains filtered or unexported methods
}
CommonGenOption is an option common to model generation, prompt definition, and prompt execution.
func WithMaxTurns ¶ added in v0.3.0
func WithMaxTurns(maxTurns int) CommonGenOption
WithMaxTurns sets the maximum number of tool call iterations before erroring. A tool call happens when tools are provided in the request and a model decides to call one or more as a response. Each round trip, including multiple tools in parallel, counts as one turn.
func WithMessages ¶ added in v0.1.0
func WithMessages(messages ...*Message) CommonGenOption
WithMessages adds messages to the request, placed between the system and user prompts. Repeating this option, or mixing it with WithMessagesFn, appends: messages accumulate in the order the options are passed.
Message text is used verbatim, never compiled as a dotprompt template, so history containing literal braces passes through untouched. To build message text from a prompt's input, use WithMessagesFn, which receives the typed input, or WithMessagesTemplate for a multi-turn template.
Declaring the conversation makes the prompt responsible for the messages passed to Prompt.Execute: they are not spliced in automatically, because only the prompt knows where its examples end and a real conversation begins. Place them with {{history}} in a WithMessagesTemplate or HistoryFromContext in a WithMessagesFn.
func WithMessagesFn ¶ added in v0.3.0
WithMessagesFn adds messages produced by fn at request time, placed between the system and user prompts. Like WithMessages, repeating this option (or mixing the two) appends the produced messages in call order.
fn receives the prompt's input converted to In, or the zero value of In when there is none, as at Generate. Its messages are used verbatim, as with WithMessages.
Those messages are the conversation the prompt declares, so messages supplied at execution time are not appended on top of them. fn reads them from HistoryFromContext instead, and can summarize or truncate them rather than only prepend to them.
func WithMiddleware
deprecated
added in
v0.3.0
func WithMiddleware(middleware ...ModelMiddleware) CommonGenOption
WithMiddleware adds middleware to apply to the model request. Repeating this option appends to the chain.
Deprecated: Use WithUse instead, which supports Generate, Model, and Tool hooks.
func WithModel ¶ added in v0.3.0
func WithModel(model ModelArg) CommonGenOption
WithModel sets either a Model or a ModelRef that may contain a config. Passing WithConfig will take precedence over the config in WithModel. Repeating this option, or mixing it with WithModelName, takes the last model set.
func WithModelName ¶ added in v0.3.0
func WithModelName(name string) CommonGenOption
WithModelName sets the model name to call for generation. The model name will be resolved to a Model and may error if the reference is invalid. Repeating this option, or mixing it with WithModel, takes the last model set.
func WithResources ¶ added in v0.7.0
func WithResources(resources ...Resource) CommonGenOption
WithResources specifies resources to be temporarily available during generation. Repeating this option appends. Resources are unregistered resources that get attached to a temporary registry during the generation request and cleaned up afterward.
func WithReturnToolRequests ¶ added in v0.3.0
func WithReturnToolRequests(returnReqs bool) CommonGenOption
WithReturnToolRequests configures whether to return tool requests instead of making the tool calls and continuing the generation.
func WithToolChoice ¶ added in v0.3.0
func WithToolChoice(toolChoice ToolChoice) CommonGenOption
WithToolChoice configures whether by default tool calls are required, disabled, or optional for the prompt.
func WithTools ¶ added in v0.1.0
func WithTools(tools ...ToolRef) CommonGenOption
WithTools adds tools to use for the generate request. Repeating this option appends; duplicate tools (by name) are rejected when the request runs.
func WithUse ¶ added in v1.7.0
func WithUse(middleware ...Middleware) CommonGenOption
WithUse adds middleware to apply to generation. Middleware hooks wrap the generate loop, model calls, and tool executions. Repeating this option appends to the chain.
Accepts either a middleware config struct (produced by a plugin) or an inline adapter via MiddlewareFunc. The chain applies outer-to-inner, so WithUse(A, B) expands to A { B { ... } }.
type ConfigOption ¶ added in v0.5.0
type ConfigOption interface {
CommonGenOption
EmbedderOption
RetrieverOption
EvaluatorOption
// contains filtered or unexported methods
}
ConfigOption is an option for model configuration. It is accepted anywhere a primitive takes a config: generation, prompt definition and execution, embedding, retrieval, and evaluation.
func WithConfig ¶ added in v0.1.0
func WithConfig(config any) ConfigOption
WithConfig sets the configuration. Repeating this option takes the last config set.
type ConstrainedSupport ¶ added in v0.3.0
type ConstrainedSupport string
ConstrainedSupport indicates the level of constrained generation support.
const ( ConstrainedSupportNone ConstrainedSupport = "none" ConstrainedSupportAll ConstrainedSupport = "all" ConstrainedSupportNoTools ConstrainedSupport = "no-tools" )
type DataPrompt ¶ added in v1.3.0
type DataPrompt[In, Out any] struct { // contains filtered or unexported fields }
DataPrompt is a prompt with strongly-typed input and output. It wraps an underlying Prompt and provides type-safe Execute and Render methods. The Out type parameter can be string for text outputs or any struct type for JSON outputs.
func AsDataPrompt ¶ added in v1.3.0
func AsDataPrompt[In, Out any](p Prompt) *DataPrompt[In, Out]
AsDataPrompt wraps an existing Prompt with type information, returning a DataPrompt. This is useful for adding strong typing to a dynamically obtained prompt.
func DefineDataPrompt ¶ added in v1.3.0
func DefineDataPrompt[In, Out any](r api.Registry, name string, opts ...PromptOption) *DataPrompt[In, Out]
DefineDataPrompt creates a new data prompt and registers it. It automatically infers input schema from the In type parameter and configures output schema and JSON format from the Out type parameter (unless Out is string).
func LookupDataPrompt ¶ added in v1.3.0
func LookupDataPrompt[In, Out any](r api.Registry, name string) *DataPrompt[In, Out]
LookupDataPrompt looks up a prompt by name and wraps it with type information. This is useful for wrapping prompts loaded from .prompt files with strong types. It returns nil if the prompt was not found.
func (*DataPrompt) Desc ¶ added in v1.3.0
func (p *DataPrompt) Desc() api.ActionDesc
Desc returns a descriptor of the prompt with resolved schema references.
func (*DataPrompt[In, Out]) Execute ¶ added in v1.3.0
func (dp *DataPrompt[In, Out]) Execute(ctx context.Context, input In, opts ...PromptExecuteOption) (Out, *ModelResponse, error)
Execute executes the typed prompt and returns the strongly-typed output along with the full model response. For structured output types (non-string Out), the prompt must be configured with the appropriate output schema, either through DefineDataPrompt or by using WithOutputType when defining the prompt. The typed input argument fills the input slot last, so it wins over any WithInput passed in opts.
func (*DataPrompt[In, Out]) ExecuteStream ¶ added in v1.3.0
func (dp *DataPrompt[In, Out]) ExecuteStream(ctx context.Context, input In, opts ...PromptExecuteOption) iter.Seq2[*StreamValue[Out, Out], error]
ExecuteStream executes the typed prompt with streaming and returns an iterator.
If the yield function is passed a non-nil error, execution has failed with that error; the yield function will not be called again.
If the yield function's StreamValue argument has Done == true, the value's Output and Response fields contain the final typed output and response; the yield function will not be called again.
Otherwise the Chunk field of the passed StreamValue holds a streamed chunk.
For structured output types (non-string Out), the prompt must be configured with the appropriate output schema, either through DefineDataPrompt or by using WithOutputType when defining the prompt. The typed input argument fills the input slot last, so it wins over any WithInput passed in opts.
func (*DataPrompt[In, Out]) Render ¶ added in v1.3.0
func (dp *DataPrompt[In, Out]) Render(ctx context.Context, input In) (*GenerateActionOptions, error)
Render renders the typed prompt template with the given input.
type DocsFn ¶ added in v1.12.0
DocsFn is a function that generates context documents from a prompt's input.
The input is untyped. WithDocsFn takes a function with a concrete input type and converts for you.
type Document ¶
type Document struct {
// The data that is part of this document.
Content []*Part `json:"content,omitempty"`
// The metadata for this document.
Metadata map[string]any `json:"metadata,omitempty"`
}
A Document is a piece of data that can be embedded, indexed, or retrieved. It includes metadata. It can contain multiple parts.
Example ¶
This example demonstrates the Document type used in RAG applications.
package main
import (
"fmt"
"github.com/firebase/genkit/go/ai"
)
func main() {
// Create a document with text content
doc := &ai.Document{
Content: []*ai.Part{
ai.NewTextPart("This is the document content."),
},
Metadata: map[string]any{
"source": "knowledge-base",
"page": 42,
},
}
fmt.Println("Content:", doc.Content[0].Text)
fmt.Println("Source:", doc.Metadata["source"])
}
Output: Content: This is the document content. Source: knowledge-base
type DocumentOption ¶ added in v0.5.0
type DocumentOption interface {
PromptOption
GenerateOption
PromptExecuteOption
EmbedderOption
RetrieverOption
// contains filtered or unexported methods
}
DocumentOption is an option for providing context or input documents. It applies to DefinePrompt, Generate, Prompt.Execute, Embed, and Retrieve.
func WithDocs ¶ added in v0.3.0
func WithDocs(docs ...*Document) DocumentOption
WithDocs adds documents as context for generation or as input to an embedder. Repeating this option (or mixing it with WithTextDocs) appends.
func WithTextDocs ¶ added in v0.5.0
func WithTextDocs(text ...string) DocumentOption
WithTextDocs adds text as context documents for generation or as input to an embedder. Repeating this option (or mixing it with WithDocs) appends.
type DownloadMediaOptions ¶ added in v0.3.0
type DownloadMediaOptions struct {
MaxBytes int64 // Maximum number of bytes to download.
Filter func(part *Part) bool // Filter to apply to parts that are media URLs.
}
DownloadMediaOptions configures how media is downloaded in the DownloadRequestMedia middleware.
type EmbedRequest ¶
type EmbedRequest struct {
// Input is the array of documents to generate embeddings for.
Input []*Document `json:"input,omitempty"`
// Options contains embedder-specific configuration parameters.
Options any `json:"options,omitempty"`
}
EmbedRequest represents a request to generate embeddings for documents.
type EmbedResponse ¶ added in v0.0.2
type EmbedResponse struct {
// Embeddings is the array of generated embedding vectors with metadata.
Embeddings []*Embedding `json:"embeddings,omitempty"`
}
EmbedResponse contains the generated embeddings from an embed request.
func Embed ¶ added in v0.1.0
func Embed(ctx context.Context, r api.Registry, opts ...EmbedderOption) (*EmbedResponse, error)
Embed invokes the embedder with provided options.
type Embedder ¶
type Embedder interface {
// Name returns the registry name of the embedder.
Name() string
// Embed embeds to content as part of the [EmbedRequest].
Embed(ctx context.Context, req *EmbedRequest) (*EmbedResponse, error)
// Register registers the embedder with the given registry.
Register(r api.Registry)
}
Embedder represents an embedder that can perform content embedding. It is the type to accept as an argument and to look up by name; implementations are created with NewEmbedderAction, or [genkit.DefineEmbedderAction] in an application.
func LookupEmbedder ¶
LookupEmbedder looks up a registered Embedder by name. It will try to resolve the embedder dynamically if the embedder is not found. It returns nil if the embedder was not resolved.
func NewEmbedder
deprecated
added in
v0.7.0
func NewEmbedder(name string, opts *EmbedderOptions, fn EmbedderFunc) Embedder
NewEmbedder creates a new Embedder.
Deprecated: Use NewEmbedderAction, which passes the request's options to fn as a typed value instead of leaving them type-erased on the request.
type EmbedderAction ¶ added in v1.12.0
type EmbedderAction struct {
// contains filtered or unexported fields
}
EmbedderAction is an embedder backed by a registry action. It is the concrete type returned by NewEmbedderAction; pass it to WithEmbedder to use it for embedding, or return it from a plugin's Init for the framework to register.
It implements Embedder and api.Action, so it can be passed anywhere either is accepted. It also promotes core.Action.Run, the typed equivalent of EmbedderAction.Embed.
func NewEmbedderAction ¶ added in v1.12.0
func NewEmbedderAction[Config any]( name string, opts *EmbedderOptions, fn EmbedderActionFunc[Config], ) *EmbedderAction
NewEmbedderAction creates an unregistered EmbedderAction: return it from a plugin's Init for the framework to register, or call EmbedderAction.Register directly. Applications should define embedders with [genkit.DefineEmbedderAction].
Config is the embedder's typed configuration; it is usually inferred from fn's signature. The framework deserializes the request's raw options into Config before calling fn: the exact Config type (or a pointer to it) and map[string]any (from the Dev UI and other JSON callers) are accepted, and mismatched types are rejected. The request's EmbedRequest.Options is normalized to the converted value, so it always matches the typed parameter. The config's JSON schema is inferred from Config unless EmbedderOptions.ConfigSchema overrides it.
func (*EmbedderAction) Desc ¶ added in v1.12.0
func (e *EmbedderAction) Desc() api.ActionDesc
Desc returns the embedder's action descriptor: its name, schemas, and metadata.
func (*EmbedderAction) Embed ¶ added in v1.12.0
func (e *EmbedderAction) Embed(ctx context.Context, req *EmbedRequest) (*EmbedResponse, error)
Embed runs the given Embedder.
func (*EmbedderAction) Name ¶ added in v1.12.0
func (e *EmbedderAction) Name() string
Name returns the registry name of the embedder.
func (*EmbedderAction) Register ¶ added in v1.12.0
func (e *EmbedderAction) Register(r api.Registry)
Register registers the embedder with r, making it available to lookups and to the Dev UI. A plugin that returns the embedder from its Init does not need to call this.
func (*EmbedderAction) RunJSON ¶ added in v1.12.0
func (e *EmbedderAction) RunJSON(ctx context.Context, input json.RawMessage, cb core.StreamCallback[json.RawMessage]) (json.RawMessage, error)
RunJSON runs the embedder on a JSON-encoded EmbedRequest and returns a JSON-encoded EmbedResponse. The framework uses it to serve reflection and registry-driven calls; prefer EmbedderAction.Embed.
func (*EmbedderAction) RunJSONWithTelemetry ¶ added in v1.12.0
func (e *EmbedderAction) RunJSONWithTelemetry(ctx context.Context, input json.RawMessage, cb core.StreamCallback[json.RawMessage]) (*api.ActionRunResult[json.RawMessage], error)
RunJSONWithTelemetry is EmbedderAction.RunJSON with the run's telemetry returned alongside the output.
type EmbedderActionFunc ¶ added in v1.12.0
type EmbedderActionFunc[Config any] = func(context.Context, *EmbedRequest, Config) (*EmbedResponse, error)
EmbedderActionFunc is an EmbedderFunc that additionally receives the request's typed Config: the framework deserializes the request's raw options into it before calling the function (see NewEmbedderAction).
type EmbedderArg ¶ added in v0.7.0
type EmbedderArg interface {
Name() string
}
EmbedderArg is the interface for embedder arguments. It can either be the embedder action itself or a reference to be looked up.
type EmbedderFunc ¶ added in v0.7.0
type EmbedderFunc = func(context.Context, *EmbedRequest) (*EmbedResponse, error)
EmbedderFunc is the function type for embedding documents.
type EmbedderOption ¶ added in v0.5.0
type EmbedderOption interface {
// contains filtered or unexported methods
}
EmbedderOption is an option for configuring an embedder request. It applies only to Embed.
func WithEmbedder ¶ added in v0.7.0
func WithEmbedder(embedder EmbedderArg) EmbedderOption
WithEmbedder sets either a Embedder or a EmbedderRef that may contain a config. Passing WithConfig will take precedence over the config in WithEmbedder.
func WithEmbedderName ¶ added in v0.7.0
func WithEmbedderName(name string) EmbedderOption
WithEmbedderName sets the embedder name to call for document embedding. The embedder name will be resolved to a Embedder and may error if the reference is invalid.
type EmbedderOptions ¶ added in v0.7.0
type EmbedderOptions struct {
// ConfigSchema is the JSON schema for the embedder's config.
ConfigSchema map[string]any `json:"configSchema,omitempty"`
// Label is a user-friendly name for the embedder model (e.g., "Google AI - Gemini Pro").
Label string `json:"label,omitempty"`
// Supports defines the capabilities of the embedder, such as input types and multilingual support.
Supports *EmbedderSupports `json:"supports,omitempty"`
// Dimensions specifies the number of dimensions in the embedding vector.
Dimensions int `json:"dimensions,omitempty"`
// Metadata is arbitrary key-value data attached to the action descriptor.
Metadata map[string]any `json:"-"`
}
EmbedderOptions represents the configuration options for an embedder.
func (EmbedderOptions) Overlay ¶ added in v1.12.0
func (o EmbedderOptions) Overlay(override EmbedderOptions) EmbedderOptions
Overlay returns o with every field set in override replacing o's. A field left at its zero value in override keeps o's.
type EmbedderRef ¶ added in v0.7.0
type EmbedderRef struct {
// contains filtered or unexported fields
}
EmbedderRef is a struct to hold embedder name and configuration.
func NewEmbedderRef ¶ added in v0.7.0
func NewEmbedderRef(name string, config any) EmbedderRef
NewEmbedderRef creates a new EmbedderRef with the given name and configuration.
func (EmbedderRef) Config ¶ added in v0.7.0
func (e EmbedderRef) Config() any
Config returns the configuration to use by default for this embedder.
func (EmbedderRef) Name ¶ added in v0.7.0
func (e EmbedderRef) Name() string
Name returns the name of the embedder.
type EmbedderSupports ¶ added in v0.7.0
type EmbedderSupports struct {
// Input lists the types of data the model can process (e.g., "text", "image", "video").
Input []string `json:"input,omitempty"`
// Multilingual indicates whether the model supports multiple languages.
Multilingual bool `json:"multilingual,omitempty"`
}
EmbedderSupports represents the supported capabilities of the embedder model.
type Embedding ¶ added in v0.3.0
type Embedding struct {
// Embedding is the vector representation of the input.
Embedding []float32 `json:"embedding,omitempty"`
// Metadata identifies which part of a document this embedding corresponds to.
Metadata map[string]any `json:"metadata,omitempty"`
}
Embedding represents a vector embedding with associated metadata.
Example ¶
This example demonstrates creating an Embedding for vector search.
package main
import (
"fmt"
"github.com/firebase/genkit/go/ai"
)
func main() {
// Create an embedding (typically returned by an embedder)
embedding := &ai.Embedding{
Embedding: []float32{0.1, 0.2, 0.3, 0.4, 0.5},
Metadata: map[string]any{
"source": "document-1",
},
}
fmt.Printf("Embedding dimensions: %d\n", len(embedding.Embedding))
fmt.Printf("First value: %.1f\n", embedding.Embedding[0])
}
Output: Embedding dimensions: 5 First value: 0.1
type EvaluationResult ¶ added in v0.3.0
type EvaluationResult struct {
TestCaseId string `json:"testCaseId"`
TraceID string `json:"traceId,omitempty"`
SpanID string `json:"spanId,omitempty"`
Evaluation []Score `json:"evaluation"`
}
EvaluationResult is the result of running the evaluator on a single Example. An evaluator may provide multiple scores simultaneously (e.g. if they are using an API to score on multiple criteria)
type Evaluator ¶ added in v0.3.0
type Evaluator interface {
// Name returns the name of the evaluator.
Name() string
// Evaluates a dataset.
Evaluate(ctx context.Context, req *EvaluatorRequest) (*EvaluatorResponse, error)
// Register registers the evaluator with the given registry.
Register(r api.Registry)
}
Evaluator represents an evaluator. It is the type to accept as an argument and to look up by name; implementations are created with NewEvaluatorAction or NewBatchEvaluatorAction, or their [genkit.DefineEvaluatorAction] and [genkit.DefineBatchEvaluatorAction] counterparts in an application.
func LookupEvaluator ¶ added in v0.3.0
LookupEvaluator looks up a registered Evaluator by name. It returns nil if the evaluator was not defined.
func NewBatchEvaluator
deprecated
added in
v0.7.0
func NewBatchEvaluator(name string, opts *EvaluatorOptions, fn BatchEvaluatorFunc) Evaluator
NewBatchEvaluator creates a new Evaluator. This method provides the full EvaluatorRequest to the callback function, giving more flexibility to the user for processing the data, such as batching or parallelization.
Deprecated: Use NewBatchEvaluatorAction, which passes the request's options to fn as a typed value instead of leaving them type-erased on the request.
func NewEvaluator
deprecated
added in
v0.7.0
func NewEvaluator(name string, opts *EvaluatorOptions, fn EvaluatorFunc) Evaluator
NewEvaluator creates a new Evaluator. This method processes the input dataset one-by-one.
Deprecated: Use NewEvaluatorAction, which passes the request's options to fn as a typed value instead of leaving them type-erased on the request.
type EvaluatorAction ¶ added in v1.12.0
type EvaluatorAction struct {
// contains filtered or unexported fields
}
EvaluatorAction is an evaluator backed by a registry action. It is the concrete type returned by NewEvaluatorAction and NewBatchEvaluatorAction; pass it to WithEvaluator, or return it from a plugin's Init for the framework to register.
It implements Evaluator and api.Action, so it can be passed anywhere either is accepted. It also promotes core.Action.Run, the typed equivalent of EvaluatorAction.Evaluate.
func NewBatchEvaluatorAction ¶ added in v1.12.0
func NewBatchEvaluatorAction[Config any]( name string, opts *EvaluatorOptions, fn BatchEvaluatorActionFunc[Config], ) *EvaluatorAction
NewBatchEvaluatorAction creates an unregistered EvaluatorAction: return it from a plugin's Init for the framework to register, or call EvaluatorAction.Register directly. Applications should define batch evaluators with [genkit.DefineBatchEvaluatorAction]. This method provides the full EvaluatorRequest to the callback function, giving more flexibility to the user for processing the data, such as batching or parallelization.
Config is the evaluator's typed configuration; it is usually inferred from fn's signature. See NewEvaluatorAction for how the request's options are deserialized.
EvaluatorOptions.ConfigSchema is enforced: it becomes the options slot of the action's input schema and every request is validated against it, so a schema narrower than what callers actually send now fails at the action boundary. Batch evaluators did not validate options before; leave ConfigSchema unset to accept anything.
func NewEvaluatorAction ¶ added in v1.12.0
func NewEvaluatorAction[Config any]( name string, opts *EvaluatorOptions, fn EvaluatorActionFunc[Config], ) *EvaluatorAction
NewEvaluatorAction creates an unregistered EvaluatorAction: return it from a plugin's Init for the framework to register, or call EvaluatorAction.Register directly. Applications should define evaluators with [genkit.DefineEvaluatorAction]. This method processes the input dataset one-by-one.
Config is the evaluator's typed configuration; it is usually inferred from fn's signature. The framework deserializes the request's raw options into Config before calling fn: the exact Config type (or a pointer to it) and map[string]any (from the Dev UI and other JSON callers) are accepted, and mismatched types are rejected. The config's JSON schema is inferred from Config unless EvaluatorOptions.ConfigSchema overrides it.
func (*EvaluatorAction) Desc ¶ added in v1.12.0
func (e *EvaluatorAction) Desc() api.ActionDesc
Desc returns the evaluator's action descriptor: its name, schemas, and metadata.
func (*EvaluatorAction) Evaluate ¶ added in v1.12.0
func (e *EvaluatorAction) Evaluate(ctx context.Context, req *EvaluatorRequest) (*EvaluatorResponse, error)
Evaluate runs the given Evaluator.
func (*EvaluatorAction) Name ¶ added in v1.12.0
func (e *EvaluatorAction) Name() string
Name returns the registry name of the evaluator.
func (*EvaluatorAction) Register ¶ added in v1.12.0
func (e *EvaluatorAction) Register(r api.Registry)
Register registers the evaluator with r, making it available to lookups and to the Dev UI. A plugin that returns the evaluator from its Init does not need to call this.
func (*EvaluatorAction) RunJSON ¶ added in v1.12.0
func (e *EvaluatorAction) RunJSON(ctx context.Context, input json.RawMessage, cb core.StreamCallback[json.RawMessage]) (json.RawMessage, error)
RunJSON runs the evaluator on a JSON-encoded EvaluatorRequest and returns a JSON-encoded EvaluatorResponse. The framework uses it to serve reflection and registry-driven calls; prefer EvaluatorAction.Evaluate.
func (*EvaluatorAction) RunJSONWithTelemetry ¶ added in v1.12.0
func (e *EvaluatorAction) RunJSONWithTelemetry(ctx context.Context, input json.RawMessage, cb core.StreamCallback[json.RawMessage]) (*api.ActionRunResult[json.RawMessage], error)
RunJSONWithTelemetry is EvaluatorAction.RunJSON with the run's telemetry returned alongside the output.
type EvaluatorActionFunc ¶ added in v1.12.0
type EvaluatorActionFunc[Config any] = func(context.Context, *EvaluatorCallbackRequest, Config) (*EvaluatorCallbackResponse, error)
EvaluatorActionFunc is an EvaluatorFunc that additionally receives the request's typed Config: the framework deserializes the request's raw options into it before calling the function (see NewEvaluatorAction).
type EvaluatorArg ¶ added in v0.7.0
type EvaluatorArg interface {
Name() string
}
EvaluatorArg is the interface for evaluator arguments. It can either be the evaluator action itself or a reference to be looked up.
type EvaluatorCallbackRequest ¶ added in v0.3.0
type EvaluatorCallbackRequest struct {
Input Example `json:"input"`
Options any `json:"options,omitempty"`
}
EvaluatorCallbackRequest is the data we pass to the callback function provided in defineEvaluator. The Options field is specific to the actual evaluator implementation.
type EvaluatorCallbackResponse ¶ added in v0.3.0
type EvaluatorCallbackResponse = EvaluationResult
EvaluatorCallbackResponse is the result on evaluating a single Example
type EvaluatorFunc ¶ added in v0.7.0
type EvaluatorFunc = func(context.Context, *EvaluatorCallbackRequest) (*EvaluatorCallbackResponse, error)
EvaluatorFunc is the function type for evaluator implementations.
type EvaluatorOption ¶ added in v0.5.0
type EvaluatorOption interface {
// contains filtered or unexported methods
}
EvaluatorOption is an option for providing a dataset to evaluate. It applies only to Evaluator.Evaluate.
func WithDataset ¶ added in v0.5.0
func WithDataset(examples ...*Example) EvaluatorOption
WithDataset adds examples to the dataset to evaluate. Repeating this option appends.
func WithEvaluator ¶ added in v0.7.0
func WithEvaluator(evaluator EvaluatorArg) EvaluatorOption
WithEvaluator sets either a Evaluator or a EvaluatorRef that may contain a config. Passing WithConfig will take precedence over the config in WithEvaluator.
func WithEvaluatorName ¶ added in v0.7.0
func WithEvaluatorName(name string) EvaluatorOption
WithEvaluatorName sets the evaluator name to call for document evaluation. The evaluator name will be resolved to a Evaluator and may error if the reference is invalid.
func WithID ¶ added in v0.5.0
func WithID(ID string) EvaluatorOption
WithID sets the ID of the evaluation to uniquely identify it. Repeating this option takes the last ID set.
type EvaluatorOptions ¶ added in v0.3.0
type EvaluatorOptions struct {
// ConfigSchema is the JSON schema for the evaluator's config.
ConfigSchema map[string]any `json:"configSchema,omitempty"`
// Metadata is arbitrary key-value data attached to the action descriptor.
Metadata map[string]any `json:"-"`
// DisplayName is the name of the evaluator as it appears in the UI.
DisplayName string `json:"displayName"`
// Definition is the definition of the evaluator.
Definition string `json:"definition"`
// IsBilled is a flag indicating if the evaluator is billed.
IsBilled bool `json:"isBilled,omitempty"`
}
EvaluatorOptions configures an evaluator created with NewEvaluatorAction or NewBatchEvaluatorAction.
type EvaluatorRef ¶ added in v0.7.0
type EvaluatorRef struct {
// contains filtered or unexported fields
}
EvaluatorRef is a struct to hold evaluator name and configuration.
func NewEvaluatorRef ¶ added in v0.7.0
func NewEvaluatorRef(name string, config any) EvaluatorRef
NewEvaluatorRef creates a new EvaluatorRef with the given name and configuration.
func (EvaluatorRef) Config ¶ added in v0.7.0
func (e EvaluatorRef) Config() any
Config returns the configuration to use by default for this evaluator.
func (EvaluatorRef) Name ¶ added in v0.7.0
func (e EvaluatorRef) Name() string
Name returns the name of the evaluator.
type EvaluatorRequest ¶ added in v0.3.0
type EvaluatorRequest struct {
Dataset []*Example `json:"dataset"`
EvaluationId string `json:"evalRunId"`
Options any `json:"options,omitempty"`
}
EvaluatorRequest is the data we pass to evaluate a dataset. The Options field is specific to the actual evaluator implementation.
type EvaluatorResponse ¶ added in v0.3.0
type EvaluatorResponse = []EvaluationResult
EvaluatorResponse is a collection of EvaluationResult structs, it represents the result on the entire input dataset.
func Evaluate ¶ added in v0.3.0
func Evaluate(ctx context.Context, r api.Registry, opts ...EvaluatorOption) (*EvaluatorResponse, error)
Evaluate calls the retrivers with provided options.
type Example ¶ added in v0.3.0
type Example struct {
TestCaseId string `json:"testCaseId,omitempty"`
Input any `json:"input"`
Output any `json:"output,omitempty"`
Context []any `json:"context,omitempty"`
Reference any `json:"reference,omitempty"`
TraceIds []string `json:"traceIds,omitempty"`
}
Example is a single example that requires evaluation
type ExecutionOption ¶ added in v0.3.0
type ExecutionOption interface {
GenerateOption
PromptExecuteOption
// contains filtered or unexported methods
}
ExecutionOption is an option for the execution of a prompt or generate request. It applies only to Generate() and prompt.Execute().
func WithStreaming ¶ added in v0.1.0
func WithStreaming(callback ModelStreamCallback) ExecutionOption
WithStreaming sets the stream callback for the generate request. A callback is a function that is called with each chunk of the generated response before the final response is returned. Repeating this option takes the last callback set. The stream-returning APIs (GenerateStream, Prompt.ExecuteStream, and their typed variants) attach their own iterator callback without displacing one set here; both receive every chunk.
type FinishReason ¶
type FinishReason string
FinishReason indicates why generation stopped.
const ( FinishReasonStop FinishReason = "stop" FinishReasonLength FinishReason = "length" FinishReasonBlocked FinishReason = "blocked" FinishReasonAborted FinishReason = "aborted" FinishReasonInterrupted FinishReason = "interrupted" FinishReasonOther FinishReason = "other" FinishReasonUnknown FinishReason = "unknown" )
type FormatHandler ¶ added in v0.5.0
type FormatHandler interface {
// ParseMessage parses the message and returns a new formatted message.
//
// Legacy: New format handlers should implement this as a no-op passthrough and implement [StreamingFormatHandler] instead.
ParseMessage(message *Message) (*Message, error)
// Instructions returns the formatter instructions to embed in the prompt.
Instructions() string
// Config returns the output config for the model request.
Config() ModelOutputConfig
}
FormatHandler is a handler for formatting messages. A new instance is created via Formatter.Handler for each request.
type Formatter ¶ added in v0.5.0
type Formatter interface {
// Name returns the name of the formatter.
Name() string
// Handler returns the handler for the formatter.
Handler(schema map[string]any) (FormatHandler, error)
}
Formatter represents the Formatter interface.
type GenerateActionOptions ¶ added in v0.3.0
type GenerateActionOptions struct {
// Config contains configuration parameters for the generation request.
Config any `json:"config,omitempty"`
// Docs provides retrieved documents to be used as context for this generation.
Docs []*Document `json:"docs,omitempty"`
// MaxTurns is the maximum number of tool call iterations that can be performed
// in a single generate call. Defaults to 5.
MaxTurns int `json:"maxTurns,omitempty"`
// Messages contains the conversation history for multi-turn prompting when supported.
Messages []*Message `json:"messages,omitempty"`
// Model is a model name (e.g., "vertexai/gemini-flash-latest").
Model string `json:"model,omitempty"`
// Output specifies the desired output format. Defaults to the model's default if unspecified.
Output *GenerateActionOutputConfig `json:"output,omitempty"`
Resources []string `json:"resources,omitempty"`
// Resume provides options for resuming an interrupted generation.
Resume *GenerateActionResume `json:"resume,omitempty"`
// ReturnToolRequests, when true, returns tool calls for manual processing instead of
// automatically resolving them.
ReturnToolRequests bool `json:"returnToolRequests,omitempty"`
// StepName is a custom step name for this generate call to display in trace views.
// Defaults to "generate".
StepName string `json:"stepName,omitempty"`
// ToolChoice controls tool calling mode. Auto lets the model decide, required forces
// the model to choose a tool, and none forces the model not to use any tools. Defaults to auto.
ToolChoice ToolChoice `json:"toolChoice,omitempty"`
// Tools is a list of registered tool names for this generation if supported.
Tools []string `json:"tools,omitempty"`
// Use is middleware to apply to this generation, referenced by name with optional config.
Use []*MiddlewareRef `json:"use,omitempty"`
}
GenerateActionOptions holds configuration for a generate action request.
type GenerateActionOutputConfig ¶ added in v0.3.0
type GenerateActionOutputConfig struct {
// Constrained indicates whether to enforce strict adherence to the schema.
Constrained bool `json:"constrained,omitempty"`
// ContentType specifies the MIME type of the output content.
ContentType string `json:"contentType,omitempty"`
// Format specifies the desired output format (e.g., "json", "text").
Format string `json:"format,omitempty"`
// Instructions provides additional guidance for the output format.
Instructions *string `json:"instructions,omitempty"`
// JsonSchema is a JSON Schema describing the desired structure of JSON output.
JsonSchema map[string]any `json:"jsonSchema,omitempty"`
}
GenerateActionOutputConfig specifies the desired output format for a generate action.
type GenerateActionResume ¶ added in v0.3.0
type GenerateActionResume struct {
// Metadata contains additional context for resuming the generation.
Metadata map[string]any `json:"metadata,omitempty"`
// Respond contains tool response parts to send to the model when resuming.
Respond []*Part `json:"respond,omitempty"`
// Restart contains tool request parts to restart when resuming.
Restart []*Part `json:"restart,omitempty"`
}
GenerateActionResume holds options for resuming an interrupted generation.
func NewResume ¶ added in v1.10.0
func NewResume(restarts, responds []*Part) *GenerateActionResume
NewResume constructs a GenerateActionResume from Part slices. This is useful when building GenerateActionOptions directly (e.g., from a rendered prompt) and need to set the Resume field from *Part values produced by ToolAction.RestartWith or ToolAction.RespondWith.
type GenerateNext ¶ added in v1.7.0
type GenerateNext = func(ctx context.Context, params *GenerateParams) (*ModelResponse, error)
GenerateNext is the next function in the WrapGenerate hook chain.
type GenerateOption ¶ added in v0.1.0
type GenerateOption interface {
// contains filtered or unexported methods
}
GenerateOption is an option for generating a model response. It applies only to Generate().
func WithStepName ¶ added in v1.8.0
func WithStepName(name string) GenerateOption
WithStepName sets a custom name for the generation step in traces.
func WithToolResponses ¶ added in v0.6.0
func WithToolResponses(parts ...*Part) GenerateOption
WithToolResponses provides resolved responses for interrupted tool calls. Use this when you already have the result and want to skip re-executing the tool. Repeating this option appends.
func WithToolRestarts ¶ added in v0.6.0
func WithToolRestarts(parts ...*Part) GenerateOption
WithToolRestarts re-executes interrupted tool calls with additional metadata. Use this when the original call lacked required context (e.g., auth, user confirmation) that should now allow the tool to complete successfully. Repeating this option appends.
type GenerateParams ¶ added in v1.7.0
type GenerateParams struct {
// Options is the original options passed to [Generate].
Options *GenerateActionOptions
// Request is the current model request for this iteration, with accumulated messages.
Request *ModelRequest
// Iteration is the current tool-loop iteration (0-indexed).
Iteration int
// MessageIndex is the index of the next message in the streamed response sequence.
// Middleware that streams extra messages (e.g. injected user content) should emit
// chunks at this index and advance it so downstream middleware and the model
// receive the shifted value.
MessageIndex int
// Callback is the streaming callback supplied to [Generate], or nil if not streaming.
// Middleware may invoke it to emit chunks, setting [ModelResponseChunk.Role] and
// [ModelResponseChunk.Index] explicitly.
Callback ModelStreamCallback
}
GenerateParams holds params for the WrapGenerate hook.
type GenerationCommonConfig ¶
type GenerationCommonConfig struct {
// API Key to use for the model call, overrides API key provided in plugin config.
APIKey string `json:"apiKey,omitempty"`
// MaxOutputTokens limits the maximum number of tokens generated in the response.
MaxOutputTokens int `json:"maxOutputTokens,omitempty"`
// StopSequences specifies sequences that will cause generation to stop when encountered.
StopSequences []string `json:"stopSequences,omitempty"`
// Temperature controls randomness in generation. Higher values (e.g., 0.9) make output more random,
// while lower values (e.g., 0.1) make it more deterministic. Typical range is 0.0 to 1.0.
Temperature float64 `json:"temperature,omitempty"`
// TopK limits sampling to the K most likely tokens at each step.
TopK int `json:"topK,omitempty"`
// TopP (nucleus sampling) limits sampling to tokens whose cumulative probability exceeds P.
TopP float64 `json:"topP,omitempty"`
// Version specifies a particular version of a model family,
// e.g., "gemini-3.5-flash-001" for the "gemini-3.5-flash" family.
Version string `json:"version,omitempty"`
}
GenerationCommonConfig holds configuration parameters for model generation requests.
type GenerationUsage ¶
type GenerationUsage struct {
// CachedContentTokens counts tokens that were served from cache.
CachedContentTokens int `json:"cachedContentTokens,omitempty"`
// Custom contains additional usage metrics specific to the model provider.
Custom map[string]float64 `json:"custom,omitempty"`
// InputAudioFiles is the number of audio files in the input.
InputAudioFiles int `json:"inputAudioFiles,omitempty"`
// InputCharacters is the number of characters in the input.
InputCharacters int `json:"inputCharacters,omitempty"`
// InputImages is the number of images in the input.
InputImages int `json:"inputImages,omitempty"`
// InputTokens is the number of tokens in the input prompt.
InputTokens int `json:"inputTokens,omitempty"`
// InputVideos is the number of videos in the input.
InputVideos int `json:"inputVideos,omitempty"`
// OutputAudioFiles is the number of audio files generated in the output.
OutputAudioFiles int `json:"outputAudioFiles,omitempty"`
// OutputCharacters is the number of characters generated in the output.
OutputCharacters int `json:"outputCharacters,omitempty"`
// OutputImages is the number of images generated in the output.
OutputImages int `json:"outputImages,omitempty"`
// OutputTokens is the number of tokens generated in the response.
OutputTokens int `json:"outputTokens,omitempty"`
// OutputVideos is the number of videos generated in the output.
OutputVideos int `json:"outputVideos,omitempty"`
// ThoughtsTokens counts tokens used in reasoning or thinking processes.
ThoughtsTokens int `json:"thoughtsTokens,omitempty"`
// TotalTokens is the sum of input and output tokens.
TotalTokens int `json:"totalTokens,omitempty"`
}
GenerationUsage provides information about resource consumption during generation.
type Hooks ¶ added in v1.7.0
type Hooks struct {
// Tools are additional tools to register during the generation this
// middleware is attached to. They are available to the model alongside
// any user-supplied tools.
Tools []Tool
// WrapGenerate wraps each iteration of the tool loop. It sees the
// accumulated request, the iteration index, and the streaming callback.
// A single Generate() with N tool-call turns invokes this hook N+1 times.
WrapGenerate func(ctx context.Context, params *GenerateParams, next GenerateNext) (*ModelResponse, error)
// WrapModel wraps each model API call. Retry, fallback, and caching
// middleware typically hook here.
WrapModel func(ctx context.Context, params *ModelParams, next ModelNext) (*ModelResponse, error)
// WrapTool wraps each tool execution. It may be called concurrently when
// multiple tools execute in parallel for the same Generate() call; any
// state closed over from the enclosing scope that this hook mutates must
// be guarded with sync primitives.
WrapTool func(ctx context.Context, params *ToolParams, next ToolNext) (*MultipartToolResponse, error)
}
Hooks is the per-call bundle of hook functions produced by a Middleware's New method. Each field is optional; a nil hook is treated as a pass-through.
type InputOption ¶ added in v1.3.0
type InputOption interface {
PromptOption
// contains filtered or unexported methods
}
InputOption is an option for the input of a prompt. It applies only to DefinePrompt().
type InputSchemaOption ¶ added in v1.12.0
type InputSchemaOption interface {
InputOption
ToolOption
}
InputSchemaOption is an InputOption that also applies to the tool constructors, where the input schema it supplies stands in for an In type parameter of 'any'. A tool whose input shape a Go type can express should use the type parameter instead.
func WithInputSchema ¶ added in v1.3.0
func WithInputSchema(schema map[string]any) InputSchemaOption
WithInputSchema manually provides a schema map for the input.
func WithInputSchemaName ¶ added in v1.3.0
func WithInputSchemaName(name string) InputSchemaOption
WithInputSchemaName sets a pre-registered schema by name for the input. The schema is resolved from the registry at execution time; register it with github.com/firebase/genkit/go/genkit.DefineSchema.
func WithInputType ¶ added in v0.3.0
func WithInputType(input any) InputSchemaOption
WithInputType uses the type provided to derive the input schema. The inputted value may serve as the default input if no input is given at generation time depending on the action. Only supports structs and map[string]any.
It applies to the tool constructors too, where the derived schema stands in for an In type parameter of 'any'. Prefer the type parameter there: it is the same schema plus a typed function signature.
type InterruptOptions ¶ added in v0.3.0
InterruptOptions provides configuration for tool interruption.
type Media ¶ added in v0.3.0
type Media struct {
// ContentType specifies the MIME type of the media. Inferred from the data URI if not provided.
ContentType string `json:"contentType,omitempty"`
// Url is a "data:" or "https:" URI containing the media content.
Url string `json:"url,omitempty"`
}
Media represents media content with a URL and content type.
type Message ¶
type Message struct {
// Content holds the message parts (text, media, tool calls, etc.).
Content []*Part `json:"content,omitempty"`
// Metadata contains arbitrary key-value data associated with this message.
Metadata map[string]any `json:"metadata,omitempty"`
// Role indicates which entity (system, user, model, or tool) generated this message.
Role Role `json:"role,omitempty"`
}
Message represents the contents of a model message in a conversation.
func HistoryFromContext ¶ added in v1.12.0
HistoryFromContext returns the conversation history passed to Prompt.Execute via WithMessages or WithMessagesFn, or nil if there is none.
Call it from a prompt's own content functions, where it is the function-form counterpart of {{history}}. A prompt that declares a conversation owns the history: the caller's messages are not appended on top, so a function that wants them must read them here and return them, which is what makes summarizing or truncating possible. A prompt that declares no messages of its own has the caller's history used as the conversation directly.
The returned slice is the caller's; treat it as read-only.
func NewMessage ¶
NewMessage creates a new Message with the provided role, metadata and parts. Use NewTextMessage if you have a text-only message.
func NewModelMessage ¶
NewModelMessage creates a new Message with role "model" and provided parts. Use NewModelTextMessage if you have a text-only message.
func NewModelTextMessage ¶
NewModelTextMessage creates a new Message with role "model" and content with a single text part with the content of provided text.
func NewSystemMessage ¶
NewSystemMessage creates a new Message with role "system" and provided parts. Use NewSystemTextMessage if you have a text-only message.
func NewSystemTextMessage ¶
NewSystemTextMessage creates a new Message with role "system" and content with a single text part with the content of provided text.
Example ¶
This example demonstrates creating system and model messages.
package main
import (
"fmt"
"github.com/firebase/genkit/go/ai"
)
func main() {
// Create a system message
sysMsg := ai.NewSystemTextMessage("You are a helpful assistant.")
fmt.Println("System role:", sysMsg.Role)
// Create a model response message
modelMsg := ai.NewModelTextMessage("I'm here to help!")
fmt.Println("Model role:", modelMsg.Role)
}
Output: System role: system Model role: model
func NewTextMessage ¶
NewTextMessage creates a new Message with the provided role and content with a single part containint provided text.
func NewUserMessage ¶
NewUserMessage creates a new Message with role "user" and provided parts. Use NewUserTextMessage if you have a text-only message.
Example ¶
This example demonstrates building a multi-turn conversation.
package main
import (
"fmt"
"github.com/firebase/genkit/go/ai"
)
func main() {
// Build a conversation with multiple parts
userMsg := ai.NewUserMessage(
ai.NewTextPart("What's in this image?"),
ai.NewMediaPart("image/jpeg", "base64data..."),
)
fmt.Println("Role:", userMsg.Role)
fmt.Println("Parts:", len(userMsg.Content))
}
Output: Role: user Parts: 2
func NewUserMessageWithMetadata ¶ added in v0.3.0
NewUserMessageWithMetadata creates a new Message with role "user" with provided metadata and parts. Use NewUserTextMessage if you have a text-only message.
func NewUserTextMessage ¶
NewUserTextMessage creates a new Message with role "user" and content with a single text part with the content of provided text.
Example ¶
This example demonstrates creating a message with text content.
package main
import (
"fmt"
"github.com/firebase/genkit/go/ai"
)
func main() {
// Create a user message with text
msg := ai.NewUserTextMessage("What is the capital of France?")
fmt.Println("Role:", msg.Role)
fmt.Println("Text:", msg.Content[0].Text)
}
Output: Role: user Text: What is the capital of France?
func (*Message) Clone ¶ added in v1.7.0
Clone returns a shallow copy of the Message with its own Content slice and Metadata map. Callers can replace parts or add metadata keys without mutating the original.
func (*Message) MediaParts ¶ added in v1.12.0
MediaParts returns every media part of a Message, each carrying its content type alongside its data. It returns nil if the message has none.
func (*Message) Text ¶ added in v0.3.0
Text returns the textual contents of a Message as a string, concatenating its text parts and skipping every other kind. It returns an empty string if the message has none, which is what a message carrying only an image, only raw data, or only a tool request comes back as; read those with Message.MediaParts and ToolRequests instead.
A data part is deliberately not text: it holds a blob, which the plugins send as bytes and plugins/internal/uri.Data decodes as a data: URI, so concatenating one here would splice base64 into prose. If you want to get reasoning from the message, use Reasoning() instead.
func (*Message) WithCacheName ¶ added in v0.3.0
WithCacheName adds cache name to use in the generate request
func (*Message) WithCacheTTL ¶ added in v0.3.0
WithCacheTTL adds cache TTL configuration for the desired message
type MessagesFn ¶ added in v0.5.0
MessagesFn is a function that generates messages from a prompt's input.
The input is untyped. WithMessagesFn takes a function with a concrete input type and converts for you.
type Middleware ¶ added in v1.7.0
type Middleware interface {
// Name returns the registered middleware's unique identifier. Must be a
// stable constant, since it is read from a zero value of the config type
// during descriptor creation.
Name() string
// New produces a fresh [Hooks] bundle for one Generate() call. It is
// invoked per-Generate, so any state the bundle's hooks need to share
// (counters, caches) may be allocated in this method and closed over by
// the returned hooks.
New(ctx context.Context) (*Hooks, error)
}
Middleware is the contract every value passed to WithUse satisfies. The config struct both identifies the middleware (via [Name]) and produces a per-call Hooks bundle (via [New]).
Plugin-level state belongs on unexported fields of the config type. A plugin's MiddlewarePlugin.Middlewares sets those fields on a prototype that is preserved across JSON dispatch by value-copy inside the descriptor.
type MiddlewareDesc ¶ added in v1.5.0
type MiddlewareDesc struct {
// ConfigSchema is a JSON Schema describing the middleware's configuration.
ConfigSchema map[string]any `json:"configSchema,omitempty"`
// Description explains what the middleware does.
Description string `json:"description,omitempty"`
// Metadata contains additional context for the middleware.
Metadata map[string]any `json:"metadata,omitempty"`
// Name is the middleware's unique identifier.
Name string `json:"name,omitempty"`
// contains filtered or unexported fields
}
MiddlewareDesc is the registered descriptor for a middleware.
func LookupMiddleware ¶ added in v1.7.0
func LookupMiddleware(r api.Registry, name string) *MiddlewareDesc
LookupMiddleware returns the registered middleware descriptor with the given name, or nil if no such descriptor exists in the registry or any ancestor. Primarily useful for inspection and for the reflection API; callers dispatching middleware should do so through WithUse.
func NewMiddleware ¶ added in v1.7.0
func NewMiddleware[M Middleware](description string, prototype M) *MiddlewareDesc
NewMiddleware constructs a descriptor without registering it. Useful for MiddlewarePlugin.Middlewares implementations that defer registration to [genkit.Init]. The prototype argument supplies both the registered name (via its Middleware.Name method) and any plugin-level state that should flow into JSON-dispatched invocations via unexported fields preserved by value-copy.
func (*MiddlewareDesc) Register ¶ added in v1.7.0
func (d *MiddlewareDesc) Register(r api.Registry)
Register records this descriptor in the registry under its name so it can be resolved during JSON dispatch and surfaced to the Dev UI.
type MiddlewareFunc ¶ added in v1.7.0
MiddlewareFunc adapts a per-call factory closure to the Middleware interface for ad-hoc inline use, without a registered descriptor or plugin wiring. The adapted middleware does not appear in the Dev UI.
Example:
ai.WithUse(ai.MiddlewareFunc(func(ctx context.Context) (*ai.Hooks, error) {
return &ai.Hooks{WrapModel: ...}, nil
}))
func (MiddlewareFunc) Name ¶ added in v1.7.0
func (MiddlewareFunc) Name() string
Name returns the placeholder name shared by all MiddlewareFunc values. Uniqueness is unnecessary: inline middleware is resolved via the fast path in [resolveRefs] and never goes through a name-keyed registry lookup.
func (MiddlewareFunc) New ¶ added in v1.7.0
func (f MiddlewareFunc) New(ctx context.Context) (*Hooks, error)
New implements Middleware by calling f.
type MiddlewarePlugin ¶ added in v1.7.0
type MiddlewarePlugin interface {
Middlewares(ctx context.Context) ([]*MiddlewareDesc, error)
}
MiddlewarePlugin is implemented by plugins that provide middleware. The returned descriptors are registered in the registry during [genkit.Init], with any plugin-level state captured by the descriptor's build closure via the prototype passed to NewMiddleware.
type MiddlewareRef ¶ added in v1.5.0
type MiddlewareRef struct {
// Config contains the middleware configuration.
Config any `json:"config,omitempty"`
// Name is the name of the registered middleware.
Name string `json:"name,omitempty"`
}
MiddlewareRef is a serializable reference to a registered middleware with config.
type Model ¶
type Model interface {
// Name returns the registry name of the model.
Name() string
// Generate applies the [Model] to provided request, handling tool requests and handles streaming.
Generate(ctx context.Context, req *ModelRequest, cb ModelStreamCallback) (*ModelResponse, error)
// Register registers the model with the given registry.
Register(r api.Registry)
}
Model represents a model that can generate content based on a request. It is the type to accept as an argument and to look up by name; implementations are created with NewModelAction, or [genkit.DefineModelAction] in an application.
func LookupModel ¶
LookupModel looks up a registered Model by name. It will try to resolve the model dynamically if the model is not found. It returns nil if the model was not resolved.
func NewModel
deprecated
added in
v0.7.0
func NewModel(name string, opts *ModelOptions, fn ModelFunc) Model
NewModel creates a new Model.
Deprecated: Use NewModelAction, which passes the request's config to fn as a typed value instead of leaving it type-erased on the request.
type ModelAction ¶ added in v0.3.0
type ModelAction struct {
// contains filtered or unexported fields
}
ModelAction is a generative model backed by a registry action. It is the concrete type returned by NewModelAction; pass it to WithModel to use it for generation, or return it from a plugin's Init for the framework to register.
It implements Model and api.Action, so it can be passed anywhere either is accepted. It also promotes core.Action.Run, the typed equivalent of ModelAction.Generate.
func NewModelAction ¶ added in v1.12.0
func NewModelAction[Config any]( name string, opts *ModelOptions, fn ModelActionFunc[Config], ) *ModelAction
NewModelAction creates an unregistered ModelAction: return it from a plugin's Init for the framework to register, or call ModelAction.Register directly. Applications should define models with [genkit.DefineModelAction].
Config is the model's typed configuration; it is usually inferred from fn's signature. The framework deserializes the request's raw config into Config before calling fn: the exact Config type (or a pointer to it) and map[string]any (from the Dev UI and other JSON callers) are accepted, and mismatched types are rejected. The request's ModelRequest.Config is normalized to the converted value, so it always matches the typed parameter. The config's JSON schema is inferred from Config unless ModelOptions.ConfigSchema overrides it.
The config schema is enforced by input validation on every call, so if Config's JSON marshaling diverges from its reflected schema (e.g. SDK wrapper types like Opt[float64] that marshal to primitives but reflect as objects), set ModelOptions.ConfigSchema explicitly or requests will be rejected at the action boundary.
func (*ModelAction) Desc ¶ added in v1.12.0
func (m *ModelAction) Desc() api.ActionDesc
Desc returns the model's action descriptor: its name, schemas, and metadata.
func (*ModelAction) Generate ¶ added in v1.12.0
func (m *ModelAction) Generate(ctx context.Context, req *ModelRequest, cb ModelStreamCallback) (*ModelResponse, error)
Generate applies the [Action] to provided request.
func (*ModelAction) Name ¶ added in v1.12.0
func (m *ModelAction) Name() string
Name returns the registry name of the model.
func (*ModelAction) Register ¶ added in v1.12.0
func (m *ModelAction) Register(r api.Registry)
Register registers the model with r, making it available to lookups and to the Dev UI. A plugin that returns the model from its Init does not need to call this.
func (*ModelAction) RunJSON ¶ added in v1.12.0
func (m *ModelAction) RunJSON(ctx context.Context, input json.RawMessage, cb core.StreamCallback[json.RawMessage]) (json.RawMessage, error)
RunJSON runs the model on a JSON-encoded ModelRequest and returns a JSON-encoded ModelResponse. The framework uses it to serve reflection and registry-driven calls; prefer ModelAction.Generate.
func (*ModelAction) RunJSONWithTelemetry ¶ added in v1.12.0
func (m *ModelAction) RunJSONWithTelemetry(ctx context.Context, input json.RawMessage, cb core.StreamCallback[json.RawMessage]) (*api.ActionRunResult[json.RawMessage], error)
RunJSONWithTelemetry is ModelAction.RunJSON with the run's telemetry returned alongside the output.
type ModelActionFunc ¶ added in v1.12.0
type ModelActionFunc[Config any] = func(context.Context, *ModelRequest, Config, ModelStreamCallback) (*ModelResponse, error)
ModelActionFunc is a ModelFunc that additionally receives the request's typed Config: the framework deserializes the request's raw config into it before calling the function (see NewModelAction).
type ModelArg ¶ added in v0.5.0
type ModelArg interface {
Name() string
}
ModelArg is the interface for model arguments. It can either be the retriever action itself or a reference to be looked up.
type ModelFunc ¶ added in v0.3.0
type ModelFunc = core.StreamingFunc[*ModelRequest, *ModelResponse, *ModelResponseChunk]
ModelFunc is a streaming function that takes in a ModelRequest and generates a ModelResponse, optionally streaming ModelResponseChunks.
type ModelInfo ¶
type ModelInfo struct {
// ConfigSchema defines the model-specific configuration schema.
ConfigSchema map[string]any `json:"configSchema,omitempty"`
// Label is a friendly display name for this model (e.g., "Google AI - Gemini Pro").
Label string `json:"label,omitempty"`
// Stage indicates the development stage of this model.
// Featured models are recommended for general use, stable models are well-tested,
// unstable models are experimental, legacy models are not recommended for new projects,
// and deprecated models may be removed in future versions.
Stage ModelStage `json:"stage,omitempty"`
// Supports describes the capabilities that this model supports.
Supports *ModelSupports `json:"supports,omitempty"`
// Versions lists acceptable names for this model (e.g., different versions).
Versions []string `json:"versions,omitempty"`
}
ModelInfo contains metadata about a model's capabilities and characteristics.
type ModelMiddleware
deprecated
added in
v0.3.0
type ModelMiddleware = core.Middleware[*ModelRequest, *ModelResponse, *ModelResponseChunk]
ModelMiddleware is middleware for model generate requests that takes in a ModelFunc, does something, then returns another ModelFunc.
Deprecated: Use Middleware interface with WithUse instead, which supports Generate, Model, and Tool hooks.
func DownloadRequestMedia ¶ added in v0.3.0
func DownloadRequestMedia(opts *DownloadMediaOptions) ModelMiddleware
DownloadRequestMedia downloads media from a URL and replaces the media part with a base64 encoded string.
type ModelNext ¶ added in v1.7.0
type ModelNext = func(ctx context.Context, params *ModelParams) (*ModelResponse, error)
ModelNext is the next function in the WrapModel hook chain.
type ModelOperation ¶ added in v1.3.0
type ModelOperation = core.Operation[*ModelResponse]
ModelOperation is a background operation for a model.
func CheckModelOperation ¶ added in v1.3.0
func CheckModelOperation(ctx context.Context, r api.Registry, op *ModelOperation) (*ModelOperation, error)
CheckModelOperation checks the status of a background model operation by looking up the model and calling its Check method.
func GenerateOperation ¶ added in v1.3.0
func GenerateOperation(ctx context.Context, r *registry.Registry, opts ...GenerateOption) (*ModelOperation, error)
GenerateOperation generates a model response as a long-running operation based on the provided options.
type ModelOptions ¶ added in v0.7.0
type ModelOptions struct {
// ConfigSchema is the JSON schema for the model's config. Inferred from the
// constructor's Config type parameter when nil.
ConfigSchema map[string]any
// Label is a user-friendly name for the model. Defaults to its name.
Label string
// Stage indicates the maturity stage of the model.
Stage ModelStage
// Supports describes what the model can do. A nil value claims nothing.
Supports *ModelSupports
// Versions lists the model versions a request may pin through its config.
Versions []string
// Metadata is arbitrary key-value data attached to the action descriptor.
Metadata map[string]any
}
ModelOptions represents the configuration options for a model.
func (ModelOptions) Overlay ¶ added in v1.12.0
func (o ModelOptions) Overlay(override ModelOptions) ModelOptions
Overlay returns o with every field set in override replacing o's. A field left at its zero value in override keeps o's.
type ModelOutputConfig ¶ added in v0.3.0
type ModelOutputConfig struct {
// Constrained indicates whether to enforce strict adherence to the schema.
Constrained bool `json:"constrained,omitempty"`
// ContentType specifies the MIME type of the output content.
ContentType string `json:"contentType,omitempty"`
// Format specifies the desired output format (e.g., "json", "text").
Format string `json:"format,omitempty"`
// Schema is a JSON Schema describing the desired structure of the output.
Schema map[string]any `json:"schema,omitempty"`
}
OutputConfig describes the structure that the model's output should conform to. If Format is OutputFormatJSON, then Schema can describe the desired form of the generated JSON.
type ModelParams ¶ added in v1.7.0
type ModelParams struct {
// Request is the model request about to be sent.
Request *ModelRequest
// Callback is the streaming callback, or nil if not streaming.
Callback ModelStreamCallback
}
ModelParams holds params for the WrapModel hook.
type ModelRef ¶ added in v0.5.0
type ModelRef struct {
// contains filtered or unexported fields
}
ModelRef is a struct to hold model name and configuration.
ModelRef supports JSON marshaling: it serializes as a plain string when only a name is present, or as {"name": "...", "config": ...} when configuration is also set.
func NewModelRef ¶ added in v0.5.0
NewModelRef creates a new ModelRef with the given name and configuration.
Example ¶
This example demonstrates creating a model reference with configuration.
package main
import (
"fmt"
"github.com/firebase/genkit/go/ai"
)
func main() {
// Create a reference to a model with custom configuration
// The config type depends on the model provider
modelRef := ai.NewModelRef("googleai/gemini-flash-latest", map[string]any{
"temperature": 0.7,
})
fmt.Println("Model name:", modelRef.Name())
}
Output: Model name: googleai/gemini-flash-latest
func (ModelRef) Config ¶ added in v0.5.0
Config returns the configuration to use by default for this model.
func (ModelRef) JSONSchema ¶ added in v1.7.0
func (ModelRef) JSONSchema() *jsonschema.Schema
JSONSchema implements the invopop/jsonschema customSchemaImpl interface so that schema reflection produces the correct object schema instead of an empty object (ModelRef has only unexported fields).
func (ModelRef) MarshalJSON ¶ added in v1.7.0
MarshalJSON implements json.Marshaler. ModelRef always marshals as a JSON object with "name" and optional "config" fields.
func (*ModelRef) UnmarshalJSON ¶ added in v1.7.0
UnmarshalJSON implements json.Unmarshaler. It accepts either a JSON object with "name" and optional "config" fields, or a plain string (interpreted as the model name).
type ModelReference ¶ added in v1.5.0
type ModelRequest ¶ added in v0.3.0
type ModelRequest struct {
// Config holds model-specific configuration parameters.
Config any `json:"config,omitempty"`
// Docs provides retrieved documents to be used as context for this generation.
Docs []*Document `json:"docs,omitempty"`
// Messages contains the conversation history for the model.
Messages []*Message `json:"messages,omitempty"`
// Output describes the desired response format.
Output *ModelOutputConfig `json:"output,omitempty"`
// ToolChoice controls how the model uses tools (auto, required, or none).
ToolChoice ToolChoice `json:"toolChoice,omitempty"`
// Tools lists the available tools that the model can ask the client to run.
Tools []*ToolDefinition `json:"tools,omitempty"`
}
A ModelRequest is a request to generate completions from a model.
func NewModelRequest ¶ added in v0.3.0
func NewModelRequest(config any, messages ...*Message) *ModelRequest
NewModelRequest create a new ModelRequest with provided config and messages.
type ModelResponse ¶ added in v0.3.0
type ModelResponse struct {
// Custom contains model-specific extra information. Deprecated: use Raw instead.
Custom any `json:"custom,omitempty"`
// FinishMessage provides additional details about why generation finished.
FinishMessage string `json:"finishMessage,omitempty"`
// FinishReason indicates why generation stopped (e.g., stop, length, blocked).
FinishReason FinishReason `json:"finishReason,omitempty"`
// LatencyMs is the time the request took in milliseconds.
LatencyMs float64 `json:"latencyMs,omitempty"`
// Message contains the generated response content.
Message *Message `json:"message,omitempty"`
// Operation provides information about a long-running background task if applicable.
Operation *Operation `json:"operation,omitempty"`
// Raw contains the unprocessed model-specific response data.
Raw any `json:"raw,omitempty"`
// Request is the ModelRequest struct used to trigger this response.
Request *ModelRequest `json:"request,omitempty"`
// Usage describes how many resources were used by this generation request.
Usage *GenerationUsage `json:"usage,omitempty"`
// contains filtered or unexported fields
}
A ModelResponse is a model's response to a ModelRequest.
func Generate ¶ added in v0.1.0
func Generate(ctx context.Context, r api.Registry, opts ...GenerateOption) (*ModelResponse, error)
Generate generates a model response based on the provided options.
func GenerateData ¶ added in v0.1.0
func GenerateData[Out any](ctx context.Context, r api.Registry, opts ...GenerateOption) (*Out, *ModelResponse, error)
GenerateData runs a generate request and returns strongly-typed output. If the response doesn't contain text output (e.g., contains tool requests or interrupts instead), the output will be nil and no error is returned. Check resp.Interrupts() or resp.ToolRequests() to handle these cases.
The output format is JSON with a schema inferred from Out; an explicit WithOutputSchema or WithOutputSchemaName overrides the schema while extraction into Out keeps working. Overriding the format itself with a non-JSON WithOutputFormat or WithOutputEnums breaks that extraction: the response text will not parse into Out.
func GenerateWithRequest ¶ added in v0.3.0
func GenerateWithRequest(ctx context.Context, r api.Registry, opts *GenerateActionOptions, mmws []ModelMiddleware, cb ModelStreamCallback) (*ModelResponse, error)
GenerateWithRequest is the central generation implementation for ai.Generate(), prompt.Execute(), and the GenerateAction direct call.
func (*ModelResponse) History ¶ added in v0.3.0
func (mr *ModelResponse) History() []*Message
History returns messages from the request combined with the response message to represent the conversation history. The result is always freshly allocated, so callers may retain or append to it without disturbing Request.Messages.
func (*ModelResponse) Interrupts ¶ added in v1.0.3
func (mr *ModelResponse) Interrupts() []*Part
Interrupts returns the interrupted tool request parts from the response.
func (*ModelResponse) Media ¶ added in v1.1.0
func (mr *ModelResponse) Media() string
Media returns the media content of the ModelResponse as a string.
Only the first media part is returned, and its content type is left behind, so a response carrying more than one image, or one whose content type is needed to render it, is better read with ModelResponse.MediaParts.
func (*ModelResponse) MediaParts ¶ added in v1.12.0
func (mr *ModelResponse) MediaParts() []*Part
MediaParts returns every media part of the ModelResponse, each carrying its content type alongside its data. It returns nil if the response has none.
A model that may answer with media often answers with media alone, so this pairs with ModelResponse.Text rather than replacing it: the two read disjoint halves of a response and either may come back empty.
func (*ModelResponse) Output ¶ added in v0.3.0
func (mr *ModelResponse) Output(v any) error
Output parses the structured output from the response and unmarshals it into v. If a format handler is set, it uses the handler's ParseOutput method. Otherwise, it falls back to parsing the response text as JSON.
func (*ModelResponse) Reasoning ¶ added in v0.6.0
func (mr *ModelResponse) Reasoning() string
Reasoning concatenates all reasoning parts present in the message
func (*ModelResponse) Text ¶ added in v0.3.0
func (mr *ModelResponse) Text() string
Text returns the contents of the first candidate in a ModelResponse as a string. It returns an empty string if there are no candidates or if the candidate has no message.
func (*ModelResponse) ToolRequests ¶ added in v0.5.0
func (mr *ModelResponse) ToolRequests() []*Part
ToolRequests returns the tool requests from the response.
type ModelResponseChunk ¶ added in v0.3.0
type ModelResponseChunk struct {
// Aggregated indicates whether the chunk includes all data from previous chunks.
// If false, the chunk is considered incremental.
Aggregated bool `json:"aggregated,omitempty"`
// Content is the chunk of message parts to stream right now.
Content []*Part `json:"content,omitempty"`
// Custom contains model-specific extra information attached to this chunk.
Custom any `json:"custom,omitempty"`
// Index of the message this chunk belongs to.
Index int `json:"index"`
// Role indicates the entity that generated this chunk.
Role Role `json:"role,omitempty"`
// contains filtered or unexported fields
}
A ModelResponseChunk is the portion of the ModelResponse that is passed to a streaming callback.
func (*ModelResponseChunk) Interrupts ¶ added in v1.10.0
func (c *ModelResponseChunk) Interrupts() []*Part
Interrupts returns the interrupted tool request parts from the chunk.
func (*ModelResponseChunk) Output ¶ added in v1.3.0
func (c *ModelResponseChunk) Output(v any) error
Output parses the chunk using the format handler and unmarshals the result into v. Returns an error if the format handler is not set or does not support parsing chunks.
func (*ModelResponseChunk) Reasoning ¶ added in v1.3.0
func (c *ModelResponseChunk) Reasoning() string
Reasoning returns the reasoning content of the ModelResponseChunk as a string. It returns an empty string if there is no Content in the response chunk.
func (*ModelResponseChunk) Text ¶ added in v0.3.0
func (c *ModelResponseChunk) Text() string
Text returns the text content of the ModelResponseChunk as a string, concatenating its text parts and skipping every other kind, as Message.Text does. It returns an empty string if the chunk has none. For the parsed structured output, use ModelResponseChunk.Output instead.
func (*ModelResponseChunk) ToolResponses ¶ added in v1.10.0
func (c *ModelResponseChunk) ToolResponses() []*Part
ToolResponses returns the tool response parts from the chunk. Use Part.IsPartial to distinguish streaming progress updates from final tool results.
type ModelStage ¶ added in v0.3.0
type ModelStage string
ModelStage indicates the development stage of a model.
const ( ModelStageFeatured ModelStage = "featured" ModelStageStable ModelStage = "stable" ModelStageUnstable ModelStage = "unstable" ModelStageLegacy ModelStage = "legacy" ModelStageDeprecated ModelStage = "deprecated" )
type ModelStreamCallback ¶ added in v0.3.0
type ModelStreamCallback = func(context.Context, *ModelResponseChunk) error
ModelStreamCallback is a stream callback of a ModelAction.
type ModelStreamValue ¶ added in v1.3.0
type ModelStreamValue = StreamValue[struct{}, *ModelResponseChunk]
ModelStreamValue is a stream value for a model response. Out is never set because the output is already available in the Response field.
type ModelSupports ¶ added in v0.3.0
type ModelSupports struct {
// Constrained indicates the level of constrained generation support (none, all, or no-tools).
Constrained ConstrainedSupport `json:"constrained,omitempty"`
// ContentType lists the content types the model supports for output.
ContentType []string `json:"contentType,omitempty"`
// Context indicates whether the model can natively support document-based context grounding.
Context bool `json:"context,omitempty"`
// LongRunning indicates whether the model supports long-running operations.
LongRunning bool `json:"longRunning,omitempty"`
// Media indicates whether the model can process media as part of the prompt (multimodal input).
Media bool `json:"media,omitempty"`
// Multiturn indicates whether the model can process historical messages passed with a prompt.
Multiturn bool `json:"multiturn,omitempty"`
// Output lists the types of data the model can generate.
Output []string `json:"output,omitempty"`
// SystemRole indicates whether the model can accept messages with role "system".
SystemRole bool `json:"systemRole,omitempty"`
// ToolChoice indicates whether the model supports controlling tool choice (e.g., forced tool calling).
ToolChoice bool `json:"toolChoice,omitempty"`
// Tools indicates whether the model can perform tool calls.
Tools bool `json:"tools,omitempty"`
}
ModelSupports describes the capabilities that a model supports.
type MultipartToolFunc ¶ added in v1.3.0
type MultipartToolFunc[In any] = func(ctx *ToolContext, input In) (*MultipartToolResponse, error)
MultipartToolFunc is the function type for multipart tool implementations. Unlike regular tools that return just an output value, multipart tools can return both an output value and additional content parts (like media).
type MultipartToolResponse ¶ added in v1.3.0
type MultipartToolResponse struct {
// Content holds additional message parts providing context or details.
Content []*Part `json:"content,omitempty"`
Metadata map[string]any `json:"metadata,omitempty"`
// Output contains the structured output data from the tool.
Output any `json:"output,omitempty"`
}
MultipartToolResponse represents a tool response with both structured output and content parts.
type Operation ¶ added in v1.3.0
type Operation struct {
// Action is the name of the action being performed by this operation.
Action string `json:"action"`
// Done indicates whether the operation has completed.
Done bool `json:"done"`
// Error contains error information if the operation failed.
Error *OperationError `json:"error,omitempty"`
// Id is the unique identifier for this operation.
Id string `json:"id"`
// Metadata contains additional information about the operation.
Metadata map[string]any `json:"metadata,omitempty"`
// Output contains the result of the operation if it has completed successfully.
Output any `json:"output,omitempty"`
}
Operation represents a long-running background task.
type OperationError ¶ added in v1.3.0
type OperationError struct {
// Message describes the error that occurred.
Message string `json:"message,omitempty"`
}
OperationError contains error information for a failed operation.
type OutputOption ¶ added in v0.3.0
type OutputOption interface {
PromptOption
GenerateOption
// contains filtered or unexported methods
}
OutputOption is an option for the output of a prompt or generate request. It applies only to DefinePrompt() and Generate().
func WithCustomConstrainedOutput ¶ added in v0.5.0
func WithCustomConstrainedOutput() OutputOption
WithCustomConstrainedOutput opts out of using the model's native constrained output generation.
By default, the system will use the model's native constrained output capabilities when available. When this option is set, or when the model doesn't support native constraints, the system will use custom implementation to guide the model toward producing properly formatted output.
func WithOutputEnums ¶ added in v1.3.0
func WithOutputEnums[T ~string](values ...T) OutputOption
WithOutputEnums sets the output format to enum and the schema based on the given values. Accepts any string-based type (e.g. type MyEnum string).
func WithOutputFormat ¶ added in v0.1.0
func WithOutputFormat(format string) OutputOption
WithOutputFormat sets the format of the output.
func WithOutputInstructions ¶ added in v0.5.0
func WithOutputInstructions(instructions string) OutputOption
WithOutputInstructions sets custom instructions for constraining output format in the prompt.
When WithOutputType is used without this option, default instructions will be automatically set. If you provide empty instructions, no instructions will be added to the prompt.
This will automatically set WithCustomConstrainedOutput.
func WithOutputType ¶ added in v0.3.0
func WithOutputType(output any) OutputOption
WithOutputType sets the output format to JSON and the schema derived from the given value.
type OutputSchemaOption ¶ added in v1.12.0
type OutputSchemaOption interface {
OutputOption
ToolOption
}
OutputSchemaOption is an OutputOption that provides an explicit output schema, inline or by name. Unlike the schema-inference and generation-steering options, it also applies to the tool constructors: a tool's output shape is decided by its function, so the only output options that make sense there are explicit schemas standing in for an Out type parameter of 'any'; a tool whose output a Go type can express should use the type parameter instead.
func WithOutputSchema ¶ added in v0.1.0
func WithOutputSchema(schema map[string]any) OutputSchemaOption
WithOutputSchema manually provides a schema map for the output.
func WithOutputSchemaName ¶ added in v1.3.0
func WithOutputSchemaName(name string) OutputSchemaOption
WithOutputSchemaName sets the schema name that will be resolved at execution time. Register the schema with github.com/firebase/genkit/go/genkit.DefineSchema.
type Part ¶
type Part struct {
Kind PartKind `json:"kind,omitempty"`
ContentType string `json:"contentType,omitempty"` // valid for kind==blob
Text string `json:"text,omitempty"` // valid for kind∈{text,blob}
ToolRequest *ToolRequest `json:"toolRequest,omitempty"` // valid for kind==partToolRequest
ToolResponse *ToolResponse `json:"toolResponse,omitempty"` // valid for kind==partToolResponse
Resource *ResourcePart `json:"resource,omitempty"` // valid for kind==partResource
Custom map[string]any `json:"custom,omitempty"` // valid for plugin-specific custom parts
Metadata map[string]any `json:"metadata,omitempty"` // valid for all kinds
}
A Part is one part of a Document. This may be plain text or it may be a URL (possibly a "data:" URL with embedded data).
func NewCustomPart ¶ added in v0.5.1
NewCustomPart returns a Part containing custom plugin-specific data.
func NewDataPart ¶
NewDataPart returns a Part containing raw string data.
Example ¶
This example demonstrates creating a data part for raw string content.
package main
import (
"fmt"
"github.com/firebase/genkit/go/ai"
)
func main() {
// Create a data part with raw string content
part := ai.NewDataPart(`{"name": "Alice", "age": 30}`)
fmt.Println("Is data part:", part.IsData())
fmt.Println("Content:", part.Text)
}
Output: Is data part: true Content: {"name": "Alice", "age": 30}
func NewMediaPart ¶
NewMediaPart returns a Part containing structured data described by the given mimeType.
Example ¶
This example demonstrates creating a media part for images or other media.
package main
import (
"fmt"
"github.com/firebase/genkit/go/ai"
)
func main() {
// Create a media part with base64-encoded image data
// In practice, you would encode actual image bytes
imageData := "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJ..."
part := ai.NewMediaPart("image/png", imageData)
fmt.Println("Is media:", part.IsMedia())
fmt.Println("Content type:", part.ContentType)
}
Output: Is media: true Content type: image/png
func NewPartialToolResponsePart ¶ added in v1.10.0
func NewPartialToolResponsePart(r *ToolResponse) *Part
NewPartialToolResponsePart returns a Part containing a partial tool response. Partial tool responses are streamed during tool execution for client-side display (e.g., progress indicators) and are not included in conversation history.
func NewReasoningPart ¶ added in v0.6.0
NewReasoningPart returns a Part containing reasoning text
func NewResourcePart ¶ added in v0.7.0
NewResourcePart returns a Part containing a resource reference.
func NewResponseForToolRequest ¶ added in v0.6.0
NewResponseForToolRequest returns a Part containing the results of executing the tool request part.
func NewTextPart ¶
NewTextPart returns a Part containing text.
Example ¶
This example demonstrates creating different types of message parts.
package main
import (
"fmt"
"github.com/firebase/genkit/go/ai"
)
func main() {
// Create a text part
part := ai.NewTextPart("Hello, world!")
fmt.Println(part.Text)
}
Output: Hello, world!
func NewToolRequestPart ¶
func NewToolRequestPart(r *ToolRequest) *Part
NewToolRequestPart returns a Part containing a request from the model to the client to run a Tool. (Only genkit plugins should need to use this function.)
func NewToolResponsePart ¶
func NewToolResponsePart(r *ToolResponse) *Part
NewToolResponsePart returns a Part containing the results of applying a Tool that the model requested.
func (*Part) Clone ¶ added in v1.7.0
Clone returns a shallow copy of the Part with its own Metadata and Custom maps. Callers can add or remove map keys without mutating the original.
func (*Part) IsCustom ¶ added in v0.5.1
IsCustom reports whether the Part contains custom plugin-specific data.
func (*Part) IsInterrupt ¶ added in v1.0.3
IsInterrupt reports whether the Part contains a tool request that was interrupted.
func (*Part) IsPartial ¶ added in v1.10.0
IsPartial reports whether the Part contains a partial tool response streamed during tool execution (e.g., a progress update).
func (*Part) IsReasoning ¶ added in v0.6.0
IsReasoning reports whether the Part contains a reasoning text
func (*Part) IsResource ¶ added in v0.7.0
IsResource reports whether the Part contains a resource reference.
func (*Part) IsToolRequest ¶
IsToolRequest reports whether the Part contains a request to run a tool.
func (*Part) IsToolResponse ¶
IsToolResponse reports whether the Part contains the result of running a tool.
func (Part) JSONSchemaAlias ¶
JSONSchemaAlias tells the JSON schema reflection code to use a different type for the schema for this type. This is needed because the JSON marshaling of Part uses a schema that matches the TypeScript code, rather than the natural JSON marshaling. This matters because the current JSON validation code works by marshaling the JSON.
func (*Part) MarshalJSON ¶
MarshalJSON is called by the JSON marshaler to write out a Part.
func (*Part) UnmarshalJSON ¶
UnmarshalJSON is called by the JSON unmarshaler to read a Part.
type PartKind ¶
type PartKind int8
PartKind is what a Part carries: text, media, a tool request, and so on.
type PartsFn ¶ added in v1.12.0
PartsFn is a function that generates message content from a prompt's input.
The input is untyped. WithPromptPartsFn and WithSystemPartsFn take a function with a concrete input type and convert for you.
type PathMetadata ¶ added in v0.3.0
type PathMetadata struct {
// Error contains error information if the path failed.
Error string `json:"error,omitempty"`
// Latency is the execution time for this path in milliseconds.
Latency float64 `json:"latency,omitempty"`
// Path is the identifier for this execution path.
Path string `json:"path,omitempty"`
// Status indicates the outcome of this path.
Status string `json:"status,omitempty"`
}
PathMetadata contains metadata about a single execution path in a trace.
type Prompt ¶
type Prompt interface {
// Name returns the name of the prompt.
Name() string
// Execute executes the prompt with the given options and returns a [ModelResponse].
//
// # Options
//
// Input:
//
// - [WithInput]: Supply the prompt's input, overriding the default from [WithInputType]
//
// Conversation:
//
// - [WithMessages]: Supply the conversation this execution continues
// - [WithMessagesFn]: As above, computed from the input
//
// A prompt that declares a conversation of its own decides where these go,
// with {{history}} or [HistoryFromContext]. One that declares none uses
// them directly, between the system message and the user prompt.
//
// Overrides, each replacing what the prompt was defined with:
//
// - [WithModel], [WithModelName]: Call a different model
// - [WithConfig]: Replace the generation config
// - [WithDocs], [WithTextDocs]: Replace the context documents, skipping any [WithDocsFn]
// - [WithTools], [WithToolChoice], [WithMaxTurns], [WithReturnToolRequests]: Change tool behavior
// - [WithMiddleware], [WithUse]: Add middleware for this execution
// - [WithStreaming]: Receive streamed chunks
Execute(ctx context.Context, opts ...PromptExecuteOption) (*ModelResponse, error)
// ExecuteStream executes the prompt with streaming and returns an iterator.
// It accepts the same options as Execute.
ExecuteStream(ctx context.Context, opts ...PromptExecuteOption) iter.Seq2[*ModelStreamValue, error]
// Render renders the prompt with the given input and returns a [GenerateActionOptions] to be used with [GenerateWithRequest].
Render(ctx context.Context, input any) (*GenerateActionOptions, error)
}
Prompt is the interface for a prompt that can be executed and rendered.
func DefinePrompt ¶
func DefinePrompt(r api.Registry, name string, opts ...PromptOption) Prompt
DefinePrompt creates a new Prompt and registers it.
func LoadPrompt ¶ added in v0.3.0
LoadPrompt loads a single prompt from a directory on the local filesystem into the registry.
func LoadPromptFromFS ¶ added in v1.3.0
LoadPromptFromFS loads a single prompt from a filesystem into the registry. The fsys parameter should be an fs.FS implementation (e.g., embed.FS or os.DirFS). The dir parameter specifies the directory within the filesystem where the prompt is located.
func LoadPromptFromSource ¶ added in v1.3.0
LoadPromptFromSource loads a prompt from raw .prompt file content. The source parameter should contain the complete .prompt file text (frontmatter + template). The name parameter is the prompt name (may include variant suffix like "myPrompt.variant").
func LookupPrompt ¶
LookupPrompt looks up a Prompt registered by DefinePrompt. It returns nil if the prompt was not defined.
type PromptExecuteOption ¶ added in v0.5.0
type PromptExecuteOption interface {
// contains filtered or unexported methods
}
PromptExecuteOption is an option for executing a prompt. It applies only to [prompt.Execute].
func WithInput ¶ added in v0.3.0
func WithInput(input any) PromptExecuteOption
WithInput sets the input for the prompt request. Input must conform to the prompt's input schema and can either be a map[string]any or a struct of the same api. Repeating this option takes the last input set. APIs that take the input as a typed argument (DataPrompt.Execute, DataPrompt.ExecuteStream) apply that argument after these options, so the typed argument wins.
type PromptFn ¶ added in v0.5.0
PromptFn is a function that generates a prompt from a prompt's input.
The input is untyped. WithPromptFn and WithSystemFn take a function with a concrete input type and convert for you.
type PromptOption ¶ added in v0.3.0
type PromptOption interface {
// contains filtered or unexported methods
}
PromptOption is an option for defining a prompt. It applies only to DefinePrompt().
func WithDescription ¶ added in v0.3.0
func WithDescription(description string) PromptOption
WithDescription sets the description of the prompt. Repeating this option takes the last description set.
func WithDocsFn ¶ added in v1.12.0
WithDocsFn sets the function that selects the context documents for a prompt, such as by querying a retriever. It applies only to DefinePrompt, since only a prompt has input to give it.
fn receives the prompt's input converted to In.
Documents accumulate: repeating this option, or combining it with WithDocs or WithTextDocs, adds to the set rather than replacing it. The fixed documents resolve first, then the computed ones, each in call order.
func WithMessagesTemplate ¶ added in v1.12.0
func WithMessagesTemplate(text string, args ...any) PromptOption
WithMessagesTemplate sets the conversation to a dotprompt template. Optional args are applied to text with fmt.Sprintf.
This is the multi-turn form: each {{role "..."}} block starts a new message, so one template can express a system preamble, few-shot examples, and the user turn. It is what a .prompt file's body compiles to.
Messages passed to Prompt.Execute reach the template at {{history}}, or, without an explicit {{history}}, are inserted before its final user message.
Unlike WithMessages, the text here is compiled, so literal braces in it are template syntax. Content from a user or a remote source belongs in WithMessages or WithMessagesFn, which never compile it.
The template is the whole conversation, so nothing else can contribute to it: passing this in the same DefinePrompt call as WithMessages or WithMessagesFn panics. Write those messages as {{role}} blocks instead. Repeating this option alone fills one slot, so the last one set wins.
Compiling needs a prompt, so this is a PromptOption: passing it to Generate or Prompt.Execute does not compile. Use WithMessages there.
func WithMetadata ¶ added in v0.3.0
func WithMetadata(metadata map[string]any) PromptOption
WithMetadata sets arbitrary metadata for the prompt. Repeating this option replaces the metadata rather than merging it.
type PromptingOption ¶ added in v0.3.0
type PromptingOption interface {
PromptOption
GenerateOption
// contains filtered or unexported methods
}
PromptingOption is an option for the system and user prompts of a prompt or generate request. It applies only to DefinePrompt() and Generate().
func WithPrompt ¶ added in v0.5.0
func WithPrompt(text string, args ...any) PromptingOption
WithPrompt sets the user prompt message. The user prompt is always the last message in the list.
With DefinePrompt, the text is compiled as a dotprompt template against the prompt's input, so it may reference input fields such as {{name}}. args, if given, are applied with fmt.Sprintf first.
A {{role}} marker in the text is an error: this slot is one user message. WithMessagesTemplate is where turns with their own roles belong.
func WithPromptFn ¶ added in v0.3.0
WithPromptFn sets the function that generates the user prompt message. The user prompt is always the last message in the list.
fn receives the prompt's input converted to In, or the zero value of In when there is none, as at Generate. Its string is used verbatim, never compiled as a template, so it may safely hold user content and literal braces. Use WithPromptPartsFn to return non-text content.
It shares one slot with WithPrompt, WithPromptParts, and WithPromptPartsFn: the last one set wins.
func WithPromptParts ¶ added in v1.12.0
func WithPromptParts(parts ...*Part) PromptingOption
WithPromptParts sets the content of the user prompt message. The user prompt is always the last message in the list.
It is the multi-part form of WithPrompt, for prompts that mix text with media or other non-text parts. The parts are used verbatim, never compiled as a template. Use WithPromptPartsFn when the content depends on the prompt's input.
genkit.Generate(ctx, g,
ai.WithModelName("googleai/gemini-flash-latest"),
ai.WithPromptParts(
ai.NewTextPart("What is in this image?"),
ai.NewMediaPart("image/png", imageURL),
),
)
It shares one slot with WithPrompt, WithPromptFn, and WithPromptPartsFn: the last one set wins.
func WithPromptPartsFn ¶ added in v1.12.0
WithPromptPartsFn sets the function that generates the content of the user prompt message. The user prompt is always the last message in the list.
It is the multi-part form of WithPromptFn. The returned parts are used verbatim.
It shares one slot with WithPrompt, WithPromptParts, and WithPromptFn: the last one set wins.
func WithSystem ¶ added in v0.5.0
func WithSystem(text string, args ...any) PromptingOption
WithSystem sets the system prompt message. The system prompt is always the first message in the list.
With DefinePrompt, the text is compiled as a dotprompt template against the prompt's input, so it may reference input fields such as {{name}}. args, if given, are applied with fmt.Sprintf first.
A {{role}} marker in the text is an error: this slot is one system message. WithMessagesTemplate is where turns with their own roles belong.
func WithSystemFn ¶ added in v0.3.0
WithSystemFn sets the function that generates the system prompt message. The system prompt is always the first message in the list.
fn receives the prompt's input converted to In, or the zero value of In when there is none, as at Generate. Its string is used verbatim, never compiled as a template, so it may safely hold user content and literal braces. Use WithSystemPartsFn to return non-text content.
It shares one slot with WithSystem, WithSystemParts, and WithSystemPartsFn: the last one set wins.
func WithSystemParts ¶ added in v1.12.0
func WithSystemParts(parts ...*Part) PromptingOption
WithSystemParts sets the content of the system prompt message. The system prompt is always the first message in the list.
It is the multi-part form of WithSystem, for instructions that mix text with media or other non-text parts. The parts are used verbatim, never compiled as a template. Use WithSystemPartsFn when the content depends on the prompt's input.
It shares one slot with WithSystem, WithSystemFn, and WithSystemPartsFn: the last one set wins.
func WithSystemPartsFn ¶ added in v1.12.0
WithSystemPartsFn sets the function that generates the content of the system prompt message. The system prompt is always the first message in the list.
It is the multi-part form of WithSystemFn. The returned parts are used verbatim.
It shares one slot with WithSystem, WithSystemParts, and WithSystemFn: the last one set wins.
type RankedDocumentData ¶ added in v0.3.0
type RankedDocumentData struct {
// Content holds the document's parts (text and media).
Content []*Part `json:"content,omitempty"`
// Metadata contains the reranking score and other arbitrary key-value data.
Metadata *RankedDocumentMetadata `json:"metadata,omitempty"`
}
RankedDocumentData represents a document with a relevance score from reranking.
type RankedDocumentMetadata ¶ added in v0.3.0
type RankedDocumentMetadata struct {
// Score is the relevance score assigned by the reranker.
Score float64 `json:"score,omitempty"`
}
RankedDocumentMetadata contains the relevance score and other metadata for a reranked document.
type RerankerRequest ¶ added in v0.3.0
type RerankerRequest struct {
// Documents is the array of documents to rerank.
Documents []*Document `json:"documents,omitempty"`
// Options contains reranker-specific configuration parameters.
Options any `json:"options,omitempty"`
// Query is the document to use for reranking.
Query *Document `json:"query,omitempty"`
}
RerankerRequest represents a request to rerank documents based on relevance.
type RerankerResponse ¶ added in v0.3.0
type RerankerResponse struct {
// Documents is the array of reranked documents with scores.
Documents []*RankedDocumentData `json:"documents,omitempty"`
}
RerankerResponse contains the reranked documents with relevance scores.
type Resource ¶ added in v0.7.0
type Resource interface {
// Name returns the name of the resource.
Name() string
// Matches reports whether this resource matches the given URI.
Matches(uri string) bool
// ExtractVariables extracts variables from a URI using this resource's template.
ExtractVariables(uri string) (map[string]string, error)
// Execute runs the resource with the given input.
Execute(ctx context.Context, input *ResourceInput) (*ResourceOutput, error)
// Register registers the resource with the given registry.
Register(r api.Registry)
}
Resource represents an instance of a resource.
func LookupResource ¶ added in v0.7.0
LookupResource looks up the resource in the registry by provided name and returns it.
func NewResource ¶ added in v0.7.0
func NewResource(name string, opts *ResourceOptions, fn ResourceFunc) Resource
NewResource creates a resource but does not register it in the registry. It can be registered later via the Register method.
type ResourceFunc ¶ added in v0.7.0
type ResourceFunc = func(context.Context, *ResourceInput) (*ResourceOutput, error)
ResourceFunc is a function that loads content for a resource.
type ResourceInput ¶ added in v0.7.0
type ResourceInput struct {
URI string `json:"uri"` // The resource URI
Variables map[string]string `json:"variables"` // Extracted variables from URI template matching
}
ResourceInput represents the input to a resource function.
type ResourceOptions ¶ added in v0.7.0
type ResourceOptions struct {
// URI is the resource's static URI. Mutually exclusive with Template.
URI string
// Template is a URI template the resource matches. Mutually exclusive with URI.
Template string
// Description is the resource's human-readable description.
Description string
// Metadata is arbitrary key-value data attached to the action descriptor.
Metadata map[string]any
}
ResourceOptions configures a resource definition.
type ResourceOutput ¶ added in v0.7.0
type ResourceOutput struct {
Content []*Part `json:"content"` // The content parts returned by the resource
}
ResourceOutput represents the output from a resource function.
type ResourcePart ¶ added in v0.7.0
type ResourcePart struct {
// Uri is the URI of the external resource.
Uri string `json:"uri,omitempty"`
}
type RespondOptions ¶ added in v0.6.0
type RespondOptions struct {
// Metadata is additional metadata to include in the response.
Metadata map[string]any
}
RespondOptions provides configuration options for responding to a tool request.
type RespondWithOption ¶ added in v1.4.0
type RespondWithOption[Out any] interface { // contains filtered or unexported methods }
RespondWithOption is a functional option for ToolAction.RespondWith.
func WithResponseMetadata ¶ added in v1.4.0
func WithResponseMetadata[Out any](meta map[string]any) RespondWithOption[Out]
WithResponseMetadata sets metadata for the response. Repeating this option replaces the metadata rather than merging it.
type RestartOptions ¶ added in v0.6.0
type RestartOptions struct {
// ReplaceInput allows replacing the existing input arguments to the tool with different ones,
// for example if the user revised an action before confirming. When input is replaced,
// the existing tool request will be amended in the message history.
ReplaceInput any
// ResumedMetadata is the metadata you want to provide to the tool to aide in reprocessing.
// Defaults to true if none is supplied.
ResumedMetadata any
}
RestartOptions provides configuration options for restarting a tool.
type RestartWithOption ¶ added in v1.4.0
type RestartWithOption[In any] interface { // contains filtered or unexported methods }
RestartWithOption is a functional option for ToolAction.RestartWith.
func WithNewInput ¶ added in v1.4.0
func WithNewInput[In any](input In) RestartWithOption[In]
WithNewInput sets a new input value to replace the original tool request input. Repeating this option takes the last input set.
func WithResumedMetadata ¶ added in v1.4.0
func WithResumedMetadata[In any](meta map[string]any) RestartWithOption[In]
WithResumedMetadata sets metadata to pass to the resumed tool execution. The metadata will be available in the tool's ToolContext.Resumed field. Repeating this option replaces the metadata rather than merging it.
type Retriever ¶
type Retriever interface {
// Name returns the name of the retriever.
Name() string
// Retrieve retrieves the documents.
Retrieve(ctx context.Context, req *RetrieverRequest) (*RetrieverResponse, error)
// Register registers the retriever with the given registry.
Register(r api.Registry)
}
Retriever represents a document retriever. It is the type to accept as an argument and to look up by name; implementations are created with NewRetrieverAction, or [genkit.DefineRetrieverAction] in an application.
func LookupRetriever ¶
LookupRetriever looks up a registered Retriever by name. It will try to resolve the retriever dynamically if the retriever is not found. It returns nil if the retriever was not resolved.
func NewRetriever
deprecated
added in
v0.7.0
func NewRetriever(name string, opts *RetrieverOptions, fn RetrieverFunc) Retriever
NewRetriever creates a new Retriever.
Deprecated: Use NewRetrieverAction, which passes the request's options to fn as a typed value instead of leaving them type-erased on the request.
type RetrieverAction ¶ added in v1.12.0
type RetrieverAction struct {
// contains filtered or unexported fields
}
RetrieverAction is a retriever backed by a registry action. It is the concrete type returned by NewRetrieverAction; pass it to WithRetriever to retrieve with it, or return it from a plugin's Init for the framework to register.
It implements Retriever and api.Action, so it can be passed anywhere either is accepted. It also promotes core.Action.Run, the typed equivalent of RetrieverAction.Retrieve.
func NewRetrieverAction ¶ added in v1.12.0
func NewRetrieverAction[Config any]( name string, opts *RetrieverOptions, fn RetrieverActionFunc[Config], ) *RetrieverAction
NewRetrieverAction creates an unregistered RetrieverAction: return it from a plugin's Init for the framework to register, or call RetrieverAction.Register directly. Applications should define retrievers with [genkit.DefineRetrieverAction].
Config is the retriever's typed configuration; it is usually inferred from fn's signature. The framework deserializes the request's raw options into Config before calling fn: the exact Config type (or a pointer to it) and map[string]any (from the Dev UI and other JSON callers) are accepted, and mismatched types are rejected. The request's RetrieverRequest.Options is normalized to the converted value, so it always matches the typed parameter. The config's JSON schema is inferred from Config unless RetrieverOptions.ConfigSchema overrides it.
func (*RetrieverAction) Desc ¶ added in v1.12.0
func (r *RetrieverAction) Desc() api.ActionDesc
Desc returns the retriever's action descriptor: its name, schemas, and metadata.
func (*RetrieverAction) Name ¶ added in v1.12.0
func (r *RetrieverAction) Name() string
Name returns the registry name of the retriever.
func (*RetrieverAction) Register ¶ added in v1.12.0
func (r *RetrieverAction) Register(reg api.Registry)
Register registers the retriever with reg, making it available to lookups and to the Dev UI. A plugin that returns the retriever from its Init does not need to call this.
func (*RetrieverAction) Retrieve ¶ added in v1.12.0
func (r *RetrieverAction) Retrieve(ctx context.Context, req *RetrieverRequest) (*RetrieverResponse, error)
Retrieve runs the given Retriever.
func (*RetrieverAction) RunJSON ¶ added in v1.12.0
func (r *RetrieverAction) RunJSON(ctx context.Context, input json.RawMessage, cb core.StreamCallback[json.RawMessage]) (json.RawMessage, error)
RunJSON runs the retriever on a JSON-encoded RetrieverRequest and returns a JSON-encoded RetrieverResponse. The framework uses it to serve reflection and registry-driven calls; prefer RetrieverAction.Retrieve.
func (*RetrieverAction) RunJSONWithTelemetry ¶ added in v1.12.0
func (r *RetrieverAction) RunJSONWithTelemetry(ctx context.Context, input json.RawMessage, cb core.StreamCallback[json.RawMessage]) (*api.ActionRunResult[json.RawMessage], error)
RunJSONWithTelemetry is RetrieverAction.RunJSON with the run's telemetry returned alongside the output.
type RetrieverActionFunc ¶ added in v1.12.0
type RetrieverActionFunc[Config any] = func(context.Context, *RetrieverRequest, Config) (*RetrieverResponse, error)
RetrieverActionFunc is a RetrieverFunc that additionally receives the request's typed Config: the framework deserializes the request's raw options into it before calling the function (see NewRetrieverAction).
type RetrieverArg ¶ added in v0.7.0
type RetrieverArg interface {
Name() string
}
RetrieverArg is the interface for retriever arguments. It can either be the retriever action itself or a reference to be looked up.
type RetrieverFunc ¶ added in v0.5.0
type RetrieverFunc = func(context.Context, *RetrieverRequest) (*RetrieverResponse, error)
RetrieverFunc is the function type for retriever implementations.
type RetrieverOption ¶ added in v0.5.0
type RetrieverOption interface {
// contains filtered or unexported methods
}
RetrieverOption is an option for configuring a retriever request. It applies only to Retriever.Retrieve.
func WithRetriever ¶ added in v0.7.0
func WithRetriever(retriever RetrieverArg) RetrieverOption
WithRetriever sets either a Retriever or a RetrieverRef that may contain a config. Passing WithConfig will take precedence over the config in WithRetriever.
func WithRetrieverName ¶ added in v0.7.0
func WithRetrieverName(name string) RetrieverOption
WithRetrieverName sets the retriever name to call for document retrieval. The retriever name will be resolved to a Retriever and may error if the reference is invalid.
type RetrieverOptions ¶ added in v0.7.0
type RetrieverOptions struct {
// ConfigSchema is the JSON schema for the retriever's config.
ConfigSchema map[string]any `json:"configSchema,omitempty"`
// Label is a user-friendly name for the retriever.
Label string `json:"label,omitempty"`
// Supports defines the capabilities of the retriever, such as media support.
Supports *RetrieverSupports `json:"supports,omitempty"`
// Metadata is arbitrary key-value data attached to the action descriptor.
Metadata map[string]any `json:"-"`
}
RetrieverOptions represents the configuration options for a retriever.
type RetrieverRef ¶ added in v0.7.0
type RetrieverRef struct {
// contains filtered or unexported fields
}
RetrieverRef is a struct to hold retriever name and configuration.
func NewRetrieverRef ¶ added in v0.7.0
func NewRetrieverRef(name string, config any) RetrieverRef
NewRetrieverRef creates a new RetrieverRef with the given name and configuration.
func (RetrieverRef) Config ¶ added in v0.7.0
func (r RetrieverRef) Config() any
Config returns the configuration to use by default for this retriever.
func (RetrieverRef) Name ¶ added in v0.7.0
func (r RetrieverRef) Name() string
Name returns the name of the retriever.
type RetrieverRequest ¶
type RetrieverRequest struct {
// Options contains retriever-specific configuration parameters.
Options any `json:"options,omitempty"`
// Query is the document to use for retrieval.
Query *Document `json:"query,omitempty"`
}
RetrieverRequest represents a request to retrieve relevant documents.
type RetrieverResponse ¶
type RetrieverResponse struct {
// Documents is the array of retrieved documents.
Documents []*Document `json:"documents,omitempty"`
}
RetrieverResponse contains the retrieved documents from a retriever request.
func Retrieve ¶ added in v0.1.0
func Retrieve(ctx context.Context, r api.Registry, opts ...RetrieverOption) (*RetrieverResponse, error)
Retrieve calls the retriever with the provided options.
type RetrieverSupports ¶ added in v0.7.0
type RetrieverSupports struct {
// Media indicates whether the retriever supports media content.
Media bool `json:"media,omitempty"`
}
RetrieverSupports defines the supported capabilities of the retriever.
type Role ¶
type Role string
Role indicates which entity is responsible for the content of a message.
const ( // RoleSystem indicates this message is user-independent context. RoleSystem Role = "system" // RoleUser indicates this message was generated by the client. RoleUser Role = "user" // RoleModel indicates this message was generated by the model during a previous interaction. RoleModel Role = "model" // RoleTool indicates this message was generated by a local tool, likely triggered by a request // from the model in one of its previous responses. RoleTool Role = "tool" )
type Score ¶ added in v0.3.0
type Score struct {
Id string `json:"id,omitempty"`
Score any `json:"score,omitempty"`
Status string `json:"status,omitempty" jsonschema:"enum=UNKNOWN,enum=FAIL,enum=PASS"`
Error string `json:"error,omitempty"`
Details map[string]any `json:"details,omitempty"`
}
Score is the evaluation score that represents the result of an evaluator. This struct includes information such as the score (numeric, string or other types), the reasoning provided for this score (if any), the score status (if any) and other details.
type ScoreDetails ¶ added in v0.3.0
type ScoreDetails struct {
// Reasoning explains the rationale behind the score.
Reasoning string `json:"reasoning,omitempty"`
}
ScoreDetails provides additional context and explanation for an evaluation score.
type ScoreStatus ¶ added in v0.3.0
type ScoreStatus int
ScoreStatus is an enum used to indicate if a Score has passed or failed. This drives additional features in tooling / the Dev UI.
const ( ScoreStatusUnknown ScoreStatus = iota ScoreStatusFail ScoreStatusPass )
func (ScoreStatus) String ¶ added in v0.3.0
func (ss ScoreStatus) String() string
String returns the wire name of the score status ("PASS", "FAIL", or "UNKNOWN").
type StartModelOpFunc ¶ added in v1.3.0
type StartModelOpFunc = func(ctx context.Context, req *ModelRequest) (*ModelOperation, error)
StartModelOpFunc starts a background model operation.
type StreamValue ¶ added in v1.3.0
type StreamValue[Out, Stream any] struct { Done bool Chunk Stream // valid if Done is false Output Out // valid if Done is true Response *ModelResponse // valid if Done is true }
StreamValue is either a streamed chunk or the final response of a generate request.
type StreamingFormatHandler ¶ added in v1.3.0
type StreamingFormatHandler interface {
// ParseOutput parses the final output and returns parsed output.
ParseOutput(message *Message) (any, error)
// ParseChunk processes a streaming chunk and returns parsed output.
// The handler maintains its own internal state. When the chunk's index changes, the state is reset for the new turn.
// Returns parsed output, or nil if nothing can be parsed yet.
ParseChunk(chunk *ModelResponseChunk) (any, error)
}
StreamingFormatHandler is a handler for formatting messages that supports streaming. This interface must be implemented to be able to use ModelResponse.Output and ModelResponseChunk.Output.
type Tool ¶
type Tool interface {
// Name returns the name of the tool.
Name() string
// Definition returns the definition for this tool to be passed to models.
Definition() *ToolDefinition
// RunRaw runs this tool using the provided raw input and returns just the output.
RunRaw(ctx context.Context, input any) (any, error)
// RunRawMultipart runs this tool and returns the full [MultipartToolResponse].
RunRawMultipart(ctx context.Context, input any) (*MultipartToolResponse, error)
// Respond constructs a [Part] with a [ToolResponse] for a given interrupted tool request.
Respond(toolReq *Part, outputData any, opts *RespondOptions) *Part
// Restart constructs a [Part] with a new [ToolRequest] to re-trigger a tool,
// potentially with new input and metadata.
Restart(toolReq *Part, opts *RestartOptions) *Part
// Register registers the tool with the given registry.
Register(r api.Registry)
}
Tool represents a tool that can be called by a model. It is the type to accept as an argument and to look up by name; implementations are created with NewTool or NewMultipartTool, or their [genkit.DefineTool] and [genkit.DefineMultipartTool] counterparts in an application.
func LookupTool ¶ added in v0.1.0
LookupTool looks up the tool in the registry by provided name and returns it. It checks for "tool.v2" first, then falls back to "tool" for legacy compatibility. Since the types are not known at lookup time, it returns a type-erased tool.
type ToolAction ¶ added in v1.12.0
type ToolAction[In, Out any] struct { // contains filtered or unexported fields }
ToolAction is a tool backed by a registry action. It is the concrete type returned by NewTool and NewMultipartTool. Internally, all tools use the v2 format (returning MultipartToolResponse). For regular tools, RunRaw unwraps the Output field for backward compatibility.
It implements Tool and api.Action, so it can be passed anywhere either is accepted, including the action slice a plugin returns from Init. Unlike the other primitives it holds its action in a named field rather than embedding it, so its documented methods are its whole surface.
func NewMultipartTool ¶ added in v1.3.0
func NewMultipartTool[In any](name, description string, fn MultipartToolFunc[In], opts ...ToolOption) *ToolAction[In, *MultipartToolResponse]
NewMultipartTool creates a new multipart ToolAction. It can be passed directly to Generate. Multipart tools can return both output data and additional content parts (like media). Use WithInputSchema to provide a custom JSON schema instead of inferring from the type parameter. Use WithOutputSchema or WithOutputSchemaName to advertise the logical output the tool produces (the envelope's output field); the wire format stays the multipart response envelope.
func NewTool ¶ added in v0.6.0
func NewTool[In, Out any](name, description string, fn ToolFunc[In, Out], opts ...ToolOption) *ToolAction[In, Out]
NewTool creates a new ToolAction. It can be passed directly to Generate. Use WithInputSchema or WithOutputSchema to provide custom JSON schemas instead of inferring them from the type parameters.
func NewToolWithInputSchema
deprecated
added in
v1.0.0
func NewToolWithInputSchema[Out any](name, description string, inputSchema map[string]any, fn ToolFunc[any, Out]) *ToolAction[any, Out]
NewToolWithInputSchema creates a new ToolAction with a custom input schema. It can be passed directly to Generate.
Deprecated: Use NewTool with WithInputSchema instead.
func (*ToolAction[In, Out]) Definition ¶ added in v1.12.0
func (t *ToolAction[In, Out]) Definition() *ToolDefinition
Definition returns ToolDefinition for for this tool.
func (*ToolAction[In, Out]) Desc ¶ added in v1.12.0
func (t *ToolAction[In, Out]) Desc() api.ActionDesc
Desc returns the tool's action descriptor: its name, schemas, and metadata.
func (*ToolAction[In, Out]) IsMultipart ¶ added in v1.12.0
func (t *ToolAction[In, Out]) IsMultipart() bool
IsMultipart returns true if the tool is a multipart tool (tool.v2 only).
func (*ToolAction[In, Out]) Name ¶ added in v1.12.0
func (t *ToolAction[In, Out]) Name() string
Name returns the name of the tool.
func (*ToolAction[In, Out]) Register ¶ added in v1.12.0
func (t *ToolAction[In, Out]) Register(r api.Registry)
Register registers the tool with the given registry.
func (*ToolAction[In, Out]) Respond
deprecated
added in
v1.12.0
func (t *ToolAction[In, Out]) Respond(toolReq *Part, output any, opts *RespondOptions) *Part
Respond creates a part for WithToolResponses to provide a resolved response for an interrupted tool call. Returns nil if the part is not a tool request.
Deprecated: Use ToolAction.RespondWith instead for strongly-typed options.
func (*ToolAction[In, Out]) RespondWith ¶ added in v1.12.0
func (t *ToolAction[In, Out]) RespondWith(toolReq *Part, output Out, opts ...RespondWithOption[Out]) (*Part, error)
RespondWith creates a part for WithToolResponses to provide a resolved response for an interrupted tool call.
Example:
part, err := myTool.RespondWith(toolReq, output, WithResponseMetadata[MyOutput](meta))
func (*ToolAction[In, Out]) Restart
deprecated
added in
v1.12.0
func (t *ToolAction[In, Out]) Restart(p *Part, opts *RestartOptions) *Part
Restart creates a part for WithToolRestarts to re-execute an interrupted tool call with additional context. Returns nil if the part is not a tool request.
Deprecated: Use ToolAction.RestartWith instead for strongly-typed options.
func (*ToolAction[In, Out]) RestartWith ¶ added in v1.12.0
func (t *ToolAction[In, Out]) RestartWith(toolReq *Part, opts ...RestartWithOption[In]) (*Part, error)
RestartWith creates a part for WithToolRestarts to re-execute an interrupted tool call with additional context.
Example:
part, err := myTool.RestartWith(toolReq, WithNewInput(newInput), WithResumedMetadata[MyInput](meta))
func (*ToolAction[In, Out]) RunJSON ¶ added in v1.12.0
func (t *ToolAction[In, Out]) RunJSON(ctx context.Context, input json.RawMessage, cb core.StreamCallback[json.RawMessage]) (json.RawMessage, error)
RunJSON runs the tool on JSON-encoded input and returns the JSON-encoded multipart response envelope, which is what the registry serves for this tool. Prefer ToolAction.RunRaw, which unwraps the envelope's output for a regular tool.
func (*ToolAction[In, Out]) RunJSONWithTelemetry ¶ added in v1.12.0
func (t *ToolAction[In, Out]) RunJSONWithTelemetry(ctx context.Context, input json.RawMessage, cb core.StreamCallback[json.RawMessage]) (*api.ActionRunResult[json.RawMessage], error)
RunJSONWithTelemetry is ToolAction.RunJSON with the run's telemetry returned alongside the output.
func (*ToolAction[In, Out]) RunRaw ¶ added in v1.12.0
RunRaw runs this tool using the provided raw map format data (JSON parsed as map[string]any).
func (*ToolAction[In, Out]) RunRawMultipart ¶ added in v1.12.0
func (t *ToolAction[In, Out]) RunRawMultipart(ctx context.Context, input any) (*MultipartToolResponse, error)
RunRawMultipart runs this tool using the provided raw map format data (JSON parsed as map[string]any). It returns the full multipart response.
type ToolChoice ¶ added in v0.3.0
type ToolChoice string
ToolChoice controls how the model uses tools.
const ( ToolChoiceAuto ToolChoice = "auto" ToolChoiceRequired ToolChoice = "required" ToolChoiceNone ToolChoice = "none" )
type ToolConfig ¶ added in v0.3.0
type ToolConfig struct {
MaxTurns int // Maximum number of tool call iterations before erroring.
ReturnToolRequests bool // Whether to return tool requests instead of making the tool calls and continuing the generation.
}
ToolConfig handles configuration around tool calls during generation.
type ToolContext ¶ added in v0.3.0
type ToolContext struct {
context.Context
// Resumed is optional metadata that can be used to resume the tool execution.
// Map is not nil only if the tool was interrupted.
Resumed map[string]any
// OriginalInput is the original input to the tool if the tool was interrupted, otherwise nil.
OriginalInput any
}
ToolContext provides context and utility functions for tool execution.
func (*ToolContext) Interrupt ¶ added in v0.3.0
func (tc *ToolContext) Interrupt(opts *InterruptOptions) error
Interrupt interrupts the tool execution and returns control to the caller with the total model response so far. The provided metadata is preserved and passed back via ToolContext.Resumed when the tool is restarted.
func (*ToolContext) IsResumed ¶ added in v1.4.0
func (tc *ToolContext) IsResumed() bool
IsResumed returns true if this tool execution is a resumption after an interrupt.
type ToolDef
deprecated
added in
v0.1.0
type ToolDef[In, Out any] = ToolAction[In, Out]
ToolDef is the previous name for ToolAction. It was renamed because it read as a sibling of ToolDefinition, the wire type a tool advertises to the model, which it is not.
Deprecated: use ToolAction.
type ToolDefinition ¶
type ToolDefinition struct {
// Description explains what the tool does and when to use it.
Description string `json:"description,omitempty"`
// InputSchema is a valid JSON Schema representing the input parameters of the tool.
InputSchema map[string]any `json:"inputSchema,omitempty"`
Key string `json:"key,omitempty"`
// Metadata contains additional information about this tool definition.
Metadata map[string]any `json:"metadata,omitempty"`
// Name is the unique identifier for this tool.
Name string `json:"name,omitempty"`
// OutputSchema is a valid JSON Schema describing the output of the tool.
OutputSchema map[string]any `json:"outputSchema,omitempty"`
}
A ToolDefinition describes a tool.
type ToolFunc ¶ added in v0.7.0
type ToolFunc[In, Out any] = func(ctx *ToolContext, input In) (Out, error)
ToolFunc is the function type for tool implementations.
type ToolName ¶ added in v0.3.0
type ToolName string
ToolName is a distinct type for a tool name. It is meant to be passed where a ToolRef is expected but no Tool is had.
type ToolNext ¶ added in v1.7.0
type ToolNext = func(ctx context.Context, params *ToolParams) (*MultipartToolResponse, error)
ToolNext is the next function in the WrapTool hook chain.
type ToolOption ¶ added in v1.3.0
type ToolOption interface {
// contains filtered or unexported methods
}
ToolOption is an option for defining a tool.
func WithStrictSchema ¶ added in v1.9.0
func WithStrictSchema(strict bool) ToolOption
WithStrictSchema controls whether the provider enforces strict JSON schema validation on this tool's input. Strict mode requires recursive additionalProperties: false and may reject some JSON Schema keywords (e.g. minItems/maxItems on Anthropic).
When unset, the provider's default applies. Providers without strict-tool support ignore this option.
type ToolParams ¶ added in v1.7.0
type ToolParams struct {
// Request is the tool request about to be executed.
Request *ToolRequest
// Tool is the resolved tool being called.
Tool Tool
}
ToolParams holds params for the WrapTool hook.
type ToolRef ¶ added in v0.3.0
type ToolRef interface {
Name() string
}
ToolRef is a reference to a tool.
type ToolRequest ¶
type ToolRequest struct {
// Input is a JSON object containing the input parameters for the tool.
// For example: map[string]any{"country":"USA", "president":3}.
Input any `json:"input,omitempty"`
// Name is the name of the tool to call.
Name string `json:"name,omitempty"`
// Partial indicates whether this is a partial streaming chunk.
Partial bool `json:"partial,omitempty"`
// Ref is the call ID or reference for this specific request.
Ref string `json:"ref,omitempty"`
}
A ToolRequest is a message from the model to the client that it should run a specific tool and pass a ToolResponse to the model on the next chat request it makes. Any ToolRequest will correspond to some ToolDefinition previously sent by the client.
type ToolResponse ¶
type ToolResponse struct {
// Content holds additional message parts that provide context or details about the tool response.
Content []*Part `json:"content,omitempty"`
// Name is the name of the tool that was executed.
Name string `json:"name,omitempty"`
// Output is a JSON object describing the results of running the tool.
// For example: map[string]any{"name":"Thomas Jefferson", "born":1743}.
Output any `json:"output,omitempty"`
// Ref is the call ID or reference matching the original request.
Ref string `json:"ref,omitempty"`
}
A ToolResponse is a message from the client to the model containing the results of running a specific tool on the arguments passed to the client by the model in a ToolRequest.
type TraceMetadata ¶ added in v0.3.0
type TraceMetadata struct {
// FeatureName identifies the feature being traced.
FeatureName string `json:"featureName,omitempty"`
// Paths contains metadata for each path executed during the trace.
Paths []*PathMetadata `json:"paths,omitempty"`
// Timestamp is when the trace was created.
Timestamp float64 `json:"timestamp,omitempty"`
}
TraceMetadata contains metadata about a trace execution.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package exp provides experimental AI primitives for Genkit.
|
Package exp provides experimental AI primitives for Genkit. |
|
localstore
Package localstore provides single-process exp.SessionStore implementations suitable for local development, tests, and single-instance apps (CLI tools, desktop apps, local web services).
|
Package localstore provides single-process exp.SessionStore implementations suitable for local development, tests, and single-instance apps (CLI tools, desktop apps, local web services). |
|
tool
Package tool provides runtime helpers for use inside tool functions.
|
Package tool provides runtime helpers for use inside tool functions. |