googleai

package
v0.1.14-update.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 4, 2026 License: MIT Imports: 21 Imported by: 0

README

This directory contains langchaingo provider for Google's models.

Both the googleai and vertex providers give access to Gemini-family multi-modal LLMs. The code between these providers is very similar; therefore, most of the vertex package is code-generated from the googleai package using a tool:

go run ./llms/googleai/internal/cmd/generate-vertex.go < llms/googleai/googleai.go > llms/googleai/vertex/vertex.go

Testing:

The test code between googleai and vertex is also shared, and lives in the shared_test directory. The same tests are run for both providers.

Documentation

Overview

Package googleai provides caching support for Google AI models.

package googleai implements a langchaingo provider for Google AI LLMs. See https://ai.google.dev/ for more details.

Index

Constants

View Source 
const (
	CITATIONS                         = "citations"
	SAFETY                            = "safety"
	RoleSystem                        = "system"
	RoleModel                         = "model"
	RoleUser                          = "user"
	RoleTool                          = "tool"
	ResponseMIMETypeJson              = "application/json"
	GENERATED_FUNCTION_CALL_ID_PREFIX = "fcall_"
)

Variables

View Source 
var (
	ErrNoContentInResponse   = errors.New("no content in generation response")
	ErrUnknownPartInResponse = errors.New("unknown part type in generation response")
	ErrInvalidMimeType       = errors.New("invalid mime type on content")
)

Functions

func MapError 

func MapError(err error) error

MapError maps Google AI-specific errors to standardized error codes.

func WithCachedContent 

func WithCachedContent(name string) llms.CallOption

WithCachedContent enables the use of pre-created cached content. The cached content must be created separately using Client.CreateCachedContent. This is different from Anthropic's inline cache control.

Types

type CachingHelper 

type CachingHelper struct {
	// contains filtered or unexported fields
}

CachingHelper provides utilities for working with Google AI's cached content feature. Unlike Anthropic which supports inline cache control, Google AI requires pre-creating cached content through the API.

Google AI caching is particularly useful for: - Large system prompts that are reused across multiple requests - Extensive context documents (e.g., knowledge bases, documentation) - Long conversation histories

The minimum cacheable content size is 32,768 tokens (~24,000 words). Cached content has a TTL (time-to-live) and will be automatically deleted after expiration.

func NewCachingHelper 

func NewCachingHelper(ctx context.Context, opts ...Option) (*CachingHelper, error)

NewCachingHelper creates a helper for managing cached content.

func (*CachingHelper) AllCachedContents 

func (ch *CachingHelper) AllCachedContents(ctx context.Context) func(func(*genai.CachedContent, error) bool)

AllCachedContents returns an iterator that yields all cached contents. This handles pagination automatically.

func (*CachingHelper) CreateCachedContent 

func (ch *CachingHelper) CreateCachedContent(
	ctx context.Context,
	modelName string,
	messages []llms.MessageContent,
	ttl time.Duration,
	displayName string,
) (*genai.CachedContent, error)

CreateCachedContent creates cached content that can be reused across multiple requests. This is useful for caching large system prompts, context documents, or frequently used instructions.

Parameters:

  • modelName: The model to use (e.g., "gemini-2.0-flash-exp")
  • messages: The content to cache (must be at least 32,768 tokens)
  • ttl: Time-to-live for the cached content (e.g., 1*time.Hour)
  • displayName: Optional human-readable name for the cache

Example usage: 

helper, _ := NewCachingHelper(ctx, WithAPIKey(apiKey))
cached, _ := helper.CreateCachedContent(ctx, "gemini-2.0-flash-exp",
    []llms.MessageContent{
        {
            Role: llms.ChatMessageTypeSystem,
            Parts: []llms.ContentPart{
                llms.TextPart("You are an expert assistant..."),
            },
        },
    },
    1*time.Hour,
    "my-expert-system-prompt",
)

// Use the cached content in requests
model, _ := New(ctx, WithAPIKey(apiKey))
resp, _ := model.GenerateContent(ctx, messages, WithCachedContent(cached.Name))

func (*CachingHelper) DeleteCachedContent 

func (ch *CachingHelper) DeleteCachedContent(ctx context.Context, name string) error

DeleteCachedContent removes cached content.

func (*CachingHelper) GetCachedContent 

func (ch *CachingHelper) GetCachedContent(ctx context.Context, name string) (*genai.CachedContent, error)

GetCachedContent retrieves existing cached content by name.

func (*CachingHelper) ListCachedContents 

func (ch *CachingHelper) ListCachedContents(ctx context.Context, pageSize int32) (genai.Page[genai.CachedContent], error)

ListCachedContents lists all cached content with pagination support. Returns a Page that can be iterated to fetch all cached contents.

func (*CachingHelper) UpdateCachedContent 

func (ch *CachingHelper) UpdateCachedContent(ctx context.Context, name string, ttl time.Duration) (*genai.CachedContent, error)

UpdateCachedContent updates the TTL or expiration time of cached content.

type GoogleAI 

type GoogleAI struct {
	CallbacksHandler callbacks.Handler
	// contains filtered or unexported fields
}

GoogleAI is a type that represents a Google AI API client.

func New 

func New(ctx context.Context, opts ...Option) (*GoogleAI, error)

New creates a new GoogleAI client.

func (*GoogleAI) Call 

func (g *GoogleAI) Call(ctx context.Context, prompt string, options ...llms.CallOption) (string, error)

Call implements the llms.Model interface.

func (*GoogleAI) CreateEmbedding 

func (g *GoogleAI) CreateEmbedding(ctx context.Context, texts []string) ([][]float32, error)

CreateEmbedding creates embeddings from texts.

func (*GoogleAI) GenerateContent 

func (g *GoogleAI) GenerateContent(
	ctx context.Context,
	messages []llms.MessageContent,
	options ...llms.CallOption,
) (*llms.ContentResponse, error)

GenerateContent implements the llms.Model interface.

type HarmBlockThreshold 

type HarmBlockThreshold string
const (
	// HarmBlockUnspecified means threshold is unspecified.
	HarmBlockUnspecified HarmBlockThreshold = "HARM_BLOCK_THRESHOLD_UNSPECIFIED"
	// HarmBlockLowAndAbove means content with NEGLIGIBLE will be allowed.
	HarmBlockLowAndAbove HarmBlockThreshold = "BLOCK_LOW_AND_ABOVE"
	// HarmBlockMediumAndAbove means content with NEGLIGIBLE and LOW will be allowed.
	HarmBlockMediumAndAbove HarmBlockThreshold = "BLOCK_MEDIUM_AND_ABOVE"
	// HarmBlockOnlyHigh means content with NEGLIGIBLE, LOW, and MEDIUM will be allowed.
	HarmBlockOnlyHigh HarmBlockThreshold = "BLOCK_ONLY_HIGH"
	// HarmBlockNone means all content will be allowed.
	HarmBlockNone HarmBlockThreshold = "BLOCK_NONE"
)

type Option 

type Option func(*Options)

func WithAPIKey 

func WithAPIKey(apiKey string) Option

WithAPIKey passes the API KEY (token) to the client. This is useful for googleai clients.

func WithCloudLocation 

func WithCloudLocation(l string) Option

WithCloudLocation passes the GCP cloud location (region) name to the client. This is useful for vertex clients.

func WithCloudProject 

func WithCloudProject(p string) Option

WithCloudProject passes the GCP cloud project name to the client. This is useful for vertex clients.

func WithCredentialsFile 

func WithCredentialsFile(credentialsFile string) Option

WithCredentialsFile append a ClientOption that authenticates API calls with the given service account or refresh token JSON credentials file.

func WithCredentialsJSON 

func WithCredentialsJSON(credentialsJSON []byte) Option

WithCredentialsJSON append a ClientOption that authenticates API calls with the given service account or refresh token JSON credentials.

func WithDefaultCandidateCount 

func WithDefaultCandidateCount(defaultCandidateCount int) Option

WithDefaultCandidateCount sets the candidate count for the model.

func WithDefaultEmbeddingModel 

func WithDefaultEmbeddingModel(defaultEmbeddingModel string) Option

WithDefaultModel passes a default embedding model name to the client. This model name is used if not explicitly provided in specific client invocations.

func WithDefaultMaxTokens 

func WithDefaultMaxTokens(maxTokens int) Option

WithDefaultMaxTokens sets the maximum token count for the model.

func WithDefaultModel 

func WithDefaultModel(defaultModel string) Option

WithDefaultModel passes a default content model name to the client. This model name is used if not explicitly provided in specific client invocations.

func WithDefaultTemperature 

func WithDefaultTemperature(defaultTemperature float64) Option

WithDefaultTemperature sets the maximum token count for the model.

func WithDefaultTopK 

func WithDefaultTopK(defaultTopK int) Option

WithDefaultTopK sets the TopK for the model.

func WithDefaultTopP 

func WithDefaultTopP(defaultTopP float64) Option

WithDefaultTopP sets the TopP for the model.

func WithEndpoint 

func WithEndpoint(endpoint string) Option

WithEndpoint append a ClientOption that uses the provided endpoint to make requests. This is useful for gemini clients.

func WithGRPCClient 

func WithGRPCClient(grpcClient *grpc.ClientConn) Option

WithGRPCClient append a ClientOption that uses the provided gRPC client to make requests. This is useful for gemini clients.

func WithGRPCConn 

func WithGRPCConn(conn *grpc.ClientConn) Option

WithGRPCConn appends a ClientOption that uses the provided gRPC client connection to make requests. This is useful for testing embeddings in vertex clients.

func WithHTTPClient 

func WithHTTPClient(httpClient *http.Client) Option

WithHTTPClient append a ClientOption that uses the provided HTTP client to make requests. This is useful for vertex clients.

func WithHarmThreshold 

func WithHarmThreshold(ht HarmBlockThreshold) Option

WithHarmThreshold sets the safety/harm setting for the model, potentially limiting any harmful content it may generate.

func WithRest 

func WithRest() Option

WithRest configures the client to use the REST API.

type Options 

type Options struct {
	CloudProject          string
	CloudLocation         string
	DefaultModel          string
	DefaultEmbeddingModel string
	DefaultCandidateCount int
	DefaultMaxTokens      int
	DefaultTemperature    float64
	DefaultTopK           int
	DefaultTopP           float64
	HarmThreshold         HarmBlockThreshold
	APIKey                string
	HTTPClient            *http.Client

	ClientOptions []option.ClientOption
}

Options is a set of options for GoogleAI and Vertex clients.

func DefaultOptions 

func DefaultOptions() Options

func (*Options) EnsureAuthPresent 

func (o *Options) EnsureAuthPresent()

EnsureAuthPresent attempts to ensure that the client has authentication information. If it does not, it will attempt to use the GOOGLE_API_KEY environment variable.

Source Files

View all Source files

Directories

Path Synopsis
internal
cmd command
Code generator for vertex.go from googleai.go nolint
 Code generator for vertex.go from googleai.go nolint
palmclient
package palm implements a langchaingo provider for Google Vertex AI legacy PaLM models.
 package palm implements a langchaingo provider for Google Vertex AI legacy PaLM models.
package vertex implements a langchaingo provider for Google Vertex AI LLMs, including the new Gemini models.
 package vertex implements a langchaingo provider for Google Vertex AI LLMs, including the new Gemini models.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.