protocol

package
v0.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 30, 2026 License: Apache-2.0 Imports: 22 Imported by: 0

Documentation

Overview

Package protocol implements the Google Gen AI wire protocol reused by Gemini and Vertex AI provider endpoints inside the models module.

Constructors:

  • NewChat — native genai chat. Full Gemini surface: thinking budget, response modalities, system instructions, safety settings, structured output, tool calling, grounding with Google Search;
  • NewEmbeddingModel — gemini-embedding-2 with output_dimensionality truncation;
  • NewImageModel — Gemini image generation through Interactions;
  • NewAudioTTSModel — Gemini-TTS via generate_content with audio response modality;
  • NewAudioTranscriptionModel — audio-input → text via generate_content (Gemini transcribes any audio attachment).

Token estimation: NewTextEstimator wraps CountTokens for model-specific tokenizer-based counts.

Gemini's Context Caching API (cheaper repeated prompts) doesn't fit core/chat's request model and is not exposed.

genai supports two backends: Generative Language (api key) and Vertex AI has its own facade package and construction config.

Index

Constants

View Source
const (
	SpeechRequestExtensionKey         = "google/speech_request"
	SpeechResponseExtensionKey        = "google/speech_response"
	TranscriptionRequestExtensionKey  = "google/transcription_request"
	TranscriptionResponseExtensionKey = "google/transcription_response"
	EmbeddingRequestExtensionKey      = "google/embedding_request"
	EmbeddingResponseExtensionKey     = "google/embedding_response"
	ImageRequestExtensionKey          = "google/image_request"
	ImageResponseExtensionKey         = "google/image_response"
)
View Source
const (
	ModelGemini36Flash      = "gemini-3.6-flash"
	ModelGemini35Flash      = "gemini-3.5-flash"
	ModelGemini35FlashLite  = "gemini-3.5-flash-lite"
	ModelGemini31ProPreview = "gemini-3.1-pro-preview"

	ModelGemini25FlashPreviewTTS = "gemini-2.5-flash-preview-tts"
	ModelGemini25ProPreviewTTS   = "gemini-2.5-pro-preview-tts"
	ModelGemini31FlashTTSPreview = "gemini-3.1-flash-tts-preview"

	ModelGemini25FlashImage     = "gemini-2.5-flash-image"
	ModelGemini3ProImage        = "gemini-3-pro-image"
	ModelGemini31FlashImage     = "gemini-3.1-flash-image"
	ModelGemini31FlashLiteImage = "gemini-3.1-flash-lite-image"

	ModelGeminiEmbedding2 = "gemini-embedding-2"
)
View Source
const DefaultBaseURL = "https://generativelanguage.googleapis.com"
View Source
const (
	// RequestExtensionKey stores an official [genai.GenerateContentConfig].
	// Core options, messages, and tools take precedence over overlapping fields.
	RequestExtensionKey = "google/request"
)
View Source
const (
	// ResponseExtensionKey preserves the complete official GenerateContent
	// response (or the current official stream chunk).
	ResponseExtensionKey = "google/response"
)

Variables

This section is empty.

Functions

This section is empty.

Types

type AudioTTSModel

type AudioTTSModel struct {
	// contains filtered or unexported fields
}

AudioTTSModel wraps Gemini's native TTS through GenerateContent with ResponseModalities=AUDIO. Current supported models are declared in constant.go; only Gemini 3.1 Flash TTS supports incremental streaming.

Speed and OutputFormat are not honored: Gemini's TTS has no playback-rate knob. GenerateContent returns 24 kHz signed 16-bit little-endian PCM; callers choose their own container at the application boundary.

func NewAudioTTSModel

func NewAudioTTSModel(config AudioTTSModelConfig) (*AudioTTSModel, error)

func (*AudioTTSModel) Call

func (a *AudioTTSModel) Call(ctx context.Context, req *tts.Request) (*tts.Response, error)

func (*AudioTTSModel) Stream

func (a *AudioTTSModel) Stream(ctx context.Context, req *tts.Request) iter.Seq2[*tts.Response, error]

type AudioTTSModelConfig

type AudioTTSModelConfig struct {
	Provider       string
	Client         ClientConfig
	DefaultOptions tts.Options
}

func (AudioTTSModelConfig) Validate

func (a AudioTTSModelConfig) Validate() error

type AudioTranscriptionModel

type AudioTranscriptionModel struct {
	// contains filtered or unexported fields
}

AudioTranscriptionModel exposes Gemini's multimodal chat through the transcription interface. Gemini has no /transcribe endpoint — any audio-accepting model returns a transcript when prompted. This adapter uses the stable instruction "Transcribe this audio.".

func (*AudioTranscriptionModel) Call

type AudioTranscriptionModelConfig

type AudioTranscriptionModelConfig struct {
	Provider       string
	Client         ClientConfig
	DefaultOptions transcription.Options
}

func (AudioTranscriptionModelConfig) Validate

func (a AudioTranscriptionModelConfig) Validate() error

type Chat

type Chat struct {
	// contains filtered or unexported fields
}

Chat implements the minimal Core Model and optional Streamer capabilities.

func NewChat

func NewChat(config ChatConfig) (*Chat, error)

func (*Chat) Call

func (c *Chat) Call(ctx context.Context, req *corechat.Request) (*corechat.Response, error)

func (*Chat) Stream

func (c *Chat) Stream(ctx context.Context, req *corechat.Request) iter.Seq2[*corechat.Response, error]

Stream performs one streaming GenerateContent request. Candidate and logical part offsets are retained only for the lifetime of this stream.

type ChatConfig

type ChatConfig struct {
	Provider       string
	Client         ClientConfig
	DefaultOptions corechat.Options
}

func (ChatConfig) Validate

func (c ChatConfig) Validate() error

type ClientConfig

type ClientConfig struct {
	APIKey string

	// Backend selects the genai backend. Zero value falls back to
	// [genai.BackendGeminiAPI] — the public Gemini API. Set to
	// [genai.BackendVertexAI] for GCP-hosted enterprise deployments;
	// Project and Location become required in that mode and APIKey
	// is ignored in favor of the supplied [genai.ClientConfig.Credentials]
	// (or ADC).
	Backend genai.Backend

	// Project is the GCP project id, required when Backend ==
	// BackendVertexAI. Ignored otherwise.
	Project string

	// Location is the GCP region (e.g. "us-central1"), required when
	// Backend == BackendVertexAI. Ignored otherwise.
	Location string

	// BaseURL overrides the genai client endpoint. Optional —
	// production users should leave it empty (the SDK picks the right
	// host per Backend). Useful for mock servers / corporate proxies.
	BaseURL string

	// HTTPClient is shared by the official Gen AI SDK and the Interactions
	// transport. Optional.
	HTTPClient *http.Client
}

func (ClientConfig) Validate

func (c ClientConfig) Validate() error

type EmbeddingModel

type EmbeddingModel struct {
	// contains filtered or unexported fields
}

EmbeddingModel wraps Gemini's embed_content endpoint. New integrations use gemini-embedding-2, whose output dimensionality is configurable from 128 to 3072. Core's text-only request intentionally exposes only that model's text input capability; richer multimodal embedding inputs belong in a dedicated protocol rather than being hidden inside text.

func NewEmbeddingModel

func NewEmbeddingModel(config EmbeddingModelConfig) (*EmbeddingModel, error)

func (*EmbeddingModel) Call

type EmbeddingModelConfig

type EmbeddingModelConfig struct {
	Provider       string
	Client         ClientConfig
	DefaultOptions embedding.Options
}

func (EmbeddingModelConfig) Validate

func (e EmbeddingModelConfig) Validate() error

type ImageGenerationOptions

type ImageGenerationOptions struct {
	AspectRatio           string                    `json:"aspect_ratio,omitempty"`
	ImageSize             string                    `json:"image_size,omitempty"`
	Delivery              string                    `json:"delivery,omitempty"`
	PreviousInteractionID string                    `json:"previous_interaction_id,omitempty"`
	Store                 *bool                     `json:"store,omitempty"`
	ThinkingLevel         string                    `json:"thinking_level,omitempty"`
	ThinkingSummaries     string                    `json:"thinking_summaries,omitempty"`
	ServiceTier           string                    `json:"service_tier,omitempty"`
	Labels                map[string]string         `json:"labels,omitempty"`
	InputImages           []*media.Media            `json:"input_images,omitempty"`
	GoogleSearch          *ImageGoogleSearchOptions `json:"google_search,omitempty"`
	SafetySettings        []ImageSafetySetting      `json:"safety_settings,omitempty"`
}

ImageGenerationOptions carries the current Interactions API controls that do not have provider-neutral equivalents in image.Options. Store it under ImageRequestExtensionKey.

type ImageGoogleSearchOptions

type ImageGoogleSearchOptions struct {
	SearchTypes []string `json:"search_types,omitempty"`
}

ImageGoogleSearchOptions configures the image-generation guide's google_search tool. SearchTypes accepts "web_search" and "image_search".

type ImageModel

type ImageModel struct {
	// contains filtered or unexported fields
}

ImageModel uses the current Gemini Interactions API. Imagen's legacy GenerateImages endpoint is deliberately not exposed: Google has deprecated Imagen and scheduled it for shutdown on 2026-08-17.

func NewImageModel

func NewImageModel(config ImageModelConfig) (*ImageModel, error)

func (*ImageModel) Call

func (i *ImageModel) Call(ctx context.Context, req *image.Request) (*image.Response, error)

type ImageModelConfig

type ImageModelConfig struct {
	Client         ClientConfig
	DefaultOptions image.Options
}

func (ImageModelConfig) Validate

func (i ImageModelConfig) Validate() error

type ImageSafetySetting

type ImageSafetySetting struct {
	Type      string `json:"type"`
	Threshold string `json:"threshold"`
	Method    string `json:"method,omitempty"`
}

ImageSafetySetting mirrors the Interactions API safety-setting shape.

type TextEstimator

type TextEstimator struct {
	// contains filtered or unexported fields
}

TextEstimator reports input-token counts via Gemini's count_tokens endpoint. Implements tokenizer.TextEstimator so it drops into code paths gating on token budgets (RAG chunking, prompt-window checks).

func NewTextEstimator

func NewTextEstimator(config TextEstimatorConfig) (*TextEstimator, error)

func (*TextEstimator) EstimateText

func (t *TextEstimator) EstimateText(ctx context.Context, text string) (int, error)

EstimateText returns the prompt-token count Gemini would charge if text were sent as a single user message under the configured model.

type TextEstimatorConfig

type TextEstimatorConfig struct {
	Client ClientConfig
	Model  string
}

TextEstimatorConfig configures a Gemini-backed token estimator. Token counts vary across model families — supply the same Model name you intend to send chat requests under so the count matches the real billing.

func (TextEstimatorConfig) Validate

func (t TextEstimatorConfig) Validate() error

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL