translator

package
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 21, 2026 License: Apache-2.0 Imports: 38 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	LineFeedSSEDelimiter               = "\n\n"
	CarriageReturnSSEDelimiter         = "\r\r"
	CarriageReturnLineFeedSSEDelimiter = "\r\n\r\n"
)
View Source 
const BedrockDefaultVersion = "bedrock-2023-05-31"

https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages.html

Variables

This section is empty.

Functions

This section is empty.

Types

type AnthropicCountTokensTranslator added in v1.1.0 

type AnthropicCountTokensTranslator = Translator[anthropicschema.CountTokensRequest, tracingapi.CountTokensSpan]

AnthropicCountTokensTranslator translates the Anthropic's /v1/messages/count_tokens endpoint.

func NewCountTokensToAWSAnthropicTranslator added in v1.1.0 

func NewCountTokensToAWSAnthropicTranslator(apiVersion string, modelNameOverride internalapi.ModelNameOverride) AnthropicCountTokensTranslator

NewCountTokensToAWSAnthropicTranslator creates a translator for count_tokens to AWS Bedrock. AWS Bedrock has a dedicated CountTokens API at /model/{modelId}/count-tokens. The request wraps the Anthropic body in {"input":{"invokeModel":{"body":"<base64>"}}} and the response returns {"inputTokens": N}.

func NewCountTokensToAnthropicTranslator added in v1.1.0 

func NewCountTokensToAnthropicTranslator(modelNameOverride internalapi.ModelNameOverride) AnthropicCountTokensTranslator

NewCountTokensToAnthropicTranslator creates a passthrough translator for Anthropic count_tokens.

func NewCountTokensToGCPAnthropicTranslator added in v1.1.0 

func NewCountTokensToGCPAnthropicTranslator(apiVersion string, modelNameOverride internalapi.ModelNameOverride) AnthropicCountTokensTranslator

NewCountTokensToGCPAnthropicTranslator creates a translator for count_tokens to GCP Anthropic.

type AnthropicMessagesTranslator 

type AnthropicMessagesTranslator = Translator[anthropicschema.MessagesRequest, tracingapi.MessageSpan]

AnthropicMessagesTranslator translates the Anthropic's /messages endpoint.

func NewAnthropicToAWSAnthropicTranslator 

func NewAnthropicToAWSAnthropicTranslator(apiVersion string, modelNameOverride internalapi.ModelNameOverride) AnthropicMessagesTranslator

NewAnthropicToAWSAnthropicTranslator creates a translator for Anthropic to AWS Bedrock Anthropic format. AWS Bedrock supports the native Anthropic Messages API, so this is essentially a passthrough translator with AWS-specific path modifications.

func NewAnthropicToAWSBedrockTranslator added in v0.7.0 

func NewAnthropicToAWSBedrockTranslator(modelNameOverride internalapi.ModelNameOverride) AnthropicMessagesTranslator

NewAnthropicToAWSBedrockTranslator creates a translator for Anthropic Messages API to AWS Bedrock Converse API.

func NewAnthropicToAnthropicTranslator 

func NewAnthropicToAnthropicTranslator(prefix string, modelNameOverride internalapi.ModelNameOverride) AnthropicMessagesTranslator

NewAnthropicToAnthropicTranslator creates a passthrough translator for Anthropic. The prefix defaults to "v1" via schemaToFilterAPI, producing "/v1/messages". AWS and GCP Anthropic wrappers pass empty prefix as they override the request path entirely.

func NewAnthropicToChatCompletionOpenAITranslator added in v0.6.0 

func NewAnthropicToChatCompletionOpenAITranslator(prefix string, modelNameOverride internalapi.ModelNameOverride) AnthropicMessagesTranslator

NewAnthropicToChatCompletionOpenAITranslator implements [Factory] for Anthropic to OpenAI ChatCompletion translation. This translator converts Anthropic API format to OpenAI ChatCompletion API requests. The prefix parameter is the prefix field set in the OpenAI VersionAPISchema used to construct the translated path (e.g., "v1" produces "/v1/chat/completions", "gateway/v1" produces "/gateway/v1/chat/completions").

func NewAnthropicToGCPAnthropicTranslator 

func NewAnthropicToGCPAnthropicTranslator(apiVersion string, modelNameOverride internalapi.ModelNameOverride) AnthropicMessagesTranslator

NewAnthropicToGCPAnthropicTranslator creates a translator for Anthropic to GCP Anthropic format. This is essentially a passthrough translator with GCP-specific modifications.

type AnthropicResponseRedactor added in v0.6.0 

type AnthropicResponseRedactor interface {
	// SetRedactionConfig configures redaction settings for the translator.
	SetRedactionConfig(debugLogEnabled, enableRedaction bool, logger *slog.Logger)
	// RedactAnthropicBody creates a redacted copy of the Anthropic response for safe logging.
	// Returns a new MessagesResponse with sensitive fields redacted.
	// The original response is never modified.
	RedactAnthropicBody(resp *anthropicschema.MessagesResponse) *anthropicschema.MessagesResponse
}

AnthropicResponseRedactor is an optional interface that Anthropic translators can implement to support response body redaction for debug logging.

type CohereRerankTranslator 

type CohereRerankTranslator = Translator[cohereschema.RerankV2Request, tracingapi.RerankSpan]

CohereRerankTranslator translates the Cohere's /v2/rerank endpoint.

func NewRerankCohereToCohereTranslator 

func NewRerankCohereToCohereTranslator(apiVersion string, modelNameOverride internalapi.ModelNameOverride) CohereRerankTranslator

NewRerankCohereToCohereTranslator implements [Factory] for Cohere Rerank v2 translation.

type ContentTypeSetter added in v0.7.0 

type ContentTypeSetter interface {
	SetContentType(ct string)
}

ContentTypeSetter is an optional interface that translators can implement to receive the original request Content-Type header. This is needed for multipart/form-data endpoints where the translator needs the boundary to re-encode the body during model name override.

type OpenAIAudioTranscriptionTranslator added in v0.7.0 

type OpenAIAudioTranscriptionTranslator = Translator[openai.TranscriptionRequest, tracingapi.TranscriptionSpan]

OpenAIAudioTranscriptionTranslator translates the OpenAI's /v1/audio/transcriptions endpoint.

func NewTranscriptionOpenAIToOpenAITranslator added in v0.7.0 

func NewTranscriptionOpenAIToOpenAITranslator(prefix string, modelNameOverride internalapi.ModelNameOverride) OpenAIAudioTranscriptionTranslator

NewTranscriptionOpenAIToOpenAITranslator implements OpenAIAudioTranscriptionTranslator for OpenAI to OpenAI translation for audio transcription.

type OpenAIAudioTranslationTranslator added in v0.7.0 

type OpenAIAudioTranslationTranslator = Translator[openai.TranslationRequest, tracingapi.TranslationSpan]

OpenAIAudioTranslationTranslator translates the OpenAI's /v1/audio/translations endpoint.

func NewTranslationOpenAIToOpenAITranslator added in v0.7.0 

func NewTranslationOpenAIToOpenAITranslator(prefix string, modelNameOverride internalapi.ModelNameOverride) OpenAIAudioTranslationTranslator

NewTranslationOpenAIToOpenAITranslator implements OpenAIAudioTranslationTranslator for OpenAI to OpenAI translation for audio translations.

type OpenAIChatCompletionTranslator 

type OpenAIChatCompletionTranslator = Translator[openai.ChatCompletionRequest, tracingapi.ChatCompletionSpan]

OpenAIChatCompletionTranslator translates the OpenAI's /chat/completions endpoint.

func NewChatCompletionOpenAIToAWSAnthropicTranslator added in v0.6.0 

func NewChatCompletionOpenAIToAWSAnthropicTranslator(apiVersion string, modelNameOverride internalapi.ModelNameOverride) OpenAIChatCompletionTranslator

NewChatCompletionOpenAIToAWSAnthropicTranslator implements [Factory] for OpenAI to AWS Anthropic translation. This translator converts OpenAI ChatCompletion API requests to AWS Anthropic API format.

func NewChatCompletionOpenAIToAWSBedrockTranslator 

func NewChatCompletionOpenAIToAWSBedrockTranslator(modelNameOverride internalapi.ModelNameOverride) OpenAIChatCompletionTranslator

NewChatCompletionOpenAIToAWSBedrockTranslator implements [Factory] for OpenAI to AWS Bedrock translation.

func NewChatCompletionOpenAIToAzureOpenAITranslator 

func NewChatCompletionOpenAIToAzureOpenAITranslator(apiVersion string, modelNameOverride internalapi.ModelNameOverride) OpenAIChatCompletionTranslator

NewChatCompletionOpenAIToAzureOpenAITranslator implements [Factory] for OpenAI to Azure OpenAI translations. Except RequestBody method requires modification to satisfy Microsoft Azure OpenAI spec https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#chat-completions, other interface methods are identical to NewChatCompletionOpenAIToOpenAITranslator's interface implementations.

func NewChatCompletionOpenAIToGCPAnthropicTranslator 

func NewChatCompletionOpenAIToGCPAnthropicTranslator(apiVersion string, modelNameOverride internalapi.ModelNameOverride) OpenAIChatCompletionTranslator

NewChatCompletionOpenAIToGCPAnthropicTranslator implements [Factory] for OpenAI to GCP Anthropic translation. This translator converts OpenAI ChatCompletion API requests to GCP Anthropic API format.

func NewChatCompletionOpenAIToGCPVertexAITranslator 

func NewChatCompletionOpenAIToGCPVertexAITranslator(modelNameOverride internalapi.ModelNameOverride) OpenAIChatCompletionTranslator

NewChatCompletionOpenAIToGCPVertexAITranslator implements [Factory] for OpenAI to GCP Gemini translation. This translator converts OpenAI ChatCompletion API requests to GCP Gemini API format.

func NewChatCompletionOpenAIToOpenAITranslator 

func NewChatCompletionOpenAIToOpenAITranslator(prefix string, modelNameOverride internalapi.ModelNameOverride) OpenAIChatCompletionTranslator

NewChatCompletionOpenAIToOpenAITranslator implements [Factory] for OpenAI to OpenAI translation.

type OpenAICompletionTranslator 

type OpenAICompletionTranslator = Translator[openai.CompletionRequest, tracingapi.CompletionSpan]

OpenAICompletionTranslator translates the OpenAI's /completions endpoint.

func NewCompletionOpenAIToOpenAITranslator 

func NewCompletionOpenAIToOpenAITranslator(apiVersion string, modelNameOverride internalapi.ModelNameOverride) OpenAICompletionTranslator

NewCompletionOpenAIToOpenAITranslator implements [Factory] for OpenAI to OpenAI translation for completions.

type OpenAIEmbeddingTranslator 

type OpenAIEmbeddingTranslator = Translator[openai.EmbeddingRequest, tracingapi.EmbeddingsSpan]

OpenAIEmbeddingTranslator translates the OpenAI's /embeddings endpoint.

func NewEmbeddingOpenAIToAWSBedrockTranslator added in v0.6.0 

func NewEmbeddingOpenAIToAWSBedrockTranslator(modelNameOverride internalapi.ModelNameOverride) OpenAIEmbeddingTranslator

NewEmbeddingOpenAIToAWSBedrockTranslator implements [Factory] for OpenAI to AWS Bedrock embedding translation.

func NewEmbeddingOpenAIToAzureOpenAITranslator 

func NewEmbeddingOpenAIToAzureOpenAITranslator(apiVersion string, modelNameOverride internalapi.ModelNameOverride) OpenAIEmbeddingTranslator

NewEmbeddingOpenAIToAzureOpenAITranslator implements [Factory] for OpenAI to Azure OpenAI translation for embeddings.

func NewEmbeddingOpenAIToGCPVertexAITranslator added in v0.6.0 

func NewEmbeddingOpenAIToGCPVertexAITranslator(requestModel internalapi.RequestModel, modelNameOverride internalapi.ModelNameOverride) OpenAIEmbeddingTranslator

NewEmbeddingOpenAIToGCPVertexAITranslator implements [Factory] for OpenAI to GCP VertexAI translation for embeddings.

func NewEmbeddingOpenAIToOpenAITranslator 

func NewEmbeddingOpenAIToOpenAITranslator(apiVersion string, modelNameOverride internalapi.ModelNameOverride) OpenAIEmbeddingTranslator

NewEmbeddingOpenAIToOpenAITranslator implements [Factory] for OpenAI to OpenAI translation for embeddings.

type OpenAIImageGenerationTranslator 

type OpenAIImageGenerationTranslator = Translator[openai.ImageGenerationRequest, tracingapi.ImageGenerationSpan]

OpenAIImageGenerationTranslator translates the OpenAI's /images/generations endpoint.

func NewImageGenerationOpenAIToOpenAITranslator 

func NewImageGenerationOpenAIToOpenAITranslator(apiVersion string, modelNameOverride internalapi.ModelNameOverride) OpenAIImageGenerationTranslator

NewImageGenerationOpenAIToOpenAITranslator implements [Factory] for OpenAI to OpenAI image generation translation.

type OpenAIResponsesInputTokensTranslator added in v1.1.0 

type OpenAIResponsesInputTokensTranslator = Translator[openai.ResponseRequest, tracingapi.ResponsesInputTokensSpan]

OpenAIResponsesInputTokensTranslator translates the OpenAI's /v1/responses/input_tokens endpoint.

func NewResponsesInputTokensOpenAIToAzureOpenAITranslator added in v1.1.0 

func NewResponsesInputTokensOpenAIToAzureOpenAITranslator(apiVersion string, modelNameOverride internalapi.ModelNameOverride) OpenAIResponsesInputTokensTranslator

NewResponsesInputTokensOpenAIToAzureOpenAITranslator implements [Factory] for OpenAI to Azure OpenAI translation for /v1/responses/input_tokens.

func NewResponsesInputTokensOpenAIToOpenAITranslator added in v1.1.0 

func NewResponsesInputTokensOpenAIToOpenAITranslator(prefix string, modelNameOverride internalapi.ModelNameOverride) OpenAIResponsesInputTokensTranslator

NewResponsesInputTokensOpenAIToOpenAITranslator creates a passthrough translator for OpenAI /v1/responses/input_tokens.

type OpenAIResponsesTranslator 

type OpenAIResponsesTranslator = Translator[openai.ResponseRequest, tracingapi.ResponsesSpan]

OpenAIResponsesTranslator translates the OpenAI's /responses endpoint.

func NewResponsesOpenAIToAzureOpenAITranslator added in v0.7.0 

func NewResponsesOpenAIToAzureOpenAITranslator(apiVersion string, modelNameOverride internalapi.ModelNameOverride) OpenAIResponsesTranslator

NewResponsesOpenAIToAzureOpenAITranslator implements [Factory] for OpenAI to Azure OpenAI translation for responses.

func NewResponsesOpenAIToOpenAITranslator 

func NewResponsesOpenAIToOpenAITranslator(prefix string, modelNameOverride internalapi.ModelNameOverride) OpenAIResponsesTranslator

NewResponsesOpenAIToOpenAITranslator implements OpenAIResponsesTranslator for OpenAI to OpenAI translation for responses.

type OpenAISpeechTranslator added in v0.6.0 

type OpenAISpeechTranslator = Translator[openai.SpeechRequest, tracingapi.SpeechSpan]

OpenAISpeechTranslator translates the OpenAI's /v1/audio/speech endpoint.

func NewSpeechOpenAIToOpenAITranslator added in v0.6.0 

func NewSpeechOpenAIToOpenAITranslator(prefix string, modelNameOverride internalapi.ModelNameOverride) OpenAISpeechTranslator

NewSpeechOpenAIToOpenAITranslator implements OpenAISpeechTranslator for OpenAI to OpenAI translation for speech.

type RequestHeadersSetter added in v0.7.0 

type RequestHeadersSetter interface {
	SetRequestHeaders(headers map[string]string)
}

RequestHeadersSetter is an optional interface for translators that need request headers to translate provider-specific request fields.

type ResponseRedactor added in v0.6.0 

type ResponseRedactor interface {
	// SetRedactionConfig configures redaction settings for the translator.
	SetRedactionConfig(debugLogEnabled, enableRedaction bool, logger *slog.Logger)
	// RedactBody creates a redacted copy of the response for safe logging.
	// Returns a new ChatCompletionResponse with sensitive fields redacted.
	// The original response is never modified.
	RedactBody(resp *openai.ChatCompletionResponse) *openai.ChatCompletionResponse
}

ResponseRedactor is an optional interface that translators can implement to support response body redaction for debug logging.

type ToAWSAnthropicV1Tokenize added in v1.1.0 

type ToAWSAnthropicV1Tokenize struct {
	// contains filtered or unexported fields
}

ToAWSAnthropicV1Tokenize translates OpenAI tokenize requests to AWS Bedrock CountTokens API using the InvokeModel format: {"input":{"invokeModel":{"body":"<base64>"}}}

func (*ToAWSAnthropicV1Tokenize) RequestBody added in v1.1.0 

func (o *ToAWSAnthropicV1Tokenize) RequestBody(_ []byte, tokenizeReq *tokenize.RequestUnion, _ bool) (
	newHeaders []internalapi.Header, newBody []byte, err error,
)

RequestBody implements [TokenizeTranslator.RequestBody] for AWS Anthropic.

func (*ToAWSAnthropicV1Tokenize) ResponseBody added in v1.1.0 

func (o *ToAWSAnthropicV1Tokenize) ResponseBody(_ map[string]string, body io.Reader, _ bool, span tracingapi.TokenizeSpan) (
	newHeaders []internalapi.Header, newBody []byte, tokenUsage metrics.TokenUsage, responseModel string, err error,
)

ResponseBody implements [TokenizeTranslator.ResponseBody] for AWS Anthropic.

func (*ToAWSAnthropicV1Tokenize) ResponseError added in v1.1.0 

func (o *ToAWSAnthropicV1Tokenize) ResponseError(respHeaders map[string]string, body io.Reader) (
	newHeaders []internalapi.Header, newBody []byte, err error,
)

ResponseError implements [TokenizeTranslator.ResponseError] for AWS Anthropic.

func (*ToAWSAnthropicV1Tokenize) ResponseHeaders added in v1.1.0 

func (o *ToAWSAnthropicV1Tokenize) ResponseHeaders(map[string]string) (newHeaders []internalapi.Header, err error)

ResponseHeaders implements [TokenizeTranslator.ResponseHeaders].

type ToAWSBedrockV1Tokenize added in v1.1.0 

type ToAWSBedrockV1Tokenize struct {
	// contains filtered or unexported fields
}

ToAWSBedrockV1Tokenize translates tokenize API requests to AWS Bedrock format. Converts OpenAI-compatible tokenize requests to AWS Bedrock Converse API format for token counting. Uses the Converse API with minimal token generation to extract input token usage statistics.

func (*ToAWSBedrockV1Tokenize) RequestBody added in v1.1.0 

func (o *ToAWSBedrockV1Tokenize) RequestBody(_ []byte, tokenizeReq *tokenize.RequestUnion, _ bool) (
	newHeaders []internalapi.Header, newBody []byte, err error,
)

RequestBody implements [TokenizeTranslator.RequestBody] for AWS Bedrock. This method translates an OpenAI tokenize request to AWS Bedrock CountTokens format.

func (*ToAWSBedrockV1Tokenize) ResponseBody added in v1.1.0 

func (o *ToAWSBedrockV1Tokenize) ResponseBody(_ map[string]string, body io.Reader, _ bool, span tracingapi.TokenizeSpan) (
	newHeaders []internalapi.Header, newBody []byte, tokenUsage metrics.TokenUsage, responseModel string, err error,
)

ResponseBody implements [TokenizeTranslator.ResponseBody] for AWS Bedrock. This method translates an AWS Bedrock CountTokens response to OpenAI tokenize format. AWS Bedrock uses static model execution without virtualization, where the requested model is exactly what gets executed. We extract token count from the CountTokens response.

func (*ToAWSBedrockV1Tokenize) ResponseError added in v1.1.0 

func (o *ToAWSBedrockV1Tokenize) ResponseError(respHeaders map[string]string, body io.Reader) (
	newHeaders []internalapi.Header, newBody []byte, err error,
)

ResponseError implements [TokenizeTranslator.ResponseError] for AWS Bedrock. Translate AWS Bedrock exceptions to OpenAI error type.

func (*ToAWSBedrockV1Tokenize) ResponseHeaders added in v1.1.0 

func (o *ToAWSBedrockV1Tokenize) ResponseHeaders(map[string]string) (newHeaders []internalapi.Header, err error)

ResponseHeaders implements [TokenizeTranslator.ResponseHeaders].

type ToGCPAnthropicV1Tokenize added in v1.1.0 

type ToGCPAnthropicV1Tokenize struct {
	// contains filtered or unexported fields
}

ToGCPAnthropicV1Tokenize translates tokenize API requests to GCP Anthropic format. Converts OpenAI-compatible tokenize requests to GCP Anthropic Messages API format for token counting. Uses the count-tokens model with rawPredict method for token counting.

func (*ToGCPAnthropicV1Tokenize) RequestBody added in v1.1.0 

func (o *ToGCPAnthropicV1Tokenize) RequestBody(_ []byte, tokenizeReq *tokenize.RequestUnion, _ bool) (
	newHeaders []internalapi.Header, newBody []byte, err error,
)

RequestBody implements [TokenizeTranslator.RequestBody] for GCP Anthropic. This method translates an OpenAI tokenize request to GCP Anthropic Messages format. Only the fields that affect the token count are mapped (messages, model, system, tools); see openAIToAnthropicCountTokensParams for why the remaining MessageCountTokensParams fields (cache_control, output_config, thinking, tool_choice) are intentionally omitted.

func (*ToGCPAnthropicV1Tokenize) ResponseBody added in v1.1.0 

func (o *ToGCPAnthropicV1Tokenize) ResponseBody(_ map[string]string, body io.Reader, _ bool, span tracingapi.TokenizeSpan) (
	newHeaders []internalapi.Header, newBody []byte, tokenUsage metrics.TokenUsage, responseModel string, err error,
)

ResponseBody implements [TokenizeTranslator.ResponseBody] for GCP Anthropic. This method translates a GCP Anthropic MessageTokensCount response to OpenAI tokenize format. GCP Anthropic uses deterministic model mapping without virtualization, where the requested model is exactly what gets executed. We extract token count from the token counting response.

func (*ToGCPAnthropicV1Tokenize) ResponseError added in v1.1.0 

func (o *ToGCPAnthropicV1Tokenize) ResponseError(respHeaders map[string]string, body io.Reader) (
	newHeaders []internalapi.Header, newBody []byte, err error,
)

ResponseError implements [TokenizeTranslator.ResponseError] for GCP Anthropic. Translate GCP Anthropic exceptions to OpenAI error type.

func (*ToGCPAnthropicV1Tokenize) ResponseHeaders added in v1.1.0 

func (o *ToGCPAnthropicV1Tokenize) ResponseHeaders(map[string]string) (newHeaders []internalapi.Header, err error)

ResponseHeaders implements [TokenizeTranslator.ResponseHeaders].

type ToGCPVertexAIV1Tokenize added in v1.1.0 

type ToGCPVertexAIV1Tokenize struct {
	// contains filtered or unexported fields
}

ToGCPVertexAIV1Tokenize translates tokenize API requests to GCP Vertex AI format. Converts OpenAI-compatible tokenize requests to GCP Gemini CountTokens API format. https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/count-tokens

func (*ToGCPVertexAIV1Tokenize) RequestBody added in v1.1.0 

func (o *ToGCPVertexAIV1Tokenize) RequestBody(_ []byte, tokenizeReq *tokenize.RequestUnion, _ bool) (
	newHeaders []internalapi.Header, newBody []byte, err error,
)

RequestBody implements [TokenizeTranslator.RequestBody] for GCP Vertex AI. This method translates an OpenAI tokenize request to GCP Gemini CountTokens format.

func (*ToGCPVertexAIV1Tokenize) ResponseBody added in v1.1.0 

func (o *ToGCPVertexAIV1Tokenize) ResponseBody(_ map[string]string, body io.Reader, _ bool, span tracingapi.TokenizeSpan) (
	newHeaders []internalapi.Header, newBody []byte, tokenUsage metrics.TokenUsage, responseModel string, err error,
)

ResponseBody implements [TokenizeTranslator.ResponseBody] for GCP Vertex AI. This method translates a GCP Gemini CountTokens response to OpenAI tokenize format. GCP Vertex AI uses deterministic model mapping without virtualization, where the requested model is exactly what gets executed. The response does not contain a model field, so we return the request model that was originally sent.

func (*ToGCPVertexAIV1Tokenize) ResponseError added in v1.1.0 

func (o *ToGCPVertexAIV1Tokenize) ResponseError(respHeaders map[string]string, body io.Reader) (
	newHeaders []internalapi.Header, newBody []byte, err error,
)

ResponseError implements [TokenizeTranslator.ResponseError] for GCP Vertex AI. Translate GCP Vertex AI exceptions to OpenAI error type.

func (*ToGCPVertexAIV1Tokenize) ResponseHeaders added in v1.1.0 

func (o *ToGCPVertexAIV1Tokenize) ResponseHeaders(map[string]string) (newHeaders []internalapi.Header, err error)

ResponseHeaders implements [TokenizeTranslator.ResponseHeaders].

type ToOpenAITokenize added in v1.1.0 

type ToOpenAITokenize struct {
	// contains filtered or unexported fields
}

ToOpenAITokenize is a passthrough translator for OpenAI Tokenize API. It may apply model overrides but otherwise preserves the tokenization requests in OpenAI format. Supports both chat and completion tokenize requests as defined in the tokenize API spec.

func (*ToOpenAITokenize) RequestBody added in v1.1.0 

func (o *ToOpenAITokenize) RequestBody(original []byte, req *tokenize.RequestUnion, forceBodyMutation bool) (
	newHeaders []internalapi.Header, newBody []byte, err error,
)

RequestBody implements [TokenizeTranslator.RequestBody]. This method validates the tokenize request union, applies model overrides if specified, and sets the appropriate routing headers for the tokenize endpoint.

func (*ToOpenAITokenize) ResponseBody added in v1.1.0 

func (o *ToOpenAITokenize) ResponseBody(_ map[string]string, body io.Reader, _ bool, span tracingapi.TokenizeSpan) (
	newHeaders []internalapi.Header, newBody []byte, tokenUsage metrics.TokenUsage, responseModel string, err error,
)

ResponseBody implements [TokenizeTranslator.ResponseBody]. OpenAI tokenize responses are passed through unchanged. The response does not contain a model field, so we fallback to the request model for metrics and tracing consistency.

func (*ToOpenAITokenize) ResponseError added in v1.1.0 

func (o *ToOpenAITokenize) ResponseError(respHeaders map[string]string, body io.Reader) (
	newHeaders []internalapi.Header, newBody []byte, err error,
)

ResponseError implements [TokenizeTranslator.ResponseError]. For OpenAI-based backends we return the OpenAI error type as is. If connection fails the error body is translated to OpenAI error type for events such as HTTP 503 or 504.

func (*ToOpenAITokenize) ResponseHeaders added in v1.1.0 

func (o *ToOpenAITokenize) ResponseHeaders(map[string]string) (newHeaders []internalapi.Header, err error)

ResponseHeaders implements [TokenizeTranslator.ResponseHeaders]. For OpenAI tokenize responses, no header modifications are needed so this returns nil.

type TokenizeTranslator added in v1.1.0 

type TokenizeTranslator = Translator[tokenize.RequestUnion, tracingapi.TokenizeSpan]

TokenizeTranslator translates the tokenize endpoint.

func NewTokenizeToAWSAnthropicTranslator added in v1.1.0 

func NewTokenizeToAWSAnthropicTranslator(apiVersion string, modelNameOverride internalapi.ModelNameOverride) TokenizeTranslator

NewTokenizeToAWSAnthropicTranslator creates a translator for tokenize requests to AWS Bedrock CountTokens API using the InvokeModel format.

func NewTokenizeToAWSBedrockTranslator added in v1.1.0 

func NewTokenizeToAWSBedrockTranslator(modelNameOverride internalapi.ModelNameOverride) TokenizeTranslator

NewTokenizeToAWSBedrockTranslator implements [Factory] for tokenize to AWS Bedrock translation.

func NewTokenizeToGCPAnthropicTranslator added in v1.1.0 

func NewTokenizeToGCPAnthropicTranslator(apiVersion string, modelNameOverride internalapi.ModelNameOverride) TokenizeTranslator

NewTokenizeToGCPAnthropicTranslator implements [Factory] for tokenize to GCP Anthropic translation.

func NewTokenizeToGCPVertexAITranslator added in v1.1.0 

func NewTokenizeToGCPVertexAITranslator(modelNameOverride internalapi.ModelNameOverride) TokenizeTranslator

NewTokenizeToGCPVertexAITranslator implements [Factory] for tokenize to GCP Vertex AI translation.

func NewTokenizeTranslator added in v1.1.0 

func NewTokenizeTranslator(modelNameOverride internalapi.ModelNameOverride) TokenizeTranslator

NewTokenizeTranslator implements [Factory] for OpenAI to OpenAI tokenize translation.

type Translator 

type Translator[ReqT any, SpanT any] interface {
	// RequestBody translates the request body.
	//     - `raw` is the raw request body.
	//     - `body` is the parsed request body of type *ReqT.
	//     - `flag` is a boolean context flag. Depending on the specific implementation,
	//       this represents either `forceBodyMutation` or `onRetry`.
	RequestBody(raw []byte, body *ReqT, flag bool) (
		newHeaders []internalapi.Header,
		mutatedBody []byte,
		err error,
	)

	// ResponseHeaders translates the response headers.
	ResponseHeaders(headers map[string]string) (
		newHeaders []internalapi.Header,
		err error,
	)

	// ResponseBody translates the response body.
	//     - `span` is the tracing span of type SpanT. Implementations that do not
	//       require tracing in this step can ignore this argument.
	ResponseBody(respHeaders map[string]string, body io.Reader, endOfStream bool, span SpanT) (
		newHeaders []internalapi.Header,
		mutatedBody []byte,
		tokenUsage metrics.TokenUsage,
		responseModel internalapi.ResponseModel,
		err error,
	)

	// ResponseError translates the response error (non-2xx status codes).
	ResponseError(respHeaders map[string]string, body io.Reader) (
		newHeaders []internalapi.Header,
		mutatedBody []byte,
		err error,
	)
}

Translator translates the request and response messages between the client and the backend API schemas.

ReqT represents the structured request body type. SpanT represents the tracing span type passed to ResponseBody. Use any if the implementation does not have tracing span yet.

This is created per request and is not thread-safe.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.