schemas

package
v1.7.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 21, 2026 License: Apache-2.0 Imports: 32 Imported by: 34

Documentation

Overview

Package schemas defines the core schemas and types used by the Bifrost system.

Package schemas defines the core schemas and types used by the Bifrost system.

Package schemas defines the core schemas and types used by the Bifrost system.

Package schemas defines the core schemas and types used by the Bifrost system.

Package schemas defines the core schemas and types used by the Bifrost system.

Package schemas defines the core schemas and types used by the Bifrost system.

Package schemas defines the core schemas and types used by the Bifrost system.

Package schemas defines the core schemas and types used by the Bifrost system.

Package schemas defines the core schemas and types used by the Bifrost system.

Package schemas defines the core schemas and types used by the Bifrost system.

Package schemas defines the core schemas and types used by the Bifrost system.

Package schemas defines the core schemas and types used by the Bifrost system.

Index

Constants

View Source 
const (
	// AsyncHeaderResultTTL is the header containing the result TTL for async job retrieval.
	AsyncHeaderResultTTL = "x-bf-async-job-result-ttl"
	// AsyncHeaderCreate is the header that triggers async job creation on integration routes.
	AsyncHeaderCreate = "x-bf-async"
	// AsyncHeaderGetID is the header containing the job ID for async job retrieval on integration routes.
	AsyncHeaderGetID = "x-bf-async-id"
)
View Source 
const (
	RoutingEngineGovernance     = "governance"
	RoutingEngineRoutingRule    = "routing-rule"
	RoutingEngineLoadbalancing  = "loadbalancing"
	RoutingEngineModelCatalog   = "model-catalog"
	RoutingEngineCircuitBreaker = "circuit-breaker"
	// RoutingEngineCore represents the Bifrost core orchestrator's own
	// routing decisions — primarily fallback transitions. Emitted when the
	// primary attempt fails and core advances through the fallback chain so
	// the per-request audit trail closes the loop on what plugin-level
	// engines (governance, loadbalancing, etc.) selected upstream.
	RoutingEngineCore = "core"
)

RoutingEngine constants

View Source 
const (
	RequestCancelled         = "request_cancelled"
	RequestTimedOut          = "request_timed_out"
	RequestDropped           = "request_dropped"
	ProviderConnectionFailed = "provider_connection_failed"
)
View Source 
const (
	MCPAuthRequiredKindOAuth   = "oauth"
	MCPAuthRequiredKindHeaders = "headers"
)

MCPAuthRequiredKind discriminates the kind of inline-401 auth flow surfaced to the caller. The value lands in MCPAuthRequiredError.Kind and on the wire under extra_fields.mcp_auth_required.kind.

View Source 
const (
	DefaultMaxAgentDepth        = 10
	DefaultToolExecutionTimeout = 30 * time.Second
)
View Source 
const (
	PluginStatusActive        = "active"
	PluginStatusError         = "error"
	PluginStatusDisabled      = "disabled"
	PluginStatusLoading       = "loading"
	PluginStatusUninitialized = "uninitialized"
	PluginStatusUnloaded      = "unloaded"
	PluginStatusLoaded        = "loaded"
)

PluginStatus constants

View Source 
const (
	DefaultMaxRetries                 = 0
	DefaultRetryBackoffInitial        = 500 * time.Millisecond
	DefaultRetryBackoffMax            = 5 * time.Second
	DefaultRequestTimeoutInSeconds    = 300
	DefaultMaxConnDurationInSeconds   = 300 // 5 minutes — forces connection recycling to prevent stale connections from NAT/LB silent drops
	DefaultBufferSize                 = 5000
	DefaultConcurrency                = 1000
	DefaultStreamBufferSize           = 256
	DefaultStreamIdleTimeoutInSeconds = 120 // Idle timeout per stream chunk — if no data for this many seconds, bifrost closes the connection
	DefaultKeepAliveTimeoutInSeconds  = 30  // Idle keep-alive for pooled connections — how long an idle connection is kept for reuse before being closed
	DefaultMaxConnsPerHost            = 5000
	MaxConnsPerHostUpperBound         = 10000
	DefaultMaxIdleConnsPerHost        = 40
)
View Source 
const (
	ErrProviderRequestTimedOut      = "" /* 190-byte string literal not displayed */
	ErrRequestCancelled             = "request cancelled by caller"
	ErrRequestBodyConversion        = "failed to convert bifrost request to the expected provider request body"
	ErrProviderRequestMarshal       = "failed to marshal request body to JSON"
	ErrProviderCreateRequest        = "failed to create HTTP request to provider API"
	ErrProviderDoRequest            = "failed to execute HTTP request to provider API"
	ErrProviderNetworkError         = "network error occurred while connecting to provider API (DNS lookup, connection refused, etc.)"
	ErrProviderResponseDecode       = "failed to decode response body from provider API"
	ErrProviderResponseUnmarshal    = "failed to unmarshal response from provider API"
	ErrProviderResponseEmpty        = "empty response received from provider"
	ErrProviderResponseHTML         = "HTML response received from provider"
	ErrProviderRawRequestUnmarshal  = "failed to unmarshal raw request from provider API"
	ErrProviderRawResponseUnmarshal = "failed to unmarshal raw response from provider API"
	ErrProviderResponseDecompress   = "failed to decompress provider's response"
)

Pre-defined errors for provider operations

View Source 
const (
	ResponsesResponseStatusInProgress = "in_progress"
	ResponsesResponseStatusCompleted  = "completed"
	ResponsesResponseStatusIncomplete = "incomplete"
	ResponsesResponseStatusFailed     = "failed"
	ResponsesResponseStatusCancelled  = "cancelled"
	ResponsesResponseStatusQueued     = "queued"
)

ResponsesResponse.Status values (OpenAI Responses API).

View Source 
const (
	ResponsesResponseIncompleteReasonMaxOutputTokens = "max_output_tokens"
	ResponsesResponseIncompleteReasonContentFilter   = "content_filter"
)

ResponsesResponseIncompleteDetails.Reason values.

View Source 
const (
	// TraceAttrSessionID holds the session ID from the x-bf-session-id request
	// header. The key matches the header name because connectors already read it.
	TraceAttrSessionID = "x-bf-session-id"
	// TraceAttrDimensions holds the map[string]string of request dimensions
	// parsed from x-bf-dim-* headers, keyed by bare dimension name.
	TraceAttrDimensions = "bifrost.dimensions"
)

Trace-level attribute keys. Unlike span attributes, trace attributes are never exported as OTEL/Datadog span attributes — observability connectors (BigQuery, Datadog) read them directly off the completed trace.

View Source 
const (
	// Provider and Model Attributes
	AttrProviderName  = "gen_ai.provider.name"
	AttrRequestModel  = "gen_ai.request.model"
	AttrOperationName = "gen_ai.operation.name"

	// Request Parameter Attributes
	AttrMaxTokens        = "gen_ai.request.max_tokens"
	AttrTemperature      = "gen_ai.request.temperature"
	AttrTopP             = "gen_ai.request.top_p"
	AttrStopSequences    = "gen_ai.request.stop_sequences"
	AttrPresencePenalty  = "gen_ai.request.presence_penalty"
	AttrFrequencyPenalty = "gen_ai.request.frequency_penalty"
	AttrParallelToolCall = "gen_ai.request.parallel_tool_calls"
	AttrRequestUser      = "gen_ai.request.user"
	AttrBestOf           = "gen_ai.request.best_of"
	AttrEcho             = "gen_ai.request.echo"
	AttrLogitBias        = "gen_ai.request.logit_bias"
	AttrLogProbs         = "gen_ai.request.logprobs"
	AttrN                = "gen_ai.request.n" // legacy: replaced by AttrChoiceCount
	AttrChoiceCount      = "gen_ai.request.choice.count"
	// AttrEmbeddingsDimensionCount is the OTel spec key for embedding dimensions
	// (Bifrost historically emitted AttrDimensions = gen_ai.request.dimensions).
	AttrEmbeddingsDimensionCount = "gen_ai.embeddings.dimension.count"
	AttrSeed                     = "gen_ai.request.seed"
	AttrSuffix                   = "gen_ai.request.suffix"
	AttrDimensions               = "gen_ai.request.dimensions"      // legacy: replaced by AttrEmbeddingsDimensionCount
	AttrEncodingFormat           = "gen_ai.request.encoding_format" // legacy: singular form; replaced by AttrEncodingFormats (string[])
	AttrEncodingFormats          = "gen_ai.request.encoding_formats"
	AttrLanguage                 = "gen_ai.request.language"
	AttrPrompt                   = "gen_ai.request.prompt"
	AttrResponseFormat           = "gen_ai.request.response_format"
	AttrFormat                   = "gen_ai.request.format"
	AttrVoice                    = "gen_ai.request.voice"
	AttrMultiVoiceConfig         = "gen_ai.request.multi_voice_config"
	AttrInstructions             = "gen_ai.request.instructions"
	AttrSpeed                    = "gen_ai.request.speed"
	AttrMessageCount             = "gen_ai.request.message_count"

	// Response Attributes
	AttrResponseID       = "gen_ai.response.id"
	AttrResponseModel    = "gen_ai.response.model"
	AttrFinishReason     = "gen_ai.response.finish_reason"
	AttrFinishReasons    = "gen_ai.response.finish_reasons"
	AttrSystemFprint     = "gen_ai.response.system_fingerprint"
	AttrServiceTier      = "gen_ai.response.service_tier"
	AttrCreated          = "gen_ai.response.created"
	AttrObject           = "gen_ai.response.object"
	AttrTimeToFirstToken = "gen_ai.response.time_to_first_token" // legacy: nanoseconds; replaced by gen_ai.response.time_to_first_chunk (seconds)
	AttrTimeToFirstChunk = "gen_ai.response.time_to_first_chunk"
	AttrTotalChunks      = "gen_ai.response.total_chunks"

	// Plugin Attributes (for aggregated streaming post-hook spans)
	AttrPluginInvocations     = "plugin.invocation_count"
	AttrPluginAvgDurationMs   = "plugin.avg_duration_ms"
	AttrPluginTotalDurationMs = "plugin.total_duration_ms"
	AttrPluginErrorCount      = "plugin.error_count"

	// Usage Attributes
	// legacy: AttrPromptTokens / AttrCompletionTokens are the deprecated OTel names;
	// new code should use AttrInputTokens / AttrOutputTokens. Kept for dashboards.
	AttrPromptTokens     = "gen_ai.usage.prompt_tokens"
	AttrCompletionTokens = "gen_ai.usage.completion_tokens"
	AttrTotalTokens      = "gen_ai.usage.total_tokens"
	AttrInputTokens      = "gen_ai.usage.input_tokens"
	AttrOutputTokens     = "gen_ai.usage.output_tokens"
	AttrUsageCost        = "gen_ai.usage.cost"
	// OTel GenAI spec keys for cache tokens (flat namespace).
	AttrUsageCacheReadInputTokens     = "gen_ai.usage.cache_read.input_tokens"
	AttrUsageCacheCreationInputTokens = "gen_ai.usage.cache_creation.input_tokens"
	// OTel GenAI spec key for reasoning tokens (flat namespace).
	AttrUsageReasoningOutputTokens = "gen_ai.usage.reasoning.output_tokens"
	// Chat completion usage detail attributes
	// legacy: nested namespace; OTel spec uses flat gen_ai.usage.cache_read.input_tokens
	// and gen_ai.usage.cache_creation.input_tokens for the cached_* entries. The
	// non-cached fields below have no spec equivalent and stay as-is.
	AttrPromptTokenDetailsText          = "gen_ai.usage.prompt_token_details.text_tokens"
	AttrPromptTokenDetailsAudio         = "gen_ai.usage.prompt_token_details.audio_tokens"
	AttrPromptTokenDetailsImage         = "gen_ai.usage.prompt_token_details.image_tokens"
	AttrPromptTokenDetailsCachedRead    = "gen_ai.usage.prompt_token_details.cached_read_tokens"  // legacy: see AttrUsageCacheReadInputTokens
	AttrPromptTokenDetailsCachedWrite   = "gen_ai.usage.prompt_token_details.cached_write_tokens" // legacy: see AttrUsageCacheCreationInputTokens
	AttrPromptTokenDetailsCachedWrite5m = "gen_ai.usage.prompt_token_details.cached_write_tokens_5m"
	AttrPromptTokenDetailsCachedWrite1h = "gen_ai.usage.prompt_token_details.cached_write_tokens_1h"
	AttrCompletionTokenDetailsText      = "gen_ai.usage.completion_token_details.text_tokens"
	AttrCompletionTokenDetailsAudio     = "gen_ai.usage.completion_token_details.audio_tokens"
	AttrCompletionTokenDetailsImage     = "gen_ai.usage.completion_token_details.image_tokens"
	AttrCompletionTokenDetailsReason    = "gen_ai.usage.completion_token_details.reasoning_tokens"
	AttrCompletionTokenDetailsAccept    = "gen_ai.usage.completion_token_details.accepted_prediction_tokens"
	AttrCompletionTokenDetailsReject    = "gen_ai.usage.completion_token_details.rejected_prediction_tokens"
	AttrCompletionTokenDetailsCite      = "gen_ai.usage.completion_token_details.citation_tokens"
	AttrCompletionTokenDetailsSearch    = "gen_ai.usage.completion_token_details.num_search_queries"

	// Error Attributes
	AttrError = "gen_ai.error"
	// legacy: AttrErrorType is the gen_ai.* placement; OTel general semconv uses the
	// unprefixed "error.type". Emitted in parallel from PopulateErrorAttributes.
	AttrErrorType = "gen_ai.error.type"
	AttrErrorCode = "gen_ai.error.code"
	// AttrHTTPResponseStatusCode is the OTel semconv HTTP response status code (e.g. 400).
	// Sourced from BifrostError.StatusCode; used as the status_code dimension on error metrics.
	AttrHTTPResponseStatusCode = "http.response.status_code"

	// Input/Output Attributes
	AttrInputText      = "gen_ai.input.text"
	AttrInputMessages  = "gen_ai.input.messages"
	AttrInputSpeech    = "gen_ai.input.speech"
	AttrInputEmbedding = "gen_ai.input.embedding"
	AttrOutputMessages = "gen_ai.output.messages"

	// Bifrost Context Attributes
	// legacy: every key below sits under gen_ai.* but represents a Bifrost-internal
	// concept (governance / routing). The bifrost.* mirrors are the canonical home
	// going forward; these will be dropped once dashboards migrate.
	AttrRequestID       = "gen_ai.request_id"
	AttrVirtualKeyID    = "gen_ai.virtual_key_id"
	AttrVirtualKeyName  = "gen_ai.virtual_key_name"
	AttrSelectedKeyID   = "gen_ai.selected_key_id"
	AttrSelectedKeyName = "gen_ai.selected_key_name"
	AttrRoutingRuleID   = "gen_ai.routing_rule_id"
	AttrRoutingRuleName = "gen_ai.routing_rule_name"
	AttrTeamID          = "gen_ai.team_id"
	AttrTeamName        = "gen_ai.team_name"
	AttrCustomerID      = "gen_ai.customer_id"
	AttrCustomerName    = "gen_ai.customer_name"
	AttrNumberOfRetries = "gen_ai.number_of_retries"
	AttrFallbackIndex   = "gen_ai.fallback_index"

	// Extra Header Attributes
	AttrExtraHeaderPrefix = "gen_ai.request.extra_header."

	// Responses API Request Attributes
	AttrPromptCacheKey      = "gen_ai.request.prompt_cache_key"
	AttrReasoningEffort     = "gen_ai.request.reasoning_effort"
	AttrReasoningSummary    = "gen_ai.request.reasoning_summary"
	AttrReasoningGenSummary = "gen_ai.request.reasoning_generate_summary"
	AttrSafetyIdentifier    = "gen_ai.request.safety_identifier"
	AttrStore               = "gen_ai.request.store"
	AttrTextVerbosity       = "gen_ai.request.text_verbosity"
	AttrTextFormatType      = "gen_ai.request.text_format_type"
	AttrTopLogProbs         = "gen_ai.request.top_logprobs"
	AttrToolChoiceType      = "gen_ai.request.tool_choice_type"
	AttrToolChoiceName      = "gen_ai.request.tool_choice_name"
	AttrTools               = "gen_ai.request.tools"
	AttrTruncation          = "gen_ai.request.truncation"

	// Responses API Response Attributes
	AttrRespInclude          = "gen_ai.responses.include"
	AttrRespMaxOutputTokens  = "gen_ai.responses.max_output_tokens"
	AttrRespMaxToolCalls     = "gen_ai.responses.max_tool_calls"
	AttrRespMetadata         = "gen_ai.responses.metadata"
	AttrRespPreviousRespID   = "gen_ai.responses.previous_response_id"
	AttrRespPromptCacheKey   = "gen_ai.responses.prompt_cache_key"
	AttrRespReasoningText    = "gen_ai.responses.reasoning"
	AttrRespReasoningEffort  = "gen_ai.responses.reasoning_effort"
	AttrRespReasoningGenSum  = "gen_ai.responses.reasoning_generate_summary"
	AttrRespSafetyIdentifier = "gen_ai.responses.safety_identifier"
	AttrRespStore            = "gen_ai.responses.store"
	AttrRespTemperature      = "gen_ai.responses.temperature"
	AttrRespTextVerbosity    = "gen_ai.responses.text_verbosity"
	AttrRespTextFormatType   = "gen_ai.responses.text_format_type"
	AttrRespTopLogProbs      = "gen_ai.responses.top_logprobs"
	AttrRespTopP             = "gen_ai.responses.top_p"
	AttrRespToolChoiceType   = "gen_ai.responses.tool_choice_type"
	AttrRespToolChoiceName   = "gen_ai.responses.tool_choice_name"
	AttrRespTruncation       = "gen_ai.responses.truncation"
	AttrRespTools            = "gen_ai.responses.tools"

	// Batch Operation Attributes
	AttrBatchID             = "gen_ai.batch.id"
	AttrBatchStatus         = "gen_ai.batch.status"
	AttrBatchObject         = "gen_ai.batch.object"
	AttrBatchEndpoint       = "gen_ai.batch.endpoint"
	AttrBatchInputFileID    = "gen_ai.batch.input_file_id"
	AttrBatchOutputFileID   = "gen_ai.batch.output_file_id"
	AttrBatchErrorFileID    = "gen_ai.batch.error_file_id"
	AttrBatchCompletionWin  = "gen_ai.batch.completion_window"
	AttrBatchCreatedAt      = "gen_ai.batch.created_at"
	AttrBatchExpiresAt      = "gen_ai.batch.expires_at"
	AttrBatchRequestsCount  = "gen_ai.batch.requests_count"
	AttrBatchDataCount      = "gen_ai.batch.data_count"
	AttrBatchResultsCount   = "gen_ai.batch.results_count"
	AttrBatchHasMore        = "gen_ai.batch.has_more"
	AttrBatchMetadata       = "gen_ai.batch.metadata"
	AttrBatchLimit          = "gen_ai.batch.limit"
	AttrBatchAfter          = "gen_ai.batch.after"
	AttrBatchBeforeID       = "gen_ai.batch.before_id"
	AttrBatchAfterID        = "gen_ai.batch.after_id"
	AttrBatchPageToken      = "gen_ai.batch.page_token"
	AttrBatchPageSize       = "gen_ai.batch.page_size"
	AttrBatchCountTotal     = "gen_ai.batch.request_counts.total"
	AttrBatchCountCompleted = "gen_ai.batch.request_counts.completed"
	AttrBatchCountFailed    = "gen_ai.batch.request_counts.failed"
	AttrBatchFirstID        = "gen_ai.batch.first_id"
	AttrBatchLastID         = "gen_ai.batch.last_id"
	AttrBatchInProgressAt   = "gen_ai.batch.in_progress_at"
	AttrBatchFinalizingAt   = "gen_ai.batch.finalizing_at"
	AttrBatchCompletedAt    = "gen_ai.batch.completed_at"
	AttrBatchFailedAt       = "gen_ai.batch.failed_at"
	AttrBatchExpiredAt      = "gen_ai.batch.expired_at"
	AttrBatchCancellingAt   = "gen_ai.batch.cancelling_at"
	AttrBatchCancelledAt    = "gen_ai.batch.cancelled_at"
	AttrBatchNextCursor     = "gen_ai.batch.next_cursor"

	// Transcription Response Attributes
	AttrInputTokenDetailsText  = "gen_ai.usage.input_token_details.text_tokens"
	AttrInputTokenDetailsAudio = "gen_ai.usage.input_token_details.audio_tokens"

	// Responses API usage detail attributes
	AttrInputTokenDetailsImage         = "gen_ai.usage.input_token_details.image_tokens"
	AttrInputTokenDetailsCachedRead    = "gen_ai.usage.input_token_details.cached_read_tokens"
	AttrInputTokenDetailsCachedWrite   = "gen_ai.usage.input_token_details.cached_write_tokens"
	AttrInputTokenDetailsCachedWrite5m = "gen_ai.usage.input_token_details.cached_write_tokens_5m"
	AttrInputTokenDetailsCachedWrite1h = "gen_ai.usage.input_token_details.cached_write_tokens_1h"
	AttrOutputTokenDetailsText         = "gen_ai.usage.output_token_details.text_tokens"
	AttrOutputTokenDetailsAudio        = "gen_ai.usage.output_token_details.audio_tokens"
	AttrOutputTokenDetailsImage        = "gen_ai.usage.output_token_details.image_tokens"
	AttrOutputTokenDetailsReason       = "gen_ai.usage.output_token_details.reasoning_tokens"
	AttrOutputTokenDetailsAccept       = "gen_ai.usage.output_token_details.accepted_prediction_tokens"
	AttrOutputTokenDetailsReject       = "gen_ai.usage.output_token_details.rejected_prediction_tokens"
	AttrOutputTokenDetailsCite         = "gen_ai.usage.output_token_details.citation_tokens"
	AttrOutputTokenDetailsSearch       = "gen_ai.usage.output_token_details.num_search_queries"

	// Tool execution attributes (OTel GenAI spec) used on MCP tool spans.
	AttrToolName          = "gen_ai.tool.name"
	AttrToolCallID        = "gen_ai.tool.call.id"
	AttrToolCallArguments = "gen_ai.tool.call.arguments"
	AttrToolCallResult    = "gen_ai.tool.call.result"
	AttrToolType          = "gen_ai.tool.type"

	// OTel MCP semconv attributes on mcp.client spans, read by the duration metric.
	AttrMCPMethodName    = "mcp.method.name"   // e.g. tools/call, tools/list, ping
	AttrNetworkTransport = "network.transport" // pipe (stdio) | tcp (http/sse)

	// Tool-execution latency (ms) — the raw CallTool round-trip — so the duration metric
	// measures it, not span wall-time (which covers the PostHooks). Bifrost-namespaced; not
	// OTel MCP semconv.
	AttrBifrostMCPToolDurationMs = "bifrost.mcp.tool.duration_ms"

	// =====================================================================
	// Bifrost-namespaced attributes (bifrost.*)
	//
	// Canonical home for everything that is NOT part of the OTel GenAI spec:
	//   - Bifrost-internal concepts (routing/governance, request id, retry counters)
	//   - Raw Bifrost short names that mirror canonicalized gen_ai.* values
	//   - Back-compat fallbacks for shape changes (e.g. comma-joined stop_sequences)
	//
	// The corresponding legacy gen_ai.* emissions are tagged "// legacy:" at their
	// call sites and will be removed once dashboards migrate over.
	// =====================================================================
	AttrBifrostProviderName        = "bifrost.provider.name"
	AttrBifrostRequestID           = "bifrost.request.id"
	AttrBifrostVirtualKeyID        = "bifrost.virtual_key.id"
	AttrBifrostVirtualKeyName      = "bifrost.virtual_key.name"
	AttrBifrostSelectedKeyID       = "bifrost.selected_key.id"
	AttrBifrostSelectedKeyName     = "bifrost.selected_key.name"
	AttrBifrostRoutingRuleID       = "bifrost.routing_rule.id"
	AttrBifrostRoutingRuleName     = "bifrost.routing_rule.name"
	AttrBifrostTeamID              = "bifrost.team.id"
	AttrBifrostTeamName            = "bifrost.team.name"
	AttrBifrostCustomerID          = "bifrost.customer.id"
	AttrBifrostCustomerName        = "bifrost.customer.name"
	AttrBifrostBusinessUnitID      = "bifrost.business_unit.id"
	AttrBifrostBusinessUnitName    = "bifrost.business_unit.name"
	AttrBifrostTeamIDs             = "bifrost.team.ids"
	AttrBifrostTeamNames           = "bifrost.team.names"
	AttrBifrostCustomerIDs         = "bifrost.customer.ids"
	AttrBifrostCustomerNames       = "bifrost.customer.names"
	AttrBifrostBusinessUnitIDs     = "bifrost.business_unit.ids"
	AttrBifrostBusinessUnitNames   = "bifrost.business_unit.names"
	AttrBifrostUserID              = "bifrost.user.id"
	AttrBifrostUserName            = "bifrost.user.name"
	AttrBifrostUserEmail           = "bifrost.user.email"
	AttrBifrostRetries             = "bifrost.retries"
	AttrBifrostFallbackIndex       = "bifrost.fallback_index"
	AttrBifrostAlias               = "bifrost.alias"               // original requested model when it differs from the resolved model
	AttrBifrostRoutingEngineUsed   = "bifrost.routing_engine_used" // comma-joined routing engines that handled the request
	AttrBifrostStopSequencesJoined = "bifrost.request.stop_sequences"

	// OTel general semconv (no gen_ai prefix). Emitted alongside the legacy
	// gen_ai.error.type from PopulateErrorAttributes.
	AttrErrorTypeSpec = "error.type"

	// legacy: bare unprefixed keys retained for back-compat with existing dashboards.
	// "request.type" is superseded by AttrOperationName; "retry.count" has no spec
	// equivalent but stays under bifrost.retries going forward.
	AttrLegacyRequestType = "request.type"
	AttrLegacyRetryCount  = "retry.count"

	// File Operation Attributes
	AttrFileID             = "gen_ai.file.id"
	AttrFileObject         = "gen_ai.file.object"
	AttrFileFilename       = "gen_ai.file.filename"
	AttrFilePurpose        = "gen_ai.file.purpose"
	AttrFileBytes          = "gen_ai.file.bytes"
	AttrFileCreatedAt      = "gen_ai.file.created_at"
	AttrFileStatus         = "gen_ai.file.status"
	AttrFileStorageBackend = "gen_ai.file.storage_backend"
	AttrFileDataCount      = "gen_ai.file.data_count"
	AttrFileHasMore        = "gen_ai.file.has_more"
	AttrFileDeleted        = "gen_ai.file.deleted"
	AttrFileContentType    = "gen_ai.file.content_type"
	AttrFileContentBytes   = "gen_ai.file.content_bytes"
	AttrFileLimit          = "gen_ai.file.limit"
	AttrFileAfter          = "gen_ai.file.after"
	AttrFileOrder          = "gen_ai.file.order"
)

LLM Attribute Keys (gen_ai.* namespace) These follow the OpenTelemetry semantic conventions for GenAI and are compatible with both OTEL and Datadog backends.

View Source 
const (
	DefaultWSMaxIdlePerKey                = 50
	DefaultWSMaxTotalConnections          = 1000
	DefaultWSIdleTimeoutSeconds           = 600
	DefaultWSMaxConnectionLifetimeSeconds = 7200
	DefaultWSMaxConnections               = 100
	DefaultWSTranscriptBufferSize         = 100
)

Default pool configuration values (set high for production workloads)

View Source 
const (
	DefaultInitialPoolSize = 5000
)
View Source 
const (
	// DefaultLargePayloadRequestThresholdBytes is the default request-size heuristic
	// used by transport guards when no enterprise threshold is present on context.
	DefaultLargePayloadRequestThresholdBytes = 10 * 1024 * 1024 // 10MB
)
View Source 
const DefaultPageSize = 1000

DefaultPageSize is the default page size for listing models

View Source 
const (
	DefaultSessionStickyTTL = time.Hour
)
View Source 
const DefaultVideoDuration = "8"

DefaultVideoDuration is the default video duration in seconds for Gemini/Vertex when not specified.

View Source 
const MCPAuthTempTokenReminder = "" /* 221-byte string literal not displayed */

MCPAuthTempTokenReminder is appended to MCPAuthRequiredError.Message when the auth URL carries a `#t=<token>` temp-token fragment (see MCPAuthURLHasTempTokenFragment). The fragment is deliberately never sent to the server (unlike a query param, it's not logged or forwarded as a Referer), but that also makes it easy for an LLM relaying the link to a human to mistake it for a non-essential anchor and drop it, breaking the link. Spelling this out in the message itself is the cheapest way to stop that from happening.

View Source 
const MaxPaginationRequests = 20

MaxPaginationRequests is the maximum number of pagination requests to make

View Source 
const OTelOperationNameExecuteTool = "execute_tool"

OTelOperationNameExecuteTool is the gen_ai.operation.name value for MCP tool executions. execute_tool is an MCPRequestType, not a Bifrost RequestType, so it can't flow through OTelOperationName.

View Source 
const RedactedAttrValue = "REDACTED"

RedactedAttrValue is the placeholder recorded in place of a sensitive header value, following the OpenTelemetry HTTP semantic-convention guidance for redacting credentials.

View Source 
const ResponsesToolTypeOpenRouterPrefix = "openrouter:"

ResponsesToolTypeOpenRouterPrefix is the namespace prefix for OpenRouter server tools (e.g. "openrouter:web_search", "openrouter:web_fetch", "openrouter:datetime", "openrouter:image_generation", "openrouter:apply_patch", "openrouter:subagent"). These are executed server-side by OpenRouter and are not part of the OpenAI spec.

Variables

View Source 
var (
	ErrOAuth2ConfigNotFound       = errors.New("oauth2 config not found")
	ErrOAuth2ProviderNotAvailable = errors.New("oauth2 provider not available")
	ErrOAuth2TokenExpired         = errors.New("oauth2 token expired")
	ErrOAuth2TokenInvalid         = errors.New("oauth2 token invalid")
	ErrOAuth2RefreshFailed        = errors.New("oauth2 token refresh failed")
	ErrOAuth2NotPerUserSession    = errors.New("state does not match a per-user oauth session")
	ErrOAuth2TokenNotFound        = errors.New("per-user oauth token not found for this identity and mcp server")
	ErrOAuth2FlowNotPending       = errors.New("oauth flow is not in pending state")
	ErrOAuth2FlowExpired          = errors.New("oauth flow has expired")
	// ErrMCPReconnectNotApplicable signals that the reconnect operation is not
	// meaningful for this client type — e.g. per-user OAuth clients, where
	// each user manages their own auth and there is no shared upstream
	// connection to "reconnect". Distinct from "not implemented".
	ErrMCPReconnectNotApplicable = errors.New("reconnect is not applicable for this client type")
)

OAuth-related errors

View Source 
var (
	// ErrHeadersCredentialProviderNotAvailable signals that the headers
	// provider isn't wired up — typically a misconfiguration (per_user_headers
	// auth type used while running without a configstore-backed provider).
	ErrHeadersCredentialProviderNotAvailable = errors.New("per-user headers credential provider not available")

	// ErrHeadersCredentialNotFound is the sentinel returned by
	// MCPHeadersProvider.GetCredentialByMode when no row exists for the
	// (mode, identity, mcp_client) triple. The resolver fans this out into an
	// inline MCPAuthRequiredError so the caller can complete the submission
	// flow.
	ErrHeadersCredentialNotFound = errors.New("per-user headers credential not found for this identity and mcp client")

	// ErrHeadersCredentialNeedsUpdate signals that the stored credential is
	// stale relative to the current MCPClientConfig.PerUserHeaderKeys schema
	// (e.g. admin added a new required key). The resolver treats this like
	// "not found" for inline-401 purposes but the row is preserved so the UI
	// can prefill known values.
	ErrHeadersCredentialNeedsUpdate = errors.New("per-user headers credential is missing keys required by the current schema")
)

Per-user-headers errors. Mirrors the OAuth sentinels at the top of mcp.go; kept in this file so the headers feature surface is self-contained.

View Source 
var (
	// ClaudeCLI — Anthropic Claude Code / Claude CLI (identifiers vary by release).
	ClaudeCLI   = UserAgentIdentifiers{"claude-cli", "claude-code", "claude-vscode"}
	GeminiCLI   = UserAgentIdentifiers{"geminicli"}
	CodexCLI    = UserAgentIdentifiers{"codex-tui"}
	QwenCodeCLI = UserAgentIdentifiers{"qwencode"}
	OpenCode    = UserAgentIdentifiers{"opencode"}
	Cursor      = UserAgentIdentifiers{"cursor"}
)
View Source 
var DefaultConcurrencyAndBufferSize = ConcurrencyAndBufferSize{
	Concurrency: DefaultConcurrency,
	BufferSize:  DefaultBufferSize,
}

DefaultConcurrencyAndBufferSize is the default concurrency and buffer size for provider operations.

View Source 
var DefaultNetworkConfig = NetworkConfig{
	DefaultRequestTimeoutInSeconds: DefaultRequestTimeoutInSeconds,
	MaxRetries:                     DefaultMaxRetries,
	RetryBackoffInitial:            DefaultRetryBackoffInitial,
	RetryBackoffMax:                DefaultRetryBackoffMax,
	StreamIdleTimeoutInSeconds:     DefaultStreamIdleTimeoutInSeconds,
	KeepAliveTimeoutInSeconds:      DefaultKeepAliveTimeoutInSeconds,
	MaxConnsPerHost:                DefaultMaxConnsPerHost,
}

DefaultNetworkConfig is the default network configuration for provider connections.

View Source 
var EnrichmentDims = []EnrichmentDim{

	{Name: "provider", SpanAttr: AttrBifrostProviderName, MetricSafe: true},
	{Name: "model", SpanAttr: AttrRequestModel, MetricSafe: true},
	{Name: "method", Column: "request_type", SpanAttr: AttrLegacyRequestType, MetricSafe: true},

	{Name: "alias", SpanAttr: AttrBifrostAlias, MetricSafe: true},
	{Name: "routing_engine_used", SpanAttr: AttrBifrostRoutingEngineUsed, MetricSafe: true},
	{Name: "virtual_key_id", SpanAttr: AttrBifrostVirtualKeyID, MetricSafe: true},
	{Name: "virtual_key_name", SpanAttr: AttrBifrostVirtualKeyName, MetricSafe: true},
	{Name: "selected_key_id", SpanAttr: AttrBifrostSelectedKeyID, MetricSafe: true},
	{Name: "selected_key_name", SpanAttr: AttrBifrostSelectedKeyName, MetricSafe: true},
	{Name: "routing_rule_id", SpanAttr: AttrBifrostRoutingRuleID, MetricSafe: true},
	{Name: "routing_rule_name", SpanAttr: AttrBifrostRoutingRuleName, MetricSafe: true},
	{Name: "team_id", SpanAttr: AttrBifrostTeamID, MetricSafe: true},
	{Name: "team_name", SpanAttr: AttrBifrostTeamName, MetricSafe: true},
	{Name: "customer_id", SpanAttr: AttrBifrostCustomerID, MetricSafe: true},
	{Name: "customer_name", SpanAttr: AttrBifrostCustomerName, MetricSafe: true},
	{Name: "business_unit_id", SpanAttr: AttrBifrostBusinessUnitID, MetricSafe: true},
	{Name: "business_unit_name", SpanAttr: AttrBifrostBusinessUnitName, MetricSafe: true},
	{Name: "fallback_index", SpanAttr: AttrBifrostFallbackIndex, MetricSafe: true},

	{Name: "user_id", SpanAttr: AttrBifrostUserID},
	{Name: "user_name", SpanAttr: AttrBifrostUserName},
	{Name: "user_email", SpanAttr: AttrBifrostUserEmail},
	{Name: "team_ids", SpanAttr: AttrBifrostTeamIDs, Multi: true},
	{Name: "team_names", SpanAttr: AttrBifrostTeamNames, Multi: true},
	{Name: "customer_ids", SpanAttr: AttrBifrostCustomerIDs, Multi: true},
	{Name: "customer_names", SpanAttr: AttrBifrostCustomerNames, Multi: true},
	{Name: "business_unit_ids", SpanAttr: AttrBifrostBusinessUnitIDs, Multi: true},
	{Name: "business_unit_names", SpanAttr: AttrBifrostBusinessUnitNames, Multi: true},
}

EnrichmentDims is the canonical, ordered registry of identity/context dimensions. Order is stable so derived lists (labels/tags/columns) are deterministic. Add a dimension here once and every curated connector picks it up via its derivation + conformance test.

View Source 
var ErrUnsatisfiableSchema = errors.New("json schema is the boolean schema 'false', which no output can satisfy")

ErrUnsatisfiableSchema is returned when a request carries the boolean JSON Schema `false`, which no value can satisfy.

View Source 
var NoDeadline time.Time
View Source 
var StandardProviders = []ModelProvider{
	Anthropic,
	Azure,
	Bedrock,
	BedrockMantle,
	Cerebras,
	Cohere,
	DeepSeek,
	Gemini,
	Groq,
	Mistral,
	Ollama,
	OpencodeGo,
	OpencodeZen,
	OpenAI,
	Parasail,
	Perplexity,
	SGL,
	Vertex,
	OpenRouter,
	Elevenlabs,
	HuggingFace,
	Nebius,
	XAI,
	Replicate,
	VLLM,
	Runway,
	Runware,
	Fireworks,
	Sarvam,
	Wafer,
}

StandardProviders is the list of all built-in (non-custom) providers.

View Source 
var SupportedBaseProviders = []ModelProvider{
	Anthropic,
	Bedrock,
	Cohere,
	Gemini,
	OpenAI,
	HuggingFace,
	Replicate,
}

SupportedBaseProviders is the list of base providers allowed for custom providers.

View Source 
var VaultPrefixHook func() string

VaultPrefixHook returns the configured vault path prefix (e.g. "bifrost"). It is nil in OSS deployments; VaultPrefix() falls back to "bifrost".

View Source 
var VaultRemoveHook func(ctx context.Context, path string) error

VaultRemoveHook deletes the secret at path (best-effort; errors are ignored by callers). It is nil in OSS deployments.

View Source 
var VaultResolveHook func(ctx context.Context, value *string) error

VaultResolveHook is wired by enterprise startup to the vault registry's ResolveString. It is nil in OSS deployments; GetValue() is a no-op when nil.

View Source 
var VaultStoreHook func(ctx context.Context, path string, value *string) error

VaultStoreHook stores a plaintext secret at path and rewrites *value to a "vault.<canonical>" reference. It is wired by enterprise startup to the vault registry's StoreString and is nil in OSS deployments (store helpers no-op).

Functions

func AppendToContextList added in v1.4.3 

func AppendToContextList[T comparable](ctx *BifrostContext, key BifrostContextKey, value T)

AppendToContextList appends value to the context list at key, skipping the append when value already exists in the list. Downstream consumers of these lists (notably `routing_engines_used` → Prometheus labels) treat duplicate entries as bugs, so set semantics are enforced at the write site.

func ApplyLiteralReplacements added in v1.6.4 

func ApplyLiteralReplacements(text string, replacements map[string]string) string

ApplyLiteralReplacements performs deterministic best-effort string redaction.

func BaseModelName added in v1.2.33 

func BaseModelName(id string) string

BaseModelName returns the model id with any recognized version suffix stripped.

This is your "model name without version".

func BedrockModelSupportsCachePoints added in v1.5.13 

func BedrockModelSupportsCachePoints(model string) bool

BedrockModelSupportsCachePoints reports whether the Bedrock model supports explicit prompt-caching cache points in the Converse API request.

func BedrockModelSupportsExtendedCacheTTL added in v1.5.22 

func BedrockModelSupportsExtendedCacheTTL(model string) bool

BedrockModelSupportsExtendedCacheTTL reports whether the Bedrock model supports the 1h (extended) prompt-caching TTL. This is Anthropic-only; other cache-point models (e.g. Nova) support caching but only the default 5m TTL and 400 with "Extended TTL prompt caching is only supported for Anthropic models".

func Compact added in v1.4.7 

func Compact(dst *bytes.Buffer, src []byte) error

Compact removes insignificant whitespace from JSON-encoded src and appends the result to dst.

func ConvertViaJSON added in v1.4.15 

func ConvertViaJSON[T any](src interface{}) (T, error)

ConvertViaJSON converts src to type T via JSON round-trip using sorted marshaling. Use as fallback when direct type assertion fails (e.g., map[string]interface{} from JSON).

func DeepCopy added in v1.2.26 

func DeepCopy(in interface{}) interface{}

func EncodeSerialCursor added in v1.2.39 

func EncodeSerialCursor(cursor *SerialCursor) string

EncodeSerialCursor encodes a SerialCursor to a base64 string for transport.

func EnrichmentDimColumnNames added in v1.6.4 

func EnrichmentDimColumnNames() []string

EnrichmentDimColumnNames returns the BigQuery column name for every dimension, in registry order (the record/trace-tier set).

func EnrichmentDimNames added in v1.6.4 

func EnrichmentDimNames() []string

EnrichmentDimNames returns every dimension name, in registry order (the record/trace-tier set).

func ExtractAndSetUserAgentFromHeaders added in v1.5.3 

func ExtractAndSetUserAgentFromHeaders(headers map[string][]string, bifrostCtx *BifrostContext)

func ExtractTopLevelKeyOrder added in v1.4.4 

func ExtractTopLevelKeyOrder(data []byte) []string

ExtractTopLevelKeyOrder parses a JSON object and returns its top-level keys in document order. Useful for capturing key order before struct deserialization loses it, so that re-serialization can preserve the original order.

func FilterHeaders added in v1.5.17 

func FilterHeaders(headers map[string]string, patterns []string) map[string]string

FilterHeaders returns the subset of headers whose (lowercased) keys match any of the given patterns (exact name or wildcard like "x-custom-*" or "*"). Header keys are expected to already be lowercased by the capture layer. Returns nil when nothing matches.

func GetRandomString added in v1.3.10 

func GetRandomString(length int) string

GetRandomString generates a random alphanumeric string of the given length.

func GroupPluginLogsByName added in v1.5.0 

func GroupPluginLogsByName(logs []PluginLogEntry) map[string][]PluginLogEntry

GroupPluginLogsByName groups a flat slice of plugin log entries by plugin name. Returns nil if the input is empty.

func IsAllDigitsASCII added in v1.2.23 

func IsAllDigitsASCII(s string) bool

IsAllDigitsASCII checks if a string contains only ASCII digits (0-9).

func IsAnthropicModel added in v1.2.32 

func IsAnthropicModel(model string) bool

IsAnthropicModel checks if the model is an Anthropic model.

func IsAnthropicModelFamily added in v1.5.19 

func IsAnthropicModelFamily(ctx *BifrostContext, model string) bool

IsAnthropicModelFamily reports whether the current attempt resolves to the Anthropic model family. Thin wrapper over ResolveFamily so provider code reads uniformly at the many call sites that branch on Anthropic vs non-Anthropic (request shape, response parsing, anthropic-version header, URL path construction). model is passed as the substring-match fallback used when no alias is resolved in ctx — typically request.Model.

func IsAzureModelRouter added in v1.7.3 

func IsAzureModelRouter(model string) bool

IsAzureModelRouter reports whether model is Azure's model-router model.

func IsCohereModel added in v1.5.19 

func IsCohereModel(model string) bool

IsCohereModel checks if the model is a Cohere model. Matches the Bedrock identifier prefix ("cohere.embed-*", "cohere.command-*") which is the wire shape that flows through alias resolution.

func IsCohereModelFamily added in v1.5.19 

func IsCohereModelFamily(ctx *BifrostContext, model string) bool

IsCohereModelFamily reports whether the current attempt resolves to the Cohere model family. Used by Bedrock to pick the Cohere request/response shape for embeddings (vs. the Titan envelope).

func IsContentAttribute added in v1.6.4 

func IsContentAttribute(key string) bool

IsContentAttribute reports whether a span attribute may carry user or model content.

func IsDiarizedTranscriptionFormat added in v1.6.4 

func IsDiarizedTranscriptionFormat(format *string) bool

IsDiarizedTranscriptionFormat returns true if the given response format produces speaker-diarized segments (OpenAI's response_format=diarized_json, used by models like gpt-4o-transcribe-diarize).

func IsElevenlabsSoundModel added in v1.6.4 

func IsElevenlabsSoundModel(model string) bool

IsElevenlabsSoundModel checks if the model targets ElevenLabs' text-to-sound effects API (POST /v1/sound-generation, e.g. "eleven_text_to_sound_v2") rather than text-to-speech. These models are not tied to a voice.

func IsElevenlabsSoundModelFamily added in v1.6.4 

func IsElevenlabsSoundModelFamily(ctx *BifrostContext, model string) bool

IsElevenlabsSoundModelFamily reports whether the current attempt resolves to an ElevenLabs sound-effects (text-to-sound) model. It honors aliases by resolving the canonical model name first, so an alias whose ModelName/ModelID is a sound model is detected the same as a raw model id. See IsAnthropicModelFamily for usage notes.

func IsGATranscriptionSessionBody added in v1.6.4 

func IsGATranscriptionSessionBody(root map[string]json.RawMessage) bool

IsGATranscriptionSessionBody reports whether root represents a GA transcription-only session (RealtimeTranscriptionSessionCreateRequestGA) rather than a full realtime session. This is the single canonical classifier shared by extraction (above) and both normalization layers (transports/bifrost-http/handlers and core/providers/openai) so they cannot diverge on the same input — each layer previously ran its own ad hoc classification logic, which could disagree.

Priority (checked in order, first match wins):

  1. explicit session.type is present -> authoritative, since it's the schema's actual discriminator (oneOf RealtimeSessionCreateRequestGA / RealtimeTranscriptionSessionCreateRequestGA): "transcription" -> true, anything else -> false, regardless of what other fields are also present (e.g. a stray session.model alongside type=="transcription" is invalid input the normalizer should clean up, not a signal to reclassify the session).
  2. no explicit type: session.model present -> full realtime session (false).
  3. no explicit type: legacy top-level "model" present -> full realtime session (false). Checked before any transcription-shape inspection so a full session that also enables live input-audio transcription as a sibling feature is never misclassified.
  4. no explicit type, no model anywhere: session.audio.input.transcription is present and non-null -> transcription session (true); otherwise false.

func IsGLMModel added in v1.6.0 

func IsGLMModel(model string) bool

IsGLMModel checks if the model is a Z.AI GLM model.

func IsGeminiModel added in v1.2.37 

func IsGeminiModel(model string) bool

func IsGeminiModelFamily added in v1.5.19 

func IsGeminiModelFamily(ctx *BifrostContext, model string) bool

IsGeminiModelFamily reports whether the current attempt resolves to the Google Gemini model family. Used by Vertex to pick Gemini-shaped request transforms and the publishers/google URL prefix.

func IsGemmaModel added in v1.5.6 

func IsGemmaModel(model string) bool

func IsGemmaModelFamily added in v1.5.19 

func IsGemmaModelFamily(ctx *BifrostContext, model string) bool

IsGemmaModelFamily reports whether the current attempt resolves to the Gemma model family. Vertex routes Gemma via the publishers/google path alongside Gemini.

func IsGrokModel added in v1.6.3 

func IsGrokModel(model string) bool

IsGrokModel checks if the model is an xAI Grok model.

func IsGrokReasoningModel added in v1.3.6 

func IsGrokReasoningModel(model string) bool

IsGrokReasoningModel checks if the given model is a grok reasoning model

func IsImagenModel added in v1.3.9 

func IsImagenModel(model string) bool

IsImagenModel checks if the model is an Imagen model.

func IsImagenModelFamily added in v1.5.19 

func IsImagenModelFamily(ctx *BifrostContext, model string) bool

IsImagenModelFamily reports whether the current attempt resolves to the Imagen model family. Used by Vertex for the :predict endpoint and Imagen- specific request shaping.

func IsKnownProvider added in v1.4.1 

func IsKnownProvider(provider string) bool

IsKnownProvider checks if a provider string is known.

func IsLlamaModel added in v1.5.8 

func IsLlamaModel(model string) bool

IsLlamaModel checks if the model is a Meta Llama model.

Used by the Bedrock provider to gate tool_choice handling: Bedrock Converse rejects toolConfig.toolChoice.tool on Meta Llama variants with HTTP 400 ("This model doesn't support the toolConfig.toolChoice.tool field"). See AWS docs for the per-model tool_choice support matrix: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ToolChoice.html

func IsLlamaModelFamily added in v1.5.19 

func IsLlamaModelFamily(ctx *BifrostContext, model string) bool

IsLlamaModelFamily reports whether the current attempt resolves to the Llama model family. Used by Bedrock to gate tool_choice handling — AWS Bedrock Converse rejects toolConfig.toolChoice.tool on Meta Llama variants.

func IsMistralModel added in v1.2.32 

func IsMistralModel(model string) bool

IsMistralModel checks if the model is a Mistral or Codestral model.

func IsMistralModelFamily added in v1.5.19 

func IsMistralModelFamily(ctx *BifrostContext, model string) bool

IsMistralModelFamily reports whether the current attempt resolves to the Mistral model family. See IsAnthropicModelFamily for usage notes.

func IsNova2Model added in v1.5.11 

func IsNova2Model(model string) bool

func IsNovaModel added in v1.2.39 

func IsNovaModel(model string) bool

IsNovaModel checks if the model is a Nova model.

func IsNovaModelFamily added in v1.5.19 

func IsNovaModelFamily(ctx *BifrostContext, model string) bool

IsNovaModelFamily reports whether the current attempt resolves to the Amazon Nova model family. Used by Bedrock to gate cache-point insertion and tool shaping that differs from Anthropic.

func IsOpenAIModel added in v1.6.3 

func IsOpenAIModel(model string) bool

IsOpenAIModel checks if the model is an OpenAI model.

func IsOpenAIModelFamily added in v1.6.3 

func IsOpenAIModelFamily(ctx *BifrostContext, model string) bool

func IsPlainTextTranscriptionFormat added in v1.5.0 

func IsPlainTextTranscriptionFormat(format *string) bool

IsPlainTextTranscriptionFormat returns true if the given response format produces a plain-text response body (not JSON).

func IsRealtimeConversationItemEventType added in v1.5.1 

func IsRealtimeConversationItemEventType(eventType RealtimeEventType) bool

IsRealtimeConversationItemEventType reports whether the event carries a canonical conversation item payload after provider translation.

func IsRealtimeInputTranscriptEvent added in v1.5.1 

func IsRealtimeInputTranscriptEvent(event *BifrostRealtimeEvent) bool

IsRealtimeInputTranscriptEvent reports whether the event carries a finalized input-audio transcript in the canonical Bifrost realtime schema.

func IsRealtimeToolOutputEvent added in v1.5.1 

func IsRealtimeToolOutputEvent(event *BifrostRealtimeEvent) bool

IsRealtimeToolOutputEvent reports whether the event represents a finalized tool output item in the canonical Bifrost realtime schema.

func IsRealtimeUserInputEvent added in v1.5.1 

func IsRealtimeUserInputEvent(event *BifrostRealtimeEvent) bool

IsRealtimeUserInputEvent reports whether the event represents a finalized user input item in the canonical Bifrost realtime schema.

func IsSecretRef added in v1.6.3 

func IsSecretRef(value string) bool

IsSecretRef reports whether value is a secret reference (env.* or vault.* prefix, or a JSON-encoded SecretVar with an env/vault type) without resolving it. Use this instead of NewSecretVar(...).IsFromSecret() when resolution side-effects (vault HTTP calls, env lookups) must be avoided.

func IsSensitiveHeader added in v1.5.13 

func IsSensitiveHeader(name string) bool

IsSensitiveHeader reports whether a header name carries credentials that must not be exported verbatim into span attributes. The match is case-insensitive and trims surrounding whitespace so callers using the core SDK directly (which bypass the transport-layer security denylist) are still protected. Beyond the well-known exact names, substring/suffix patterns catch credential-bearing variants like x-auth-token, x-amz-security-token, and provider-specific *-api-key headers.

func IsTitanModel added in v1.5.19 

func IsTitanModel(model string) bool

IsTitanModel checks if the model is an Amazon Titan model. Matches the Bedrock identifier prefix ("amazon.titan-*").

func IsTitanModelFamily added in v1.5.19 

func IsTitanModelFamily(ctx *BifrostContext, model string) bool

IsTitanModelFamily reports whether the current attempt resolves to the Amazon Titan model family. Used by Bedrock to pick the Titan embedding request/response envelope.

func IsVeoModel added in v1.4.4 

func IsVeoModel(model string) bool

func IsVeoModelFamily added in v1.5.19 

func IsVeoModelFamily(ctx *BifrostContext, model string) bool

IsVeoModelFamily reports whether the current attempt resolves to the Veo model family. Used by Vertex for video-generation request shaping.

func JsonifyInput added in v1.2.0 

func JsonifyInput(input interface{}) string

JsonifyInput converts an interface{} to a JSON string

func LookupVault added in v1.6.0 

func LookupVault(ref string) (string, bool)

LookupVault resolves a vault reference string (e.g. "vault.path/to/secret") via the registered resolver, returning the resolved secret and true on success — analogous to os.LookupEnv. Returns ("", false) when ref doesn't have the "vault." prefix or no resolver is registered (OSS deployments / before enterprise startup).

func MCPAuthURLHasTempTokenFragment added in v1.6.4 

func MCPAuthURLHasTempTokenFragment(authURL string) bool

MCPAuthURLHasTempTokenFragment reports whether authURL carries the `#t=` temp-token fragment minted by InitiateUserOAuthFlow / InitiateUserSubmissionFlow. Callers use this to decide whether MCPAuthTempTokenReminder applies — the mint is best-effort (see those functions' docs), so the fragment isn't always present even when MCPEnableTempTokenAuth is on.

func Marshal added in v1.3.4 

func Marshal(v interface{}) ([]byte, error)

Marshal encodes v to JSON bytes using the high-performance sonic library.

func MarshalDeeplySorted added in v1.4.7 

func MarshalDeeplySorted(v interface{}) ([]byte, error)

MarshalDeeplySorted encodes v to JSON with all map keys sorted alphabetically, including nested maps inside OrderedMap and other custom types with MarshalJSON. This ensures fully deterministic output for hashing/caching purposes.

Unlike MarshalSorted which relies on sonic's SortMapKeys (which doesn't affect types with custom MarshalJSON like OrderedMap), this function first normalizes the entire structure to plain maps, then marshals with sorted keys.

func MarshalSorted added in v1.4.7 

func MarshalSorted(v interface{}) ([]byte, error)

MarshalSorted encodes v to JSON with map keys sorted alphabetically. Use this when deterministic output is needed (e.g., hashing, caching keys). Uses sonic.ConfigStd which has SortMapKeys enabled.

func MarshalSortedIndent added in v1.4.15 

func MarshalSortedIndent(v interface{}, prefix, indent string) ([]byte, error)

MarshalSortedIndent encodes v to indented JSON with map keys sorted alphabetically.

func MarshalString added in v1.3.4 

func MarshalString(v interface{}) (string, error)

MarshalString encodes v to a JSON string using sonic.

func MatchHeaderPattern added in v1.5.17 

func MatchHeaderPattern(headerName, pattern string) bool

MatchHeaderPattern reports whether a lowercased header name matches a pattern. Supports exact match, "*" (all), and a single trailing-wildcard prefix (e.g. "x-custom-*"). The pattern is trimmed and lowercased before comparison; the header name is expected to already be lowercased by the caller.

func MetricSafeEnrichmentDimNames added in v1.6.4 

func MetricSafeEnrichmentDimNames() []string

MetricSafeEnrichmentDimNames returns the names of the metric-tier dimensions, in registry order.

func OTelOperationName added in v1.5.13 

func OTelOperationName(rt RequestType) string

OTelOperationName maps a Bifrost RequestType to the value that should be emitted under gen_ai.operation.name. Values not modeled by the spec fall through to the raw RequestType string.

func OTelProviderName added in v1.5.13 

func OTelProviderName(p ModelProvider) string

OTelProviderName maps a Bifrost ModelProvider to the value that should be emitted under gen_ai.provider.name. Providers not covered by the spec keep their Bifrost short name.

func ParseFlexibleDuration added in v1.5.6 

func ParseFlexibleDuration(data json.RawMessage, fieldName string) (time.Duration, error)

ParseFlexibleDuration parses a JSON token (raw bytes) as a time.Duration.

Supported formats:

  • JSON string "5s", "500ms", "1m30s" — forwarded to time.ParseDuration
  • JSON integer (e.g. 5000000000) — treated as nanoseconds, identical to the default Go JSON encoding of time.Duration

fieldName is used only in error messages (e.g. "context_timeout").

func PluginNameFromSpan added in v1.5.19 

func PluginNameFromSpan(span *Span) string

PluginNameFromSpan extracts "<name>" from a plugin span whose name follows the core tracer contract "plugin.<name>.<stage>", where <stage> is one of prehook, posthook, prerequesthook, mcp_prehook, mcp_posthook, mcp_connect_prehook, or mcp_connect_posthook (see core/bifrost.go). It returns "" for non-plugin spans or names that don't match the contract (wrong prefix, or fewer than three segments), so malformed names pass through ShouldExportSpan as exported rather than being silently filtered.

The <stage> segment is intentionally not constrained to a fixed list: the tracer emits several hook stages (including the mcp_* variants above), so pinning it to just prehook/posthook would make every MCP-hook span unfilterable.

func Ptr added in v1.2.0 

func Ptr[T any](v T) *T

Ptr creates a pointer to any value. This is a helper function for creating pointers to values.

func RedactAttributeValue added in v1.6.4 

func RedactAttributeValue(value any, replacements map[string]string) any

RedactAttributeValue applies literal replacements to supported attribute value shapes.

func RegisterKnownProvider added in v1.4.1 

func RegisterKnownProvider(provider ModelProvider)

RegisterKnownProvider adds a provider to the known providers set. This allows ParseModelString to correctly parse model strings with custom provider prefixes (e.g., "my-custom-provider/gpt-4").

func ReleaseChatToResponsesStreamState added in v1.2.23 

func ReleaseChatToResponsesStreamState(state *ChatToResponsesStreamState)

ReleaseChatToResponsesStreamState returns a ChatToResponsesStreamState to the pool.

func ReleaseHTTPRequest added in v1.3.4 

func ReleaseHTTPRequest(req *HTTPRequest)

ReleaseHTTPRequest returns an HTTPRequest to the pool. The HTTPRequest is reset before being returned to the pool. Do not use the HTTPRequest after calling this function.

func ReleaseHTTPResponse added in v1.3.9 

func ReleaseHTTPResponse(resp *HTTPResponse)

ReleaseHTTPResponse returns an HTTPResponse to the pool. The HTTPResponse is reset before being returned to the pool. Do not use the HTTPResponse after calling this function.

func RemoveOwnedVaultSecretVars added in v1.6.0 

func RemoveOwnedVaultSecretVars(ctx context.Context, ownedPrefix string, model interface{}) []error

RemoveOwnedVaultSecretVars best-effort deletes the vault secret for every SecretVar / *SecretVar field in model whose VaultRef starts with ownedPrefix+"/". Refs outside that prefix are user-provided and are left alone. Returns one error per field whose deletion failed; callers should log these but must not treat them as fatal (the DB row is already deleted).

func ReorderJSONKeys added in v1.4.4 

func ReorderJSONKeys(data []byte, order []string) ([]byte, error)

ReorderJSONKeys takes serialized JSON and a desired key order, and returns the same JSON with top-level keys reordered. Keys present in `order` are emitted first in that order; any remaining keys follow in their original order. This is a general-purpose utility for preserving client-specified key order through struct serialization/deserialization round-trips.

func ResolveCanonicalModel added in v1.5.22 

func ResolveCanonicalModel(ctx *BifrostContext, fallbackModel string) string

ResolveCanonicalModel returns the model string that capability/version gating should run against for the current attempt, walking the alias hierarchy: canonical ModelName → wire ModelID → fallbackModel.

Precedence here means "first present tier wins" — a present ModelName is authoritative and we do not fall through to ModelID on it. The ModelName tier is what makes Claude-on-Azure work: there the wire ModelID is an opaque deployment id, but the admin-configured ModelName carries the real "claude-opus-4-8" string the substring checks need.

When no alias is resolved in ctx, fallbackModel (typically request.Model) is returned unchanged, preserving pre-refactor behavior.

func SafeExtractBool added in v1.2.0 

func SafeExtractBool(value interface{}) (bool, bool)

SafeExtractBool safely extracts a bool value from an interface{} with type checking

func SafeExtractBoolPointer added in v1.2.0 

func SafeExtractBoolPointer(value interface{}) (*bool, bool)

SafeExtractBoolPointer safely extracts a *bool value from an interface{} with type checking

func SafeExtractFloat64 added in v1.2.0 

func SafeExtractFloat64(value interface{}) (float64, bool)

SafeExtractFloat64 safely extracts a float64 value from an interface{} with type checking

func SafeExtractFloat64Pointer added in v1.2.0 

func SafeExtractFloat64Pointer(value interface{}) (*float64, bool)

SafeExtractFloat64Pointer safely extracts a *float64 value from an interface{} with type checking

func SafeExtractFromMap added in v1.2.0 

func SafeExtractFromMap(m map[string]interface{}, key string) (interface{}, bool)

SafeExtractFromMap safely extracts a value from a map[string]interface{} with type checking

func SafeExtractInt added in v1.2.0 

func SafeExtractInt(value interface{}) (int, bool)

SafeExtractInt safely extracts an int value from an interface{} with type checking

func SafeExtractIntPointer added in v1.2.0 

func SafeExtractIntPointer(value interface{}) (*int, bool)

SafeExtractIntPointer safely extracts an *int value from an interface{} with type checking

func SafeExtractString added in v1.2.0 

func SafeExtractString(value interface{}) (string, bool)

SafeExtractString safely extracts a string value from an interface{} with type checking

func SafeExtractStringMap added in v1.3.9 

func SafeExtractStringMap(value interface{}) (map[string]string, bool)

SafeExtractStringMap safely extracts a map[string]string from an interface{} with type checking. Handles both direct map[string]string and JSON-deserialized map[string]interface{} cases.

func SafeExtractStringPointer added in v1.2.0 

func SafeExtractStringPointer(value interface{}) (*string, bool)

SafeExtractStringPointer safely extracts a *string value from an interface{} with type checking

func SafeExtractStringSlice added in v1.2.0 

func SafeExtractStringSlice(value interface{}) ([]string, bool)

SafeExtractStringSlice safely extracts a []string value from an interface{} with type checking

func SameBaseModel added in v1.2.33 

func SameBaseModel(a, b string) bool

SameBaseModel reports whether two model ids refer to the same base model, ignoring any recognized version suffixes.

This works even if both sides are versioned, or both unversioned.

func SanitizeImageURL added in v1.2.0 

func SanitizeImageURL(rawURL string) (string, error)

SanitizeImageURL sanitizes and normalizes an image URL. It handles both data URLs and regular URLs, accepting only http/https for non-data URLs. Callers that need to accept provider-specific schemes (e.g. gs://) must use SanitizeImageURLWithAllowedSchemes.

func SanitizeImageURLWithAllowedSchemes added in v1.6.3 

func SanitizeImageURLWithAllowedSchemes(rawURL string, allowedSchemes ...string) (string, error)

SanitizeImageURLWithAllowedSchemes sanitizes and normalizes an image URL, then validates regular URL schemes against the target provider's allowlist. Passing an empty list is treated as "no non-data URL is acceptable" — call SanitizeImageURL if you want the default http/https policy.

func SanitizePluginSpanName added in v1.5.19 

func SanitizePluginSpanName(name string) string

SanitizePluginSpanName normalizes a plugin's name into the form embedded in its span names.

func SecretVarAsString added in v1.6.0 

func SecretVarAsString(e *SecretVar) string

SecretVarAsString returns the wire form used when serializing *SecretVar as a string.

func SetRedactionDataOnContext added in v1.6.4 

func SetRedactionDataOnContext(ctx *BifrostContext, data RedactionData) bool

SetRedactionDataOnContext stores non-empty redaction data on ctx.

func SplitModelAndVersion added in v1.2.33 

func SplitModelAndVersion(id string) (base, version string)

SplitModelAndVersion splits a model id into (base, versionSuffix). If no known version suffix is found, versionSuffix will be empty and base will be the original id.

Examples: 

"claude-sonnet-4"                 -> ("claude-sonnet-4", "")
"claude-sonnet-4-20250514"        -> ("claude-sonnet-4", "20250514")
"gpt-4.1-2024-09-12"              -> ("gpt-4.1", "2024-09-12")
"gpt-4.1-mini-2024-09-12"         -> ("gpt-4.1-mini", "2024-09-12")
"some-model-v2"                   -> ("some-model", "v2")
"text-embedding-3-large-beta"     -> ("text-embedding-3-large", "beta")
"claude-sonnet-4.5"               -> ("claude-sonnet-4.5", "")

func StoreOwnedVaultSecretVars added in v1.6.0 

func StoreOwnedVaultSecretVars(ctx context.Context, basePath string, model interface{}) error

StoreOwnedVaultSecretVars stores every plaintext SecretVar / *SecretVar struct field of model into the vault under basePath/<column>, converting each to a vault ref. The reflection walk mirrors RemoveOwnedVaultSecretVars.

func StoreVaultSecretVar added in v1.6.0 

func StoreVaultSecretVar(ctx context.Context, path string, e *SecretVar) error

StoreVaultSecretVar pushes a single plaintext SecretVar value into the vault at path and converts the field to a vault reference. No-op when vault disabled, field is nil, env/vault-sourced, empty, or redacted.

func Unmarshal added in v1.3.4 

func Unmarshal(data []byte, v interface{}) error

Unmarshal decodes JSON data into v using sonic.

func UnregisterKnownProvider added in v1.4.1 

func UnregisterKnownProvider(provider ModelProvider)

UnregisterKnownProvider removes a custom provider from the known providers set. Standard providers cannot be unregistered.

func VaultBasePath added in v1.6.0 

func VaultBasePath(tableName, primaryKey string) string

VaultBasePath returns the standard vault path prefix for a table row.

func VaultPrefix added in v1.6.0 

func VaultPrefix() string

VaultPrefix returns the configured vault path prefix, defaulting to "bifrost".

func VaultStoreWriteEnabled added in v1.6.0 

func VaultStoreWriteEnabled() bool

VaultStoreWriteEnabled reports whether vault write storage is available (i.e. VaultStoreHook has been wired by enterprise startup). Use this to guard StoreOwnedVaultSecretVars / RemoveOwnedVaultSecretVars calls in BeforeSave hooks, since those calls in BeforeSave hooks.

Types

type Account 

type Account interface {
	// GetConfiguredProviders returns a list of providers that are configured
	// in the account. This is used to determine which providers are available for use.
	GetConfiguredProviders() ([]ModelProvider, error)

	// GetKeysForProvider returns the API keys configured for a specific provider.
	// The keys include their values, supported models, and weights for load balancing.
	// The context can carry data from any source that sets values before the Bifrost request,
	// including but not limited to plugin pre-hooks, application logic, or any in app middleware sharing the context.
	// This enables dynamic key selection based on any context values present during the request.
	GetKeysForProvider(ctx context.Context, providerKey ModelProvider) ([]Key, error)

	// GetConfigForProvider returns the configuration for a specific provider.
	// This includes network settings, authentication details, and other provider-specific
	// configurations.
	GetConfigForProvider(providerKey ModelProvider) (*ProviderConfig, error)
}

Account defines the interface for managing provider accounts and their configurations. It provides methods to access provider-specific settings, API keys, and configurations.

type AdditionalPropertiesStruct added in v1.3.9 

type AdditionalPropertiesStruct struct {
	AdditionalPropertiesBool *bool
	AdditionalPropertiesMap  *OrderedMap
}

func (AdditionalPropertiesStruct) MarshalJSON added in v1.3.9 

func (a AdditionalPropertiesStruct) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshalling for AdditionalPropertiesStruct. It marshals either AdditionalPropertiesBool or AdditionalPropertiesMap based on which is set.

func (*AdditionalPropertiesStruct) UnmarshalJSON added in v1.3.9 

func (a *AdditionalPropertiesStruct) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshalling for AdditionalPropertiesStruct. It handles both boolean and object types for additionalProperties.

type AliasConfig added in v1.5.19 

type AliasConfig struct {
	ModelID     string       `json:"model_id"`               // wire model identifier sent to the provider
	ModelName   *string      `json:"model_name,omitempty"`   // canonical model name used for pricing, logging, and 2nd-tier family routing
	ModelFamily *ModelFamily `json:"model_family,omitempty"` // 1st-tier family routing enum
	Description string       `json:"description,omitempty"`  // description of the alias for users to understand its purpose (not used by bifrost)
	Region      *SecretVar   `json:"region,omitempty"`
	// ProjectID is a per-alias project override shared across providers (like Region).
	// Vertex uses it as the GCP project; Bedrock and Bedrock Mantle use it as the
	// AWS project sent via the OpenAI-Project / anthropic-workspace-id header. Kept
	// top-level (rather than inside each provider sub-config) so the flat "project_id"
	// JSON key does not collide between embedded sub-configs — Go/sonic silently drop
	// a field name shared by multiple same-depth anonymous structs.
	ProjectID             *SecretVar `json:"project_id,omitempty"`
	UseAnthropicEndpoints *bool      `json:"use_anthropic_endpoints,omitempty"` // Whether to use anthropic endpoints for this alias

	*AzureAliasCfg
	*VertexAliasCfg
	*BedrockAliasCfg
	*ReplicateAliasCfg
}

AliasConfig is the rich value type held by KeyAliases. It carries everything needed to call a provider for an aliased model: the wire model identifier (ModelID), the canonical model name used for pricing/logging (ModelName), the family used for provider routing decisions (ModelFamily), and optional provider-specific overrides that override the key-level defaults.

func (AliasConfig) MarshalJSON added in v1.5.19 

func (ac AliasConfig) MarshalJSON() ([]byte, error)

MarshalJSON emits the legacy string wire shape when only ModelID is set, so callers that haven't opted into the rich AliasConfig see no observable change on the wire. When any other field is populated, the full object is emitted.

type AllowedRequests added in v1.1.26 

type AllowedRequests struct {
	ListModels            bool `json:"list_models"`
	TextCompletion        bool `json:"text_completion"`
	TextCompletionStream  bool `json:"text_completion_stream"`
	ChatCompletion        bool `json:"chat_completion"`
	ChatCompletionStream  bool `json:"chat_completion_stream"`
	Responses             bool `json:"responses"`
	ResponsesStream       bool `json:"responses_stream"`
	ResponsesRetrieve     bool `json:"responses_retrieve"`
	ResponsesDelete       bool `json:"responses_delete"`
	ResponsesCancel       bool `json:"responses_cancel"`
	ResponsesInputItems   bool `json:"responses_input_items"`
	CountTokens           bool `json:"count_tokens"`
	Compaction            bool `json:"compaction"`
	Embedding             bool `json:"embedding"`
	Rerank                bool `json:"rerank"`
	OCR                   bool `json:"ocr"`
	Speech                bool `json:"speech"`
	SpeechStream          bool `json:"speech_stream"`
	Transcription         bool `json:"transcription"`
	TranscriptionStream   bool `json:"transcription_stream"`
	ImageGeneration       bool `json:"image_generation"`
	ImageGenerationStream bool `json:"image_generation_stream"`
	ImageEdit             bool `json:"image_edit"`
	ImageEditStream       bool `json:"image_edit_stream"`
	ImageVariation        bool `json:"image_variation"`
	VideoGeneration       bool `json:"video_generation"`
	VideoRetrieve         bool `json:"video_retrieve"`
	VideoDownload         bool `json:"video_download"`
	VideoDelete           bool `json:"video_delete"`
	VideoList             bool `json:"video_list"`
	VideoRemix            bool `json:"video_remix"`
	BatchCreate           bool `json:"batch_create"`
	BatchList             bool `json:"batch_list"`
	BatchRetrieve         bool `json:"batch_retrieve"`
	BatchCancel           bool `json:"batch_cancel"`
	BatchDelete           bool `json:"batch_delete"`
	BatchResults          bool `json:"batch_results"`
	FileUpload            bool `json:"file_upload"`
	FileList              bool `json:"file_list"`
	FileRetrieve          bool `json:"file_retrieve"`
	FileDelete            bool `json:"file_delete"`
	FileContent           bool `json:"file_content"`
	ContainerCreate       bool `json:"container_create"`
	ContainerList         bool `json:"container_list"`
	ContainerRetrieve     bool `json:"container_retrieve"`
	ContainerDelete       bool `json:"container_delete"`
	ContainerFileCreate   bool `json:"container_file_create"`
	ContainerFileList     bool `json:"container_file_list"`
	ContainerFileRetrieve bool `json:"container_file_retrieve"`
	ContainerFileContent  bool `json:"container_file_content"`
	ContainerFileDelete   bool `json:"container_file_delete"`
	Passthrough           bool `json:"passthrough"`
	PassthroughStream     bool `json:"passthrough_stream"`
	WebSocketResponses    bool `json:"websocket_responses"`
	Realtime              bool `json:"realtime"`
	CachedContentCreate   bool `json:"cached_content_create"`
	CachedContentList     bool `json:"cached_content_list"`
	CachedContentRetrieve bool `json:"cached_content_retrieve"`
	CachedContentUpdate   bool `json:"cached_content_update"`
	CachedContentDelete   bool `json:"cached_content_delete"`
}

AllowedRequests controls which operations are permitted. A nil *AllowedRequests means "all operations allowed." A non-nil value only allows fields set to true; omitted or false fields are disallowed.

func (*AllowedRequests) IsOperationAllowed added in v1.1.26 

func (ar *AllowedRequests) IsOperationAllowed(operation RequestType) bool

IsOperationAllowed checks if a specific operation is allowed

type Architecture added in v1.2.14 

type Architecture struct {
	Modality         *string  `json:"modality,omitempty"`
	Tokenizer        *string  `json:"tokenizer,omitempty"`
	InstructType     *string  `json:"instruct_type,omitempty"`
	InputModalities  []string `json:"input_modalities,omitempty"`
	OutputModalities []string `json:"output_modalities,omitempty"`
}

type AsyncJobResponse added in v1.4.4 

type AsyncJobResponse struct {
	ID          string         `json:"id"`
	RequestID   string         `json:"request_id,omitempty"`
	Status      AsyncJobStatus `json:"status"`
	ExpiresAt   *time.Time     `json:"expires_at,omitempty"`
	CreatedAt   time.Time      `json:"created_at"`
	CompletedAt *time.Time     `json:"completed_at,omitempty"`
	StatusCode  int            `json:"status_code,omitempty"`
	Result      interface{}    `json:"result,omitempty"`
	Error       *BifrostError  `json:"error,omitempty"`
}

AsyncJobResponse is the JSON response returned when creating or polling an async job

type AsyncJobStatus added in v1.4.4 

type AsyncJobStatus string

AsyncJobStatus represents the status of an async job 

const (
	AsyncJobStatusPending    AsyncJobStatus = "pending"
	AsyncJobStatusProcessing AsyncJobStatus = "processing"
	AsyncJobStatusCompleted  AsyncJobStatus = "completed"
	AsyncJobStatusFailed     AsyncJobStatus = "failed"
)

func (AsyncJobStatus) Value added in v1.6.3 

func (s AsyncJobStatus) Value() (driver.Value, error)

Value implements driver.Valuer so database drivers that append typed column values (e.g. clickhouse-go batch inserts) can serialize the type.

type AzureAliasCfg added in v1.5.19 

type AzureAliasCfg struct {
	APIVersion       *string    `json:"api_version,omitempty"`       // overrides the Azure OpenAI api-version query param for this alias
	AnthropicVersion *string    `json:"anthropic_version,omitempty"` // overrides the anthropic-version header for Claude-on-Azure deployments
	Endpoint         *SecretVar `json:"endpoint,omitempty"`          // overrides AzureKeyConfig.Endpoint for this alias — lets one credential span deployments on multiple Azure resources
}

AzureAliasCfg holds Azure-specific overrides that apply to a single alias. Each field, when non-nil, overrides the corresponding key-level default.

type AzureAuthType added in v1.4.5 

type AzureAuthType string
const (
	AzureAuthTypeClientSecret    AzureAuthType = "client_secret"
	AzureAuthTypeManagedIdentity AzureAuthType = "managed_identity"
)

type AzureKeyConfig added in v1.1.9 

type AzureKeyConfig struct {
	Endpoint SecretVar `json:"endpoint"` // Azure service endpoint URL

	ClientID     *SecretVar `json:"client_id,omitempty"`     // Azure client ID for authentication
	ClientSecret *SecretVar `json:"client_secret,omitempty"` // Azure client secret for authentication
	TenantID     *SecretVar `json:"tenant_id,omitempty"`     // Azure tenant ID for authentication
	Scopes       []string   `json:"scopes,omitempty"`
}

AzureKeyConfig represents the Azure-specific configuration. It contains Azure-specific settings required for service access and deployment management.

type BasePlugin added in v1.4.0 

type BasePlugin interface {
	// GetName returns the name of the plugin.
	GetName() string

	// Cleanup is called on bifrost shutdown.
	// It allows plugins to clean up any resources they have allocated.
	// Returns any error that occurred during cleanup, which will be logged as a warning by the Bifrost instance.
	Cleanup() error
}

type BatchEndpoint added in v1.2.38 

type BatchEndpoint string

BatchEndpoint represents supported batch API endpoints. 

const (
	BatchEndpointChatCompletions BatchEndpoint = "/v1/chat/completions"
	BatchEndpointEmbeddings      BatchEndpoint = "/v1/embeddings"
	BatchEndpointCompletions     BatchEndpoint = "/v1/completions"
	BatchEndpointResponses       BatchEndpoint = "/v1/responses"
	BatchEndpointMessages        BatchEndpoint = "/v1/messages" // Anthropic
)

type BatchError added in v1.2.38 

type BatchError struct {
	Code    string `json:"code,omitempty"`
	Message string `json:"message,omitempty"`
	Param   string `json:"param,omitempty"`
	Line    *int   `json:"line,omitempty"`
}

BatchError represents a single error in batch processing.

type BatchErrors added in v1.2.38 

type BatchErrors struct {
	Object string       `json:"object,omitempty"`
	Data   []BatchError `json:"data,omitempty"`
}

BatchErrors represents errors encountered during batch processing.

type BatchExpiresAfter added in v1.4.0 

type BatchExpiresAfter struct {
	Anchor  string `json:"anchor"`  // e.g., "created_at"
	Seconds int    `json:"seconds"` // 3600-2592000 (1 hour to 30 days)
}

BatchExpiresAfter represents an expiration configuration for batch output.

type BatchOutputFolder added in v1.5.9 

type BatchOutputFolder struct {
	URL string `json:"url"`
}

type BatchRequestCounts added in v1.2.38 

type BatchRequestCounts struct {
	Total     int `json:"total"`
	Completed int `json:"completed"`
	Failed    int `json:"failed"`
	Succeeded int `json:"succeeded,omitempty"` // Anthropic-specific
	Expired   int `json:"expired,omitempty"`   // Anthropic-specific
	Canceled  int `json:"canceled,omitempty"`  // Anthropic-specific
	Pending   int `json:"pending,omitempty"`   // Anthropic-specific
}

BatchRequestCounts tracks the counts of requests in different states.

type BatchRequestItem added in v1.2.38 

type BatchRequestItem struct {
	CustomID string                 `json:"custom_id"`        // User-provided unique ID for this request
	Method   string                 `json:"method,omitempty"` // HTTP method (typically "POST")
	URL      string                 `json:"url,omitempty"`    // Endpoint URL (e.g., "/v1/chat/completions")
	Body     map[string]interface{} `json:"body,omitempty"`   // Request body parameters
	Params   map[string]interface{} `json:"params,omitempty"` // Alternative to Body for Anthropic
}

BatchRequestItem represents a single request in a batch (for inline requests).

type BatchResultData added in v1.2.38 

type BatchResultData struct {
	Type    string                 `json:"type"` // "succeeded", "errored", "expired", "canceled"
	Message map[string]interface{} `json:"message,omitempty"`
}

BatchResultData represents Anthropic-style result data.

type BatchResultError added in v1.2.38 

type BatchResultError struct {
	Code    string `json:"code,omitempty"`
	Message string `json:"message,omitempty"`
}

BatchResultError represents an error for a single batch request.

type BatchResultItem added in v1.2.38 

type BatchResultItem struct {
	CustomID string `json:"custom_id"`

	// Result data (varies by request type)
	Response *BatchResultResponse `json:"response,omitempty"` // OpenAI format
	Result   *BatchResultData     `json:"result,omitempty"`   // Anthropic format

	// Error if the individual request failed
	Error *BatchResultError `json:"error,omitempty"`
}

BatchResultItem represents a single result from a batch request.

type BatchResultResponse added in v1.2.38 

type BatchResultResponse struct {
	StatusCode int                    `json:"status_code"`
	RequestID  string                 `json:"request_id,omitempty"`
	Body       map[string]interface{} `json:"body,omitempty"`
}

BatchResultResponse represents OpenAI-style result response.

type BatchS3Config added in v1.2.39 

type BatchS3Config struct {
	Buckets []S3BucketConfig `json:"buckets,omitempty"` // List of S3 bucket configurations
}

BatchS3Config holds S3 bucket configurations for Bedrock batch operations. Supports multiple buckets to allow flexible batch job routing.

type BatchStatus added in v1.2.38 

type BatchStatus string

BatchStatus represents the status of a batch job. 

const (
	BatchStatusValidating BatchStatus = "validating"
	BatchStatusFailed     BatchStatus = "failed"
	BatchStatusInProgress BatchStatus = "in_progress"
	BatchStatusFinalizing BatchStatus = "finalizing"
	BatchStatusCompleted  BatchStatus = "completed"
	BatchStatusExpired    BatchStatus = "expired"
	BatchStatusCancelling BatchStatus = "cancelling"
	BatchStatusCancelled  BatchStatus = "cancelled"
	BatchStatusEnded      BatchStatus = "ended"   // Anthropic-specific
	BatchStatusDeleted    BatchStatus = "deleted" // Gemini-specific
)

type BedrockAliasCfg added in v1.5.19 

type BedrockAliasCfg struct {
	InferenceProfileARN *SecretVar `json:"inference_profile_arn,omitempty"`
}

BedrockAliasCfg holds Bedrock-specific overrides that apply to a single alias.

type BedrockKeyConfig added in v1.1.14 

type BedrockKeyConfig struct {
	AccessKey    SecretVar  `json:"access_key,omitempty"`    // AWS access key for authentication
	SecretKey    SecretVar  `json:"secret_key,omitempty"`    // AWS secret access key for authentication
	SessionToken *SecretVar `json:"session_token,omitempty"` // AWS session token for temporary credentials
	Region       *SecretVar `json:"region,omitempty"`        // AWS region for service access
	ARN          *SecretVar `json:"arn,omitempty"`           // Amazon Resource Name for resource identification
	// IAM role for STS AssumeRole
	RoleARN         *SecretVar `json:"role_arn,omitempty"`
	ExternalID      *SecretVar `json:"external_id,omitempty"`
	RoleSessionName *SecretVar `json:"session_name,omitempty"`

	// ProjectID scopes the Bedrock Mantle sub-surface (OpenAI-compatible gpt-*/Gemma routing and the
	// mantle catalog merge in ListModels) to a specific Bedrock project via the "OpenAI-Project"
	// header. When empty, AWS routes to the account's default project. It has no effect on the
	// Converse/bedrock-runtime paths, which are not project-scoped.
	ProjectID *SecretVar `json:"project_id,omitempty"`

	BatchS3Config *BatchS3Config `json:"batch_s3_config,omitempty"` // S3 bucket configuration for batch operations
}

BedrockKeyConfig represents the AWS Bedrock-specific configuration. It contains AWS-specific settings required for authentication and service access.

type BedrockMantleKeyConfig added in v1.6.3 

type BedrockMantleKeyConfig struct {
	AccessKey    SecretVar  `json:"access_key,omitempty"`    // AWS access key for SigV4 authentication
	SecretKey    SecretVar  `json:"secret_key,omitempty"`    // AWS secret access key for SigV4 authentication
	SessionToken *SecretVar `json:"session_token,omitempty"` // AWS session token for temporary credentials
	Region       *SecretVar `json:"region,omitempty"`        // AWS region used to build the bedrock-mantle endpoint host
	// IAM role for STS AssumeRole
	RoleARN         *SecretVar `json:"role_arn,omitempty"`
	ExternalID      *SecretVar `json:"external_id,omitempty"`
	RoleSessionName *SecretVar `json:"session_name,omitempty"`

	// ProjectID scopes inference and model listing to a specific Bedrock project. It is sent as the
	// "OpenAI-Project" header on the OpenAI-compatible surface and the "anthropic-workspace-id"
	// header on the native-Anthropic (Claude) surface. When empty, AWS routes to the account's
	// default project.
	ProjectID *SecretVar `json:"project_id,omitempty"`
}

BedrockMantleKeyConfig represents the Bedrock Mantle-specific configuration. Mantle serves Claude (native-Anthropic Messages), OpenAI-compatible, and Gemma models on the bedrock-mantle.{region}.api.aws host. It carries only the credentials and region the mantle endpoints use; it intentionally omits the inference-profile ARN and batch S3 config, which apply only to the Converse/bedrock-runtime surface.

type BifrostBatchCancelRequest added in v1.2.38 

type BifrostBatchCancelRequest struct {
	Provider ModelProvider `json:"provider"`
	Model    *string       `json:"model"`
	BatchID  string        `json:"batch_id"` // ID of the batch to cancel

	RawRequestBody []byte `json:"-"` // Raw request body (not serialized)

	// Extra parameters for provider-specific features
	ExtraParams map[string]interface{} `json:"-"`
}

BifrostBatchCancelRequest represents a request to cancel a batch job.

func (*BifrostBatchCancelRequest) GetRawRequestBody added in v1.2.38 

func (request *BifrostBatchCancelRequest) GetRawRequestBody() []byte

GetRawRequestBody returns the raw request body.

type BifrostBatchCancelResponse added in v1.2.38 

type BifrostBatchCancelResponse struct {
	ID            string             `json:"id"`
	Object        string             `json:"object,omitempty"`
	Status        BatchStatus        `json:"status"`
	RequestCounts BatchRequestCounts `json:"request_counts,omitempty"`
	CancellingAt  *int64             `json:"cancelling_at,omitempty"`
	CancelledAt   *int64             `json:"cancelled_at,omitempty"`

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostBatchCancelResponse represents the response from cancelling a batch job.

type BifrostBatchCreateRequest added in v1.2.38 

type BifrostBatchCreateRequest struct {
	Provider       ModelProvider `json:"provider"`
	Model          *string       `json:"model,omitempty"` // Model hint for routing (optional for file-based) it may or may not present depending on the provider and usage of integration vs direct API
	RawRequestBody []byte        `json:"-"`               // Raw request body (not serialized)

	// OpenAI-style: file-based batching
	InputFileID string `json:"input_file_id,omitempty"` // ID of uploaded JSONL file

	// Anthropic-style: inline requests
	Requests []BatchRequestItem `json:"requests,omitempty"` // Inline request items

	// Azure-style: Blob storage input and output folder
	InputBlob    *string            `json:"input_blob,omitempty"`
	OutputFolder *BatchOutputFolder `json:"output_folder,omitempty"`

	// Common fields
	DisplayName        *string            `json:"display_name,omitempty"`         // Human-readable job name (e.g. Vertex displayName)
	Endpoint           BatchEndpoint      `json:"endpoint,omitempty"`             // Target endpoint for batch requests
	CompletionWindow   string             `json:"completion_window,omitempty"`    // Time window (e.g., "24h")
	Metadata           map[string]string  `json:"metadata,omitempty"`             // User-provided metadata
	OutputExpiresAfter *BatchExpiresAfter `json:"output_expires_after,omitempty"` // Expiration for batch output (OpenAI only)

	// Extra parameters for provider-specific features
	ExtraParams map[string]interface{} `json:"-"`
}

BifrostBatchCreateRequest represents a request to create a batch job.

func (*BifrostBatchCreateRequest) GetRawRequestBody added in v1.2.38 

func (request *BifrostBatchCreateRequest) GetRawRequestBody() []byte

GetRawRequestBody returns the raw request body.

type BifrostBatchCreateResponse added in v1.2.38 

type BifrostBatchCreateResponse struct {
	ID               string             `json:"id"`
	Object           string             `json:"object,omitempty"`       // "batch" for OpenAI
	DisplayName      *string            `json:"display_name,omitempty"` // Human-readable job name (e.g. Vertex displayName)
	Endpoint         string             `json:"endpoint,omitempty"`
	InputFileID      string             `json:"input_file_id,omitempty"`
	CompletionWindow string             `json:"completion_window,omitempty"`
	Status           BatchStatus        `json:"status"`
	RequestCounts    BatchRequestCounts `json:"request_counts,omitempty"`
	Metadata         map[string]string  `json:"metadata,omitempty"`
	CreatedAt        int64              `json:"created_at,omitempty"`
	ExpiresAt        *int64             `json:"expires_at,omitempty"`

	// Output file references (OpenAI)
	OutputFileID *string `json:"output_file_id,omitempty"`
	ErrorFileID  *string `json:"error_file_id,omitempty"`

	// Anthropic-specific
	ProcessingStatus *string `json:"processing_status,omitempty"`
	ResultsURL       *string `json:"results_url,omitempty"`

	// Gemini-specific (operation response)
	OperationName *string `json:"operation_name,omitempty"`

	// Azure-specific Blob Storage URLs (returned when using blob storage input/output)
	InputBlob  *string `json:"input_blob,omitempty"`
	OutputBlob *string `json:"output_blob,omitempty"`
	ErrorBlob  *string `json:"error_blob,omitempty"`

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostBatchCreateResponse represents the response from creating a batch job.

type BifrostBatchDeleteRequest added in v1.4.8 

type BifrostBatchDeleteRequest struct {
	Provider ModelProvider `json:"provider"`
	Model    *string       `json:"model"`
	BatchID  string        `json:"batch_id"` // ID of the batch to delete

	RawRequestBody []byte `json:"-"` // Raw request body (not serialized)

	// Extra parameters for provider-specific features
	ExtraParams map[string]interface{} `json:"-"`
}

BifrostBatchDeleteRequest represents a request to delete a batch job.

func (*BifrostBatchDeleteRequest) GetRawRequestBody added in v1.4.8 

func (request *BifrostBatchDeleteRequest) GetRawRequestBody() []byte

GetRawRequestBody returns the raw request body.

type BifrostBatchDeleteResponse added in v1.4.8 

type BifrostBatchDeleteResponse struct {
	ID            string             `json:"id"`
	Object        string             `json:"object,omitempty"`
	Status        BatchStatus        `json:"status"`
	RequestCounts BatchRequestCounts `json:"request_counts,omitempty"`

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostBatchDeleteResponse represents the response from deleting a batch job.

type BifrostBatchListRequest added in v1.2.38 

type BifrostBatchListRequest struct {
	Provider ModelProvider `json:"provider"`
	Model    *string       `json:"model"`

	// Pagination
	Limit      int     `json:"limit,omitempty"`       // Max results to return
	After      *string `json:"after,omitempty"`       // Cursor for pagination (OpenAI)
	BeforeID   *string `json:"before_id,omitempty"`   // Pagination cursor (Anthropic)
	AfterID    *string `json:"after_id,omitempty"`    // Pagination cursor (Anthropic)
	PageToken  *string `json:"page_token,omitempty"`  // For Gemini pagination
	PageSize   int     `json:"page_size,omitempty"`   // For Gemini pagination
	NextCursor *string `json:"next_cursor,omitempty"` // For Gemini pagination

	// Extra parameters for provider-specific features
	ExtraParams map[string]interface{} `json:"-"`
}

BifrostBatchListRequest represents a request to list batch jobs.

type BifrostBatchListResponse added in v1.2.38 

type BifrostBatchListResponse struct {
	Object  string                         `json:"object,omitempty"` // "list"
	Data    []BifrostBatchRetrieveResponse `json:"data"`
	FirstID *string                        `json:"first_id,omitempty"`
	LastID  *string                        `json:"last_id,omitempty"`
	HasMore bool                           `json:"has_more,omitempty"`

	// Anthropic pagination
	NextCursor *string `json:"next_cursor,omitempty"` // For cursor-based pagination

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostBatchListResponse represents the response from listing batch jobs.

type BifrostBatchResultsRequest added in v1.2.38 

type BifrostBatchResultsRequest struct {
	Provider ModelProvider `json:"provider"`
	Model    *string       `json:"model"`
	BatchID  string        `json:"batch_id"` // ID of the batch to get results for

	RawRequestBody []byte `json:"-"` // Raw request body (not serialized)

	// Extra parameters for provider-specific features
	ExtraParams map[string]interface{} `json:"-"`
}

BifrostBatchResultsRequest represents a request to retrieve batch results.

func (*BifrostBatchResultsRequest) GetRawRequestBody added in v1.2.38 

func (request *BifrostBatchResultsRequest) GetRawRequestBody() []byte

GetRawRequestBody returns the raw request body.

type BifrostBatchResultsResponse added in v1.2.38 

type BifrostBatchResultsResponse struct {
	BatchID string            `json:"batch_id"`
	Results []BatchResultItem `json:"results"`

	// For streaming results (Anthropic)
	HasMore    bool    `json:"has_more,omitempty"`
	NextCursor *string `json:"next_cursor,omitempty"`

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostBatchResultsResponse represents the response from retrieving batch results.

type BifrostBatchRetrieveRequest added in v1.2.38 

type BifrostBatchRetrieveRequest struct {
	Provider ModelProvider `json:"provider"`
	Model    *string       `json:"model"`
	BatchID  string        `json:"batch_id"` // ID of the batch to retrieve

	RawRequestBody []byte `json:"-"` // Raw request body (not serialized)

	// Extra parameters for provider-specific features
	ExtraParams map[string]interface{} `json:"-"`
}

BifrostBatchRetrieveRequest represents a request to retrieve a batch job.

func (*BifrostBatchRetrieveRequest) GetRawRequestBody added in v1.2.38 

func (request *BifrostBatchRetrieveRequest) GetRawRequestBody() []byte

GetRawRequestBody returns the raw request body.

type BifrostBatchRetrieveResponse added in v1.2.38 

type BifrostBatchRetrieveResponse struct {
	ID               string             `json:"id"`
	Object           string             `json:"object,omitempty"`
	DisplayName      *string            `json:"display_name,omitempty"` // Human-readable job name (e.g. Vertex displayName)
	Endpoint         string             `json:"endpoint,omitempty"`
	InputFileID      string             `json:"input_file_id,omitempty"`
	CompletionWindow string             `json:"completion_window,omitempty"`
	Status           BatchStatus        `json:"status"`
	RequestCounts    BatchRequestCounts `json:"request_counts,omitempty"`
	Metadata         map[string]string  `json:"metadata,omitempty"`
	CreatedAt        int64              `json:"created_at,omitempty"`
	ExpiresAt        *int64             `json:"expires_at,omitempty"`
	InProgressAt     *int64             `json:"in_progress_at,omitempty"`
	FinalizingAt     *int64             `json:"finalizing_at,omitempty"`
	CompletedAt      *int64             `json:"completed_at,omitempty"`
	FailedAt         *int64             `json:"failed_at,omitempty"`
	ExpiredAt        *int64             `json:"expired_at,omitempty"`
	CancellingAt     *int64             `json:"cancelling_at,omitempty"`
	CancelledAt      *int64             `json:"cancelled_at,omitempty"`

	// Output references
	OutputFileID *string      `json:"output_file_id,omitempty"`
	ErrorFileID  *string      `json:"error_file_id,omitempty"`
	Errors       *BatchErrors `json:"errors,omitempty"`

	// Anthropic-specific
	ProcessingStatus *string `json:"processing_status,omitempty"`
	ResultsURL       *string `json:"results_url,omitempty"`
	ArchivedAt       *int64  `json:"archived_at,omitempty"`

	// Gemini-specific
	OperationName *string `json:"operation_name,omitempty"`
	Done          *bool   `json:"done,omitempty"`
	Progress      *int    `json:"progress,omitempty"` // Percentage progress

	// Azure-specific Blob Storage URLs (returned when using blob storage input/output)
	InputBlob  *string `json:"input_blob,omitempty"`
	OutputBlob *string `json:"output_blob,omitempty"`
	ErrorBlob  *string `json:"error_blob,omitempty"`

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostBatchRetrieveResponse represents the response from retrieving a batch job.

type BifrostCacheDebug added in v1.1.25 

type BifrostCacheDebug struct {
	CacheHit bool `json:"cache_hit"`

	CacheID *string `json:"cache_id,omitempty"`
	HitType *string `json:"hit_type,omitempty"`

	RequestedProvider *string `json:"requested_provider,omitempty"`
	RequestedModel    *string `json:"requested_model,omitempty"`

	// Semantic cache only (provider, model, and input tokens will be present for semantic cache, even if cache is not hit)
	ProviderUsed *string `json:"provider_used,omitempty"`
	ModelUsed    *string `json:"model_used,omitempty"`
	InputTokens  *int    `json:"input_tokens,omitempty"`

	// Semantic cache only (only when cache is hit)
	Threshold  *float64 `json:"threshold,omitempty"`
	Similarity *float64 `json:"similarity,omitempty"`

	// CacheHitLatency is the time in milliseconds spent serving the cache hit
	// (lookup + response build). Only set when CacheHit is true.
	CacheHitLatency *int64 `json:"cache_hit_latency,omitempty"`
}

BifrostCacheDebug represents debug information about the cache.

type BifrostCachedContentCreateRequest added in v1.5.8 

type BifrostCachedContentCreateRequest struct {
	Provider          ModelProvider `json:"provider"`
	Model             string        `json:"model"`
	DisplayName       *string       `json:"display_name,omitempty"`
	SystemInstruction any           `json:"system_instruction,omitempty"`
	Contents          []any         `json:"contents,omitempty"`
	Tools             []any         `json:"tools,omitempty"`
	ToolConfig        any           `json:"tool_config,omitempty"`
	TTL               *string       `json:"ttl,omitempty"`         // duration like "3600s"
	ExpireTime        *string       `json:"expire_time,omitempty"` // RFC3339 timestamp

	RawRequestBody []byte         `json:"-"`
	ExtraParams    map[string]any `json:"-"`
}

BifrostCachedContentCreateRequest creates a new cached content. TTL and ExpireTime are mutually exclusive — providers must error if both are set.

func (*BifrostCachedContentCreateRequest) GetRawRequestBody added in v1.5.8 

func (r *BifrostCachedContentCreateRequest) GetRawRequestBody() []byte

GetRawRequestBody returns the raw request body.

type BifrostCachedContentCreateResponse added in v1.5.8 

type BifrostCachedContentCreateResponse struct {
	Name              string         `json:"name"`
	DisplayName       string         `json:"display_name,omitempty"`
	Model             string         `json:"model"`
	SystemInstruction any            `json:"system_instruction,omitempty"`
	Contents          []any          `json:"contents,omitempty"`
	Tools             []any          `json:"tools,omitempty"`
	ToolConfig        any            `json:"tool_config,omitempty"`
	CreateTime        string         `json:"create_time,omitempty"`
	UpdateTime        string         `json:"update_time,omitempty"`
	ExpireTime        string         `json:"expire_time,omitempty"`
	UsageMetadata     map[string]any `json:"usage_metadata,omitempty"`

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostCachedContentCreateResponse is the response from creating a cached content.

type BifrostCachedContentDeleteRequest added in v1.5.8 

type BifrostCachedContentDeleteRequest struct {
	Provider ModelProvider `json:"provider"`
	Model    *string       `json:"model"`

	// Name is the identifier of the cached content to delete (see Retrieve.Name).
	Name string `json:"name"`

	RawRequestBody []byte         `json:"-"`
	ExtraParams    map[string]any `json:"-"`
}

BifrostCachedContentDeleteRequest deletes a cached content by name.

func (*BifrostCachedContentDeleteRequest) GetRawRequestBody added in v1.5.8 

func (r *BifrostCachedContentDeleteRequest) GetRawRequestBody() []byte

GetRawRequestBody returns the raw request body.

type BifrostCachedContentDeleteResponse added in v1.5.8 

type BifrostCachedContentDeleteResponse struct {
	Name    string `json:"name,omitempty"`
	Deleted bool   `json:"deleted"`

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostCachedContentDeleteResponse is the response from deleting a cached content. Providers typically return an empty body on success; this struct carries a Deleted flag set by bifrost plus ExtraFields for diagnostics.

type BifrostCachedContentListRequest added in v1.5.8 

type BifrostCachedContentListRequest struct {
	Provider ModelProvider `json:"provider"`
	Model    *string       `json:"model"`

	// Pagination
	PageSize  int     `json:"page_size,omitempty"`
	PageToken *string `json:"page_token,omitempty"`

	RawRequestBody []byte         `json:"-"`
	ExtraParams    map[string]any `json:"-"`
}

BifrostCachedContentListRequest lists cached contents in the project.

func (*BifrostCachedContentListRequest) GetRawRequestBody added in v1.5.8 

func (r *BifrostCachedContentListRequest) GetRawRequestBody() []byte

GetRawRequestBody returns the raw request body.

type BifrostCachedContentListResponse added in v1.5.8 

type BifrostCachedContentListResponse struct {
	CachedContents []CachedContentObject `json:"cached_contents"`
	NextPageToken  string                `json:"next_page_token,omitempty"`

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostCachedContentListResponse is the response from listing cached contents.

type BifrostCachedContentRetrieveRequest added in v1.5.8 

type BifrostCachedContentRetrieveRequest struct {
	Provider ModelProvider `json:"provider"`
	Model    *string       `json:"model"`

	// Name is the identifier of the cached content.
	//   - Google AI Studio: "cachedContents/{id}" or just "{id}"
	//   - Vertex AI:        "projects/{p}/locations/{l}/cachedContents/{id}" or just "{id}"
	Name string `json:"name"`

	RawRequestBody []byte         `json:"-"`
	ExtraParams    map[string]any `json:"-"`
}

BifrostCachedContentRetrieveRequest retrieves a single cached content by name.

func (*BifrostCachedContentRetrieveRequest) GetRawRequestBody added in v1.5.8 

func (r *BifrostCachedContentRetrieveRequest) GetRawRequestBody() []byte

GetRawRequestBody returns the raw request body.

type BifrostCachedContentRetrieveResponse added in v1.5.8 

type BifrostCachedContentRetrieveResponse struct {
	Name              string         `json:"name"`
	DisplayName       string         `json:"display_name,omitempty"`
	Model             string         `json:"model"`
	SystemInstruction any            `json:"system_instruction,omitempty"`
	Contents          []any          `json:"contents,omitempty"`
	Tools             []any          `json:"tools,omitempty"`
	ToolConfig        any            `json:"tool_config,omitempty"`
	CreateTime        string         `json:"create_time,omitempty"`
	UpdateTime        string         `json:"update_time,omitempty"`
	ExpireTime        string         `json:"expire_time,omitempty"`
	UsageMetadata     map[string]any `json:"usage_metadata,omitempty"`

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostCachedContentRetrieveResponse is the response from retrieving one cached content.

type BifrostCachedContentUpdateRequest added in v1.5.8 

type BifrostCachedContentUpdateRequest struct {
	Provider ModelProvider `json:"provider"`
	Model    *string       `json:"model"`

	// Name is the identifier of the cached content to update (see Retrieve.Name).
	Name string `json:"name"`

	TTL        *string `json:"ttl,omitempty"`
	ExpireTime *string `json:"expire_time,omitempty"`

	RawRequestBody []byte         `json:"-"`
	ExtraParams    map[string]any `json:"-"`
}

BifrostCachedContentUpdateRequest updates a cached content's expiration. Only TTL or ExpireTime may be set — they are mutually exclusive.

func (*BifrostCachedContentUpdateRequest) GetRawRequestBody added in v1.5.8 

func (r *BifrostCachedContentUpdateRequest) GetRawRequestBody() []byte

GetRawRequestBody returns the raw request body.

type BifrostCachedContentUpdateResponse added in v1.5.8 

type BifrostCachedContentUpdateResponse struct {
	Name              string         `json:"name"`
	DisplayName       string         `json:"display_name,omitempty"`
	Model             string         `json:"model"`
	SystemInstruction any            `json:"system_instruction,omitempty"`
	Contents          []any          `json:"contents,omitempty"`
	Tools             []any          `json:"tools,omitempty"`
	ToolConfig        any            `json:"tool_config,omitempty"`
	CreateTime        string         `json:"create_time,omitempty"`
	UpdateTime        string         `json:"update_time,omitempty"`
	ExpireTime        string         `json:"expire_time,omitempty"`
	UsageMetadata     map[string]any `json:"usage_metadata,omitempty"`

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostCachedContentUpdateResponse is the response from updating a cached content.

type BifrostChatRequest added in v1.2.0 

type BifrostChatRequest struct {
	Provider       ModelProvider   `json:"provider"`
	Model          string          `json:"model"`
	Input          []ChatMessage   `json:"input,omitempty"`
	Params         *ChatParameters `json:"params,omitempty"`
	Fallbacks      []Fallback      `json:"fallbacks,omitempty"`
	RawRequestBody []byte          `json:"-"` // set bifrost-use-raw-request-body to true in ctx to use the raw request body. Bifrost will directly send this to the downstream provider.
}

BifrostChatRequest is the request struct for chat completion requests

func (*BifrostChatRequest) GetExtraParams added in v1.4.0 

func (cr *BifrostChatRequest) GetExtraParams() map[string]interface{}

func (*BifrostChatRequest) GetRawRequestBody added in v1.2.17 

func (cr *BifrostChatRequest) GetRawRequestBody() []byte

GetRawRequestBody returns the raw request body

func (*BifrostChatRequest) ToResponsesRequest added in v1.2.0 

func (cr *BifrostChatRequest) ToResponsesRequest() *BifrostResponsesRequest

ToResponsesRequest converts a BifrostChatRequest to BifrostResponsesRequest format

type BifrostChatResponse added in v1.2.7 

type BifrostChatResponse struct {
	ID                string                     `json:"id"`
	Choices           []BifrostResponseChoice    `json:"choices"`
	Created           int                        `json:"created"` // The Unix timestamp (in seconds).
	Model             string                     `json:"model"`
	Object            string                     `json:"object"` // "chat.completion" or "chat.completion.chunk"
	ServiceTier       *BifrostServiceTier        `json:"service_tier,omitempty"`
	Speed             *string                    `json:"speed,omitempty"`         // "fast" | "standard" — speed actually served (Anthropic fast mode); drives fast-mode billing
	InferenceGeo      *string                    `json:"inference_geo,omitempty"` // "us" | "global" — inference geography served (Anthropic data residency); drives the 1.1x US multiplier
	Diagnostics       *CacheDiagnostics          `json:"diagnostics,omitempty"`   // Anthropic cache diagnostics (cache-diagnosis-2026-04-07); first prompt-cache prefix divergence point
	SystemFingerprint string                     `json:"system_fingerprint"`
	Usage             *BifrostLLMUsage           `json:"usage"`
	ExtraFields       BifrostResponseExtraFields `json:"extra_fields"`
	ExtraParams       map[string]interface{}     `json:"-"`

	// Perplexity-specific fields
	SearchResults []SearchResult `json:"search_results,omitempty"`
	Videos        []VideoResult  `json:"videos,omitempty"`
	Citations     []string       `json:"citations,omitempty"`
}

BifrostChatResponse represents the complete result from a chat completion request.

func (*BifrostChatResponse) BackfillParams added in v1.4.19 

func (cr *BifrostChatResponse) BackfillParams(request *BifrostChatRequest)

BackfillParams populates response fields from the request that are needed

func (*BifrostChatResponse) ToBifrostResponsesResponse added in v1.2.7 

func (cr *BifrostChatResponse) ToBifrostResponsesResponse() *BifrostResponsesResponse

ToBifrostResponsesResponse converts the BifrostChatResponse to BifrostResponsesResponse format This converts Chat-style fields (Choices) to Responses API format

func (*BifrostChatResponse) ToBifrostResponsesStreamResponse added in v1.2.7 

func (cr *BifrostChatResponse) ToBifrostResponsesStreamResponse(state *ChatToResponsesStreamState) []*BifrostResponsesStreamResponse

ToBifrostResponsesStreamResponse converts the BifrostChatResponse from Chat streaming format to Responses streaming format This converts Chat stream chunks (Choices with Deltas) to BifrostResponsesStreamResponse format Returns a slice of responses to support cases where a single event produces multiple responses

func (*BifrostChatResponse) ToBifrostTextCompletionResponse added in v1.5.2 

func (cr *BifrostChatResponse) ToBifrostTextCompletionResponse() *BifrostTextCompletionResponse

ToBifrostTextCompletionResponse converts a BifrostChatResponse to a BifrostTextCompletionResponse

func (*BifrostChatResponse) ToTextCompletionResponse added in v1.2.7 

func (cr *BifrostChatResponse) ToTextCompletionResponse() *BifrostTextCompletionResponse

ToTextCompletionResponse converts a BifrostChatResponse to a BifrostTextCompletionResponse

type BifrostCompactionRequest added in v1.5.17 

type BifrostCompactionRequest struct {
	Provider             ModelProvider          `json:"provider"`
	Model                string                 `json:"model"`
	Input                []ResponsesMessage     `json:"input,omitempty"`
	Instructions         *string                `json:"instructions,omitempty"`
	PreviousResponseID   *string                `json:"previous_response_id,omitempty"`
	PromptCacheKey       *string                `json:"prompt_cache_key,omitempty"`
	PromptCacheRetention *string                `json:"prompt_cache_retention,omitempty"`
	PromptCacheOptions   *PromptCacheOptions    `json:"prompt_cache_options,omitempty"`
	ServiceTier          *BifrostServiceTier    `json:"service_tier,omitempty"`
	Fallbacks            []Fallback             `json:"fallbacks,omitempty"`
	ExtraParams          map[string]interface{} `json:"-"`
	RawRequestBody       []byte                 `json:"-"`
}

BifrostCompactionRequest is the request for the context compaction endpoint (POST /v1/responses/compact). It is a strict subset of BifrostResponsesRequest — tools, sampling params, and streaming are not supported.

func (*BifrostCompactionRequest) GetRawRequestBody added in v1.5.17 

func (r *BifrostCompactionRequest) GetRawRequestBody() []byte

type BifrostCompactionResponse added in v1.5.17 

type BifrostCompactionResponse struct {
	ID          *string                    `json:"id,omitempty"`
	Object      string                     `json:"object"` // always "response.compaction"
	Model       string                     `json:"model,omitempty"`
	CreatedAt   int                        `json:"created_at"`
	Output      []ResponsesMessage         `json:"output"`
	Usage       *ResponsesResponseUsage    `json:"usage,omitempty"`
	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostCompactionResponse is the response from the context compaction endpoint. object is always "response.compaction". output contains user messages plus one encrypted compaction item.

func (*BifrostCompactionResponse) WithDefaults added in v1.5.17 

func (resp *BifrostCompactionResponse) WithDefaults() *BifrostCompactionResponse

type BifrostConfig 

type BifrostConfig struct {
	Account            Account
	LLMPlugins         []LLMPlugin
	MCPPlugins         []MCPPlugin
	OAuth2Provider     OAuth2Provider
	MCPHeadersProvider MCPHeadersProvider // Backend for MCPAuthTypePerUserHeaders credential storage; nil disables per-user-headers auth (resolver errors at use)
	Logger             Logger
	Tracer             Tracer        // Tracer for distributed tracing (nil = NoOpTracer)
	InitialPoolSize    int           // Initial pool size for sync pools in Bifrost. Higher values will reduce memory allocations but will increase memory usage.
	DropExcessRequests bool          // If true, in cases where the queue is full, requests will not wait for the queue to be empty and will be dropped instead.
	MCPConfig          *MCPConfig    // MCP (Model Context Protocol) configuration for tool integration
	KeySelector        KeySelector   // Custom key selector function
	KeyPoolFilter      KeyPoolFilter // Optional hook to filter available keys before selection; nil = all keys eligible
	KVStore            KVStore       // shared KV store for clustering/session stickiness; nil = disabled
}

BifrostConfig represents the configuration for initializing a Bifrost instance. It contains the necessary components for setting up the system including account details, plugins, logging, and initial pool size.

type BifrostContainerCreateRequest added in v1.3.12 

type BifrostContainerCreateRequest struct {
	Provider ModelProvider `json:"provider"`

	// Required fields
	Name string `json:"name"` // Name of the container

	// Optional fields
	ExpiresAfter *ContainerExpiresAfter `json:"expires_after,omitempty"` // Expiration configuration
	FileIDs      []string               `json:"file_ids,omitempty"`      // IDs of existing files to copy into this container
	MemoryLimit  string                 `json:"memory_limit,omitempty"`  // Memory limit (e.g., "1g", "4g")
	Metadata     map[string]string      `json:"metadata,omitempty"`      // User-provided metadata

	// Extra parameters for provider-specific features
	ExtraParams map[string]interface{} `json:"-"`
}

BifrostContainerCreateRequest represents a request to create a container.

type BifrostContainerCreateResponse added in v1.3.12 

type BifrostContainerCreateResponse struct {
	ID           string                 `json:"id"`
	Object       string                 `json:"object,omitempty"` // "container"
	Name         string                 `json:"name"`
	CreatedAt    int64                  `json:"created_at"`
	Status       ContainerStatus        `json:"status,omitempty"`
	ExpiresAfter *ContainerExpiresAfter `json:"expires_after,omitempty"`
	LastActiveAt *int64                 `json:"last_active_at,omitempty"`
	MemoryLimit  string                 `json:"memory_limit,omitempty"`
	Metadata     map[string]string      `json:"metadata,omitempty"`

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostContainerCreateResponse represents the response from creating a container.

type BifrostContainerDeleteRequest added in v1.3.12 

type BifrostContainerDeleteRequest struct {
	Provider    ModelProvider `json:"provider"`
	ContainerID string        `json:"container_id"` // ID of the container to delete

	// Extra parameters for provider-specific features
	ExtraParams map[string]interface{} `json:"-"`
}

BifrostContainerDeleteRequest represents a request to delete a container.

type BifrostContainerDeleteResponse added in v1.3.12 

type BifrostContainerDeleteResponse struct {
	ID      string `json:"id"`
	Object  string `json:"object,omitempty"` // "container.deleted"
	Deleted bool   `json:"deleted"`

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostContainerDeleteResponse represents the response from deleting a container.

type BifrostContainerFileContentRequest added in v1.3.12 

type BifrostContainerFileContentRequest struct {
	Provider    ModelProvider `json:"provider"`
	ContainerID string        `json:"container_id"` // ID of the container
	FileID      string        `json:"file_id"`      // ID of the file

	// Extra parameters for provider-specific features
	ExtraParams map[string]interface{} `json:"-"`
}

BifrostContainerFileContentRequest represents a request to retrieve the content of a container file.

type BifrostContainerFileContentResponse added in v1.3.12 

type BifrostContainerFileContentResponse struct {
	Content     []byte `json:"content"`      // Raw file content
	ContentType string `json:"content_type"` // MIME type of the content

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostContainerFileContentResponse represents the response from retrieving container file content.

type BifrostContainerFileCreateRequest added in v1.3.12 

type BifrostContainerFileCreateRequest struct {
	Provider    ModelProvider `json:"provider"`
	ContainerID string        `json:"container_id"` // ID of the container

	// One of these must be provided
	File   []byte  `json:"-"`                   // File content (for multipart upload)
	FileID *string `json:"file_id,omitempty"`   // Reference to existing file
	Path   *string `json:"file_path,omitempty"` // Path for the file in the container

	// Extra parameters for provider-specific features
	ExtraParams map[string]interface{} `json:"-"`
}

BifrostContainerFileCreateRequest represents a request to create a file in a container.

type BifrostContainerFileCreateResponse added in v1.3.12 

type BifrostContainerFileCreateResponse struct {
	ID          string `json:"id"`
	Object      string `json:"object,omitempty"` // "container.file"
	Bytes       int64  `json:"bytes"`
	CreatedAt   int64  `json:"created_at"`
	ContainerID string `json:"container_id"`
	Path        string `json:"path"`
	Source      string `json:"source"`

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostContainerFileCreateResponse represents the response from creating a container file.

type BifrostContainerFileDeleteRequest added in v1.3.12 

type BifrostContainerFileDeleteRequest struct {
	Provider    ModelProvider `json:"provider"`
	ContainerID string        `json:"container_id"` // ID of the container
	FileID      string        `json:"file_id"`      // ID of the file to delete

	// Extra parameters for provider-specific features
	ExtraParams map[string]interface{} `json:"-"`
}

BifrostContainerFileDeleteRequest represents a request to delete a container file.

type BifrostContainerFileDeleteResponse added in v1.3.12 

type BifrostContainerFileDeleteResponse struct {
	ID      string `json:"id"`
	Object  string `json:"object,omitempty"` // "container.file.deleted"
	Deleted bool   `json:"deleted"`

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostContainerFileDeleteResponse represents the response from deleting a container file.

type BifrostContainerFileListRequest added in v1.3.12 

type BifrostContainerFileListRequest struct {
	Provider    ModelProvider `json:"provider"`
	ContainerID string        `json:"container_id"` // ID of the container

	// Pagination
	Limit int     `json:"limit,omitempty"` // Max results to return (1-100, default 20)
	After *string `json:"after,omitempty"` // Cursor for pagination
	Order *string `json:"order,omitempty"` // Sort order (asc/desc), default desc

	// Extra parameters for provider-specific features
	ExtraParams map[string]interface{} `json:"-"`
}

BifrostContainerFileListRequest represents a request to list files in a container.

type BifrostContainerFileListResponse added in v1.3.12 

type BifrostContainerFileListResponse struct {
	Object  string                `json:"object,omitempty"` // "list"
	Data    []ContainerFileObject `json:"data"`
	FirstID *string               `json:"first_id,omitempty"`
	LastID  *string               `json:"last_id,omitempty"`
	HasMore bool                  `json:"has_more,omitempty"`
	After   *string               `json:"after,omitempty"` // Encoded cursor for next page (includes key index for multi-key pagination)

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostContainerFileListResponse represents the response from listing container files.

type BifrostContainerFileRetrieveRequest added in v1.3.12 

type BifrostContainerFileRetrieveRequest struct {
	Provider    ModelProvider `json:"provider"`
	ContainerID string        `json:"container_id"` // ID of the container
	FileID      string        `json:"file_id"`      // ID of the file to retrieve

	// Extra parameters for provider-specific features
	ExtraParams map[string]interface{} `json:"-"`
}

BifrostContainerFileRetrieveRequest represents a request to retrieve a container file.

type BifrostContainerFileRetrieveResponse added in v1.3.12 

type BifrostContainerFileRetrieveResponse struct {
	ID          string `json:"id"`
	Object      string `json:"object,omitempty"` // "container.file"
	Bytes       int64  `json:"bytes"`
	CreatedAt   int64  `json:"created_at"`
	ContainerID string `json:"container_id"`
	Path        string `json:"path"`
	Source      string `json:"source"`

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostContainerFileRetrieveResponse represents the response from retrieving a container file.

type BifrostContainerListRequest added in v1.3.12 

type BifrostContainerListRequest struct {
	Provider ModelProvider `json:"provider"`

	// Pagination
	Limit int     `json:"limit,omitempty"` // Max results to return (1-100, default 20)
	After *string `json:"after,omitempty"` // Cursor for pagination
	Order *string `json:"order,omitempty"` // Sort order (asc/desc), default desc

	// Extra parameters for provider-specific features
	ExtraParams map[string]interface{} `json:"-"`
}

BifrostContainerListRequest represents a request to list containers.

type BifrostContainerListResponse added in v1.3.12 

type BifrostContainerListResponse struct {
	Object  string            `json:"object,omitempty"` // "list"
	Data    []ContainerObject `json:"data"`
	FirstID *string           `json:"first_id,omitempty"`
	LastID  *string           `json:"last_id,omitempty"`
	HasMore bool              `json:"has_more,omitempty"`
	After   *string           `json:"after,omitempty"` // Encoded cursor for next page (includes key index for multi-key pagination)

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostContainerListResponse represents the response from listing containers.

type BifrostContainerRetrieveRequest added in v1.3.12 

type BifrostContainerRetrieveRequest struct {
	Provider    ModelProvider `json:"provider"`
	ContainerID string        `json:"container_id"` // ID of the container to retrieve

	// Extra parameters for provider-specific features
	ExtraParams map[string]interface{} `json:"-"`
}

BifrostContainerRetrieveRequest represents a request to retrieve a container.

type BifrostContainerRetrieveResponse added in v1.3.12 

type BifrostContainerRetrieveResponse struct {
	ID           string                 `json:"id"`
	Object       string                 `json:"object,omitempty"` // "container"
	Name         string                 `json:"name"`
	CreatedAt    int64                  `json:"created_at"`
	Status       ContainerStatus        `json:"status,omitempty"`
	ExpiresAfter *ContainerExpiresAfter `json:"expires_after,omitempty"`
	LastActiveAt *int64                 `json:"last_active_at,omitempty"`
	MemoryLimit  string                 `json:"memory_limit,omitempty"`
	Metadata     map[string]string      `json:"metadata,omitempty"`

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostContainerRetrieveResponse represents the response from retrieving a container.

type BifrostContext added in v1.1.17 

type BifrostContext struct {
	// contains filtered or unexported fields
}

BifrostContext is a custom context.Context implementation that tracks user-set values. It supports deadlines, can be derived from other contexts, and provides layered value inheritance when derived from another BifrostContext.

func NewBifrostContext added in v1.2.31 

func NewBifrostContext(parent context.Context, deadline time.Time) *BifrostContext

NewBifrostContext creates a new BifrostContext with the given parent context and deadline. If the deadline is zero, no deadline is set on this context (though the parent may have one). The context will be cancelled when the deadline expires or when the parent context is cancelled.

func NewBifrostContextWithCancel added in v1.3.4 

func NewBifrostContextWithCancel(parent context.Context) (*BifrostContext, context.CancelFunc)

NewBifrostContextWithCancel creates a new BifrostContext with a cancel function. This is a convenience wrapper around NewBifrostContext. Returns the context and a cancel function that should be called to release resources.

func NewBifrostContextWithTimeout added in v1.2.31 

func NewBifrostContextWithTimeout(parent context.Context, timeout time.Duration) (*BifrostContext, context.CancelFunc)

NewBifrostContextWithTimeout creates a new BifrostContext with a timeout duration. This is a convenience wrapper around NewBifrostContext. Returns the context and a cancel function that should be called to release resources.

func NewBifrostContextWithValue added in v1.3.4 

func NewBifrostContextWithValue(parent context.Context, deadline time.Time, key any, value any) *BifrostContext

NewBifrostContextWithValue creates a new BifrostContext with the given value set.

func (*BifrostContext) AppendRoutingEngineLog added in v1.4.3 

func (bc *BifrostContext) AppendRoutingEngineLog(engineName string, level LogLevel, message string)

AppendRoutingEngineLog appends a routing engine log entry to the context. Parameters:

  • ctx: The Bifrost context
  • engineName: Name of the routing engine (e.g., "governance", "routing-rule")
  • message: Human-readable log message describing the decision/action

func (*BifrostContext) BlockRestrictedWrites added in v1.3.4 

func (bc *BifrostContext) BlockRestrictedWrites()

BlockRestrictedWrites returns true if restricted writes are blocked.

func (*BifrostContext) Cancel added in v1.2.31 

func (bc *BifrostContext) Cancel()

Cancel cancels the context, closing the Done channel and setting the error to context.Canceled.

func (*BifrostContext) ClearPausedStreamBuffer added in v1.6.4 

func (bc *BifrostContext) ClearPausedStreamBuffer() error

ClearPausedStreamBuffer drops chunks buffered while the active stream is paused.

func (*BifrostContext) ClearValue added in v1.4.8 

func (bc *BifrostContext) ClearValue(key any)

ClearValue clears a value from the internal userValues map. For scoped contexts, delegates to the root context via valueDelegate.

func (*BifrostContext) Deadline added in v1.2.31 

func (bc *BifrostContext) Deadline() (time.Time, bool)

Deadline returns the deadline for this context. For scoped contexts, delegates to the root context. If both this context and the parent have deadlines, the earlier one is returned.

func (*BifrostContext) Done added in v1.2.31 

func (bc *BifrostContext) Done() <-chan struct{}

Done returns a channel that is closed when the context is cancelled.

func (*BifrostContext) DrainPluginLogs added in v1.5.0 

func (bc *BifrostContext) DrainPluginLogs() []PluginLogEntry

DrainPluginLogs transfers ownership of the plugin log slice to the caller. The internal log store is returned to the pool after draining. Returns nil if no logs have been recorded. This should be called once on the root context after all plugin hooks have completed.

func (*BifrostContext) EndStream added in v1.6.0 

func (bc *BifrostContext) EndStream(err *BifrostError)

EndStream terminates the active streaming response. Any buffered chunks are flushed first; if err is non-nil it is then delivered as a final error chunk. After EndStream returns, all further provider chunks for this stream are dropped (PostLLMHook still fires, but no client delivery happens). Idempotent. No-op if no Tracer or trace ID is present in ctx.

Engages the pause/resume gate (see PauseStream). Requires a real Tracer (e.g. `framework/streaming/Accumulator`); inert under `DefaultTracer()`.

func (*BifrostContext) Err added in v1.2.31 

func (bc *BifrostContext) Err() error

Err returns the error explaining why the context was cancelled. For scoped contexts, delegates to the root context. Returns nil if the context has not been cancelled.

func (*BifrostContext) GetAccumulatedResponse added in v1.6.0 

func (bc *BifrostContext) GetAccumulatedResponse() *BifrostResponse

GetAccumulatedResponse returns a snapshot of the BifrostResponse assembled from chunks received so far on the streaming response associated with this context. Built on demand — useful from PostLLMHook (including while paused) to inspect the assembled output before deciding next steps. Read-only: does not engage the gate or mutate accumulator state. Returns nil if no Tracer or trace ID is present, no accumulator exists, no chunks have been accumulated yet, or the stream type is indeterminable.

func (*BifrostContext) GetAndSetValue added in v1.3.8 

func (bc *BifrostContext) GetAndSetValue(key any, value any) any

GetAndSetValue gets a value from the internal userValues map and sets it. For scoped contexts, delegates to the root context via valueDelegate.

func (*BifrostContext) GetParentCtxWithUserValues added in v1.2.31 

func (bc *BifrostContext) GetParentCtxWithUserValues() context.Context

GetParentCtxWithUserValues returns a copy of the parent context with all user-set values merged in.

func (*BifrostContext) GetPluginLogs added in v1.5.0 

func (bc *BifrostContext) GetPluginLogs() []PluginLogEntry

GetPluginLogs returns a deep copy of all accumulated plugin log entries. Thread-safe. Returns nil if no logs have been recorded.

func (*BifrostContext) GetRoutingEngineLogs added in v1.4.3 

func (bc *BifrostContext) GetRoutingEngineLogs() []RoutingEngineLogEntry

GetRoutingEngineLogs retrieves all routing engine logs from the context. Parameters:

  • ctx: The Bifrost context

Returns:

  • []RoutingEngineLogEntry: Slice of routing engine log entries (nil if none)

func (*BifrostContext) GetUserValues added in v1.2.31 

func (bc *BifrostContext) GetUserValues() map[any]any

GetUserValues returns a copy of all user-set values in this context. If the parent is also a PluginContext, the values are merged with parent values (this context's values take precedence over parent values).

func (*BifrostContext) IsStreamEnded added in v1.6.0 

func (bc *BifrostContext) IsStreamEnded() bool

IsStreamEnded reports whether the streaming response associated with this context has been ended via EndStream (or via a final/hard-error chunk flowing through the gate while Active). Read-only: does not engage the gate or create any tracer state. Returns false when no Tracer or trace ID is present in ctx, or when no accumulator exists for this stream.

func (*BifrostContext) IsStreamPaused added in v1.6.0 

func (bc *BifrostContext) IsStreamPaused() bool

IsStreamPaused reports whether the streaming response associated with this context is currently paused. Read-only: does not engage the gate or create any tracer state. Returns false when no Tracer or trace ID is present in ctx, or when no accumulator exists for this stream.

func (*BifrostContext) Log added in v1.5.0 

func (bc *BifrostContext) Log(level LogLevel, msg string)

Log appends a structured log entry for the current plugin scope. No-op if the context is not scoped to a plugin or has no log store.

func (*BifrostContext) MCPAuthMode added in v1.5.11 

func (bc *BifrostContext) MCPAuthMode() MCPAuthMode

AuthMode derives the per-user OAuth lookup mode from current context state. Priority: UserID > VirtualKey > session. Call this at token-lookup time, not in middleware — the governance plugin can inject UserID (via VK→owner resolution) after middleware runs, and the mode must reflect that.

Returns MCPAuthModeNone when no identity column is populated (no user, no VK, no session header), so callers that branch on the returned mode alone cannot mistake an unauthenticated request for a session-mode caller.

VK check uses BifrostContextKeyGovernanceVirtualKeyID (the resolved VK row ID) rather than BifrostContextKeyVirtualKey (the raw header value) because vk-mode token rows are keyed by the resolved VK ID.

func (*BifrostContext) PauseStream added in v1.6.0 

func (bc *BifrostContext) PauseStream()

PauseStream marks the active streaming response associated with this context as paused. While paused, chunks continue to flow through PostLLMHook (so plugins can still inspect them), but they are buffered instead of delivered to the client. Buffered chunks are flushed in order when ResumeStream is called. Idempotent. No-op if no Tracer or trace ID is present in ctx.

Calling this method engages the pause/resume gate for the stream: provider send sites switch from a direct channel send to Tracer.GateSend. Streams that never call Pause/Resume/End pay no extra cost.

Requires a real Tracer to be wired via the Bifrost config (e.g. `framework/streaming/Accumulator`). Under `DefaultTracer()` (the `*NoOpTracer` fall-back used when `config.Tracer` is nil), this call is silently inert — chunks continue to flow direct to the client. See `DefaultTracer()` for the architectural reason core cannot ship a built-in gate impl.

func (*BifrostContext) ReleasePluginScope added in v1.5.0 

func (bc *BifrostContext) ReleasePluginScope()

ReleasePluginScope marks a scoped context as released. Safe no-op if called on a non-scoped context.

We deliberately do NOT mutate the scoped struct's fields here. External watcher goroutines spawned via stdlib context helpers (e.g. propagateCancel from context.WithDeadline) may still hold references and read parent/done asynchronously. Mutating those fields would race the watchers. The struct becomes garbage and is reclaimed by the GC once all watchers have exited.

We still release the plugin log store reference so it can be drained or reclaimed independently of the scope's lifetime.

func (*BifrostContext) ResumeStream added in v1.6.0 

func (bc *BifrostContext) ResumeStream()

ResumeStream resumes a previously paused stream. Buffered chunks are flushed to the client in order, then live streaming continues. Idempotent. No-op if no Tracer or trace ID is present in ctx.

Engages the pause/resume gate (see PauseStream). Requires a real Tracer (e.g. `framework/streaming/Accumulator`); inert under `DefaultTracer()`.

func (*BifrostContext) Root added in v1.5.11 

func (bc *BifrostContext) Root() *BifrostContext

Root returns the underlying root BifrostContext. For root contexts this is the receiver itself; for plugin-scoped contexts it is the underlying root that scoped Value/SetValue calls delegate to.

PLUGIN AUTHORS: capture Root() synchronously inside Pre/PostLLMHook (or any other hook) when you need to write to the context from a goroutine that outlives the hook. The plugin-scoped *BifrostContext passed into your hook is reclaimed by an internal sync.Pool the moment the hook returns — any later SetValue/Value call on it lands in detached storage that nobody downstream can read (and can leak into a future pool reuse). The root, in contrast, lives for the entire request, so a pointer captured here is safe to use for the lifetime of the request even after your hook returns.

Example: 

func (p *Plugin) PreLLMHook(ctx *schemas.BifrostContext, req ...) (...) {
    rootCtx := ctx.Root() // capture before the scope is released
    go func() {
        // ... long-running work that produces stream chunks ...
        rootCtx.SetValue(schemas.BifrostContextKeyStreamEndIndicator, true)
    }()
    return req, &schemas.LLMPluginShortCircuit{Stream: ch}, nil
}

func (*BifrostContext) SetTraceAttribute added in v1.5.8 

func (bc *BifrostContext) SetTraceAttribute(key string, value any)

SetTraceAttribute adds an attribute to the root span for the current trace. This is thread-safe and can be called concurrently.

func (*BifrostContext) SetValue added in v1.2.31 

func (bc *BifrostContext) SetValue(key, value any)

SetValue sets a value in the internal userValues map. For scoped contexts, delegates to the root context via valueDelegate. This is thread-safe and can be called concurrently.

func (*BifrostContext) UnblockRestrictedWrites added in v1.3.4 

func (bc *BifrostContext) UnblockRestrictedWrites()

UnblockRestrictedWrites unblocks restricted writes.

func (*BifrostContext) Value added in v1.2.31 

func (bc *BifrostContext) Value(key any) any

Value returns the value associated with the key. For scoped contexts, delegates to the root context via valueDelegate. Otherwise checks the internal userValues map, then delegates to the parent context.

func (*BifrostContext) WithPluginScope added in v1.5.0 

func (bc *BifrostContext) WithPluginScope(name *string) *BifrostContext

WithPluginScope returns a scoped BifrostContext that shares the root's pluginLogs store and delegates Value/SetValue/Deadline/Err/Done operations to the root.

Scoped contexts are NOT pool-reused. Plugins routinely pass the scoped ctx to stdlib helpers (context.WithDeadline, HTTP clients, vector store SDKs) which spawn watcher goroutines via context.propagateCancel — those watchers can read the scoped struct's fields long after the plugin returns. Reusing the struct across requests would race those reads. Allocating fresh is idiomatic Go context handling (the stdlib context types are not pooled either) and keeps the lifecycle race-free without atomics.

func (*BifrostContext) WithValue added in v1.3.4 

func (bc *BifrostContext) WithValue(key any, value any) *BifrostContext

WithValue returns a new context with the given value set.

type BifrostContextKey added in v1.1.17 

type BifrostContextKey string

BifrostContextKey is a type for context keys used in Bifrost. 

const (
	BifrostContextKeySessionToken      BifrostContextKey = "bifrost-session-token" // string (session token for authentication - set by auth middleware)
	BifrostContextKeyVirtualKey        BifrostContextKey = "x-bf-vk"               // string
	BifrostContextKeyAPIKeyName        BifrostContextKey = "x-bf-api-key"          // string (explicit key name selection)
	BifrostContextKeyAPIKeyID          BifrostContextKey = "x-bf-api-key-id"       // string (explicit key ID selection, takes priority over name)
	BifrostContextKeyDirectKey         BifrostContextKey = "x-bf-direct-key"       // schemas.Key (raw key supplied via x-bf-direct-key: true header; bypasses registered key pool)
	BifrostContextKeyRequestID         BifrostContextKey = "request-id"            // string
	BifrostContextKeyFallbackRequestID BifrostContextKey = "fallback-request-id"   // string

	// NOTE: []string is used for both keys, and by default all clients/tools are included (when nil).
	// If "*" is present, all clients/tools are included, and [] means no clients/tools are included.
	// Request context filtering takes priority over client config - context can override client exclusions.
	MCPContextKeyIncludeClients BifrostContextKey = "mcp-include-clients" // Context key for whitelist client filtering
	MCPContextKeyIncludeTools   BifrostContextKey = "mcp-include-tools"   // Context key for whitelist tool filtering (Note: toolName should be in "clientName-toolName" format for individual tools, or "clientName-*" for wildcard)

	BifrostContextKeySelectedKeyID                       BifrostContextKey = "bifrost-selected-key-id"                // string (to store the selected key ID (set by bifrost governance plugin - DO NOT SET THIS MANUALLY))
	BifrostContextKeySelectedKeyName                     BifrostContextKey = "bifrost-selected-key-name"              // string (to store the selected key name (set by bifrost governance plugin - DO NOT SET THIS MANUALLY))
	BifrostContextKeyGovernanceVirtualKeyID              BifrostContextKey = "bifrost-governance-virtual-key-id"      // string (to store the virtual key ID (set by bifrost governance plugin - DO NOT SET THIS MANUALLY))
	BifrostContextKeyGovernanceVirtualKeyName            BifrostContextKey = "bifrost-governance-virtual-key-name"    // string (to store the virtual key name (set by bifrost governance plugin - DO NOT SET THIS MANUALLY))
	BifrostContextKeyGovernanceTeamID                    BifrostContextKey = "bifrost-governance-team-id"             // string (to store the team ID (set by bifrost governance plugin - DO NOT SET THIS MANUALLY))
	BifrostContextKeyGovernanceTeamName                  BifrostContextKey = "bifrost-governance-team-name"           // string (to store the team name (set by bifrost governance plugin - DO NOT SET THIS MANUALLY))
	BifrostContextKeyGovernanceCustomerID                BifrostContextKey = "bifrost-governance-customer-id"         // string (to store the customer ID (set by bifrost governance plugin - DO NOT SET THIS MANUALLY))
	BifrostContextKeyGovernanceCustomerName              BifrostContextKey = "bifrost-governance-customer-name"       // string (to store the customer name (set by bifrost governance plugin - DO NOT SET THIS MANUALLY))
	BifrostContextKeyGovernanceBusinessUnitID            BifrostContextKey = "bifrost-governance-business-unit-id"    // string (to store the business unit ID (set by enterprise governance plugin - DO NOT SET THIS MANUALLY))
	BifrostContextKeyGovernanceBusinessUnitName          BifrostContextKey = "bifrost-governance-business-unit-name"  // string (to store the business unit name (set by enterprise governance plugin - DO NOT SET THIS MANUALLY))
	BifrostContextKeyGovernanceTeamIDs                   BifrostContextKey = "bifrost-governance-team-ids"            // []string (all teams a user/AP request belongs to; set by enterprise governance plugin - DO NOT SET THIS MANUALLY)
	BifrostContextKeyGovernanceTeamNames                 BifrostContextKey = "bifrost-governance-team-names"          // []string (display names, aligned with team-ids; set by enterprise governance plugin - DO NOT SET THIS MANUALLY)
	BifrostContextKeyGovernanceBusinessUnitIDs           BifrostContextKey = "bifrost-governance-business-unit-ids"   // []string (distinct BUs across the user's teams; set by enterprise governance plugin - DO NOT SET THIS MANUALLY)
	BifrostContextKeyGovernanceBusinessUnitNames         BifrostContextKey = "bifrost-governance-business-unit-names" // []string (display names, aligned with business-unit-ids; set by enterprise governance plugin - DO NOT SET THIS MANUALLY)
	BifrostContextKeyGovernanceCustomerIDs               BifrostContextKey = "bifrost-governance-customer-ids"        // []string (distinct customers a user/team request belongs to; set by enterprise governance plugin - DO NOT SET THIS MANUALLY)
	BifrostContextKeyGovernanceCustomerNames             BifrostContextKey = "bifrost-governance-customer-names"      // []string (display names, aligned with customer-ids; set by enterprise governance plugin - DO NOT SET THIS MANUALLY)
	BifrostContextKeyGovernanceScopedCustomerID          BifrostContextKey = "bifrost-governance-scoped-customer-id"  // string (resolved customer the request is scoped to via the x-bf-customer-id / x-bf-customer-name header on a team-VK path; set by the enterprise governance plugin - DO NOT SET THIS MANUALLY)
	BifrostContextKeyGovernanceRoutingRuleID             BifrostContextKey = "bifrost-governance-routing-rule-id"     // string (to store the routing rule ID (set by bifrost governance plugin - DO NOT SET THIS MANUALLY))
	BifrostContextKeyGovernanceRoutingRuleName           BifrostContextKey = "bifrost-governance-routing-rule-name"   // string (to store the routing rule name (set by bifrost governance plugin - DO NOT SET THIS MANUALLY))
	BifrostContextKeyRoutingPinnedAPIKeyID               BifrostContextKey = "bifrost-routing-pinned-api-key-id"      // string (provider key ID pinned by a matched routing rule target; resolved against the configured key pool during key selection and takes precedence over a caller-supplied pin (set by bifrost governance plugin - DO NOT SET THIS MANUALLY))
	BifrostContextKeySelectedPromptName                  BifrostContextKey = "bifrost-selected-prompt-name"           // string (display name of the selected prompt (set by prompts plugin - DO NOT SET THIS MANUALLY))
	BifrostContextKeySelectedPromptVersion               BifrostContextKey = "bifrost-selected-prompt-version"        // string (numeric version as string, e.g. "3" (set by prompts plugin - DO NOT SET THIS MANUALLY))
	BifrostContextKeySelectedPromptID                    BifrostContextKey = "bifrost-selected-prompt-id"             // string (id of the selected prompt (set by prompts plugin - DO NOT SET THIS MANUALLY))
	BifrostContextKeyGovernanceIncludeOnlyKeys           BifrostContextKey = "bf-governance-include-only-keys"        // []string (to store the include-only key IDs for provider config routing (set by bifrost governance plugin - DO NOT SET THIS MANUALLY))
	BifrostContextKeyNumberOfRetries                     BifrostContextKey = "bifrost-number-of-retries"              // int (to store the number of retries (set by bifrost - DO NOT SET THIS MANUALLY))
	BifrostContextKeyFallbackIndex                       BifrostContextKey = "bifrost-fallback-index"                 // int (to store the fallback index (set by bifrost - DO NOT SET THIS MANUALLY)) 0 for primary, 1 for first fallback, etc.
	BifrostContextKeyResolvedAlias                       BifrostContextKey = "bifrost-resolved-alias"                 // *ResolvedAlias (set by bifrost after key-level alias resolution — providers read this for model_family routing and provider-specific overrides; nil/absent when no alias matched)
	BifrostContextKeyStreamEndIndicator                  BifrostContextKey = "bifrost-stream-end-indicator"           // bool (set by bifrost - DO NOT SET THIS MANUALLY)
	BifrostContextKeyStreamGated                         BifrostContextKey = "bifrost-stream-gated"                   // bool (set by ctx.PauseStream/ResumeStream/EndStream when a plugin first engages the pause/resume gate; provider helpers use this as a fast-path check to skip Tracer.GateSend on streams that never engage the gate)
	BifrostContextKeyStreamIdleTimeout                   BifrostContextKey = "bifrost-stream-idle-timeout"            // time.Duration (per-chunk idle timeout for streaming)
	BifrostContextKeySkipKeySelection                    BifrostContextKey = "bifrost-skip-key-selection"             // bool (will pass an empty key to the provider)
	BifrostContextKeyExtraHeaders                        BifrostContextKey = "bifrost-extra-headers"                  // map[string][]string
	BifrostContextKeyURLPath                             BifrostContextKey = "bifrost-extra-url-path"                 // string
	BifrostContextKeyUseRawRequestBody                   BifrostContextKey = "bifrost-use-raw-request-body"
	BifrostContextKeyChangeRequestType                   BifrostContextKey = "bifrost-change-request-type"                      // RequestType (set by plugins to trigger request type conversion in core, e.g. text->chat or chat->responses)
	BifrostContextKeySendBackRawRequest                  BifrostContextKey = "bifrost-send-back-raw-request"                    // bool (per-request override — read by bifrost.go, never overwritten)
	BifrostContextKeySendBackRawResponse                 BifrostContextKey = "bifrost-send-back-raw-response"                   // bool (per-request override — read by bifrost.go, never overwritten)
	BifrostContextKeyIntegrationType                     BifrostContextKey = "bifrost-integration-type"                         // integration used in gateway (e.g. openai, anthropic, bedrock, etc.)
	BifrostContextKeyIsResponsesToChatCompletionFallback BifrostContextKey = "bifrost-is-responses-to-chat-completion-fallback" // bool (set by bifrost - DO NOT SET THIS MANUALLY)
	BifrostMCPAgentOriginalRequestID                     BifrostContextKey = "bifrost-mcp-agent-original-request-id"            // string (to store the original request ID for MCP agent mode)
	BifrostContextKeyParentMCPRequestID                  BifrostContextKey = "bf-parent-mcp-request-id"                         // string (parent request ID for nested tool calls from executeCode)
	BifrostContextKeyStructuredOutputToolName            BifrostContextKey = "bifrost-structured-output-tool-name"              // string (to store the name of the structured output tool (set by bifrost))
	BifrostContextKeyUserAgent                           BifrostContextKey = "bifrost-user-agent"                               // string (set by bifrost)
	BifrostContextKeySkipBudgetAndRateLimits             BifrostContextKey = "bifrost-skip-budget-and-rate-limits"              // bool (set by bifrost for read-only requests like list models that don't consume quota)
	BifrostContextKeySkipVirtualKeyUsageTracking         BifrostContextKey = "bifrost-skip-virtual-key-usage-tracking"          // bool (set by governance callers to skip VK usage while preserving VK auth/attribution)
	BifrostContextKeyTraceID                             BifrostContextKey = "bifrost-trace-id"                                 // string (trace ID for distributed tracing - set by tracing middleware)
	BifrostContextKeySpanID                              BifrostContextKey = "bifrost-span-id"                                  // string (current span ID for child span creation - set by tracer)
	BifrostContextKeyParentSpanID                        BifrostContextKey = "bifrost-parent-span-id"                           // string (parent span ID from W3C traceparent header - set by tracing middleware)
	BifrostContextKeyStreamStartTime                     BifrostContextKey = "bifrost-stream-start-time"                        // time.Time (start time for streaming TTFT calculation - set by bifrost)
	BifrostContextKeyTracer                              BifrostContextKey = "bifrost-tracer"                                   // Tracer (tracer instance for completing deferred spans - set by bifrost)
	BifrostContextKeyDeferTraceCompletion                BifrostContextKey = "bifrost-defer-trace-completion"                   // bool (signals trace completion should be deferred for streaming - set by streaming handlers)
	BifrostContextKeyTraceCompleter                      BifrostContextKey = "bifrost-trace-completer"                          // func([]PluginLogEntry) (callback to complete trace after streaming, receives transport plugin logs - set by tracing middleware)
	BifrostContextKeyAccumulatorID                       BifrostContextKey = "bifrost-accumulator-id"                           // string (ID for streaming accumulator lookup - set by tracer for accumulator operations)
	BifrostContextKeyMCPSessionID                        BifrostContextKey = "bifrost-mcp-session-id"                           // string (session-mode identity: any opaque value asserted by the caller via x-bf-mcp-session-id; binds the OAuth token row to subsequent /mcp calls when no VK or user is present)
	BifrostContextKeyMCPCallbackBaseURL                  BifrostContextKey = "bifrost-mcp-callback-base-url"                    // string (base URL like "https://host" — set by HTTP middleware. OAuth resolver appends /api/oauth/callback; headers resolver appends the workspace submit path. Used for both per-user OAuth and per-user headers auth flows)
	BifrostContextKeyIsMCPGateway                        BifrostContextKey = "bifrost-is-mcp-gateway"                           // bool (true when request is being handled via the MCP gateway path)
	BifrostContextKeyHasEmittedMessageDelta              BifrostContextKey = "bifrost-has-emitted-message-delta"                // bool (tracks whether message_delta was already emitted during streaming - avoids duplicates)
	BifrostContextKeySkipDBUpdate                        BifrostContextKey = "bifrost-skip-db-update"                           // bool (set by bifrost - DO NOT SET THIS MANUALLY)
	BifrostContextKeyGovernancePluginName                BifrostContextKey = "governance-plugin-name"                           // string (name of the governance plugin that processed the request - set by bifrost)
	BifrostContextKeyClusterNodeID                       BifrostContextKey = "bifrost-cluster-node-id"                          // string (cluster node ID for log attribution - set by enterprise server)
	BifrostContextKeyGovernanceBudgetIDs                 BifrostContextKey = "bifrost-governance-budget-ids"                    // []string (budget IDs applicable to this request - set by governance plugin)
	BifrostContextKeyGovernanceRateLimitIDs              BifrostContextKey = "bifrost-governance-rate-limit-ids"                // []string (rate limit IDs applicable to this request - set by governance plugin)
	BifrostContextKeyPromptsPluginName                   BifrostContextKey = "prompts-plugin-name"                              // string (name of the prompts plugin to use - set by bifrost - DO NOT SET THIS MANUALLY))
	BifrostContextKeyIsEnterprise                        BifrostContextKey = "is-enterprise"                                    // bool (set by bifrost - DO NOT SET THIS MANUALLY)
	BifrostContextKeyAvailableProviders                  BifrostContextKey = "available-providers"                              // []ModelProvider (set by internal bifrost components - DO NOT SET THIS MANUALLY))
	BifrostContextKeyStoreRawRequestResponse             BifrostContextKey = "bifrost-store-raw-request-response"               // bool (per-request override — read by bifrost.go, never overwritten)
	BifrostContextKeyCaptureRawRequest                   BifrostContextKey = "bifrost-capture-raw-request"                      // bool (set by bifrost - DO NOT SET THIS MANUALLY) — true when providers should capture raw request bytes
	BifrostContextKeyCaptureRawResponse                  BifrostContextKey = "bifrost-capture-raw-response"                     // bool (set by bifrost - DO NOT SET THIS MANUALLY) — true when providers should capture raw response bytes
	BifrostContextKeyDropRawRequestFromClient            BifrostContextKey = "bifrost-drop-raw-request-from-client"             // bool (set by bifrost - DO NOT SET THIS MANUALLY) — true when raw request should be stripped from the client-facing response
	BifrostContextKeyDropRawResponseFromClient           BifrostContextKey = "bifrost-drop-raw-response-from-client"            // bool (set by bifrost - DO NOT SET THIS MANUALLY) — true when raw response should be stripped from the client-facing response
	BifrostContextKeyShouldStoreRawInLogs                BifrostContextKey = "bifrost-should-store-raw-in-logs"                 // bool (set by bifrost - DO NOT SET THIS MANUALLY) — true when raw request/response should be persisted in log records
	BifrostContextKeyRetryDBFetch                        BifrostContextKey = "bifrost-retry-db-fetch"                           // bool (set by bifrost - DO NOT SET THIS MANUALLY)
	BifrostContextKeyIsCustomProvider                    BifrostContextKey = "bifrost-is-custom-provider"                       // bool (set by bifrost - DO NOT SET THIS MANUALLY)
	BifrostContextKeyHTTPRequestType                     BifrostContextKey = "bifrost-http-request-type"                        // RequestType (set by bifrost - DO NOT SET THIS MANUALLY)
	BifrostContextKeyHTTPRoute                           BifrostContextKey = "bifrost-http-route"                               // string (set by bifrost - DO NOT SET THIS MANUALLY — matched route template, set by HTTP transport; used as the low-cardinality metrics `path` label)
	BifrostContextKeyPassthroughExtraParams              BifrostContextKey = "bifrost-passthrough-extra-params"                 // bool
	BifrostContextKeyRoutingEnginesUsed                  BifrostContextKey = "bifrost-routing-engines-used"                     // []string (set by bifrost - DO NOT SET THIS MANUALLY) - list of routing engines used ("routing-rule", "governance", "loadbalancing", etc.)
	BifrostContextKeyRoutingEngineLogs                   BifrostContextKey = "bifrost-routing-engine-logs"                      // []RoutingEngineLogEntry (set by bifrost - DO NOT SET THIS MANUALLY) - list of routing engine log entries
	BifrostContextKeyTransportPluginLogs                 BifrostContextKey = "bifrost-transport-plugin-logs"                    // []PluginLogEntry (transport-layer plugin logs accumulated during HTTP transport hooks)
	BifrostContextKeyTransportPostHookCompleter          BifrostContextKey = "bifrost-transport-posthook-completer"             // func() (callback to run HTTPTransportPostHook after streaming - set by transport interceptor middleware)
	BifrostContextKeySkipPluginPipeline                  BifrostContextKey = "bifrost-skip-plugin-pipeline"                     // bool - skip plugin pipeline for the request
	BifrostContextKeyParentRequestID                     BifrostContextKey = "bifrost-parent-request-id"                        // string (parent linkage for grouped request logs like realtime turns)
	BifrostContextKeyRealtimeSessionID                   BifrostContextKey = "bifrost-realtime-session-id"                      // string
	BifrostContextKeyRealtimeProviderSessionID           BifrostContextKey = "bifrost-realtime-provider-session-id"             // string
	BifrostContextKeyRealtimeSource                      BifrostContextKey = "bifrost-realtime-source"                          // string ("ei" or "lm")
	BifrostContextKeyRealtimeEventType                   BifrostContextKey = "bifrost-realtime-event-type"                      // string
	BifrostContextKeyRealtimeTransport                   BifrostContextKey = "bifrost-realtime-transport"                       // string ("websocket" or "webrtc")
	BifrostContextKeyRealtimeVoice                       BifrostContextKey = "bifrost-realtime-voice"                           // string
	BifrostIsAsyncRequest                                BifrostContextKey = "bifrost-is-async-request"                         // bool (set by bifrost - DO NOT SET THIS MANUALLY)) - whether the request is an async request (only used in gateway)
	BifrostContextKeyRequestHeaders                      BifrostContextKey = "bifrost-request-headers"                          // map[string]string (all request headers with lowercased keys)
	BifrostContextKeyRequestQuery                        BifrostContextKey = "bifrost-request-query"                            // map[string]string (request query params with lowercased keys; consumed by governance routing CEL rules)
	BifrostContextKeyRoutingAllowedProviders             BifrostContextKey = "bifrost-routing-allowed-providers"                // []ModelProvider; when set, downstream routing layers (enterprise LB, model-catalog-resolver) must intersect their candidate providers with this set. Plugins set this when they have an opinion about which providers are valid for the request — even if they couldn't pick one themselves. Empty slice means "no provider is permitted" (fail-closed).
	BifrostContextKeyAllowPerRequestStorageOverride      BifrostContextKey = "bifrost-allow-per-request-storage-override"       // bool (set by transport from config — gates whether x-bf-disable-content-logging and x-bf-store-raw-request-response per-request overrides are honored)
	BifrostContextKeyAllowPerRequestRawOverride          BifrostContextKey = "bifrost-allow-per-request-raw-override"           // bool (set by transport from config — gates whether x-bf-send-back-raw-request and x-bf-send-back-raw-response per-request overrides are honored)
	BifrostContextKeyRedactionData                       BifrostContextKey = "bifrost-redaction-data"                           // RedactionData (set by enterprise guardrails plugin - DO NOT SET THIS MANUALLY)
	BifrostContextKeyDisableContentLogging               BifrostContextKey = "x-bf-disable-content-logging"                     // bool (per-request override for content logging; only honored when BifrostContextKeyAllowPerRequestStorageOverride is true. When retain_content_in_object_storage is on, disabled content is still offloaded to object storage as hidden instead of dropped)
	BifrostContextKeySkipListModelsGovernanceFiltering   BifrostContextKey = "bifrost-skip-list-models-governance-filtering"    // bool (set by bifrost - DO NOT SET THIS MANUALLY))
	BifrostContextKeySCIMClaims                          BifrostContextKey = "scim_claims"
	BifrostContextKeyUserID                              BifrostContextKey = "bifrost-user-id"                    // string (to store the user ID (set by enterprise auth middleware - DO NOT SET THIS MANUALLY))
	BifrostContextKeyUserName                            BifrostContextKey = "bifrost-user-name"                  // string (to store the user name (set by enterprise auth middleware - DO NOT SET THIS MANUALLY))
	BifrostContextKeyUserEmail                           BifrostContextKey = "bifrost-user-email"                 // string (to store the user email (set by enterprise auth middleware - DO NOT SET THIS MANUALLY))
	BifrostContextKeyQueryScope                          BifrostContextKey = "bifrost-query-scope"                // configstore.QueryScope (func that mutates a query; set by upstream wrapper - DO NOT SET THIS MANUALLY)
	BifrostContextKeyVisibilityFilterProvider            BifrostContextKey = "bifrost-visibility-filter-provider" // DEPRECATED: replaced by BifrostContextKeyQueryScope. Will be removed once all callers migrate.
	BifrostContextKeyTargetUserID                        BifrostContextKey = "target_user_id"
	BifrostContextKeyIsAzureUserAgent                    BifrostContextKey = "bifrost-is-azure-user-agent" // bool (set by bifrost - DO NOT SET THIS MANUALLY)) - whether the request is an Azure user agent (only used in gateway)
	BifrostContextKeyUserRoleID                          BifrostContextKey = "bifrost-user-role-id"
	BifrostContextKeyVideoOutputRequested                BifrostContextKey = "bifrost-video-output-requested"
	BifrostContextKeyValidateKeys                        BifrostContextKey = "bifrost-validate-keys"                      // bool (triggers additional key validation during provider add/update)
	BifrostContextKeyProviderResponseHeaders             BifrostContextKey = "bifrost-provider-response-headers"          // map[string]string (set by provider handlers for response header forwarding)
	BifrostContextKeyMCPAddedTools                       BifrostContextKey = "bifrost-mcp-added-tools"                    // []string (set by bifrost - DO NOT SET THIS MANUALLY)) - list of tools added to the request by MCP, all the tool are in the format "clientName-toolName"
	BifrostContextKeyLargePayloadMode                    BifrostContextKey = "bifrost-large-payload-mode"                 // bool (set by bifrost - DO NOT SET THIS MANUALLY)) indicates large payload streaming mode is active
	BifrostContextKeyLargePayloadReader                  BifrostContextKey = "bifrost-large-payload-reader"               // io.Reader (set by bifrost - DO NOT SET THIS MANUALLY)) upstream reader for large payloads
	BifrostContextKeyLargePayloadContentLength           BifrostContextKey = "bifrost-large-payload-content-length"       // int (set by bifrost - DO NOT SET THIS MANUALLY)) content length for large payloads
	BifrostContextKeyLargePayloadContentType             BifrostContextKey = "bifrost-large-payload-content-type"         // string (set by enterprise - DO NOT SET THIS MANUALLY)) original content type for large payload passthrough
	BifrostContextKeyLargePayloadMetadata                BifrostContextKey = "bifrost-large-payload-metadata"             // *LargePayloadMetadata (set by bifrost - DO NOT SET THIS MANUALLY)) routing metadata for large payloads
	BifrostContextKeyLargePayloadRequestThreshold        BifrostContextKey = "bifrost-large-payload-request-threshold"    // int64 (set by enterprise - DO NOT SET THIS MANUALLY)) request threshold used by transport heuristics
	BifrostContextKeyLargeResponseMode                   BifrostContextKey = "bifrost-large-response-mode"                // bool (set by bifrost - DO NOT SET THIS MANUALLY)) indicates large response streaming mode is active
	BifrostContextKeyLargePayloadRequestPreview          BifrostContextKey = "bifrost-large-payload-request-preview"      // string (set by bifrost - DO NOT SET THIS MANUALLY)) truncated request body preview for logging
	BifrostContextKeyLargePayloadResponsePreview         BifrostContextKey = "bifrost-large-payload-response-preview"     // string (set by bifrost - DO NOT SET THIS MANUALLY)) truncated response body preview for logging
	BifrostContextKeyLargeResponseReader                 BifrostContextKey = "bifrost-large-response-reader"              // io.ReadCloser (set by bifrost - DO NOT SET THIS MANUALLY)) upstream reader for large responses
	BifrostContextKeyLargeResponseContentLength          BifrostContextKey = "bifrost-large-response-content-length"      // int (set by bifrost - DO NOT SET THIS MANUALLY)) content length for large responses
	BifrostContextKeyLargeResponseContentType            BifrostContextKey = "bifrost-large-response-content-type"        // string (set by bifrost - DO NOT SET THIS MANUALLY)) upstream content type for large responses
	BifrostContextKeyLargeResponseContentDisposition     BifrostContextKey = "bifrost-large-response-content-disposition" // string (set by bifrost - DO NOT SET THIS MANUALLY)) downstream content disposition for large responses
	BifrostContextKeyLargeResponseThreshold              BifrostContextKey = "bifrost-large-response-threshold"           // int64 (set by enterprise - DO NOT SET THIS MANUALLY)) threshold for response streaming
	BifrostContextKeyLargePayloadPrefetchSize            BifrostContextKey = "bifrost-large-payload-prefetch-size"        // int (set by enterprise - DO NOT SET THIS MANUALLY)) prefetch buffer size for metadata extraction from large responses
	BifrostContextKeyDeferredUsage                       BifrostContextKey = "bifrost-deferred-usage"                     // chan *BifrostLLMUsage (set by provider Phase B — delivers usage after response streaming completes)
	BifrostContextKeyStreamAccumulatedUsage              BifrostContextKey = "bifrost-stream-accumulated-usage"           // *BifrostLLMUsage handle, set ONCE by a streaming provider and mutated in place as usage arrives; read on cancel/timeout to bill partial usage that the provider already consumed
	BifrostContextKeyDeferredLargePayloadMetadata        BifrostContextKey = "bifrost-deferred-large-payload-metadata"    // <-chan *LargePayloadMetadata (set by enterprise Phase B request — delivers metadata after body streaming)
	BifrostContextKeySSEReaderFactory                    BifrostContextKey = "bifrost-sse-reader-factory"                 // *providerUtils.SSEReaderFactory (set by enterprise — replaces default bufio.Scanner SSE readers with streaming readers)
	BifrostContextKeySessionID                           BifrostContextKey = "bifrost-session-id"                         // string session ID for the request (session stickiness)
	BifrostContextKeySessionTTL                          BifrostContextKey = "bifrost-session-ttl"                        // time.Duration session TTL for the request (session stickiness)
	BifrostContextKeyMCPExtraHeaders                     BifrostContextKey = "bifrost-mcp-extra-headers"                  // map[string][]string (these headers are forwarded only to the MCP while tool execution if they are in the allowlist of the MCP client)
	BifrostContextKeyMCPLogID                            BifrostContextKey = "bifrost-mcp-log-id"                         // string (unique UUID for each MCP tool log entry - set per goroutine by agent executor - DO NOT SET THIS MANUALLY)
	BifrostContextKeyMCPHealthCheckRequest               BifrostContextKey = "bifrost-mcp-health-check-request"           // bool (set by bifrost - DO NOT SET THIS MANUALLY) - true when the MCP ping/list-tools request was generated by bifrost itself for health checks rather than originating from a caller
	BifrostContextKeyCompatConvertTextToChat             BifrostContextKey = "bifrost-compat-convert-text-to-chat"        // bool (per-request override from x-bf-compat header)
	BifrostContextKeyCompatConvertChatToResponses        BifrostContextKey = "bifrost-compat-convert-chat-to-responses"   // bool (per-request override from x-bf-compat header)
	BifrostContextKeyCompatShouldDropParams              BifrostContextKey = "bifrost-compat-should-drop-params"          // bool (per-request override from x-bf-compat header)
	BifrostContextKeyCompatShouldConvertParams           BifrostContextKey = "bifrost-compat-should-convert-params"       // bool (per-request override from x-bf-compat header)
	BifrostContextKeySupportsAssistantPrefill            BifrostContextKey = "bifrost-supports-assistant-prefill"         // bool (set by compat plugin) - if model supports assistant prefill
	BifrostContextKeyAttemptTrail                        BifrostContextKey = "bifrost-attempt-trail"                      // []KeyAttemptRecord (set by bifrost - DO NOT SET THIS MANUALLY) - per-attempt key selection history
	BifrostContextKeyDimensions                          BifrostContextKey = "bifrost-dimensions"                         // map[string]string (set by HTTP transport from x-bf-dim-* headers) BifrostContextKeyDimensions holds per-request key/value dimensions supplied via x-bf-dim-<key> request headers. These dimensions are forwarded to internal logs (as metadata)
	IsAPIKeyAuthContextKey                               BifrostContextKey = "is_api_key_auth"
	IsLocalAdminContextKey                               BifrostContextKey = "is_local_admin"                // bool (set by auth middleware when password-based auth succeeds - local admin user bypasses RBAC)
	BifrostContextKeyPassthroughOverridesPresent         BifrostContextKey = "passthrough_overrides_present" // bool (set by HTTP transport) - passthrough raw request requested
	BifrostContextKeyConnectionClosed                    BifrostContextKey = "connection_closed"
	BifrostContextKeyTempTokenScope                      BifrostContextKey = "bifrost-temp-token-scope"       // string (set by auth middleware when a temp token authorized the request - names the scope from the temptoken registry)
	BifrostContextKeyTempTokenResourceID                 BifrostContextKey = "bifrost-temp-token-resource-id" // string (set by auth middleware alongside the scope - the resource_id the token is bound to, e.g. an OAuth flow ID for mcp_auth)
	BifrostContextKeyAsyncWebhookEndpoint                BifrostContextKey = "bifrost-async-webhook-endpoint" // string (webhook endpoint name to notify when an async job finishes - carried as-is from the x-bf-async-webhook header; the submit path resolves and validates it before the job is created)
)

BifrostContextKeyRequestType is a context key for the request type.

type BifrostCost added in v1.2.19 

type BifrostCost struct {
	InputTokensCost     float64 `json:"input_tokens_cost,omitempty"`
	OutputTokensCost    float64 `json:"output_tokens_cost,omitempty"`
	ReasoningTokensCost float64 `json:"reasoning_tokens_cost,omitempty"`
	CitationTokensCost  float64 `json:"citation_tokens_cost,omitempty"`
	SearchQueriesCost   float64 `json:"search_queries_cost,omitempty"`
	RequestCost         float64 `json:"request_cost,omitempty"`
	TotalCost           float64 `json:"total_cost,omitempty"`
}

func (*BifrostCost) UnmarshalJSON added in v1.2.19 

func (bc *BifrostCost) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshalling for BifrostCost.

type BifrostCountTokensResponse added in v1.2.43 

type BifrostCountTokensResponse struct {
	Object             string                        `json:"object,omitempty"`
	Model              string                        `json:"model"`
	InputTokens        int                           `json:"input_tokens"`
	InputTokensDetails *ResponsesResponseInputTokens `json:"input_tokens_details,omitempty"`
	Tokens             []int                         `json:"tokens"`
	TokenStrings       []string                      `json:"token_strings,omitempty"`
	OutputTokens       *int                          `json:"output_tokens,omitempty"`
	TotalTokens        *int                          `json:"total_tokens"`
	ExtraFields        BifrostResponseExtraFields    `json:"extra_fields"`
}

BifrostCountTokensResponse captures token counts for a provided input.

type BifrostEmbeddingRequest added in v1.2.0 

type BifrostEmbeddingRequest struct {
	Provider       ModelProvider        `json:"provider"`
	Model          string               `json:"model"`
	Input          *EmbeddingInput      `json:"input,omitempty"`
	Params         *EmbeddingParameters `json:"params,omitempty"`
	Fallbacks      []Fallback           `json:"fallbacks,omitempty"`
	RawRequestBody []byte               `json:"-"` // set bifrost-use-raw-request-body to true in ctx to use the raw request body. Bifrost will directly send this to the downstream provider.
}

func (*BifrostEmbeddingRequest) GetRawRequestBody added in v1.2.17 

func (r *BifrostEmbeddingRequest) GetRawRequestBody() []byte

type BifrostEmbeddingResponse added in v1.1.14 

type BifrostEmbeddingResponse struct {
	Data        []EmbeddingData            `json:"data"` // Maps to "data" field in provider responses (e.g., OpenAI embedding format)
	Model       string                     `json:"model"`
	Object      string                     `json:"object"` // "list"
	Usage       *BifrostLLMUsage           `json:"usage"`
	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

func (*BifrostEmbeddingResponse) BackfillParams added in v1.5.8 

func (r *BifrostEmbeddingResponse) BackfillParams(request *BifrostEmbeddingRequest)

BackfillParams copies request metadata into the response when the provider omitted it (e.g. model in JSON).

type BifrostError 

type BifrostError struct {
	EventID        *string                 `json:"event_id,omitempty"`
	Type           *string                 `json:"type,omitempty"`
	IsBifrostError bool                    `json:"is_bifrost_error"`
	StatusCode     *int                    `json:"status_code,omitempty"`
	Error          *ErrorField             `json:"error"`
	AllowFallbacks *bool                   `json:"-"` // Optional: Controls fallback behavior (nil = true by default)
	StreamControl  *StreamControl          `json:"-"` // Optional: Controls stream behavior
	ExtraFields    BifrostErrorExtraFields `json:"extra_fields"`
}

BifrostError represents an error from the Bifrost system.

PLUGIN DEVELOPERS: When creating BifrostError in PreLLMHook or PostLLMHook, you can set AllowFallbacks: - AllowFallbacks = &true: Bifrost will try fallback providers if available - AllowFallbacks = &false: Bifrost will return this error immediately, no fallbacks - AllowFallbacks = nil: Treated as true by default (fallbacks allowed for resilience)

func ExtractRealtimeClientSecretModel added in v1.5.1 

func ExtractRealtimeClientSecretModel(root map[string]json.RawMessage) (string, *BifrostError)

ExtractRealtimeClientSecretModel extracts the model from either session.model or the legacy top-level model field. Also falls back to the GA transcription shape (session.audio.input.transcription.model) since /v1/realtime/client_secrets mints both full realtime sessions (session.model, or the legacy top-level model field) AND transcription-only sessions (RealtimeTranscriptionSessionCreateRequestGA — which has no top-level session.model at all, only the nested transcription model) through this same endpoint. The GA-nested fallback is only ever consulted via IsGATranscriptionSessionBody, which itself requires both session.model and the legacy root model to be absent — a full realtime session must never be reclassified as transcription-only just because it also enables live input-audio transcription as a sibling feature.

func NewRealtimeClientSecretBodyError added in v1.5.1 

func NewRealtimeClientSecretBodyError(status int, errorType, message string, err error) *BifrostError

NewRealtimeClientSecretBodyError builds a standard invalid-request style error for HTTP realtime client-secret request parsing/validation.

func ParseRealtimeClientSecretBody added in v1.5.1 

func ParseRealtimeClientSecretBody(raw json.RawMessage) (map[string]json.RawMessage, *BifrostError)

ParseRealtimeClientSecretBody parses a realtime client-secret request body into a mutable raw JSON map while preserving unknown fields.

func (*BifrostError) GetErrorString added in v1.5.10 

func (e *BifrostError) GetErrorString() string

func (*BifrostError) PopulateExtraFields added in v1.5.1 

func (e *BifrostError) PopulateExtraFields(requestType RequestType, provider ModelProvider, originalModelRequested string, resolvedModelUsed string)

PopulateExtraFields sets RequestType, Provider, OriginalModelRequested, and ResolvedModelUsed on the error's ExtraFields. Core always calls this both before and after RunPostLLMHooks, so any plugin modifications to these 4 fields are no-ops — tampering with them inside plugins is discouraged.

func (*BifrostError) PopulateRoutingInfo added in v1.5.19 

func (e *BifrostError) PopulateRoutingInfo(info RoutingInfo)

PopulateRoutingInfo sets ExtraFields.RoutingInfo on the error and syncs the deprecated triplet. Core calls this both before and after RunPostLLMHooks alongside PopulateExtraFields.

func (*BifrostError) SetFallbackRoutingInfo added in v1.5.19 

func (e *BifrostError) SetFallbackRoutingInfo(primaryProvider ModelProvider, primaryModel string)

SetFallbackRoutingInfo is the BifrostError counterpart — see the BifrostResponse method for semantics.

func (*BifrostError) String added in v1.5.7 

func (e *BifrostError) String() string

String renders the error as JSON for logging and test diagnostics. Without this, fmt's reflection printer walks ExtraFields.RawRequest / RawResponse (which typically hold json.RawMessage = []byte) and dumps every byte as a decimal, producing unreadable output.

type BifrostErrorExtraFields added in v1.2.0 

type BifrostErrorExtraFields struct {
	RoutingInfo RoutingInfo `json:"routing_info"`
	// Deprecated: use RoutingInfo.Provider. Still populated for backward
	// compatibility; new consumers should read from RoutingInfo.
	Provider ModelProvider `json:"provider,omitempty"`
	// Deprecated: use RoutingInfo.PrimaryModel when RoutingInfo.IsFallback
	// is true, otherwise RoutingInfo.Model — both branches collapse to the
	// model string the caller sent in the request. Still populated for
	// backward compatibility; new consumers should read from RoutingInfo.
	OriginalModelRequested string `json:"original_model_requested,omitempty"`
	// Deprecated: use RoutingInfo.ResolvedKeyAlias.ModelID when an alias
	// matched (i.e. RoutingInfo.ResolvedKeyAlias != nil), otherwise
	// RoutingInfo.Model. Still populated for backward compatibility; new
	// consumers should read from RoutingInfo.
	ResolvedModelUsed         string                `json:"resolved_model_used,omitempty"`
	RequestType               RequestType           `json:"request_type,omitempty"`
	MCPRequestType            MCPRequestType        `json:"mcp_request_type,omitempty"`
	RawRequest                interface{}           `json:"raw_request,omitempty"`
	RawResponse               interface{}           `json:"raw_response,omitempty"`
	ConvertedRequestType      RequestType           `json:"converted_request_type,omitempty"`
	DroppedCompatPluginParams []string              `json:"dropped_compat_plugin_params,omitempty"`
	Latency                   int64                 `json:"latency,omitempty"` // in milliseconds
	KeyStatuses               []KeyStatus           `json:"key_statuses,omitempty"`
	MCPAuthRequired           *MCPAuthRequiredError `json:"mcp_auth_required,omitempty"` // Set when a per-user MCP tool requires the caller to complete an inline auth flow (OAuth or headers)
	// BilledUsage carries provider-reported token usage that was consumed even
	// though the request ultimately failed or was cancelled (e.g. a stream
	// aborted mid-response, or a 5xx returned after input tokens were
	// processed). Providers populate it on cancel/error paths so downstream
	// post-LLM hooks (governance billing, logging cost) can charge for tokens
	// the provider actually billed us for. Nil when the failure consumed no
	// tokens (e.g. 401/403/429 before the model ran).
	BilledUsage *BifrostLLMUsage `json:"billed_usage,omitempty"`
}

BifrostErrorExtraFields contains additional fields in an error response.

type BifrostFileContentRequest added in v1.2.38 

type BifrostFileContentRequest struct {
	Provider ModelProvider `json:"provider"`
	Model    *string       `json:"model"`
	FileID   string        `json:"file_id"` // ID of the file to download

	RawRequestBody []byte `json:"-"` // Raw request body (not serialized)

	// Storage configuration (for S3/GCS backends)
	StorageConfig *FileStorageConfig `json:"storage_config,omitempty"`

	// Extra parameters for provider-specific features
	ExtraParams map[string]interface{} `json:"-"`
}

BifrostFileContentRequest represents a request to download file content.

func (*BifrostFileContentRequest) GetRawRequestBody added in v1.2.38 

func (request *BifrostFileContentRequest) GetRawRequestBody() []byte

GetRawRequestBody returns the raw request body.

type BifrostFileContentResponse added in v1.2.38 

type BifrostFileContentResponse struct {
	FileID      string `json:"file_id"`
	Content     []byte `json:"-"`                      // Raw file content (not serialized)
	ContentType string `json:"content_type,omitempty"` // MIME type

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostFileContentResponse represents the response from downloading file content.

type BifrostFileDeleteRequest added in v1.2.38 

type BifrostFileDeleteRequest struct {
	Provider ModelProvider `json:"provider"`
	Model    *string       `json:"model"`
	FileID   string        `json:"file_id"` // ID of the file to delete

	RawRequestBody []byte `json:"-"` // Raw request body (not serialized)

	// Storage configuration (for S3/GCS backends)
	StorageConfig *FileStorageConfig `json:"storage_config,omitempty"`

	// Extra parameters for provider-specific features
	ExtraParams map[string]interface{} `json:"-"`
}

BifrostFileDeleteRequest represents a request to delete a file.

func (*BifrostFileDeleteRequest) GetRawRequestBody added in v1.2.38 

func (request *BifrostFileDeleteRequest) GetRawRequestBody() []byte

GetRawRequestBody returns the raw request body.

type BifrostFileDeleteResponse added in v1.2.38 

type BifrostFileDeleteResponse struct {
	ID      string `json:"id"`
	Object  string `json:"object,omitempty"` // "file"
	Deleted bool   `json:"deleted"`

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostFileDeleteResponse represents the response from deleting a file.

type BifrostFileListRequest added in v1.2.38 

type BifrostFileListRequest struct {
	Provider ModelProvider `json:"provider"`
	Model    *string       `json:"model"`

	RawRequestBody []byte `json:"-"` // Raw request body (not serialized)

	// Filters
	Purpose FilePurpose `json:"purpose,omitempty"` // Filter by purpose

	// Pagination
	Limit int     `json:"limit,omitempty"` // Max results to return
	After *string `json:"after,omitempty"` // Cursor for pagination
	Order *string `json:"order,omitempty"` // Sort order (asc/desc)

	// Storage configuration (for S3/GCS backends)
	StorageConfig *FileStorageConfig `json:"storage_config,omitempty"`

	// Extra parameters for provider-specific features
	ExtraParams map[string]interface{} `json:"-"`
}

BifrostFileListRequest represents a request to list files.

func (*BifrostFileListRequest) GetRawRequestBody added in v1.2.38 

func (request *BifrostFileListRequest) GetRawRequestBody() []byte

GetRawRequestBody returns the raw request body.

type BifrostFileListResponse added in v1.2.38 

type BifrostFileListResponse struct {
	Object  string       `json:"object,omitempty"` // "list"
	Data    []FileObject `json:"data"`
	HasMore bool         `json:"has_more,omitempty"`
	After   *string      `json:"after,omitempty"` // Continuation token for pagination

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostFileListResponse represents the response from listing files.

type BifrostFileRetrieveRequest added in v1.2.38 

type BifrostFileRetrieveRequest struct {
	Provider ModelProvider `json:"provider"`
	Model    *string       `json:"model"`

	RawRequestBody []byte `json:"-"` // Raw request body (not serialized)

	FileID string `json:"file_id"` // ID of the file to retrieve

	// Storage configuration (for S3/GCS backends)
	StorageConfig *FileStorageConfig `json:"storage_config,omitempty"`

	// Extra parameters for provider-specific features
	ExtraParams map[string]interface{} `json:"-"`
}

BifrostFileRetrieveRequest represents a request to retrieve file metadata.

func (*BifrostFileRetrieveRequest) GetRawRequestBody added in v1.2.38 

func (request *BifrostFileRetrieveRequest) GetRawRequestBody() []byte

GetRawRequestBody returns the raw request body.

type BifrostFileRetrieveResponse added in v1.2.38 

type BifrostFileRetrieveResponse struct {
	ID            string      `json:"id"`
	Object        string      `json:"object,omitempty"` // "file"
	Bytes         int64       `json:"bytes"`
	CreatedAt     int64       `json:"created_at"`
	UpdatedAt     int64       `json:"updated_at,omitempty"`
	Filename      string      `json:"filename"`
	Purpose       FilePurpose `json:"purpose"`
	Status        FileStatus  `json:"status,omitempty"`
	StatusDetails *string     `json:"status_details,omitempty"`
	ExpiresAt     *int64      `json:"expires_at,omitempty"`

	// Storage backend info
	StorageBackend FileStorageBackend `json:"storage_backend,omitempty"`
	StorageURI     string             `json:"storage_uri,omitempty"`

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostFileRetrieveResponse represents the response from retrieving file metadata.

type BifrostFileUploadRequest added in v1.2.38 

type BifrostFileUploadRequest struct {
	Provider ModelProvider `json:"provider"`
	Model    *string       `json:"model"`

	// File content
	File        []byte      `json:"-"`                      // Raw file content (not serialized)
	Filename    string      `json:"filename"`               // Original filename
	Purpose     FilePurpose `json:"purpose"`                // Purpose of the file (e.g., "batch")
	ContentType *string     `json:"content_type,omitempty"` // MIME type of the file

	// Storage configuration (for S3/GCS backends)
	StorageConfig *FileStorageConfig `json:"storage_config,omitempty"`

	// Expiration configuration (OpenAI only)
	ExpiresAfter *FileExpiresAfter `json:"expires_after,omitempty"`

	// Extra parameters for provider-specific features
	ExtraParams map[string]interface{} `json:"-"`
}

BifrostFileUploadRequest represents a request to upload a file.

type BifrostFileUploadResponse added in v1.2.38 

type BifrostFileUploadResponse struct {
	ID            string      `json:"id"`
	Object        string      `json:"object,omitempty"` // "file"
	Bytes         int64       `json:"bytes"`
	CreatedAt     int64       `json:"created_at"`
	Filename      string      `json:"filename"`
	Purpose       FilePurpose `json:"purpose"`
	Status        FileStatus  `json:"status,omitempty"`
	StatusDetails *string     `json:"status_details,omitempty"`
	ExpiresAt     *int64      `json:"expires_at,omitempty"`

	// Storage backend info
	StorageBackend FileStorageBackend `json:"storage_backend,omitempty"`
	StorageURI     string             `json:"storage_uri,omitempty"` // S3/GCS URI if applicable

	// GCS resumable upload session URL (Vertex only, set when File bytes are not provided).
	// Client PUTs file bytes directly to this URL; Bifrost stays out of the data path.
	UploadURL *string `json:"upload_url,omitempty"`

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostFileUploadResponse represents the response from uploading a file.

type BifrostFinishReason added in v1.3.11 

type BifrostFinishReason string

BifrostFinishReason represents the reason why the model stopped generating. 

const (
	BifrostFinishReasonStop      BifrostFinishReason = "stop"
	BifrostFinishReasonLength    BifrostFinishReason = "length"
	BifrostFinishReasonToolCalls BifrostFinishReason = "tool_calls"
)

BifrostFinishReason values

type BifrostHTTPMiddleware added in v1.3.0 

type BifrostHTTPMiddleware func(next fasthttp.RequestHandler) fasthttp.RequestHandler

BifrostHTTPMiddleware is a middleware function for the Bifrost HTTP transport. It follows the standard pattern: receives the next handler and returns a new handler. Used internally for CORS, Auth, Tracing middleware. Plugins use HTTPTransportIntercept instead.

type BifrostImageEditRequest added in v1.4.0 

type BifrostImageEditRequest struct {
	Provider       ModelProvider        `json:"provider"`
	Model          string               `json:"model"`
	Input          *ImageEditInput      `json:"input"`
	Params         *ImageEditParameters `json:"params,omitempty"`
	Fallbacks      []Fallback           `json:"fallbacks,omitempty"`
	RawRequestBody []byte               `json:"-"`
}

BifrostImageEditRequest represents an image edit request in bifrost format

func (*BifrostImageEditRequest) GetRawRequestBody added in v1.4.0 

func (b *BifrostImageEditRequest) GetRawRequestBody() []byte

GetRawRequestBody implements [utils.RequestBodyGetter].

type BifrostImageGenerationRequest added in v1.3.9 

type BifrostImageGenerationRequest struct {
	Provider       ModelProvider              `json:"provider"`
	Model          string                     `json:"model"`
	Input          *ImageGenerationInput      `json:"input"`
	Params         *ImageGenerationParameters `json:"params,omitempty"`
	Fallbacks      []Fallback                 `json:"fallbacks,omitempty"`
	RawRequestBody []byte                     `json:"-"`
}

BifrostImageGenerationRequest represents an image generation request in bifrost format

func (*BifrostImageGenerationRequest) GetRawRequestBody added in v1.3.9 

func (b *BifrostImageGenerationRequest) GetRawRequestBody() []byte

GetRawRequestBody implements utils.RequestBodyGetter.

type BifrostImageGenerationResponse added in v1.3.9 

type BifrostImageGenerationResponse struct {
	ID      string      `json:"id,omitempty"`
	Created int64       `json:"created,omitempty"`
	Model   string      `json:"model,omitempty"`
	Data    []ImageData `json:"data"`

	*ImageGenerationResponseParameters

	Usage       *ImageUsage                `json:"usage,omitempty"`
	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostImageGenerationResponse represents the image generation response in bifrost format

func (*BifrostImageGenerationResponse) BackfillParams added in v1.4.8 

func (r *BifrostImageGenerationResponse) BackfillParams(req *BifrostRequest)

BackfillParams populates response fields from the original request that are needed for cost calculation but may not be returned by the provider. - NumInputImages on ImageUsage (count of input images from the request) - Size on ImageGenerationResponseParameters (from request params if not in response) - Quality (low, medium, high, auto) only - AspectRatio on ImageGenerationResponseParameters (from request params if not in response)

type BifrostImageGenerationStreamResponse added in v1.3.9 

type BifrostImageGenerationStreamResponse struct {
	ID                string                     `json:"id,omitempty"`
	Type              ImageEventType             `json:"type,omitempty"`
	Index             int                        `json:"-"` // Which image (0-N)
	ChunkIndex        int                        `json:"-"` // Chunk order within image
	PartialImageIndex *int                       `json:"partial_image_index,omitempty"`
	SequenceNumber    int                        `json:"sequence_number,omitempty"`
	B64JSON           string                     `json:"b64_json,omitempty"`
	URL               string                     `json:"url,omitempty"`
	CreatedAt         int64                      `json:"created_at,omitempty"`
	Size              string                     `json:"size,omitempty"`
	AspectRatio       string                     `json:"aspect_ratio,omitempty"`
	Quality           string                     `json:"quality,omitempty"`
	Background        string                     `json:"background,omitempty"`
	OutputFormat      string                     `json:"output_format,omitempty"`
	RevisedPrompt     string                     `json:"revised_prompt,omitempty"`
	Usage             *ImageUsage                `json:"usage,omitempty"`
	Error             *BifrostError              `json:"error,omitempty"`
	RawRequest        string                     `json:"-"`
	RawResponse       string                     `json:"-"`
	ExtraFields       BifrostResponseExtraFields `json:"extra_fields"`
}

Streaming Response

func (*BifrostImageGenerationStreamResponse) BackfillParams added in v1.4.8 

func (r *BifrostImageGenerationStreamResponse) BackfillParams(req *BifrostRequest)

BackfillParams populates response fields from the original request that are needed for cost calculation but may not be returned by the provider. - NumInputImages on ImageUsage (count of input images from the request) - Size on ImageGenerationResponseParameters (from request params if not in response) - Quality (low, medium, high, auto) only - AspectRatio on ImageGenerationResponseParameters (from request params if not in response)

type BifrostImageVariationRequest added in v1.4.0 

type BifrostImageVariationRequest struct {
	Provider       ModelProvider             `json:"provider"`
	Model          string                    `json:"model"`
	Input          *ImageVariationInput      `json:"input"`
	Params         *ImageVariationParameters `json:"params,omitempty"`
	Fallbacks      []Fallback                `json:"fallbacks,omitempty"`
	RawRequestBody []byte                    `json:"-"`
}

BifrostImageVariationRequest represents an image variation request in bifrost format

func (*BifrostImageVariationRequest) GetRawRequestBody added in v1.4.0 

func (b *BifrostImageVariationRequest) GetRawRequestBody() []byte

GetRawRequestBody implements [utils.RequestBodyGetter].

type BifrostImageVariationResponse added in v1.4.0 

type BifrostImageVariationResponse = BifrostImageGenerationResponse

BifrostImageVariationResponse represents the image variation response in bifrost format It uses the same structure as image generation response

type BifrostLLMUsage added in v1.2.7 

type BifrostLLMUsage struct {
	PromptTokens            int                          `json:"prompt_tokens,omitempty"`
	PromptTokensDetails     *ChatPromptTokensDetails     `json:"prompt_tokens_details,omitempty"`
	CompletionTokens        int                          `json:"completion_tokens,omitempty"`
	CompletionTokensDetails *ChatCompletionTokensDetails `json:"completion_tokens_details,omitempty"`
	TotalTokens             int                          `json:"total_tokens"`
	Cost                    *BifrostCost                 `json:"cost,omitempty"` // Only for the providers which support cost calculation
	// Served Anthropic tier (fast mode / data residency), carried internally so
	// cancel/timeout billing (which reads a bare usage via BilledUsage) can apply
	// the tier multiplier. json:"-" keeps them out of every serialized usage payload.
	Speed        *string `json:"-"`
	InferenceGeo *string `json:"-"`
	// Model that actually served the turn after a server-side fallback handoff.
	// Carried here for the same reason as the two above: the bare-usage billing path
	// (CalculateCostForUsage) never sees RoutingInfo, so without it a fallback-served
	// turn is priced at the requested model's rates.
	ServerSideFallbackModel *string `json:"-"`
}

BifrostLLMUsage represents token usage information

func (*BifrostLLMUsage) ToResponsesResponseUsage added in v1.2.13 

func (cu *BifrostLLMUsage) ToResponsesResponseUsage() *ResponsesResponseUsage

type BifrostListModelsRequest added in v1.2.14 

type BifrostListModelsRequest struct {
	Provider ModelProvider `json:"provider"`

	PageSize int `json:"page_size"`

	// PageToken: Token received from previous request to retrieve next page
	PageToken string `json:"page_token"`

	// Unfiltered: If true, the response will include all models for the provider, regardless of the allowed models (internal bifrost use only, not sent to the provider)
	Unfiltered bool `json:"-"`

	// KeyID: If non-nil, scope the call to a single key (matched by Key.ID).
	// Lets callers cache list-models output per-key for fine-grained
	// invalidation. Internal bifrost use only; not sent to the provider.
	//
	// Matching runs against the already-filtered set of supported keys for the
	// provider — keys that are disabled (Enabled == false) or fail validation
	// are excluded before the lookup, so a KeyID referring to such a key
	// produces the same "no key found" error as a KeyID that does not exist
	// at all. Callers needing to distinguish those cases must check the raw
	// account configuration themselves.
	KeyID *string `json:"-"`

	// ExtraParams: Additional provider-specific query parameters
	// This allows for flexibility to pass any custom parameters that specific providers might support
	ExtraParams map[string]interface{} `json:"-"`
}

type BifrostListModelsResponse added in v1.2.14 

type BifrostListModelsResponse struct {
	Data          []Model                    `json:"data"`
	ExtraFields   BifrostResponseExtraFields `json:"extra_fields"`
	NextPageToken string                     `json:"next_page_token,omitempty"` // Token to retrieve next page

	// Key-level status tracking for multi-key providers
	KeyStatuses []KeyStatus `json:"key_statuses,omitempty"`

	// Anthropic specific fields
	FirstID *string `json:"-"`
	LastID  *string `json:"-"`
	HasMore *bool   `json:"-"`
}

func (*BifrostListModelsResponse) ApplyPagination added in v1.2.14 

func (response *BifrostListModelsResponse) ApplyPagination(pageSize int, pageToken string) *BifrostListModelsResponse

ApplyPagination applies offset-based pagination to a BifrostListModelsResponse. Uses opaque tokens with LastID validation to ensure cursor integrity. Returns the paginated response with properly set NextPageToken.

type BifrostLogProbs added in v1.2.7 

type BifrostLogProbs struct {
	Content []ContentLogProb `json:"content,omitempty"`
	Refusal []LogProb        `json:"refusal,omitempty"`

	*TextCompletionLogProb
}

BifrostLogProbs represents the log probabilities for different aspects of a response.

type BifrostMCPConnectRequest added in v1.5.10 

type BifrostMCPConnectRequest struct {
	ClientName       string            // observe-only — name of the client being connected
	ConnectionType   MCPConnectionType // observe-only — transport type being established (http/stdio/sse/inprocess)
	AuthType         MCPAuthType       // observe-only — authentication mode configured on the client
	ConnectionString *string           // mutable — URL for http/sse, nil for stdio/inprocess
	Headers          map[string]string // mutable — transport-level headers (http/sse only; stdio/inprocess ignore)
	StdioCommand     *string           // mutable — command for stdio connections (nil otherwise)
	StdioArgs        []string          // mutable — argv for stdio connections (nil otherwise)
}

BifrostMCPConnectRequest carries the prepared inputs for an MCP connect operation. Fields marked "mutable" may be modified by a plugin's PreMCPHook and the mutated values will be used for the actual transport creation; "observe-only" fields are passed to plugins for context but mutations are ignored (changing the transport type mid-flight would break the rest of the connect codepath).

type BifrostMCPConnectResponse added in v1.5.10 

type BifrostMCPConnectResponse struct {
	ConnectionInfo     *MCPClientConnectionInfo // Connection metadata after the handshake completes
	ServerInfo         *MCPServerInfo           // Name + version from the initialize handshake
	ProtocolVersion    string                   // Negotiated MCP protocol version
	ServerCapabilities *MCPServerCapabilities   // Which MCP feature groups the server claims to support
	ExtraFields        BifrostMCPResponseExtraFields
}

func (*BifrostMCPConnectResponse) PopulateExtraFields added in v1.5.10 

func (r *BifrostMCPConnectResponse) PopulateExtraFields(clientName string)

PopulateExtraFields backfills ClientName on the Connect response when it's not already set. Mirrors BifrostMCPResponse.PopulateExtraFields. Connect has no tool name, so only ClientName is populated.

type BifrostMCPExecuteToolRequest added in v1.5.10 

type BifrostMCPExecuteToolRequest struct {
}

Keeping the stub for now, will be used from the next major bump when we remove the old ChatToolCall and ResponsesToolMessage fields. Note that the tool name and arguments are not standardized in this struct yet since they are still being pulled from the old fields for backward compatibility, but they will be standardized in the future when we remove the old fields.

type BifrostMCPExecuteToolResponse added in v1.5.10 

type BifrostMCPExecuteToolResponse struct {
}

Keeping the stub for now, will be used from the next major bump when we move ChatMessage and ResponsesMessage into this struct.

type BifrostMCPListToolsRequest added in v1.5.10 

type BifrostMCPListToolsRequest struct {
}

BifrostMCPListToolsRequest is intentionally empty for the same reason as ping: list_tools reuses the existing transport's headers. Plugins observe via ClientName and may short-circuit (e.g. cached tool list).

type BifrostMCPListToolsResponse added in v1.5.10 

type BifrostMCPListToolsResponse struct {
	Tools           map[string]ChatTool // Discovered tools keyed by client-prefixed name
	ToolNameMapping map[string]string   // sanitized_name -> original_mcp_name
	RawToolCount    int                 // Count returned by the MCP server before Bifrost-side filtering
	SkippedTools    []SkippedMCPTool    // Tools Bifrost dropped during conversion + reason
}

type BifrostMCPPingRequest added in v1.5.10 

type BifrostMCPPingRequest struct {
}

BifrostMCPPingRequest is intentionally empty: the wire ping rides over the existing transport and has no per-call headers or parameters. Plugins observe via ClientName on the parent BifrostMCPRequest and may short-circuit (synthetic healthy/unhealthy).

type BifrostMCPPingResponse added in v1.5.10 

type BifrostMCPPingResponse struct {
}

type BifrostMCPRequest added in v1.4.0 

type BifrostMCPRequest struct {
	RequestType MCPRequestType
	ClientName  string // MCP client this request targets (always set, regardless of request type)

	*BifrostMCPPingRequest
	*BifrostMCPListToolsRequest

	// [DEPRECATED] these will be replaced by BifrostMCPExecuteToolRequest in the next major bump, but are kept for backward compatibility for now since some tools still rely on the old fields
	*ChatAssistantMessageToolCall
	*ResponsesToolMessage

	// Will be used in from the next major bump
	*BifrostMCPExecuteToolRequest
}

BifrostMCPRequest is the envelope for MCP requests that flow through the generic PreMCPHook/PostMCPHook pipeline (Ping, ListTools, ExecuteTool variants). Connect requests do NOT use this envelope — they are dispatched via the typed MCPConnectionPlugin interface using *BifrostMCPConnectRequest directly.

Exactly one of the embedded sub-request pointers is populated, matched by RequestType:

  • RequestType == MCPRequestTypePing → BifrostMCPPingRequest
  • RequestType == MCPRequestTypeListTools → BifrostMCPListToolsRequest
  • RequestType == MCPRequestTypeExecuteTool / MCPRequestTypeChatToolCall / MCPRequestTypeResponsesToolCall → BifrostMCPExecuteToolRequest

func (*BifrostMCPRequest) GetToolArguments added in v1.4.0 

func (r *BifrostMCPRequest) GetToolArguments() interface{}

func (*BifrostMCPRequest) GetToolName added in v1.4.0 

func (r *BifrostMCPRequest) GetToolName() string

type BifrostMCPResponse added in v1.4.0 

type BifrostMCPResponse struct {
	*BifrostMCPPingResponse
	*BifrostMCPListToolsResponse

	// [DEPRECATED] back-compat fields for execute-tool requests; will move into
	// BifrostMCPExecuteToolResponse in the next major bump.
	ChatMessage      *ChatMessage
	ResponsesMessage *ResponsesMessage

	// Empty stub today; will hold ChatMessage/ResponsesMessage in the next major bump.
	*BifrostMCPExecuteToolResponse

	ExtraFields BifrostMCPResponseExtraFields
}

BifrostMCPResponse is the envelope for MCP responses that flow through the generic PostMCPHook pipeline (Ping, ListTools, ExecuteTool variants). Connect responses do NOT use this envelope — they are dispatched via the typed MCPConnectionPlugin interface using *BifrostMCPConnectResponse directly.

Exactly one of the embedded sub-response pointers is populated, matched by the originating request's RequestType. ExtraFields (ClientName / ToolName / Latency) applies to all envelope variants. For execute-tool requests in the back-compat window, the direct ChatMessage / ResponsesMessage fields are populated instead of any embedded sub-response.

func (*BifrostMCPResponse) PopulateExtraFields added in v1.5.10 

func (r *BifrostMCPResponse) PopulateExtraFields(mcpRequestType MCPRequestType, clientName, toolName string)

PopulateExtraFields backfills ExtraFields.{MCPRequestType, ClientName, ToolName} when they aren't already set on the response. Mirrors BifrostResponse.PopulateExtraFields and is used by every MCP gate to ensure short-circuit responses carry the same attribution as real wire-call responses.

type BifrostMCPResponseExtraFields added in v1.4.0 

type BifrostMCPResponseExtraFields struct {
	MCPRequestType MCPRequestType `json:"mcp_request_type"` // request type this response corresponds to — lets PostMCPHook discriminate ping/list_tools from tool execute on success too
	ClientName     string         `json:"client_name"`
	ToolName       string         `json:"tool_name"` // empty for all but MCPRequestTypeExecuteTool requests for backwards compat, will be a pointer from next major bump.
	Latency        int64          `json:"latency"`   // in milliseconds
}

type BifrostOCRRequest added in v1.4.18 

type BifrostOCRRequest struct {
	Provider       ModelProvider  `json:"provider"`
	Model          string         `json:"model"`
	ID             *string        `json:"id,omitempty"`
	Document       OCRDocument    `json:"document"`
	Params         *OCRParameters `json:"params,omitempty"`
	Fallbacks      []Fallback     `json:"fallbacks,omitempty"`
	RawRequestBody []byte         `json:"-"`
}

BifrostOCRRequest represents a request to perform OCR on a document.

func (*BifrostOCRRequest) GetRawRequestBody added in v1.4.18 

func (r *BifrostOCRRequest) GetRawRequestBody() []byte

GetRawRequestBody returns the raw request body for the OCR request.

type BifrostOCRResponse added in v1.4.18 

type BifrostOCRResponse struct {
	Model              string                     `json:"model"`
	Pages              []OCRPage                  `json:"pages"`
	UsageInfo          *OCRUsageInfo              `json:"usage_info,omitempty"`
	DocumentAnnotation *string                    `json:"document_annotation,omitempty"`
	ExtraFields        BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostOCRResponse represents the response from an OCR request.

type BifrostPassthroughRequest added in v1.4.8 

type BifrostPassthroughRequest struct {
	Provider    ModelProvider // provider extracted from path or body, used for key selection when non-empty
	Model       string        // model extracted from path or body, used for key selection when non-empty
	Method      string
	Path        string // stripped path, e.g. "/v1/fine-tuning/jobs"
	RawQuery    string // raw query string, no "?"
	Body        []byte
	SafeHeaders map[string]string // client headers, auth already stripped
}

type BifrostPassthroughResponse added in v1.4.8 

type BifrostPassthroughResponse struct {
	StatusCode       int
	Headers          map[string]string
	Body             []byte
	ExtraFields      BifrostResponseExtraFields
	Path             string                   // stripped provider path, e.g. "/v1/chat/completions"
	PassthroughUsage *BifrostPassthroughUsage // usage extracted by the provider for billing — set on the unary response (non-streaming) or the final streaming chunk; nil when no billable usage could be extracted
}

type BifrostPassthroughUsage added in v1.5.17 

type BifrostPassthroughUsage struct {
	// Text / chat / responses / embeddings
	LLMUsage     *BifrostLLMUsage
	ServiceTier  *BifrostServiceTier // "priority" | "flex" | nil (default)
	Speed        *string             // "fast" | "standard" — speed actually served (Anthropic fast mode); drives fast-mode billing
	InferenceGeo *string             // "us" | "global" — inference geography served (Anthropic data residency); drives the 1.1x US multiplier

	// Image generation / edit / variation
	ImageUsage   *ImageUsage
	ImageSize    string // e.g. "1024x1024"
	ImageQuality string // "low" | "medium" | "high" | "auto"

	// Speech TTS — character count from request body `input` field
	AudioInputChars int

	// Transcription — token details or raw seconds as duration fallback
	AudioSeconds      *int
	AudioTokenDetails *TranscriptionUsageInputTokenDetails

	// Video generation
	VideoSeconds *int

	// Container creation (code interpreter session) — synthetic pricing identifier,
	// e.g. "container-1g", or "container" when no memory limit is reported. Maps to
	// costInput.containerIdentifierString for the flat per-session fee.
	ContainerIdentifier string
}

BifrostPassthroughUsage carries usage data extracted by the provider at stream completion. The pricing module converts this into cost using the existing compute functions — no new pricing logic is required.

type BifrostRealtimeEvent added in v1.4.8 

type BifrostRealtimeEvent struct {
	Type    RealtimeEventType `json:"type"`
	EventID string            `json:"event_id,omitempty"`

	Session *RealtimeSession `json:"session,omitempty"`
	Item    *RealtimeItem    `json:"item,omitempty"`
	Delta   *RealtimeDelta   `json:"delta,omitempty"`
	Audio   []byte           `json:"audio,omitempty"`
	Error   *RealtimeError   `json:"error,omitempty"`

	// ExtraParams preserves provider-specific top-level event fields that are not
	// promoted into the common Bifrost schema.
	ExtraParams map[string]json.RawMessage `json:"extra_params,omitempty"`

	// RawData preserves the original provider event for pass-through or debugging.
	RawData json.RawMessage `json:"raw_data,omitempty"`
}

BifrostRealtimeEvent is the unified Bifrost envelope for all Realtime events. Provider converters translate between this format and the provider-native protocol.

func ParseRealtimeEvent added in v1.5.1 

func ParseRealtimeEvent(raw []byte) (*BifrostRealtimeEvent, error)

ParseRealtimeEvent decodes a client/provider realtime event while preserving unknown top-level fields in ExtraParams for provider-specific round-tripping.

type BifrostReasoningDetailsType added in v1.2.37 

type BifrostReasoningDetailsType string
const (
	BifrostReasoningDetailsTypeSummary       BifrostReasoningDetailsType = "reasoning.summary"
	BifrostReasoningDetailsTypeEncrypted     BifrostReasoningDetailsType = "reasoning.encrypted"
	BifrostReasoningDetailsTypeText          BifrostReasoningDetailsType = "reasoning.text"
	BifrostReasoningDetailsTypeContentBlocks BifrostReasoningDetailsType = "reasoning.content_blocks"
)

type BifrostRequest 

type BifrostRequest struct {
	RequestType RequestType

	ListModelsRequest            *BifrostListModelsRequest
	TextCompletionRequest        *BifrostTextCompletionRequest
	ChatRequest                  *BifrostChatRequest
	ResponsesRequest             *BifrostResponsesRequest
	ResponsesRetrieveRequest     *BifrostResponsesRetrieveRequest
	ResponsesDeleteRequest       *BifrostResponsesDeleteRequest
	ResponsesCancelRequest       *BifrostResponsesCancelRequest
	ResponsesInputItemsRequest   *BifrostResponsesInputItemsRequest
	CountTokensRequest           *BifrostResponsesRequest
	CompactionRequest            *BifrostCompactionRequest
	EmbeddingRequest             *BifrostEmbeddingRequest
	RerankRequest                *BifrostRerankRequest
	OCRRequest                   *BifrostOCRRequest
	SpeechRequest                *BifrostSpeechRequest
	TranscriptionRequest         *BifrostTranscriptionRequest
	ImageGenerationRequest       *BifrostImageGenerationRequest
	ImageEditRequest             *BifrostImageEditRequest
	ImageVariationRequest        *BifrostImageVariationRequest
	VideoGenerationRequest       *BifrostVideoGenerationRequest
	VideoRetrieveRequest         *BifrostVideoRetrieveRequest
	VideoDownloadRequest         *BifrostVideoDownloadRequest
	VideoListRequest             *BifrostVideoListRequest
	VideoRemixRequest            *BifrostVideoRemixRequest
	VideoDeleteRequest           *BifrostVideoDeleteRequest
	FileUploadRequest            *BifrostFileUploadRequest
	FileListRequest              *BifrostFileListRequest
	FileRetrieveRequest          *BifrostFileRetrieveRequest
	FileDeleteRequest            *BifrostFileDeleteRequest
	FileContentRequest           *BifrostFileContentRequest
	CachedContentCreateRequest   *BifrostCachedContentCreateRequest
	CachedContentListRequest     *BifrostCachedContentListRequest
	CachedContentRetrieveRequest *BifrostCachedContentRetrieveRequest
	CachedContentUpdateRequest   *BifrostCachedContentUpdateRequest
	CachedContentDeleteRequest   *BifrostCachedContentDeleteRequest
	BatchCreateRequest           *BifrostBatchCreateRequest
	BatchListRequest             *BifrostBatchListRequest
	BatchRetrieveRequest         *BifrostBatchRetrieveRequest
	BatchCancelRequest           *BifrostBatchCancelRequest
	BatchResultsRequest          *BifrostBatchResultsRequest
	BatchDeleteRequest           *BifrostBatchDeleteRequest
	ContainerCreateRequest       *BifrostContainerCreateRequest
	ContainerListRequest         *BifrostContainerListRequest
	ContainerRetrieveRequest     *BifrostContainerRetrieveRequest
	ContainerDeleteRequest       *BifrostContainerDeleteRequest
	ContainerFileCreateRequest   *BifrostContainerFileCreateRequest
	ContainerFileListRequest     *BifrostContainerFileListRequest
	ContainerFileRetrieveRequest *BifrostContainerFileRetrieveRequest
	ContainerFileContentRequest  *BifrostContainerFileContentRequest
	ContainerFileDeleteRequest   *BifrostContainerFileDeleteRequest
	PassthroughRequest           *BifrostPassthroughRequest
}

BifrostRequest is the request struct for all bifrost requests. only ONE of the following fields should be set: - ListModelsRequest - TextCompletionRequest - ChatRequest - ResponsesRequest - CountTokensRequest - EmbeddingRequest - RerankRequest - SpeechRequest - TranscriptionRequest - ImageGenerationRequest NOTE: Bifrost Request is submitted back to pool after every use so DO NOT keep references to this struct after use, especially in go routines.

func (*BifrostRequest) GetRequestFields added in v1.2.7 

func (br *BifrostRequest) GetRequestFields() (provider ModelProvider, model string, fallbacks []Fallback)

GetRequestFields returns the provider, model, and fallbacks from the request.

func (*BifrostRequest) SetFallbacks added in v1.2.7 

func (br *BifrostRequest) SetFallbacks(fallbacks []Fallback)

func (*BifrostRequest) SetModel added in v1.2.7 

func (br *BifrostRequest) SetModel(model string)

func (*BifrostRequest) SetProvider added in v1.2.7 

func (br *BifrostRequest) SetProvider(provider ModelProvider)

func (*BifrostRequest) SetRawRequestBody added in v1.2.17 

func (br *BifrostRequest) SetRawRequestBody(rawRequestBody []byte)

type BifrostRerankRequest added in v1.4.4 

type BifrostRerankRequest struct {
	Provider       ModelProvider     `json:"provider"`
	Model          string            `json:"model"`
	Query          string            `json:"query"`
	Documents      []RerankDocument  `json:"documents"`
	Params         *RerankParameters `json:"params,omitempty"`
	Fallbacks      []Fallback        `json:"fallbacks,omitempty"`
	RawRequestBody []byte            `json:"-"`
}

BifrostRerankRequest represents a request to rerank documents by relevance to a query.

func (*BifrostRerankRequest) GetRawRequestBody added in v1.4.4 

func (r *BifrostRerankRequest) GetRawRequestBody() []byte

GetRawRequestBody returns the raw request body for the rerank request.

type BifrostRerankResponse added in v1.4.4 

type BifrostRerankResponse struct {
	ID          string                     `json:"id,omitempty"`
	Results     []RerankResult             `json:"results"`
	Model       string                     `json:"model"`
	Usage       *BifrostLLMUsage           `json:"usage,omitempty"`
	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostRerankResponse represents the response from a rerank request.

type BifrostResponse 

type BifrostResponse struct {
	ListModelsResponse            *BifrostListModelsResponse
	TextCompletionResponse        *BifrostTextCompletionResponse
	ChatResponse                  *BifrostChatResponse
	ResponsesResponse             *BifrostResponsesResponse
	ResponsesStreamResponse       *BifrostResponsesStreamResponse
	ResponsesDeleteResponse       *BifrostResponsesDeleteResponse
	ResponsesInputItemsResponse   *BifrostResponsesInputItemsResponse
	CountTokensResponse           *BifrostCountTokensResponse
	CompactionResponse            *BifrostCompactionResponse
	EmbeddingResponse             *BifrostEmbeddingResponse
	RerankResponse                *BifrostRerankResponse
	OCRResponse                   *BifrostOCRResponse
	SpeechResponse                *BifrostSpeechResponse
	SpeechStreamResponse          *BifrostSpeechStreamResponse
	TranscriptionResponse         *BifrostTranscriptionResponse
	TranscriptionStreamResponse   *BifrostTranscriptionStreamResponse
	ImageGenerationResponse       *BifrostImageGenerationResponse
	ImageGenerationStreamResponse *BifrostImageGenerationStreamResponse
	VideoGenerationResponse       *BifrostVideoGenerationResponse
	VideoDownloadResponse         *BifrostVideoDownloadResponse
	VideoListResponse             *BifrostVideoListResponse
	VideoDeleteResponse           *BifrostVideoDeleteResponse
	FileUploadResponse            *BifrostFileUploadResponse
	FileListResponse              *BifrostFileListResponse
	FileRetrieveResponse          *BifrostFileRetrieveResponse
	FileDeleteResponse            *BifrostFileDeleteResponse
	FileContentResponse           *BifrostFileContentResponse
	CachedContentCreateResponse   *BifrostCachedContentCreateResponse
	CachedContentListResponse     *BifrostCachedContentListResponse
	CachedContentRetrieveResponse *BifrostCachedContentRetrieveResponse
	CachedContentUpdateResponse   *BifrostCachedContentUpdateResponse
	CachedContentDeleteResponse   *BifrostCachedContentDeleteResponse
	BatchCreateResponse           *BifrostBatchCreateResponse
	BatchListResponse             *BifrostBatchListResponse
	BatchRetrieveResponse         *BifrostBatchRetrieveResponse
	BatchCancelResponse           *BifrostBatchCancelResponse
	BatchResultsResponse          *BifrostBatchResultsResponse
	BatchDeleteResponse           *BifrostBatchDeleteResponse
	ContainerCreateResponse       *BifrostContainerCreateResponse
	ContainerListResponse         *BifrostContainerListResponse
	ContainerRetrieveResponse     *BifrostContainerRetrieveResponse
	ContainerDeleteResponse       *BifrostContainerDeleteResponse
	ContainerFileCreateResponse   *BifrostContainerFileCreateResponse
	ContainerFileListResponse     *BifrostContainerFileListResponse
	ContainerFileRetrieveResponse *BifrostContainerFileRetrieveResponse
	ContainerFileContentResponse  *BifrostContainerFileContentResponse
	ContainerFileDeleteResponse   *BifrostContainerFileDeleteResponse
	PassthroughResponse           *BifrostPassthroughResponse
}

BifrostResponse represents the complete result from any bifrost request.

func (*BifrostResponse) GetExtraFields added in v1.2.7 

func (r *BifrostResponse) GetExtraFields() *BifrostResponseExtraFields

func (*BifrostResponse) PopulateExtraFields added in v1.5.1 

func (r *BifrostResponse) PopulateExtraFields(requestType RequestType, provider ModelProvider, originalModelRequested string, resolvedModelUsed string)

PopulateExtraFields sets RequestType, Provider, OriginalModelRequested, and ResolvedModelUsed on the active sub-response. Core always calls this both before and after RunPostLLMHooks, so any plugin modifications to these 4 fields are no-ops — tampering with them inside plugins is discouraged.

func (*BifrostResponse) PopulateRoutingInfo added in v1.5.19 

func (r *BifrostResponse) PopulateRoutingInfo(info RoutingInfo)

PopulateRoutingInfo sets ExtraFields.RoutingInfo on the active sub-response and keeps the deprecated Provider/OriginalModelRequested/ResolvedModelUsed triplet in sync per their documented derivation rules. Core always calls this both before and after RunPostLLMHooks so any plugin modifications are no-ops — tampering with RoutingInfo inside plugins is discouraged.

func (*BifrostResponse) SetFallbackRoutingInfo added in v1.5.19 

func (r *BifrostResponse) SetFallbackRoutingInfo(primaryProvider ModelProvider, primaryModel string)

SetFallbackRoutingInfo marks the active sub-response's RoutingInfo as a fallback attempt and records the primary attempt's provider/model. Also re-syncs the deprecated OriginalModelRequested to the primary model per its documented derivation rule. Called by the orchestrator (handleRequest) on each fallback attempt's result/error — the per-attempt code never sets these fields itself.

type BifrostResponseChoice 

type BifrostResponseChoice struct {
	Index        int              `json:"index"`
	FinishReason *string          `json:"finish_reason,omitempty"`
	LogProbs     *BifrostLogProbs `json:"logprobs,omitempty"`

	*TextCompletionResponseChoice
	*ChatNonStreamResponseChoice
	*ChatStreamResponseChoice
}

BifrostResponseChoice represents a choice in the completion result. This struct can represent either a streaming or non-streaming response choice. IMPORTANT: Only one of TextCompletionResponseChoice, NonStreamResponseChoice or StreamResponseChoice should be non-nil at a time.

type BifrostResponseExtraFields 

type BifrostResponseExtraFields struct {
	RequestType RequestType `json:"request_type"`
	RoutingInfo RoutingInfo `json:"routing_info"`
	// Deprecated: use RoutingInfo.Provider. Still populated for backward
	// compatibility; new consumers should read from RoutingInfo.
	Provider ModelProvider `json:"provider,omitempty"`
	// Deprecated: use RoutingInfo.PrimaryModel when RoutingInfo.IsFallback
	// is true, otherwise RoutingInfo.Model — both branches collapse to the
	// model string the caller sent in the request. Still populated for
	// backward compatibility; new consumers should read from RoutingInfo.
	OriginalModelRequested string `json:"original_model_requested,omitempty"`
	// Deprecated: use RoutingInfo.ResolvedKeyAlias.ModelID when an alias
	// matched (i.e. RoutingInfo.ResolvedKeyAlias != nil), otherwise
	// RoutingInfo.Model. Still populated for backward compatibility; new
	// consumers should read from RoutingInfo.
	ResolvedModelUsed         string             `json:"resolved_model_used,omitempty"`
	Latency                   int64              `json:"latency"`     // in milliseconds (for streaming responses this will be each chunk latency, and the last chunk latency will be the total latency)
	ChunkIndex                int                `json:"chunk_index"` // used for streaming responses to identify the chunk index, will be 0 for non-streaming responses
	RawRequest                interface{}        `json:"raw_request,omitempty"`
	RawResponse               interface{}        `json:"raw_response,omitempty"`
	CacheDebug                *BifrostCacheDebug `json:"cache_debug,omitempty"`
	ParseErrors               []BatchError       `json:"parse_errors,omitempty"` // errors encountered while parsing JSONL batch results
	ConvertedRequestType      RequestType        `json:"converted_request_type,omitempty"`
	DroppedCompatPluginParams []string           `json:"dropped_compat_plugin_params,omitempty"` // params dropped by the compat plugin based on model catalog
	ProviderResponseHeaders   map[string]string  `json:"provider_response_headers,omitempty"`    // HTTP response headers from the provider (filtered to exclude transport-level headers)
	PassthroughPath           string             `json:"passthrough_path,omitempty"`             // Stripped provider path for passthrough requests, e.g. "/v1/chat/completions"
}

BifrostResponseExtraFields contains additional fields in a response.

type BifrostResponsesCancelRequest added in v1.6.3 

type BifrostResponsesCancelRequest struct {
	Provider       ModelProvider `json:"provider"`
	ResponseID     string        `json:"response_id"`
	RawRequestBody []byte        `json:"-"`
}

BifrostResponsesCancelRequest cancels an in-flight stored response (OpenAI POST /v1/responses/{id}/cancel). See BifrostResponsesRetrieveRequest for multi-key pinning guidance.

func (*BifrostResponsesCancelRequest) GetRawRequestBody added in v1.6.3 

func (r *BifrostResponsesCancelRequest) GetRawRequestBody() []byte

GetRawRequestBody implements raw body passthrough when enabled on context.

type BifrostResponsesDeleteRequest added in v1.6.3 

type BifrostResponsesDeleteRequest struct {
	Provider       ModelProvider `json:"provider"`
	ResponseID     string        `json:"response_id"`
	RawRequestBody []byte        `json:"-"`
}

BifrostResponsesDeleteRequest deletes a stored response (OpenAI DELETE /v1/responses/{id}). See BifrostResponsesRetrieveRequest for multi-key pinning guidance.

func (*BifrostResponsesDeleteRequest) GetRawRequestBody added in v1.6.3 

func (r *BifrostResponsesDeleteRequest) GetRawRequestBody() []byte

GetRawRequestBody implements raw body passthrough when enabled on context.

type BifrostResponsesDeleteResponse added in v1.6.3 

type BifrostResponsesDeleteResponse struct {
	ID          string                     `json:"id"`
	Object      string                     `json:"object,omitempty"`
	Deleted     bool                       `json:"deleted"`
	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostResponsesDeleteResponse is the wire shape for a successful delete of a stored response.

type BifrostResponsesInputItemsRequest added in v1.6.3 

type BifrostResponsesInputItemsRequest struct {
	Provider       ModelProvider `json:"provider"`
	ResponseID     string        `json:"response_id"`
	After          string        `json:"after,omitempty"`
	Include        []string      `json:"include,omitempty"`
	Limit          *int          `json:"limit,omitempty"`
	Order          string        `json:"order,omitempty"`
	RawRequestBody []byte        `json:"-"`
}

BifrostResponsesInputItemsRequest lists input items for a response (OpenAI GET /v1/responses/{id}/input_items). See BifrostResponsesRetrieveRequest for multi-key pinning guidance.

func (*BifrostResponsesInputItemsRequest) GetRawRequestBody added in v1.6.3 

func (r *BifrostResponsesInputItemsRequest) GetRawRequestBody() []byte

GetRawRequestBody implements raw body passthrough when enabled on context.

type BifrostResponsesInputItemsResponse added in v1.6.3 

type BifrostResponsesInputItemsResponse struct {
	Object      string                     `json:"object"`
	Data        []ResponsesMessage         `json:"data"`
	HasMore     bool                       `json:"has_more"`
	FirstID     string                     `json:"first_id,omitempty"`
	LastID      string                     `json:"last_id,omitempty"`
	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

BifrostResponsesInputItemsResponse is the list payload for response input items.

type BifrostResponsesRequest added in v1.2.0 

type BifrostResponsesRequest struct {
	Provider       ModelProvider        `json:"provider"`
	Model          string               `json:"model"`
	Input          []ResponsesMessage   `json:"input,omitempty"`
	Params         *ResponsesParameters `json:"params,omitempty"`
	Fallbacks      []Fallback           `json:"fallbacks,omitempty"`
	RawRequestBody []byte               `json:"-"` // set bifrost-use-raw-request-body to true in ctx to use the raw request body. Bifrost will directly send this to the downstream provider.
}

func (*BifrostResponsesRequest) GetRawRequestBody added in v1.2.17 

func (r *BifrostResponsesRequest) GetRawRequestBody() []byte

func (*BifrostResponsesRequest) ToChatRequest added in v1.2.0 

func (brr *BifrostResponsesRequest) ToChatRequest() *BifrostChatRequest

ToChatRequest converts a BifrostResponsesRequest to BifrostChatRequest format

type BifrostResponsesResponse added in v1.2.7 

type BifrostResponsesResponse struct {
	ID     *string `json:"id,omitempty"` // used for internal conversions
	Object string  `json:"object"`       // "response"

	Background           *bool                               `json:"background,omitempty"`
	Conversation         *ResponsesResponseConversation      `json:"conversation,omitempty"`
	CreatedAt            int                                 `json:"created_at"`   // Unix timestamp when Response was created
	CompletedAt          *int                                `json:"completed_at"` // Unix timestamp when Response was completed
	Error                *ResponsesResponseError             `json:"error"`
	Include              []string                            `json:"include,omitempty"`  // Supported values: "web_search_call.action.sources", "code_interpreter_call.outputs", "computer_call_output.output.image_url", "file_search_call.results", "message.input_image.image_url", "message.output_text.logprobs", "reasoning.encrypted_content"
	IncompleteDetails    *ResponsesResponseIncompleteDetails `json:"incomplete_details"` // Details about why the response is incomplete
	Instructions         *ResponsesResponseInstructions      `json:"instructions"`
	MaxOutputTokens      *int                                `json:"max_output_tokens"`
	MaxToolCalls         *int                                `json:"max_tool_calls"`
	Metadata             *map[string]any                     `json:"metadata,omitempty"`
	Model                string                              `json:"model"`
	Output               []ResponsesMessage                  `json:"output"`
	ParallelToolCalls    *bool                               `json:"parallel_tool_calls,omitempty"`
	PreviousResponseID   *string                             `json:"previous_response_id"`
	Prompt               *ResponsesPrompt                    `json:"prompt,omitempty"` // Reference to a prompt template and variables
	PromptCacheKey       *string                             `json:"prompt_cache_key"` // Prompt cache key
	PromptCacheRetention *string                             `json:"prompt_cache_retention,omitempty"`
	PromptCacheOptions   *PromptCacheOptions                 `json:"prompt_cache_options,omitempty"` // Prompt-caching options applied to the response (OpenAI gpt-5.6+)
	PresencePenalty      *float64                            `json:"presence_penalty,omitempty"`
	FrequencyPenalty     *float64                            `json:"frequency_penalty,omitempty"`
	Reasoning            *ResponsesParametersReasoning       `json:"reasoning"`         // Configuration options for reasoning models
	SafetyIdentifier     *string                             `json:"safety_identifier"` // Safety identifier
	ServiceTier          *BifrostServiceTier                 `json:"service_tier"`
	Speed                *string                             `json:"speed,omitempty"`         // "fast" | "standard" — speed actually served (Anthropic fast mode); drives fast-mode billing
	InferenceGeo         *string                             `json:"inference_geo,omitempty"` // "us" | "global" — inference geography served (Anthropic data residency); drives the 1.1x US multiplier
	Diagnostics          *CacheDiagnostics                   `json:"diagnostics,omitempty"`   // Anthropic cache diagnostics (cache-diagnosis-2026-04-07); first prompt-cache prefix divergence point
	Container            *ResponsesResponseContainer         `json:"container,omitempty"`     // Code-execution sandbox container (Anthropic surfaces it on the response / final streaming message_delta). The neutral per-call id also lives on ResponsesCodeInterpreterToolCall.ContainerID.
	Status               *string                             `json:"status,omitempty"`        // completed, failed, in_progress, cancelled, queued, or incomplete
	StreamOptions        *ResponsesStreamOptions             `json:"stream_options,omitempty"`
	StopReason           *string                             `json:"stop_reason,omitempty"`  // Not in OpenAI's spec, but sent by other providers
	StopDetails          *ResponsesStopDetails               `json:"stop_details,omitempty"` // Anthropic refusal detail; null unless stop_reason is "refusal"
	Store                *bool                               `json:"store,omitempty"`
	Temperature          *float64                            `json:"temperature,omitempty"`
	Text                 *ResponsesTextConfig                `json:"text,omitempty"`
	TopLogProbs          *int                                `json:"top_logprobs,omitempty"`
	TopP                 *float64                            `json:"top_p,omitempty"`       // Controls diversity via nucleus sampling
	ToolChoice           *ResponsesToolChoice                `json:"tool_choice,omitempty"` // Whether to call a tool
	Tools                []ResponsesTool                     `json:"tools"`                 // Tools to use
	Truncation           *string                             `json:"truncation,omitempty"`
	Usage                *ResponsesResponseUsage             `json:"usage"`
	ExtraFields          BifrostResponseExtraFields          `json:"extra_fields"`
	ProviderExtraFields  map[string]interface{}              `json:"provider_extra_fields,omitempty"`

	// Perplexity-specific fields
	SearchResults []SearchResult `json:"search_results,omitempty"`
	Videos        []VideoResult  `json:"videos,omitempty"`
	Citations     []string       `json:"citations,omitempty"`
}

func (*BifrostResponsesResponse) BackfillParams added in v1.4.19 

func (resp *BifrostResponsesResponse) BackfillParams(request *BifrostResponsesRequest)

BackfillParams populates response fields from the request that are needed

func (*BifrostResponsesResponse) ToBifrostChatResponse added in v1.2.7 

func (responsesResp *BifrostResponsesResponse) ToBifrostChatResponse() *BifrostChatResponse

ToBifrostChatResponse converts a BifrostResponsesResponse to BifrostChatResponse format This converts Responses API format to Chat-style fields (Choices)

func (*BifrostResponsesResponse) UnmarshalJSON added in v1.5.11 

func (r *BifrostResponsesResponse) UnmarshalJSON(data []byte) error

UnmarshalJSON handles providers that return created_at/completed_at as floats (e.g. Bedrock mantle).

func (*BifrostResponsesResponse) WithDefaults added in v1.3.10 

func (resp *BifrostResponsesResponse) WithDefaults() *BifrostResponsesResponse

type BifrostResponsesRetrieveRequest added in v1.6.3 

type BifrostResponsesRetrieveRequest struct {
	Provider           ModelProvider `json:"provider"`
	ResponseID         string        `json:"response_id"`
	Include            []string      `json:"include,omitempty"`
	StartingAfter      *int          `json:"starting_after,omitempty"`
	IncludeObfuscation *bool         `json:"include_obfuscation,omitempty"`
	RawRequestBody     []byte        `json:"-"`
}

BifrostResponsesRetrieveRequest retrieves a stored response by ID (OpenAI GET /v1/responses/{id}).

Multi-key note: when multiple API keys are configured for the same provider, pin key selection (for example x-bf-api-key-id) on lifecycle calls so they hit the same upstream account as the create that produced response_id.

func (*BifrostResponsesRetrieveRequest) GetRawRequestBody added in v1.6.3 

func (r *BifrostResponsesRetrieveRequest) GetRawRequestBody() []byte

GetRawRequestBody implements raw body passthrough when enabled on context.

type BifrostResponsesStreamResponse added in v1.2.7 

type BifrostResponsesStreamResponse struct {
	Type           ResponsesStreamResponseType `json:"type"`
	SequenceNumber int                         `json:"sequence_number"`

	Response *BifrostResponsesResponse `json:"response,omitempty"`

	OutputIndex *int              `json:"output_index,omitempty"`
	Item        *ResponsesMessage `json:"item"`
	// SummaryIndex identifies which summary block within an item a delta belongs to.
	// Emitted on response.reasoning_summary_text.{delta,done} and
	// response.reasoning_summary_part.{added,done}.
	// See https://platform.openai.com/docs/api-reference/responses-streaming
	SummaryIndex *int `json:"summary_index,omitempty"`

	ContentIndex *int                          `json:"content_index,omitempty"`
	ItemID       *string                       `json:"item_id,omitempty"`
	Part         *ResponsesMessageContentBlock `json:"part,omitempty"`

	Delta     *string `json:"delta,omitempty"`
	Signature *string `json:"signature,omitempty"` // Not in OpenAI's spec, but sent by other providers
	// Obfuscation is random padding added to delta events to normalize payload size as a
	// side-channel mitigation. Toggle via StreamOptions.IncludeObfuscation.
	// See https://platform.openai.com/docs/api-reference/responses-streaming
	Obfuscation *string                                    `json:"obfuscation,omitempty"`
	LogProbs    []ResponsesOutputMessageContentTextLogProb `json:"logprobs"`

	Text *string `json:"text,omitempty"` // Full text of the output item, comes with event "response.output_text.done"

	Refusal *string `json:"refusal,omitempty"`

	Arguments *string `json:"arguments,omitempty"`

	PartialImageB64   *string `json:"partial_image_b64,omitempty"`
	PartialImageIndex *int    `json:"partial_image_index,omitempty"`

	Annotation      *ResponsesOutputMessageContentTextAnnotation `json:"annotation,omitempty"`
	AnnotationIndex *int                                         `json:"annotation_index,omitempty"`

	Error   *ResponsesResponseError `json:"error,omitempty"`
	Code    *string                 `json:"code,omitempty"`
	Message *string                 `json:"message,omitempty"`
	Param   *string                 `json:"param,omitempty"`

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`

	// Perplexity-specific fields
	SearchResults []SearchResult `json:"search_results,omitempty"`
	Videos        []VideoResult  `json:"videos,omitempty"`
	Citations     []string       `json:"citations,omitempty"`
}

func (*BifrostResponsesStreamResponse) ToBifrostChatResponse added in v1.5.2 

func (rsr *BifrostResponsesStreamResponse) ToBifrostChatResponse() *BifrostChatResponse

ToBifrostChatResponse converts a BifrostResponsesStreamResponse chunk to a BifrostChatResponse (chat.completion.chunk).

func (*BifrostResponsesStreamResponse) WithDefaults added in v1.3.10 

func (resp *BifrostResponsesStreamResponse) WithDefaults() *BifrostResponsesStreamResponse

type BifrostServiceTier added in v1.5.11 

type BifrostServiceTier string

BifrostServiceTier represents the service tier for a request/response. 

const (
	BifrostServiceTierAuto        BifrostServiceTier = "auto"
	BifrostServiceTierDefault     BifrostServiceTier = "default"
	BifrostServiceTierFlex        BifrostServiceTier = "flex"
	BifrostServiceTierPriority    BifrostServiceTier = "priority"
	BifrostServiceTierProvisioned BifrostServiceTier = "provisioned"
)

BifrostServiceTier values

type BifrostSpeechRequest added in v1.2.0 

type BifrostSpeechRequest struct {
	Provider       ModelProvider     `json:"provider"`
	Model          string            `json:"model"`
	Input          *SpeechInput      `json:"input,omitempty"`
	Params         *SpeechParameters `json:"params,omitempty"`
	Fallbacks      []Fallback        `json:"fallbacks,omitempty"`
	RawRequestBody []byte            `json:"-"` // set bifrost-use-raw-request-body to true in ctx to use the raw request body. Bifrost will directly send this to the downstream provider.
}

func (*BifrostSpeechRequest) GetRawRequestBody added in v1.2.17 

func (r *BifrostSpeechRequest) GetRawRequestBody() []byte

type BifrostSpeechResponse added in v1.2.7 

type BifrostSpeechResponse struct {
	Audio               []byte                     `json:"audio"`
	Usage               *SpeechUsage               `json:"usage"`
	Alignment           *SpeechAlignment           `json:"alignment,omitempty"`            // Character-level timing information
	NormalizedAlignment *SpeechAlignment           `json:"normalized_alignment,omitempty"` // Character-level timing information for normalized text
	AudioBase64         *string                    `json:"audio_base64,omitempty"`         // Base64-encoded audio (when timestamps are requested)
	ExtraFields         BifrostResponseExtraFields `json:"extra_fields"`
}

func (*BifrostSpeechResponse) BackfillParams added in v1.4.8 

func (r *BifrostSpeechResponse) BackfillParams(request *BifrostSpeechRequest)

type BifrostSpeechStreamResponse added in v1.1.11 

type BifrostSpeechStreamResponse struct {
	Type        SpeechStreamResponseType   `json:"type"`
	Audio       []byte                     `json:"audio"`
	Usage       *SpeechUsage               `json:"usage"`
	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

func (*BifrostSpeechStreamResponse) BackfillParams added in v1.4.8 

func (r *BifrostSpeechStreamResponse) BackfillParams(request *BifrostSpeechRequest)

type BifrostStreamChunk added in v1.3.13 

type BifrostStreamChunk struct {
	*BifrostTextCompletionResponse
	*BifrostChatResponse
	*BifrostResponsesStreamResponse
	*BifrostSpeechStreamResponse
	*BifrostTranscriptionStreamResponse
	*BifrostImageGenerationStreamResponse
	*BifrostPassthroughResponse
	*BifrostError
}

BifrostStreamChunk represents a stream of responses from the Bifrost system. Either BifrostResponse or BifrostError will be non-nil.

func (BifrostStreamChunk) MarshalJSON added in v1.3.13 

func (bs BifrostStreamChunk) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshaling for BifrostStreamChunk. This ensures that only the non-nil embedded struct is marshaled,

type BifrostTextCompletionRequest added in v1.2.0 

type BifrostTextCompletionRequest struct {
	Provider       ModelProvider             `json:"provider"`
	Model          string                    `json:"model"`
	Input          *TextCompletionInput      `json:"input,omitempty"`
	Params         *TextCompletionParameters `json:"params,omitempty"`
	Fallbacks      []Fallback                `json:"fallbacks,omitempty"`
	RawRequestBody []byte                    `json:"-"` // set bifrost-use-raw-request-body to true in ctx to use the raw request body. Bifrost will directly send this to the downstream provider.
}

BifrostTextCompletionRequest is the request struct for text completion requests

func (*BifrostTextCompletionRequest) GetRawRequestBody added in v1.2.17 

func (r *BifrostTextCompletionRequest) GetRawRequestBody() []byte

func (*BifrostTextCompletionRequest) ToBifrostChatRequest added in v1.2.3 

func (r *BifrostTextCompletionRequest) ToBifrostChatRequest() *BifrostChatRequest

ToBifrostChatRequest converts a Bifrost text completion request to a Bifrost chat completion request This method is discouraged to use, but is useful for litellm fallback flows

type BifrostTextCompletionResponse added in v1.2.7 

type BifrostTextCompletionResponse struct {
	ID                string                     `json:"id"`
	Choices           []BifrostResponseChoice    `json:"choices"`
	Created           int                        `json:"created,omitempty"` // The Unix timestamp (in seconds).
	Model             string                     `json:"model"`
	Object            string                     `json:"object"` // "text_completion" (same for text completion stream)
	SystemFingerprint string                     `json:"system_fingerprint"`
	Usage             *BifrostLLMUsage           `json:"usage"`
	ExtraFields       BifrostResponseExtraFields `json:"extra_fields"`
}

type BifrostTranscriptionRequest added in v1.2.0 

type BifrostTranscriptionRequest struct {
	Provider       ModelProvider            `json:"provider"`
	Model          string                   `json:"model"`
	Input          *TranscriptionInput      `json:"input,omitempty"`
	Params         *TranscriptionParameters `json:"params,omitempty"`
	Fallbacks      []Fallback               `json:"fallbacks,omitempty"`
	RawRequestBody []byte                   `json:"-"` // set bifrost-use-raw-request-body to true in ctx to use the raw request body. Bifrost will directly send this to the downstream provider.
}

func (*BifrostTranscriptionRequest) GetRawRequestBody added in v1.2.17 

func (r *BifrostTranscriptionRequest) GetRawRequestBody() []byte

type BifrostTranscriptionResponse added in v1.2.7 

type BifrostTranscriptionResponse struct {
	Duration         *float64                       `json:"duration,omitempty"` // Duration in seconds
	Language         *string                        `json:"language,omitempty"` // e.g., "english"
	LogProbs         []TranscriptionLogProb         `json:"logprobs,omitempty"`
	Segments         []TranscriptionSegment         `json:"segments,omitempty"` // Verbose-json style segments
	DiarizedSegments []TranscriptionDiarizedSegment `json:"-"`                  // Diarized-json style segments (response_format=diarized_json); see MarshalJSON
	Task             *string                        `json:"task,omitempty"`     // e.g., "transcribe"
	Text             string                         `json:"text"`
	Usage            *TranscriptionUsage            `json:"usage,omitempty"`
	Words            []TranscriptionWord            `json:"words,omitempty"`
	ResponseFormat   *string                        `json:"-"` // Set by provider for non-JSON formats (text, srt, vtt); used by integration response converters
	ExtraFields      BifrostResponseExtraFields     `json:"extra_fields"`
}

func (*BifrostTranscriptionResponse) BackfillParams added in v1.5.0 

func (r *BifrostTranscriptionResponse) BackfillParams(req *BifrostTranscriptionRequest)

func (BifrostTranscriptionResponse) MarshalJSON added in v1.6.4 

func (r BifrostTranscriptionResponse) MarshalJSON() ([]byte, error)

MarshalJSON serializes DiarizedSegments (response_format=diarized_json) under the same "segments" key that verbose_json's Segments would otherwise use, since the two shapes are mutually exclusive per request and the consuming client expects them under one field name. Segments and DiarizedSegments cannot share a literal json tag on the struct itself without an encoding/json field-name conflict, hence the shadowing here.

DiarizedSegments is checked for nil (not len>0): OpenAI's diarized_json schema treats "segments" as a required, non-optional array (unlike verbose_json's Optional segments), so a zero-segment diarized response (e.g. silent audio) must still emit "segments":[] rather than omit the key, or OpenAI SDK clients parsing it will fail on a missing required field.

An "is_diarized" marker is also written whenever DiarizedSegments is set, including when empty: an empty JSON array unmarshals successfully into either segment type, so a zero-length diarized response has no shape of its own for UnmarshalJSON to sniff. Real OpenAI SDK clients tolerate the extra field (their generated models use extra="allow"); it exists purely so Bifrost's own round-trip (e.g. framework/logstore's persist/reload) can disambiguate the empty case.

func (*BifrostTranscriptionResponse) UnmarshalJSON added in v1.6.4 

func (r *BifrostTranscriptionResponse) UnmarshalJSON(data []byte) error

UnmarshalJSON is the symmetric counterpart to MarshalJSON: since both Segments and DiarizedSegments serialize under the same "segments" key, the key alone doesn't say which shape was written. This matters beyond the initial provider response - e.g. framework/logstore persists BifrostTranscriptionResponse as JSON and reloads it via GORM's AfterFind hook on every read, so a diarized response must round-trip back into DiarizedSegments (string id/speaker/type), not get mis-decoded as TranscriptionSegment (int id) and fail or silently corrupt the data.

The "is_diarized" marker (see MarshalJSON) is authoritative when present. Data persisted before this marker existed won't have it, so as a fallback the shape is sniffed by attempting verbose_json's int-id segments first and falling back to the diarized string-id shape on a type mismatch - this fallback can't distinguish an empty verbose array from an empty diarized one, which is exactly why new writes always include the marker.

type BifrostTranscriptionStreamResponse added in v1.2.7 

type BifrostTranscriptionStreamResponse struct {
	Delta       *string                         `json:"delta,omitempty"` // For delta events
	LogProbs    []TranscriptionLogProb          `json:"logprobs,omitempty"`
	Text        string                          `json:"text"`
	Type        TranscriptionStreamResponseType `json:"type"`
	Usage       *TranscriptionUsage             `json:"usage,omitempty"`
	ExtraFields BifrostResponseExtraFields      `json:"extra_fields"`
}

BifrostTranscriptionStreamResponse represents streaming specific fields only

type BifrostVideoDeleteRequest added in v1.4.4 

type BifrostVideoDeleteRequest = BifrostVideoReferenceRequest

type BifrostVideoDeleteResponse added in v1.4.4 

type BifrostVideoDeleteResponse struct {
	ID          string                     `json:"id"`
	Deleted     bool                       `json:"deleted"`
	Object      string                     `json:"object,omitempty"` // "video.deleted"
	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

type BifrostVideoDownloadRequest added in v1.4.4 

type BifrostVideoDownloadRequest struct {
	Provider    ModelProvider         `json:"provider"`
	ID          string                `json:"id"`
	Variant     *VideoDownloadVariant `json:"variant,omitempty"`
	ExtraParams map[string]any        `json:"-"`
}

type BifrostVideoDownloadResponse added in v1.4.4 

type BifrostVideoDownloadResponse struct {
	VideoID     string `json:"video_id"`
	Content     []byte `json:"-"`                      // Raw video content (not serialized)
	ContentType string `json:"content_type,omitempty"` // MIME type (e.g., "video/mp4", "image/png" for thumbnails)

	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

type BifrostVideoGenerationRequest added in v1.4.4 

type BifrostVideoGenerationRequest struct {
	Provider       ModelProvider              `json:"provider"`
	Model          string                     `json:"model"`
	Input          *VideoGenerationInput      `json:"input"`
	Params         *VideoGenerationParameters `json:"params,omitempty"`
	Fallbacks      []Fallback                 `json:"fallbacks,omitempty"`
	RawRequestBody []byte                     `json:"-"`
}

func (*BifrostVideoGenerationRequest) GetExtraParams added in v1.4.4 

func (b *BifrostVideoGenerationRequest) GetExtraParams() map[string]interface{}

func (*BifrostVideoGenerationRequest) GetRawRequestBody added in v1.4.4 

func (b *BifrostVideoGenerationRequest) GetRawRequestBody() []byte

type BifrostVideoGenerationResponse added in v1.4.4 

type BifrostVideoGenerationResponse struct {
	ID                 string             `json:"id,omitempty"`
	CompletedAt        *int64             `json:"completed_at,omitempty"`          // Unix timestamp (seconds) when the job completed
	CreatedAt          int64              `json:"created_at,omitempty"`            // Unix timestamp (seconds) when the job was created
	Error              *VideoCreateError  `json:"error,omitempty"`                 // Error payload if generation failed
	ExpiresAt          *int64             `json:"expires_at,omitempty"`            // Unix timestamp (seconds) when downloadable assets expire
	Model              string             `json:"model,omitempty"`                 // Video generation model that produced the job
	Object             string             `json:"object,omitempty"`                // Object type, always "video"
	Progress           *float64           `json:"progress,omitempty"`              // Approximate completion percentage (0-100)
	Prompt             string             `json:"prompt,omitempty"`                // Prompt used to generate the video
	RemixedFromVideoID *string            `json:"remixed_from_video_id,omitempty"` // Source video ID if this is a remix
	Seconds            *string            `json:"seconds,omitempty"`               // Duration of the generated clip in seconds
	Size               string             `json:"size,omitempty"`                  // Resolution of the generated video
	Status             VideoStatus        `json:"status,omitempty"`                // Current lifecycle status of the video job
	Videos             []VideoOutput      `json:"videos,omitempty"`                // Generated videos (supports multiple videos)
	ContentFilter      *ContentFilterInfo `json:"content_filter,omitempty"`        // Information about content filtering (if applicable)

	ExtraFields BifrostResponseExtraFields `json:"extra_fields,omitempty"`
}

BifrostVideoGenerationResponse represents the video generation job response in bifrost format.

func (*BifrostVideoGenerationResponse) BackfillParams added in v1.4.8 

func (r *BifrostVideoGenerationResponse) BackfillParams(req *BifrostRequest)

BackfillParams populates response fields from the original request that are needed for cost calculation but may not be returned by the provider. - Seconds (duration from request params or default)

type BifrostVideoListRequest added in v1.4.4 

type BifrostVideoListRequest struct {
	Provider ModelProvider `json:"provider"`
	After    *string       `json:"after,omitempty"`
	Limit    *int          `json:"limit,omitempty"`
	Order    *string       `json:"order,omitempty"`
}

type BifrostVideoListResponse added in v1.4.4 

type BifrostVideoListResponse struct {
	Object      string                     `json:"object"` // "list"
	Data        []VideoObject              `json:"data"`
	FirstID     *string                    `json:"first_id,omitempty"`
	HasMore     *bool                      `json:"has_more,omitempty"`
	LastID      *string                    `json:"last_id,omitempty"`
	ExtraFields BifrostResponseExtraFields `json:"extra_fields"`
}

type BifrostVideoReferenceRequest added in v1.4.4 

type BifrostVideoReferenceRequest struct {
	Provider ModelProvider `json:"provider"`
	ID       string        `json:"id"`
}

type BifrostVideoRemixRequest added in v1.4.4 

type BifrostVideoRemixRequest struct {
	ID             string                `json:"id"`
	Provider       ModelProvider         `json:"provider"`
	Input          *VideoGenerationInput `json:"input"`
	ExtraParams    map[string]any        `json:"-"`
	RawRequestBody []byte                `json:"-"`
}

func (*BifrostVideoRemixRequest) GetExtraParams added in v1.4.4 

func (b *BifrostVideoRemixRequest) GetExtraParams() map[string]interface{}

func (*BifrostVideoRemixRequest) GetRawRequestBody added in v1.4.4 

func (b *BifrostVideoRemixRequest) GetRawRequestBody() []byte

type BifrostVideoRetrieveRequest added in v1.4.4 

type BifrostVideoRetrieveRequest = BifrostVideoReferenceRequest

type BlackList added in v1.5.0 

type BlackList []string

BlackList is a list of values that are denied. Semantics:

  • "*" (alone) means all values are blocked.
  • Empty list means nothing is blocked.
  • Non-empty list (without "*") means only the listed values are blocked.

func (BlackList) Contains added in v1.5.0 

func (bl BlackList) Contains(value string) bool

func (BlackList) IsBlockAll added in v1.5.0 

func (bl BlackList) IsBlockAll() bool

IsBlockAll reports whether the blacklist contains "*", meaning all values are blocked.

func (BlackList) IsBlocked added in v1.5.0 

func (bl BlackList) IsBlocked(value string) bool

IsBlocked reports whether value is blocked.

func (BlackList) IsEmpty added in v1.5.0 

func (bl BlackList) IsEmpty() bool

IsEmpty reports whether the blacklist has no entries (nothing is blocked).

func (BlackList) Validate added in v1.5.0 

func (bl BlackList) Validate() error

Validate checks that the blacklist is well-formed.

type CacheControl added in v1.2.39 

type CacheControl struct {
	Type  CacheControlType `json:"type"`
	TTL   *string          `json:"ttl,omitempty"`   // "1m" | "1h"
	Scope *string          `json:"scope,omitempty"` // "user" | "global"
}

type CacheControlType added in v1.2.39 

type CacheControlType string
const (
	CacheControlTypeEphemeral CacheControlType = "ephemeral"
)

type CacheDiagnostics added in v1.5.21 

type CacheDiagnostics struct {
	CacheMissReason *CacheMissReason `json:"cache_miss_reason"`
}

CacheDiagnostics is the Anthropic cache-diagnosis response payload (cache-diagnosis-2026-04-07 beta). CacheMissReason is null while the comparison is still pending and, when set, identifies the first prompt-cache prefix divergence point. A nil *CacheDiagnostics means no divergence / not requested.

type CacheMissReason added in v1.5.21 

type CacheMissReason struct {
	Type                   string `json:"type"`
	CacheMissedInputTokens *int   `json:"cache_missed_input_tokens,omitempty"`
}

CacheMissReason identifies the first cache-prefix divergence point. The *_changed types also carry CacheMissedInputTokens; previous_message_not_found and unavailable do not.

type CachePoint added in v1.4.2 

type CachePoint struct {
	Type string  `json:"type"`          // "default"
	TTL  *string `json:"ttl,omitempty"` // "5m" | "1h"
}

CachePoint represents a cache point marker (Bedrock-specific)

type CachedContentObject added in v1.5.8 

type CachedContentObject struct {
	Name              string         `json:"name"`
	DisplayName       string         `json:"display_name,omitempty"`
	Model             string         `json:"model"`
	SystemInstruction any            `json:"system_instruction,omitempty"`
	Contents          []any          `json:"contents,omitempty"`
	Tools             []any          `json:"tools,omitempty"`
	ToolConfig        any            `json:"tool_config,omitempty"`
	CreateTime        string         `json:"create_time,omitempty"`
	UpdateTime        string         `json:"update_time,omitempty"`
	ExpireTime        string         `json:"expire_time,omitempty"`
	UsageMetadata     map[string]any `json:"usage_metadata,omitempty"`
}

CachedContentObject represents a cached content resource as returned by the provider API (Gemini / Vertex AI). The `name` field is the canonical identifier:

  • Google AI Studio: "cachedContents/{id}"
  • Vertex AI: "projects/{p}/locations/{l}/cachedContents/{id}"

type ChatAssistantMessage added in v1.2.0 

type ChatAssistantMessage struct {
	Refusal          *string                          `json:"refusal,omitempty"`
	Audio            *ChatAudioMessageAudio           `json:"audio,omitempty"`
	Reasoning        *string                          `json:"reasoning,omitempty"`
	ReasoningDetails []ChatReasoningDetails           `json:"reasoning_details,omitempty"`
	Annotations      []ChatAssistantMessageAnnotation `json:"annotations,omitempty"`
	ToolCalls        []ChatAssistantMessageToolCall   `json:"tool_calls,omitempty"`
}

ChatAssistantMessage represents a message in a chat conversation.

func (*ChatAssistantMessage) UnmarshalJSON added in v1.2.37 

func (cm *ChatAssistantMessage) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom unmarshalling for ChatAssistantMessage. If Reasoning is non-nil and ReasoningDetails is nil/empty, it adds a single ChatReasoningDetails entry of type "reasoning.text" with the text set to Reasoning.

type ChatAssistantMessageAnnotation added in v1.2.0 

type ChatAssistantMessageAnnotation struct {
	Type        string                                 `json:"type"`
	URLCitation ChatAssistantMessageAnnotationCitation `json:"url_citation"`
}

ChatAssistantMessageAnnotation represents an annotation in a response.

type ChatAssistantMessageAnnotationCitation added in v1.2.0 

type ChatAssistantMessageAnnotationCitation struct {
	StartIndex int          `json:"start_index"`
	EndIndex   int          `json:"end_index"`
	Title      string       `json:"title"`
	URL        *string      `json:"url,omitempty"`
	Text       *string      `json:"text,omitempty"` // Cited text snippet (Bifrost extension; stripped on the OpenAI wire)
	Sources    *interface{} `json:"sources,omitempty"`
	Type       *string      `json:"type,omitempty"`
}

ChatAssistantMessageAnnotationCitation represents a citation in a response.

type ChatAssistantMessageToolCall added in v1.2.0 

type ChatAssistantMessageToolCall struct {
	Index        uint16                               `json:"index"`
	Type         *string                              `json:"type,omitempty"`
	ID           *string                              `json:"id,omitempty"`
	Function     ChatAssistantMessageToolCallFunction `json:"function"`
	ExtraContent json.RawMessage                      `json:"extra_content,omitempty"`
}

ChatAssistantMessageToolCall represents a tool call in a message. ExtraContent preserves provider-specific metadata (e.g. Gemini's thought_signature for multi-turn continuation when extended thinking is active). Stored as json.RawMessage so unknown nested fields are forwarded verbatim through any proxy/gateway layer without loss.

type ChatAssistantMessageToolCallFunction added in v1.2.0 

type ChatAssistantMessageToolCallFunction struct {
	Name      *string `json:"name"`
	Arguments string  `json:"arguments"` // stringified json as retured by OpenAI, might not be a valid JSON always
}

ChatAssistantMessageToolCallFunction represents a call to a function.

type ChatAudioMessageAudio added in v1.2.39 

type ChatAudioMessageAudio struct {
	ID         string `json:"id"`
	Data       string `json:"data"`
	ExpiresAt  int    `json:"expires_at"`
	Transcript string `json:"transcript"`
}

ChatAudioMessageAudio represents audio data in a message.

type ChatAudioParameters added in v1.2.39 

type ChatAudioParameters struct {
	Format string `json:"format,omitempty"` // Format for the audio completion
	Voice  string `json:"voice,omitempty"`  // Voice to use for the audio completion
}

ChatAudioParameters represents the parameters for a chat audio completion. (Only supported by OpenAI Models that support audio input)

type ChatCachedWriteTokenDetails added in v1.5.5 

type ChatCachedWriteTokenDetails struct {
	CachedWriteTokens5m int `json:"cached_write_tokens_5m"`
	CachedWriteTokens1h int `json:"cached_write_tokens_1h"`
}

type ChatCompletionTokensDetails added in v1.2.13 

type ChatCompletionTokensDetails struct {
	TextTokens               int  `json:"text_tokens,omitempty"`
	AcceptedPredictionTokens int  `json:"accepted_prediction_tokens,omitempty"`
	AudioTokens              int  `json:"audio_tokens,omitempty"`
	CitationTokens           *int `json:"citation_tokens,omitempty"`
	NumSearchQueries         *int `json:"num_search_queries,omitempty"`
	ReasoningTokens          int  `json:"reasoning_tokens,omitempty"`
	ImageTokens              *int `json:"image_tokens,omitempty"`
	RejectedPredictionTokens int  `json:"rejected_prediction_tokens,omitempty"`
}

type ChatContainer added in v1.4.20 

type ChatContainer struct {
	ContainerStr    *string
	ContainerObject *ChatContainerObject
}

ChatContainer is the union "container" field on a chat request. Anthropic's API accepts either a plain string (container id) or an object with id + skills[]. Mirrors AnthropicContainer in the provider package.

func (ChatContainer) MarshalJSON added in v1.4.20 

func (c ChatContainer) MarshalJSON() ([]byte, error)

MarshalJSON emits the raw string or the object form directly.

func (*ChatContainer) UnmarshalJSON added in v1.4.20 

func (c *ChatContainer) UnmarshalJSON(data []byte) error

UnmarshalJSON accepts either a plain string or the object form. Uses the build-tag-aware package-level Unmarshal (sonic on native, stdlib json on wasm/tinygo) and clears the inactive union arm on each success so repeated decodes into the same value don't leave both arms populated. JSON null clears both arms. Follows the ChatToolChoice.UnmarshalJSON pattern.

type ChatContainerObject added in v1.4.20 

type ChatContainerObject struct {
	ID     *string              `json:"id,omitempty"`
	Skills []ChatContainerSkill `json:"skills,omitempty"`
}

ChatContainerObject is the object form of ChatContainer. Both fields are optional — ID alone is a bare container reference; adding Skills makes it beta-gated.

type ChatContainerSkill added in v1.4.20 

type ChatContainerSkill struct {
	SkillID string  `json:"skill_id"`
	Type    string  `json:"type"`              // "anthropic" | "custom"
	Version *string `json:"version,omitempty"` // Optional version pin
}

ChatContainerSkill describes one skill attached to a container. Origin: Anthropic container.skills[] (beta skills-2025-10-02).

type ChatContentBlock added in v1.2.0 

type ChatContentBlock struct {
	Type           ChatContentBlockType `json:"type"`
	Text           *string              `json:"text,omitempty"`
	Refusal        *string              `json:"refusal,omitempty"`
	ImageURLStruct *ChatInputImage      `json:"image_url,omitempty"`
	InputAudio     *ChatInputAudio      `json:"input_audio,omitempty"`
	File           *ChatInputFile       `json:"file,omitempty"`

	// Not in OpenAI's schemas, but sent by a few providers (Anthropic, Bedrock are some of them)
	CacheControl *CacheControl `json:"cache_control,omitempty"`
	Citations    *Citations    `json:"citations,omitempty"`

	// PromptCacheBreakpoint marks an explicit prompt-cache breakpoint on this block (OpenAI gpt-5.6+).
	PromptCacheBreakpoint *PromptCacheBreakpoint `json:"prompt_cache_breakpoint,omitempty"`

	// CachePoint is a Bedrock-specific field for standalone cache point blocks
	// When present without other content, this indicates a cache point marker
	CachePoint *CachePoint `json:"cachePoint,omitempty"`
}

ChatContentBlock represents a content block in a message.

func (*ChatContentBlock) UnmarshalJSON added in v1.5.8 

func (c *ChatContentBlock) UnmarshalJSON(data []byte) error

UnmarshalJSON normalizes Anthropic-style document content blocks (`{"type":"document","source":{...}}`) into bifrost's canonical file shape (`{"type":"file","file":{file_data|file_url, file_type}}`) before the default unmarshal runs. This lets every code path - native /v1/chat/completions, drop-in routes, programmatic JSON callers - reuse the existing ChatContentBlockTypeFile branch in provider converters without per-handler shims.

Source variants mapped:

  • {type:"base64", media_type, data} -> File.FileData (raw base64), File.FileType (media_type)
  • {type:"url", url} -> File.FileURL, provider fetches at convert time
  • {type:"text", media_type, data} -> File.FileData (plain text), File.FileType (media_type)
  • {type:"file", file_id} -> File.FileID

Sibling fields (citations, cache_control, cachePoint, title) are preserved. Other type values pass through to the default unmarshal unchanged.

type ChatContentBlockType added in v1.2.0 

type ChatContentBlockType string

ChatContentBlockType represents the type of content block in a message. 

const (
	ChatContentBlockTypeText       ChatContentBlockType = "text"
	ChatContentBlockTypeImage      ChatContentBlockType = "image_url"
	ChatContentBlockTypeInputAudio ChatContentBlockType = "input_audio"
	ChatContentBlockTypeFile       ChatContentBlockType = "file"
	ChatContentBlockTypeRefusal    ChatContentBlockType = "refusal"
)

ChatContentBlockType values

type ChatInputAudio added in v1.2.0 

type ChatInputAudio struct {
	Data   string  `json:"data"`
	Format *string `json:"format,omitempty"`
}

ChatInputAudio represents audio data in a message. Data carries the audio payload as a string (e.g., data URL or provider-accepted encoded content). Format is optional (e.g., "wav", "mp3"); when nil, providers may attempt auto-detection.

type ChatInputFile added in v1.2.0 

type ChatInputFile struct {
	FileData *string `json:"file_data,omitempty"` // Base64 encoded file data
	FileURL  *string `json:"file_url,omitempty"`  // Direct URL to file
	FileID   *string `json:"file_id,omitempty"`   // Reference to uploaded file
	Filename *string `json:"filename,omitempty"`  // Name of the file
	FileType *string `json:"file_type,omitempty"` // Type of the file
}

ChatInputFile represents a file in a message.

type ChatInputImage added in v1.2.0 

type ChatInputImage struct {
	URL    string  `json:"url,omitempty"`
	FileID *string `json:"file_id,omitempty"` // Reference to an uploaded file (in place of URL)
	Detail *string `json:"detail,omitempty"`
}

ChatInputImage represents image data in a message.

type ChatMCPServer added in v1.4.20 

type ChatMCPServer struct {
	Type               string  `json:"type"` // "url"
	URL                string  `json:"url"`
	Name               string  `json:"name"`
	AuthorizationToken *string `json:"authorization_token,omitempty"`
}

ChatMCPServer is an MCP server definition attached to a chat request. Origin: Anthropic mcp_servers[] (mcp-client-2025-11-20 format).

type ChatMCPToolsetConfig added in v1.4.20 

type ChatMCPToolsetConfig struct {
	Enabled      *bool `json:"enabled,omitempty"`
	DeferLoading *bool `json:"defer_loading,omitempty"`
}

ChatMCPToolsetConfig configures an MCP toolset entry (mcp_toolset tool).

type ChatMessage added in v1.2.0 

type ChatMessage struct {
	Name    *string             `json:"name,omitempty"` // for chat completions
	Role    ChatMessageRole     `json:"role,omitempty"`
	Content *ChatMessageContent `json:"content,omitempty"`

	// Embedded pointer structs - when non-nil, their exported fields are flattened into the top-level JSON object
	// IMPORTANT: Only one of the following can be non-nil at a time, otherwise the JSON marshalling will override the common fields
	*ChatToolMessage
	*ChatAssistantMessage
}

ChatMessage represents a message in a chat conversation.

func DeepCopyChatMessage added in v1.2.26 

func DeepCopyChatMessage(original ChatMessage) ChatMessage

DeepCopyChatMessage creates a deep copy of a ChatMessage to prevent shared data mutation between different plugin accumulators

func ToChatMessages added in v1.2.0 

func ToChatMessages(rms []ResponsesMessage) []ChatMessage

ToChatMessages converts a slice of ResponsesMessages back to ChatMessages This handles the aggregation of function_call messages back into assistant messages with tool calls

func (*ChatMessage) ToResponsesMessages added in v1.2.0 

func (cm *ChatMessage) ToResponsesMessages() []ResponsesMessage

ToResponsesMessages converts a ChatMessage to one or more ResponsesMessages This handles the expansion of assistant messages with tool calls into separate function_call messages

func (*ChatMessage) ToResponsesToolMessage added in v1.3.0 

func (cm *ChatMessage) ToResponsesToolMessage() *ResponsesMessage

ToResponsesToolMessage converts a ChatToolMessage (tool execution result) to ResponsesToolMessage format. This creates a function_call_output message suitable for the Responses API.

Returns:

  • *ResponsesMessage: A ResponsesMessage with type=function_call_output containing the tool result

Example: 

chatToolMsg := &ChatMessage{
    Role: ChatMessageRoleTool,
    ChatToolMessage: &ChatToolMessage{
        ToolCallID: Ptr("call-123"),
    },
    Content: &ChatMessageContent{
        ContentStr: Ptr("Result: 30"),
    },
}
responsesMsg := chatToolMsg.ToResponsesToolMessage()

func (*ChatMessage) UnmarshalJSON added in v1.2.37 

func (cm *ChatMessage) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshalling for ChatMessage. This is needed because ChatAssistantMessage has a custom UnmarshalJSON method, which interferes with the JSON library's handling of other fields in ChatMessage.

type ChatMessageContent added in v1.2.0 

type ChatMessageContent struct {
	ContentStr    *string
	ContentBlocks []ChatContentBlock
}

ChatMessageContent represents a content in a message.

func (ChatMessageContent) MarshalJSON added in v1.2.0 

func (mc ChatMessageContent) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshalling for ChatMessageContent. It marshals either ContentStr or ContentBlocks directly without wrapping.

func (*ChatMessageContent) UnmarshalJSON added in v1.2.0 

func (mc *ChatMessageContent) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshalling for ChatMessageContent. It determines whether "content" is a string or array and assigns to the appropriate field. It also handles direct string/array content without a wrapper object.

type ChatMessageRole added in v1.2.0 

type ChatMessageRole string

ChatMessageRole represents the role of a chat message 

const (
	ChatMessageRoleAssistant ChatMessageRole = "assistant"
	ChatMessageRoleUser      ChatMessageRole = "user"
	ChatMessageRoleSystem    ChatMessageRole = "system"
	ChatMessageRoleTool      ChatMessageRole = "tool"
	ChatMessageRoleDeveloper ChatMessageRole = "developer"
)

ChatMessageRole values

type ChatNonStreamResponseChoice added in v1.2.7 

type ChatNonStreamResponseChoice struct {
	Message    *ChatMessage `json:"message"`
	StopString *string      `json:"stop,omitempty"`
}

ChatNonStreamResponseChoice represents a choice in the non-stream response

type ChatParameters added in v1.2.0 

type ChatParameters struct {
	Audio                *ChatAudioParameters  `json:"audio,omitempty"`                 // Audio parameters
	FrequencyPenalty     *float64              `json:"frequency_penalty,omitempty"`     // Penalizes frequent tokens
	LogitBias            *map[string]float64   `json:"logit_bias,omitempty"`            // Bias for logit values
	LogProbs             *bool                 `json:"logprobs,omitempty"`              // Number of logprobs to return
	MaxCompletionTokens  *int                  `json:"max_completion_tokens,omitempty"` // Maximum number of tokens to generate
	Metadata             *map[string]any       `json:"metadata,omitempty"`              // Metadata to be returned with the response
	Modalities           []string              `json:"modalities,omitempty"`            // Modalities to be returned with the response
	N                    *int                  `json:"n,omitempty"`                     // Number of chat completions to generate when supported
	ParallelToolCalls    *bool                 `json:"parallel_tool_calls,omitempty"`
	Prediction           *ChatPrediction       `json:"prediction,omitempty"`             // Predicted output content (OpenAI only)
	PresencePenalty      *float64              `json:"presence_penalty,omitempty"`       // Penalizes repeated tokens
	PromptCacheKey       *string               `json:"prompt_cache_key,omitempty"`       // Prompt cache key
	PromptCacheRetention *string               `json:"prompt_cache_retention,omitempty"` // Prompt cache retention ("in_memory" or "24h")
	PromptCacheOptions   *PromptCacheOptions   `json:"prompt_cache_options,omitempty"`   // Request-wide prompt cache options (OpenAI gpt-5.6+)
	Reasoning            *ChatReasoning        `json:"reasoning,omitempty"`              // Reasoning parameters
	ResponseFormat       *interface{}          `json:"response_format,omitempty"`        // Format for the response
	SafetyIdentifier     *string               `json:"safety_identifier,omitempty"`      // Safety identifier
	Seed                 *int                  `json:"seed,omitempty"`
	ServiceTier          *BifrostServiceTier   `json:"service_tier,omitempty"`
	StreamOptions        *ChatStreamOptions    `json:"stream_options,omitempty"`
	Stop                 []string              `json:"stop,omitempty"`
	Store                *bool                 `json:"store,omitempty"`
	Temperature          *float64              `json:"temperature,omitempty"`
	TopLogProbs          *int                  `json:"top_logprobs,omitempty"`
	TopP                 *float64              `json:"top_p,omitempty"`              // Controls diversity via nucleus sampling
	ToolChoice           *ChatToolChoice       `json:"tool_choice,omitempty"`        // Whether to call a tool
	Tools                []ChatTool            `json:"tools,omitempty"`              // Tools to use
	User                 *string               `json:"user,omitempty"`               // User identifier for tracking
	Verbosity            *string               `json:"verbosity,omitempty"`          // "low" | "medium" | "high"
	WebSearchOptions     *ChatWebSearchOptions `json:"web_search_options,omitempty"` // Web search options (OpenAI; mapped to Google Search grounding on Gemini)

	// Anthropic-native knobs promoted to the neutral layer. These pass through
	// typed to Anthropic-family providers (honored/stripped per ProviderFeatures
	// in core/providers/anthropic/types.go). Non-Anthropic providers (OpenAI
	// etc.) silently ignore them.
	TopK              *int            `json:"top_k,omitempty"`              // Anthropic top_k sampling
	Speed             *string         `json:"speed,omitempty"`              // "fast" (Anthropic fast-mode-2026-02-01 beta, Opus 4.6 only)
	InferenceGeo      *string         `json:"inference_geo,omitempty"`      // Anthropic inference_geo (Claude API only)
	MCPServers        []ChatMCPServer `json:"mcp_servers,omitempty"`        // Anthropic MCP connector (mcp-client-2025-11-20)
	Container         *ChatContainer  `json:"container,omitempty"`          // Anthropic container (string id, or object with skills[] — beta skills-2025-10-02)
	CacheControl      *CacheControl   `json:"cache_control,omitempty"`      // Top-level request cache control (Anthropic family)
	TaskBudget        *ChatTaskBudget `json:"task_budget,omitempty"`        // Anthropic output_config.task_budget (task-budgets-2026-03-13 beta)
	ContextManagement json.RawMessage `json:"context_management,omitempty"` // Anthropic context_management — complex union, passed as raw JSON to the provider layer

	// Dynamic parameters that can be provider-specific, they are directly
	// added to the request as is.
	ExtraParams map[string]interface{} `json:"-"`
}

ChatParameters represents the parameters for a chat completion.

func (*ChatParameters) UnmarshalJSON added in v1.2.37 

func (cp *ChatParameters) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshalling for ChatParameters.

type ChatPrediction added in v1.4.0 

type ChatPrediction struct {
	Type    string      `json:"type"`    // Always "content"
	Content interface{} `json:"content"` // String or array of content parts
}

ChatPrediction represents predicted output content for the model to reference (OpenAI only). Providing prediction content can significantly reduce latency for certain models.

type ChatPromptTokensDetails added in v1.2.13 

type ChatPromptTokensDetails struct {
	TextTokens  int `json:"text_tokens,omitempty"`
	AudioTokens int `json:"audio_tokens,omitempty"`
	ImageTokens int `json:"image_tokens,omitempty"`

	// For Providers which don't separate between cache creation and cache read tokens (like Openai, Gemini, etc), this is the total number of cached tokens read.
	CachedReadTokens        int                          `json:"cached_read_tokens,omitempty"`
	CachedWriteTokens       int                          `json:"cached_write_tokens,omitempty"`
	CachedWriteTokenDetails *ChatCachedWriteTokenDetails `json:"cached_write_token_details,omitempty"`
}

func (ChatPromptTokensDetails) MarshalJSON added in v1.4.5 

func (d ChatPromptTokensDetails) MarshalJSON() ([]byte, error)

MarshalJSON emits cached_tokens (reads only, per the OpenAI spec and mirroring UnmarshalJSON above) alongside the individual fields. Cache writes are reported separately via cached_write_tokens and are excluded from cached_tokens so that OpenAI-spec consumers do not price cache writes as cache reads.

func (*ChatPromptTokensDetails) UnmarshalJSON added in v1.4.5 

func (d *ChatPromptTokensDetails) UnmarshalJSON(data []byte) error

UnmarshalJSON maps OpenAI's cached_tokens into CachedReadTokens for compatibility.

type ChatReasoning added in v1.2.37 

type ChatReasoning struct {
	Enabled   *bool   `json:"enabled,omitempty"`    // Explicitly enable or disable reasoning (required by OpenRouter to disable reasoning for some models)
	Effort    *string `json:"effort,omitempty"`     // "none" |  "minimal" | "low" | "medium" | "high" (any value other than "none" will enable reasoning)
	MaxTokens *int    `json:"max_tokens,omitempty"` // Maximum number of tokens to generate for the reasoning output (required for anthropic)
	Display   *string `json:"display,omitempty"`    // Anthropic thinking.display: "summarized" | "omitted" (requires model support for adaptive thinking)
}

Not in OpenAI's spec, but needed to support extra parameters for reasoning.

type ChatReasoningDetails added in v1.2.37 

type ChatReasoningDetails struct {
	ID        *string                     `json:"id,omitempty"`
	Index     int                         `json:"index"`
	Type      BifrostReasoningDetailsType `json:"type"`
	Summary   *string                     `json:"summary,omitempty"`
	Text      *string                     `json:"text,omitempty"`
	Signature *string                     `json:"signature,omitempty"`
	Data      *string                     `json:"data,omitempty"` // for encrypted data
}

Not in OpenAI's spec, but needed to support inter provider reasoning capabilities.

type ChatStreamOptions added in v1.2.0 

type ChatStreamOptions struct {
	IncludeObfuscation *bool `json:"include_obfuscation,omitempty"`
	IncludeUsage       *bool `json:"include_usage,omitempty"` // Bifrost marks this as true by default
}

ChatStreamOptions represents the stream options for a chat completion.

type ChatStreamResponseChoice added in v1.2.7 

type ChatStreamResponseChoice struct {
	Delta *ChatStreamResponseChoiceDelta `json:"delta,omitempty"` // Partial message info
}

ChatStreamResponseChoice represents a choice in the stream response

type ChatStreamResponseChoiceDelta added in v1.2.7 

type ChatStreamResponseChoiceDelta struct {
	Role             *string                          `json:"role,omitempty"`      // Only in the first chunk
	Content          *string                          `json:"content,omitempty"`   // May be empty string or null
	Refusal          *string                          `json:"refusal,omitempty"`   // Refusal content if any
	Audio            *ChatAudioMessageAudio           `json:"audio,omitempty"`     // Audio data if any
	Reasoning        *string                          `json:"reasoning,omitempty"` // May be empty string or null
	ReasoningDetails []ChatReasoningDetails           `json:"reasoning_details,omitempty"`
	Annotations      []ChatAssistantMessageAnnotation `json:"annotations,omitempty"`   // URL citations from web search
	ToolCalls        []ChatAssistantMessageToolCall   `json:"tool_calls,omitempty"`    // If tool calls used (supports incremental updates)
	ExtraContent     json.RawMessage                  `json:"extra_content,omitempty"` // Provider-specific metadata (e.g. Gemini thought markers, thought_signature)
}

ChatStreamResponseChoiceDelta represents a delta in the stream response

func (*ChatStreamResponseChoiceDelta) UnmarshalJSON added in v1.2.37 

func (d *ChatStreamResponseChoiceDelta) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom unmarshalling for ChatStreamResponseChoiceDelta. If Reasoning is non-nil and ReasoningDetails is nil/empty, it adds a single ChatReasoningDetails entry of type "reasoning.text" with the text set to Reasoning.

type ChatTaskBudget added in v1.4.20 

type ChatTaskBudget struct {
	Type      string `json:"type"`                // Always "tokens"
	Total     int    `json:"total"`               // Total advisory budget
	Remaining *int   `json:"remaining,omitempty"` // Optional client-side counter
}

ChatTaskBudget advises the model of a full-loop token budget. Origin: Anthropic output_config.task_budget (beta task-budgets-2026-03-13).

type ChatToResponsesStreamState added in v1.2.23 

type ChatToResponsesStreamState struct {
	ToolArgumentBuffers   map[string]string // Maps tool call ID to accumulated argument JSON
	ItemIDs               map[string]string // Maps tool call ID to item ID
	ToolCallNames         map[string]string // Maps tool call ID to tool name
	ToolCallIndexToID     map[uint16]string // Maps tool call index to tool call ID (for lookups when ID is missing)
	MessageID             *string           // Message ID from first chunk
	Model                 *string           // Model name
	CreatedAt             int               // Timestamp for created_at consistency
	HasEmittedCreated     bool              // Whether we've emitted response.created
	HasEmittedInProgress  bool              // Whether we've emitted response.in_progress
	TextItemAdded         bool              // Whether text item has been added
	TextItemClosed        bool              // Whether text item has been closed
	TextItemHasContent    bool              // Whether text item has received any content deltas
	TextBuffer            strings.Builder   // Accumulated text deltas for output_text.done/content_part.done
	TextOutputIndex       int               // Output index assigned to the text/message item
	ReasoningItemAdded    bool              // Whether the reasoning item has been opened
	ReasoningItemClosed   bool              // Whether the reasoning item has been closed
	ReasoningOutputIndex  int               // Output index assigned to the reasoning item
	ReasoningBuffer       strings.Builder   // Accumulated reasoning deltas for reasoning output_item.done
	CurrentOutputIndex    int               // Current output index counter
	ToolCallOutputIndices map[string]int    // Maps tool call ID to output index
	SequenceNumber        int               // Monotonic sequence number across all chunks
}

ChatToResponsesStreamState tracks state during Chat-to-Responses streaming conversion

func AcquireChatToResponsesStreamState added in v1.2.23 

func AcquireChatToResponsesStreamState() *ChatToResponsesStreamState

AcquireChatToResponsesStreamState gets a ChatToResponsesStreamState from the pool.

type ChatTool added in v1.2.0 

type ChatTool struct {
	Type         ChatToolType        `json:"type"`
	Function     *ChatToolFunction   `json:"function,omitempty"`      // Function definition (shape 1)
	Custom       *ChatToolCustom     `json:"custom,omitempty"`        // Custom tool definition (shape 2)
	CacheControl *CacheControl       `json:"cache_control,omitempty"` // Cache control for the tool
	Annotations  *MCPToolAnnotations `json:"-"`                       // MCP tool annotations (Bifrost-internal, never forwarded to providers)

	// Anthropic-native tool flags promoted to the neutral layer. All optional;
	// ignored by providers that don't support them. Gating per ProviderFeatures
	// in core/providers/anthropic/types.go.
	DeferLoading        *bool                  `json:"defer_loading,omitempty"`         // Anthropic advanced-tool-use: defer loading of tool definition
	AllowedCallers      []string               `json:"allowed_callers,omitempty"`       // Anthropic advanced-tool-use: which callers can invoke this tool ("direct", "code_execution_20250825", "code_execution_20260120")
	InputExamples       []ChatToolInputExample `json:"input_examples,omitempty"`        // Anthropic tool-examples-2025-10-29: example inputs for the tool
	EagerInputStreaming *bool                  `json:"eager_input_streaming,omitempty"` // Anthropic fine-grained-tool-streaming-2025-05-14: stream input_json_delta before full args are determined (custom tools only)

	// Anthropic server-tool fields (shape 3). All optional; only populated when
	// Type is a server-tool version string. Function tools carry their name
	// inside Function.Name — use omitempty here so Name doesn't double-emit.
	Name string `json:"name,omitempty"`

	// web_search_* and web_fetch_*:
	MaxUses        *int                  `json:"max_uses,omitempty"`
	AllowedDomains []string              `json:"allowed_domains,omitempty"`
	BlockedDomains []string              `json:"blocked_domains,omitempty"`
	UserLocation   *ChatToolUserLocation `json:"user_location,omitempty"`

	// web_fetch_* only:
	MaxContentTokens *int                     `json:"max_content_tokens,omitempty"`
	Citations        *ChatToolCitationsConfig `json:"citations,omitempty"`
	UseCache         *bool                    `json:"use_cache,omitempty"` // web_fetch_20260309+ only

	// computer_*:
	DisplayWidthPx  *int  `json:"display_width_px,omitempty"`
	DisplayHeightPx *int  `json:"display_height_px,omitempty"`
	DisplayNumber   *int  `json:"display_number,omitempty"`
	EnableZoom      *bool `json:"enable_zoom,omitempty"` // computer_20251124 only

	// text_editor_20250728+:
	MaxCharacters *int `json:"max_characters,omitempty"`

	// mcp_toolset:
	MCPServerName string                           `json:"mcp_server_name,omitempty"`
	DefaultConfig *ChatMCPToolsetConfig            `json:"default_config,omitempty"`
	Configs       map[string]*ChatMCPToolsetConfig `json:"configs,omitempty"`
}

ChatTool represents a tool definition.

Three shapes coexist under this type:

  1. OpenAI function tool: Type="function", Function non-nil.
  2. Custom tool: Type="custom", Custom non-nil.
  3. Anthropic server tool: Type=server-tool version string (e.g. "web_search_20260209", "computer_20251124", "mcp_toolset"), Function/Custom nil, Name populated at top level, and the variant-specific fields (MaxUses, DisplayWidthPx, etc.) populated inline.

JSON shape for (3) matches Anthropic's native tool format directly (e.g. {"type":"web_search_20260209","name":"web_search","max_uses":5}).

Custom MarshalJSON/UnmarshalJSON enforce the union invariant:

  • On marshal, fields that don't match Type are cleared on a copy so the wire format always carries exactly one variant. Mixed caller state (e.g. Type="web_search_20260209" with Function also set) gets canonicalized instead of being forwarded ambiguously to providers.
  • On unmarshal, tolerantly accept whatever JSON shape comes in, then normalize the decoded struct so downstream code sees a canonical shape.

func DeepCopyChatTool added in v1.3.0 

func DeepCopyChatTool(original ChatTool) ChatTool

DeepCopyChatTool creates a deep copy of a ChatTool to prevent shared data mutation between different plugin accumulators

func (ChatTool) MarshalJSON added in v1.4.20 

func (t ChatTool) MarshalJSON() ([]byte, error)

MarshalJSON enforces the ChatTool union invariant: exactly one variant's fields are emitted on the wire, matching Type. A mix-state tool (e.g. Type="web_search_20260209" with Function also populated) would otherwise serialize both, and downstream provider converters — which dispatch on the top-level Type/Name shape — could misinterpret or silently forward the stray fields.

func (*ChatTool) ToResponsesTool added in v1.2.0 

func (ct *ChatTool) ToResponsesTool() *ResponsesTool

ToResponsesTool converts a ChatTool to ResponsesTool format

func (*ChatTool) UnmarshalJSON added in v1.4.20 

func (t *ChatTool) UnmarshalJSON(data []byte) error

UnmarshalJSON tolerantly decodes whatever JSON shape arrives, then canonicalizes the struct via normalizeShape so downstream code sees a single-variant result even if the input mixed multiple variants. Resets the receiver before decoding so omitted optional fields from a prior payload don't survive the new decode; mirrors ChatContainer.UnmarshalJSON.

type ChatToolChoice added in v1.2.0 

type ChatToolChoice struct {
	ChatToolChoiceStr    *string
	ChatToolChoiceStruct *ChatToolChoiceStruct
}

func (ChatToolChoice) MarshalJSON added in v1.2.0 

func (ctc ChatToolChoice) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshalling for ChatMessageContent. It marshals either ContentStr or ContentBlocks directly without wrapping.

func (*ChatToolChoice) ToResponsesToolChoice added in v1.2.0 

func (ctc *ChatToolChoice) ToResponsesToolChoice() *ResponsesToolChoice

ToResponsesToolChoice converts a ChatToolChoice to ResponsesToolChoice format

func (*ChatToolChoice) UnmarshalJSON added in v1.2.0 

func (ctc *ChatToolChoice) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshalling for ChatMessageContent. It determines whether "content" is a string or array and assigns to the appropriate field. It also handles direct string/array content without a wrapper object.

type ChatToolChoiceAllowedTools added in v1.2.0 

type ChatToolChoiceAllowedTools struct {
	Mode  string                           `json:"mode"` // "auto" | "required"
	Tools []ChatToolChoiceAllowedToolsTool `json:"tools"`
}

ChatToolChoiceAllowedTools represents a allowed tools choice.

type ChatToolChoiceAllowedToolsTool added in v1.2.0 

type ChatToolChoiceAllowedToolsTool struct {
	Type     string                 `json:"type"` // "function"
	Function ChatToolChoiceFunction `json:"function,omitempty"`
}

ChatToolChoiceAllowedToolsTool represents a allowed tools tool.

type ChatToolChoiceCustom added in v1.2.0 

type ChatToolChoiceCustom struct {
	Name string `json:"name"`
}

ChatToolChoiceCustom represents a custom choice.

type ChatToolChoiceFunction added in v1.2.0 

type ChatToolChoiceFunction struct {
	Name string `json:"name"`
}

ChatToolChoiceFunction represents a function choice.

type ChatToolChoiceStruct added in v1.2.0 

type ChatToolChoiceStruct struct {
	Type         ChatToolChoiceType          `json:"type"`                    // Type of tool choice
	Function     *ChatToolChoiceFunction     `json:"function,omitempty"`      // Function to call if type is ToolChoiceTypeFunction
	Custom       *ChatToolChoiceCustom       `json:"custom,omitempty"`        // Custom tool to call if type is ToolChoiceTypeCustom
	AllowedTools *ChatToolChoiceAllowedTools `json:"allowed_tools,omitempty"` // Allowed tools to call if type is ToolChoiceTypeAllowedTools
}

ChatToolChoiceStruct represents a tool choice.

func (ChatToolChoiceStruct) MarshalJSON added in v1.4.5 

func (s ChatToolChoiceStruct) MarshalJSON() ([]byte, error)

MarshalJSON serializes ChatToolChoiceStruct to JSON, emitting only the "type" field and the active variant. This prevents zero-value fields from unused variants (e.g., "custom", "allowed_tools") from appearing in the output, and ensures consistent field ordering with "type" always first.

func (*ChatToolChoiceStruct) UnmarshalJSON added in v1.4.5 

func (s *ChatToolChoiceStruct) UnmarshalJSON(data []byte) error

UnmarshalJSON deserializes JSON into ChatToolChoiceStruct and cleans up zero-value pointers that sonic may allocate for absent fields.

type ChatToolChoiceType added in v1.2.0 

type ChatToolChoiceType string

ChatToolChoiceType for all providers, make sure to check the provider's documentation to see which tool choices are supported. 

const (
	ChatToolChoiceTypeNone     ChatToolChoiceType = "none"
	ChatToolChoiceTypeAuto     ChatToolChoiceType = "auto"
	ChatToolChoiceTypeAny      ChatToolChoiceType = "any"
	ChatToolChoiceTypeRequired ChatToolChoiceType = "required"
	// ChatToolChoiceTypeFunction means a specific tool must be called
	ChatToolChoiceTypeFunction ChatToolChoiceType = "function"
	// ChatToolChoiceTypeAllowedTools means a specific tool must be called
	ChatToolChoiceTypeAllowedTools ChatToolChoiceType = "allowed_tools"
	// ChatToolChoiceTypeCustom means a custom tool must be called
	ChatToolChoiceTypeCustom ChatToolChoiceType = "custom"
)

ChatToolChoiceType values

type ChatToolCitationsConfig added in v1.4.20 

type ChatToolCitationsConfig struct {
	Enabled *bool `json:"enabled,omitempty"`
}

ChatToolCitationsConfig is the request-side citations config on web_fetch ({"enabled": true/false}). Distinct from response-side text citations.

type ChatToolCustom added in v1.2.0 

type ChatToolCustom struct {
	Format *ChatToolCustomFormat `json:"format,omitempty"` // The input format
}

type ChatToolCustomFormat added in v1.2.0 

type ChatToolCustomFormat struct {
	Type    string                       `json:"type"` // always "text"
	Grammar *ChatToolCustomGrammarFormat `json:"grammar,omitempty"`
}

type ChatToolCustomGrammarFormat added in v1.2.0 

type ChatToolCustomGrammarFormat struct {
	Definition string `json:"definition"` // The grammar definition
	Syntax     string `json:"syntax"`     // "lark" | "regex"
}

ChatToolCustomGrammarFormat - A grammar defined by the user

type ChatToolFunction added in v1.2.0 

type ChatToolFunction struct {
	Name        string                  `json:"name"`                  // Name of the function
	Description *string                 `json:"description,omitempty"` // Description of the parameters
	Parameters  *ToolFunctionParameters `json:"parameters,omitempty"`  // A JSON schema object describing the parameters
	Strict      *bool                   `json:"strict,omitempty"`      // Whether to enforce strict parameter validation
}

ChatToolFunction represents a function definition.

type ChatToolInputExample added in v1.4.20 

type ChatToolInputExample struct {
	Input       json.RawMessage `json:"input"`
	Description *string         `json:"description,omitempty"`
}

ChatToolInputExample is one example input for a tool, shown to the model. Origin: Anthropic tool.input_examples (beta tool-examples-2025-10-29).

type ChatToolMessage added in v1.2.0 

type ChatToolMessage struct {
	ToolCallID *string `json:"tool_call_id,omitempty"`
}

ChatToolMessage represents a tool message in a chat conversation.

type ChatToolType added in v1.2.0 

type ChatToolType string

ChatToolType represents the type of tool. 

const (
	ChatToolTypeFunction ChatToolType = "function"
	ChatToolTypeCustom   ChatToolType = "custom"
)

ChatToolType values

type ChatToolUserLocation added in v1.4.20 

type ChatToolUserLocation struct {
	Type     *string `json:"type,omitempty"` // "approximate"
	City     *string `json:"city,omitempty"`
	Region   *string `json:"region,omitempty"`
	Country  *string `json:"country,omitempty"`
	Timezone *string `json:"timezone,omitempty"`
}

ChatToolUserLocation is the neutral user_location for web_search tools.

type ChatWebSearchOptions added in v1.4.0 

type ChatWebSearchOptions struct {
	SearchContextSize *string                           `json:"search_context_size,omitempty"` // "low" | "medium" | "high"
	UserLocation      *ChatWebSearchOptionsUserLocation `json:"user_location,omitempty"`
	Filters           *ChatWebSearchOptionsFilters      `json:"filters,omitempty"` // Bifrost extension; stripped on the OpenAI wire
}

ChatWebSearchOptions represents web search options for chat completions.

type ChatWebSearchOptionsFilters added in v1.6.4 

type ChatWebSearchOptionsFilters struct {
	AllowedDomains []string `json:"allowed_domains,omitempty"`
	BlockedDomains []string `json:"blocked_domains,omitempty"`

	// Gemini only
	// Filter search results to a specific time range.
	// If users set a start time, they must set an end time (and vice versa).
	TimeRangeFilter *Interval `json:"time_range_filter,omitempty"`
}

ChatWebSearchOptionsFilters represents search filters; mirrors ResponsesToolWebSearchFilters.

type ChatWebSearchOptionsUserLocation added in v1.4.0 

type ChatWebSearchOptionsUserLocation struct {
	Type        string                                       `json:"type"` // "approximate"
	Approximate *ChatWebSearchOptionsUserLocationApproximate `json:"approximate,omitempty"`
}

ChatWebSearchOptionsUserLocation represents user location for web search.

type ChatWebSearchOptionsUserLocationApproximate added in v1.4.0 

type ChatWebSearchOptionsUserLocationApproximate struct {
	City     *string `json:"city,omitempty"`
	Country  *string `json:"country,omitempty"`  // Two-letter ISO country code (e.g., "US")
	Region   *string `json:"region,omitempty"`   // e.g., "California"
	Timezone *string `json:"timezone,omitempty"` // IANA timezone (e.g., "America/Los_Angeles")
}

ChatWebSearchOptionsUserLocationApproximate represents approximate user location details.

type Citations added in v1.3.9 

type Citations struct {
	Enabled *bool `json:"enabled,omitempty"`
}

type CodeModeBindingLevel added in v1.3.0 

type CodeModeBindingLevel string

CodeModeBindingLevel defines how tools are exposed in the VFS for code execution 

const (
	CodeModeBindingLevelServer CodeModeBindingLevel = "server"
	CodeModeBindingLevelTool   CodeModeBindingLevel = "tool"
)

type ConcurrencyAndBufferSize 

type ConcurrencyAndBufferSize struct {
	Concurrency int `json:"concurrency"` // Number of concurrent operations. Also used as the initial pool size for the provider reponses.
	BufferSize  int `json:"buffer_size"` // Size of the buffer
}

ConcurrencyAndBufferSize represents configuration for concurrent operations and buffer sizes.

type ConfigMarshallerPlugin added in v1.5.13 

type ConfigMarshallerPlugin interface {
	BasePlugin

	// MarshalConfigForStorage converts the raw config map (as received from the API)
	// into the canonical DB-storage format (e.g. *SecretVar fields as plain strings).
	MarshalConfigForStorage(config map[string]any) (map[string]any, error)
	// RedactConfig converts a stored config map into the API-response format,
	// masking sensitive literal values.
	// Returns an error if the config cannot be safely redacted; callers must not
	// return the raw map on error (fail-closed).
	RedactConfig(config map[string]any) (map[string]any, error)
}

ConfigMarshallerPlugin is optionally implemented by plugins that need custom config serialization. If a loaded plugin implements this interface, the server calls MarshalConfigForStorage before writing config to the DB, and RedactConfig when building API responses. Plugins that don't implement it are passed through unchanged — no registration or factory required.

type ContainerExpiresAfter added in v1.3.12 

type ContainerExpiresAfter struct {
	Anchor  string `json:"anchor"`  // The anchor point for expiration (e.g., "last_active_at")
	Minutes int    `json:"minutes"` // Number of minutes after anchor point
}

ContainerExpiresAfter represents the expiration configuration for a container.

type ContainerFileObject added in v1.3.12 

type ContainerFileObject struct {
	ID          string `json:"id"`
	Object      string `json:"object,omitempty"` // "container.file"
	Bytes       int64  `json:"bytes"`
	CreatedAt   int64  `json:"created_at"`
	ContainerID string `json:"container_id"`
	Path        string `json:"path"`
	Source      string `json:"source"` // "user" typically
}

ContainerFileObject represents a file within a container.

type ContainerObject added in v1.3.12 

type ContainerObject struct {
	ID           string                 `json:"id"`
	Object       string                 `json:"object,omitempty"` // "container"
	Name         string                 `json:"name"`
	CreatedAt    int64                  `json:"created_at"`
	Status       ContainerStatus        `json:"status,omitempty"`
	ExpiresAfter *ContainerExpiresAfter `json:"expires_after,omitempty"`
	LastActiveAt *int64                 `json:"last_active_at,omitempty"`
	MemoryLimit  string                 `json:"memory_limit,omitempty"` // e.g., "1g", "4g"
	Metadata     map[string]string      `json:"metadata,omitempty"`
}

ContainerObject represents a container object returned by the API.

type ContainerStatus added in v1.3.12 

type ContainerStatus string

ContainerStatus represents the status of a container. 

const (
	ContainerStatusRunning ContainerStatus = "running"
)

type ContentFilterInfo added in v1.4.4 

type ContentFilterInfo struct {
	FilteredCount int      `json:"filtered_count,omitempty"` // Number of items filtered
	Reasons       []string `json:"reasons,omitempty"`        // Human-readable reasons for filtering
}

ContentFilterInfo contains information about content that was filtered due to safety policies. This is a provider-agnostic structure for representing content filtering results.

type ContentLogProb 

type ContentLogProb struct {
	Bytes       []int     `json:"bytes"`
	LogProb     float64   `json:"logprob"`
	Token       string    `json:"token"`
	TopLogProbs []LogProb `json:"top_logprobs"`
}

ContentLogProb represents log probability information for content.

type CustomProviderConfig added in v1.1.26 

type CustomProviderConfig struct {
	CustomProviderKey    string                 `json:"-"`                                // Custom provider key, internally set by Bifrost
	IsKeyLess            bool                   `json:"is_key_less"`                      // Whether the custom provider requires a key (not allowed for Bedrock)
	BaseProviderType     ModelProvider          `json:"base_provider_type"`               // Base provider type
	AllowedRequests      *AllowedRequests       `json:"allowed_requests,omitempty"`       // Allowed requests for the custom provider
	RequestPathOverrides map[RequestType]string `json:"request_path_overrides,omitempty"` // Mapping of request type to its custom path which will override the default path of the provider (not allowed for Bedrock)
}

func (*CustomProviderConfig) IsOperationAllowed added in v1.1.26 

func (cpc *CustomProviderConfig) IsOperationAllowed(operation RequestType) bool

IsOperationAllowed checks if a specific operation is allowed for this custom provider

type DefaultParameters added in v1.2.14 

type DefaultParameters struct {
	Temperature      *float64 `json:"temperature,omitempty"`
	TopP             *float64 `json:"top_p,omitempty"`
	FrequencyPenalty *float64 `json:"frequency_penalty,omitempty"`
}

type Duration added in v1.5.6 

type Duration time.Duration

Duration is a time.Duration value that accepts both human-readable duration strings and plain integer nanosecond values on JSON unmarshal, providing backward compatibility with the default Go JSON encoding of time.Duration.

Accepted unmarshal formats:

  • String: "5s", "500ms", "1m30s", "2h", "0s" — parsed via time.ParseDuration
  • Integer: nanosecond count, identical to the default Go JSON encoding of time.Duration (e.g. 5000000000 = 5s)

MarshalJSON is intentionally NOT overridden. The type marshals as its underlying int64 nanosecond value (identical to time.Duration), preserving the existing JSON API contract for all consumers that read these fields as integers.

func (Duration) D added in v1.5.6 

func (d Duration) D() time.Duration

D returns the underlying time.Duration value.

func (Duration) String added in v1.5.6 

func (d Duration) String() string

String returns a human-readable representation of the duration.

func (*Duration) UnmarshalJSON added in v1.5.6 

func (d *Duration) UnmarshalJSON(data []byte) error

UnmarshalJSON implements json.Unmarshaler. Accepts both a Go duration string (e.g. "5s", "500ms") and an integer nanosecond value for backward compatibility. MarshalJSON is NOT overridden — values encode as int64 nanoseconds, identical to time.Duration.

type EmbeddingData added in v1.2.7 

type EmbeddingData struct {
	Index     int             `json:"index"`
	Object    string          `json:"object"`    // "embedding"
	Embedding EmbeddingStruct `json:"embedding"` // can be string, []float64, [][]float64, []int8, or []int32
}

type EmbeddingInput added in v1.1.7 

type EmbeddingInput struct {
	Text       *string
	Texts      []string
	Embedding  []int
	Embeddings [][]int
}

EmbeddingInput represents the input for an embedding request.

func (*EmbeddingInput) MarshalJSON added in v1.1.36 

func (e *EmbeddingInput) MarshalJSON() ([]byte, error)

func (*EmbeddingInput) UnmarshalJSON added in v1.1.36 

func (e *EmbeddingInput) UnmarshalJSON(data []byte) error

type EmbeddingParameters added in v1.2.0 

type EmbeddingParameters struct {
	EncodingFormat *string `json:"encoding_format,omitempty"` // Format for embedding output (e.g., "float", "base64")
	Dimensions     *int    `json:"dimensions,omitempty"`      // Number of dimensions for embedding output

	// Dynamic parameters that can be provider-specific, they are directly
	// added to the request as is.
	ExtraParams map[string]interface{} `json:"-"`
}

type EmbeddingStruct added in v1.2.7 

type EmbeddingStruct struct {
	// Embedding responses preserve provider precision in normalized API output.
	EmbeddingStr        *string
	EmbeddingArray      []float64
	Embedding2DArray    [][]float64
	EmbeddingInt8Array  []int8  // for int8 / binary formats
	EmbeddingInt32Array []int32 // for uint8 / ubinary formats
}

func (EmbeddingStruct) MarshalJSON added in v1.2.7 

func (be EmbeddingStruct) MarshalJSON() ([]byte, error)

func (*EmbeddingStruct) UnmarshalJSON added in v1.2.7 

func (be *EmbeddingStruct) UnmarshalJSON(data []byte) error

type EnrichmentDim added in v1.6.4 

type EnrichmentDim struct {
	// Name is the canonical short identifier, used verbatim as the Prometheus
	// label and the Datadog metric tag key. The BigQuery column name also equals
	// it unless Column overrides (see below).
	Name string
	// Column is the BigQuery column name when it differs from Name. BigQuery
	// predates the "method" naming and stores it as "request_type"; empty means
	// the column name equals Name.
	Column string
	// SpanAttr is the canonical bifrost.* span-attribute key the dimension is
	// stored under. It is what the record/trace-tier emitters read and what a
	// connector derives its projection from.
	SpanAttr string
	// MetricSafe marks a LOW-cardinality dimension eligible to become a Prometheus
	// label / Datadog metric tag. High-cardinality dims (per-user, arrays) are
	// false and live only on records/traces (BigQuery columns, span attributes).
	MetricSafe bool
	// Multi marks an array-valued dimension — governance can attach several teams/
	// customers/business units to a single request. Array dims are never
	// MetricSafe (they would explode metric series cardinality).
	Multi bool
}

EnrichmentDim describes one identity/context dimension that connectors attach to the telemetry they emit for a request (which team/customer/business unit/ virtual key/etc. the request belongs to).

It is the single source of truth that keeps the CURATED emitters from drifting apart — the ones that hand-pick a dimension list:

  • Prometheus labels (OSS plugins/telemetry)
  • Datadog metric tags (enterprise plugins/datadog, buildMetricTags)
  • BigQuery columns (enterprise plugins/bigquery, traceColumns)

Each of those derives its list from this registry, and a per-connector conformance test asserts the derived list matches — so adding a dimension in one place can't silently leave the others behind.

The GENERIC emitters (otel, kafka, pubsub) project the entire span attribute map and therefore already carry every dimension; they need no derivation and no conformance test here.

`alias` and `routing_engine_used` are in the metric tier but derived only post-response (once model resolution/routing has run). They are attached to the span in framework/tracing and carry a normal (non-empty) SpanAttr, so a record/trace-tier connector can read them like any other dimension.

func MetricSafeEnrichmentDims added in v1.6.4 

func MetricSafeEnrichmentDims() []EnrichmentDim

MetricSafeEnrichmentDims returns the low-cardinality dimensions eligible to be Prometheus labels / Datadog metric tags, in registry order.

func (EnrichmentDim) ColumnName added in v1.6.4 

func (d EnrichmentDim) ColumnName() string

ColumnName returns the BigQuery column name for the dimension — Column when set, otherwise Name.

type Entity added in v1.5.11 

type Entity string

Entity identifies the target row family that a VisibilityFilter is being resolved for. The provider closure stashed on the request context returns only the dimensions relevant to the requested entity, leaving every other dimension nil. This keeps OSS query helpers from having to reason about dimensions their target table cannot consume. 

const (
	EntityVirtualKey       Entity = "virtual_key"
	EntityPrompt           Entity = "prompt"
	EntityLog              Entity = "log"
	EntityRoutingRule      Entity = "routing_rule"
	EntityUser             Entity = "user"
	EntityRole             Entity = "role"
	EntityTeam             Entity = "team"
	EntityBusinessUnit     Entity = "business_unit"
	EntityCustomer         Entity = "customer"
	EntityAuditLog         Entity = "audit_log"
	EntityAccessProfile    Entity = "access_profile"
	EntityAPIKey           Entity = "api_key"
	EntityMCPToolGroup     Entity = "mcp_tool_group"
	EntityPromptDeployment Entity = "prompt_deployment"
	EntityProvider         Entity = "provider"
)

Entity values cover every list/read endpoint that participates in row-level visibility filtering. Add a new constant only when a query helper for a new table is introduced.

type ErrorField 

type ErrorField struct {
	Type    *string     `json:"type,omitempty"`
	Code    *string     `json:"code,omitempty"`
	Message string      `json:"message"`
	Error   error       `json:"-"`
	Param   interface{} `json:"param,omitempty"`
	EventID *string     `json:"event_id,omitempty"`
}

ErrorField represents detailed error information.

func (*ErrorField) MarshalJSON added in v1.2.9 

func (e *ErrorField) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshaling for ErrorField. It converts the Error field (error interface) to a string.

func (*ErrorField) UnmarshalJSON added in v1.2.9 

func (e *ErrorField) UnmarshalJSON(data []byte) error

type EventBroadcaster added in v1.4.1 

type EventBroadcaster func(eventType string, data interface{})

EventBroadcaster is a generic callback for broadcasting typed events to connected clients (e.g., via WebSocket). Any plugin or subsystem can use this to push real-time updates to the frontend. eventType identifies the message (e.g., "governance_update"), data is the JSON-serializable payload.

type Fallback 

type Fallback struct {
	Provider ModelProvider `json:"provider"`
	Model    string        `json:"model"`
}

Fallback represents a fallback model to be used if the primary model is not available.

func ParseFallbacks added in v1.2.27 

func ParseFallbacks(fallbacks []string) []Fallback

ParseFallbacks parses a slice of strings into a slice of Fallback structs

type FileExpiresAfter added in v1.4.0 

type FileExpiresAfter struct {
	Anchor  string `json:"anchor"`  // e.g., "created_at"
	Seconds int    `json:"seconds"` // 3600-2592000 (1 hour to 30 days)
}

FileExpiresAfter represents an expiration configuration for uploaded files.

type FileObject added in v1.2.38 

type FileObject struct {
	ID            string      `json:"id"`
	Object        string      `json:"object,omitempty"` // "file"
	Bytes         int64       `json:"bytes"`
	CreatedAt     int64       `json:"created_at"`
	UpdatedAt     int64       `json:"updated_at,omitempty"`
	Filename      string      `json:"filename"`
	Purpose       FilePurpose `json:"purpose"`
	Status        FileStatus  `json:"status,omitempty"`
	StatusDetails *string     `json:"status_details,omitempty"`
	ExpiresAt     *int64      `json:"expires_at,omitempty"`
}

FileObject represents a file object returned by the API.

type FilePurpose added in v1.2.38 

type FilePurpose string

FilePurpose represents the purpose of an uploaded file. 

const (
	FilePurposeBatch       FilePurpose = "batch"
	FilePurposeAssistants  FilePurpose = "assistants"
	FilePurposeFineTune    FilePurpose = "fine-tune"
	FilePurposeVision      FilePurpose = "vision"
	FilePurposeBatchOutput FilePurpose = "batch_output"
	FilePurposeUserData    FilePurpose = "user_data"
	FilePurposeResponses   FilePurpose = "responses"
	FilePurposeEvals       FilePurpose = "evals"
)

type FileStatus added in v1.2.38 

type FileStatus string

FileStatus represents the status of a file. 

const (
	FileStatusUploaded      FileStatus = "uploaded"
	FileStatusProcessed     FileStatus = "processed"
	FileStatusProcessing    FileStatus = "processing"
	FileStatusPendingUpload FileStatus = "pending_upload" // resumable session minted, bytes not yet received (Vertex)
	FileStatusError         FileStatus = "error"
	FileStatusDeleted       FileStatus = "deleted"
)

type FileStorageBackend added in v1.2.38 

type FileStorageBackend string

FileStorageBackend represents the storage backend type. 

const (
	FileStorageAPI    FileStorageBackend = "api"    // OpenAI/Azure REST API
	FileStorageS3     FileStorageBackend = "s3"     // AWS S3
	FileStorageGCS    FileStorageBackend = "gcs"    // Google Cloud Storage
	FileStorageMemory FileStorageBackend = "memory" // In-memory (for Anthropic virtual files)
)

type FileStorageConfig added in v1.2.38 

type FileStorageConfig struct {
	S3  *S3StorageConfig  `json:"s3,omitempty"`
	GCS *GCSStorageConfig `json:"gcs,omitempty"`
}

FileStorageConfig represents storage configuration for cloud storage backends.

type GCSStorageConfig added in v1.2.38 

type GCSStorageConfig struct {
	Bucket  string `json:"bucket,omitempty"`
	Project string `json:"project,omitempty"`
	Prefix  string `json:"prefix,omitempty"`
}

GCSStorageConfig represents Google Cloud Storage configuration.

type HTTPRequest added in v1.3.4 

type HTTPRequest struct {
	Method     string            `json:"method"`
	Path       string            `json:"path"`
	Headers    map[string]string `json:"headers"`
	Query      map[string]string `json:"query"`
	Body       []byte            `json:"body"`
	PathParams map[string]string `json:"path_params"` // Path variables extracted from the URL pattern (e.g., {model})
}

HTTPRequest is a serializable representation of an HTTP request. Used for plugin HTTP transport interception (supports both native .so and WASM plugins). This type is pooled for allocation control - use AcquireHTTPRequest and ReleaseHTTPRequest.

func AcquireHTTPRequest added in v1.3.4 

func AcquireHTTPRequest() *HTTPRequest

AcquireHTTPRequest gets an HTTPRequest from the pool. The returned HTTPRequest is ready to use with pre-allocated maps. Call ReleaseHTTPRequest when done to return it to the pool.

func (*HTTPRequest) CaseInsensitiveHeaderLookup added in v1.3.9 

func (req *HTTPRequest) CaseInsensitiveHeaderLookup(key string) string

CaseInsensitiveHeaderLookup looks up a header key in a case-insensitive manner

func (*HTTPRequest) CaseInsensitivePathParamLookup added in v1.3.9 

func (req *HTTPRequest) CaseInsensitivePathParamLookup(key string) string

CaseInsensitivePathParamLookup looks up a path parameter key in a case-insensitive manner

func (*HTTPRequest) CaseInsensitiveQueryLookup added in v1.3.9 

func (req *HTTPRequest) CaseInsensitiveQueryLookup(key string) string

CaseInsensitiveQueryLookup looks up a query key in a case-insensitive manner

type HTTPResponse added in v1.3.4 

type HTTPResponse struct {
	StatusCode int               `json:"status_code"`
	Headers    map[string]string `json:"headers"`
	Body       []byte            `json:"body"`
}

HTTPResponse is a serializable representation of an HTTP response. Used for short-circuit responses in plugin HTTP transport interception. This type is pooled for allocation control - use AcquireHTTPResponse and ReleaseHTTPResponse.

func AcquireHTTPResponse added in v1.3.9 

func AcquireHTTPResponse() *HTTPResponse

AcquireHTTPResponse gets an HTTPResponse from the pool. The returned HTTPResponse is ready to use with a pre-allocated Headers map. Call ReleaseHTTPResponse when done to return it to the pool.

type HTTPTransportPlugin added in v1.4.0 

type HTTPTransportPlugin interface {
	BasePlugin

	// HTTPTransportPreHook is called at the HTTP transport layer before requests enter Bifrost core.
	// It receives a serializable HTTPRequest and allows plugins to modify it in-place.
	// Only invoked when using HTTP transport (bifrost-http), not when using Bifrost as a Go SDK directly.
	// Works with both native .so plugins and WASM plugins due to serializable types.
	//
	// Return values:
	// - (nil, nil): Continue to next plugin/handler, request modifications are applied
	// - (*HTTPResponse, nil): Short-circuit with this response, skip remaining plugins and provider call
	// - (nil, error): Short-circuit with error response
	//
	// Return nil for both values if the plugin doesn't need HTTP transport interception.
	HTTPTransportPreHook(ctx *BifrostContext, req *HTTPRequest) (*HTTPResponse, error)

	// HTTPTransportPostHook is called at the HTTP transport layer after requests exit Bifrost core.
	// It receives a serializable HTTPRequest and HTTPResponse and allows plugins to modify it in-place.
	// Only invoked when using HTTP transport (bifrost-http), not when using Bifrost as a Go SDK directly.
	// Works with both native .so plugins and WASM plugins due to serializable types.
	// NOTE: This hook is NOT called for streaming responses. Use HTTPTransportStreamChunkHook instead.
	// NOTE: For large streamed responses (non-streaming APIs that switch to body streaming for memory safety),
	// resp.Body may be nil by design while StatusCode and Headers remain populated.
	//
	// Return values:
	// - nil: Continue to next plugin/handler, response modifications are applied
	// - error: Short-circuit with error response and skip remaining plugins
	//
	// Return nil if the plugin doesn't need HTTP transport interception.
	HTTPTransportPostHook(ctx *BifrostContext, req *HTTPRequest, resp *HTTPResponse) error

	// HTTPTransportStreamChunkHook is called for each chunk during streaming responses.
	// It receives the BifrostStreamChunk BEFORE they are written to the client.
	// Only invoked for streaming responses when using HTTP transport (bifrost-http).
	// Works with both native .so plugins and WASM plugins due to serializable types.
	//
	// Plugins can modify the chunk by returning a different BifrostStreamChunk.
	// Return the original chunk unchanged if no modification is needed.
	//
	// Return values:
	// - (*BifrostStreamChunk, nil): Continue with the (potentially modified) BifrostStreamChunk
	// - (nil, nil): Skip this BifrostStreamChunk entirely (don't send to client)
	// - (*BifrostStreamChunk, error): Log warning and continue with the BifrostStreamChunk
	// - (nil, error): Send back error to the client and stop the streaming
	//
	// Return (*BifrostStreamChunk, nil) unchanged if the plugin doesn't need streaming chunk interception.
	HTTPTransportStreamChunkHook(ctx *BifrostContext, req *HTTPRequest, chunk *BifrostStreamChunk) (*BifrostStreamChunk, error)
}

type ImageContentType added in v1.0.10 

type ImageContentType string

ImageContentType represents the type of image content 

const (
	ImageContentTypeBase64 ImageContentType = "base64"
	ImageContentTypeURL    ImageContentType = "url"
)

type ImageData added in v1.3.9 

type ImageData struct {
	URL           string `json:"url,omitempty"`
	B64JSON       string `json:"b64_json,omitempty"`
	RevisedPrompt string `json:"revised_prompt,omitempty"`
	Index         int    `json:"index"`
}

type ImageEditInput added in v1.4.0 

type ImageEditInput struct {
	Images []ImageInput `json:"images"`
	Prompt string       `json:"prompt"`
}

type ImageEditParameters added in v1.4.0 

type ImageEditParameters struct {
	Type              *string                `json:"type,omitempty"`           // "inpainting", "outpainting", "background_removal", "remove_background", "erase_object", "recolor", "search_replace", "control_sketch", "control_structure", "style_guide", "style_transfer", "upscale_fast", "upscale_creative", "upscale_conservative"
	Background        *string                `json:"background,omitempty"`     // "transparent", "opaque", "auto"
	InputFidelity     *string                `json:"input_fidelity,omitempty"` // "low", "high"
	Mask              []byte                 `json:"mask,omitempty"`
	N                 *int                   `json:"n,omitempty"`                  // number of images to generate (1-10)
	OutputCompression *int                   `json:"output_compression,omitempty"` // compression level (0-100%)
	OutputFormat      *string                `json:"output_format,omitempty"`      // "png", "webp", "jpeg"
	PartialImages     *int                   `json:"partial_images,omitempty"`     // 0-3
	Quality           *string                `json:"quality,omitempty"`            // "auto", "high", "medium", "low", "standard"
	ResponseFormat    *string                `json:"response_format,omitempty"`    // "url", "b64_json"
	Size              *string                `json:"size,omitempty"`               // "256x256", "512x512", "1024x1024", "1536x1024", "1024x1536", "auto"
	User              *string                `json:"user,omitempty"`
	NegativePrompt    *string                `json:"negative_prompt,omitempty"`     // negative prompt for image editing
	Seed              *int                   `json:"seed,omitempty"`                // seed for image editing
	NumInferenceSteps *int                   `json:"num_inference_steps,omitempty"` // number of inference steps
	ExtraParams       map[string]interface{} `json:"-"`
}

type ImageEventType added in v1.4.0 

type ImageEventType string
const (
	ImageGenerationEventTypePartial   ImageEventType = "image_generation.partial_image"
	ImageGenerationEventTypeCompleted ImageEventType = "image_generation.completed"
	ImageGenerationEventTypeError     ImageEventType = "error"
	ImageEditEventTypePartial         ImageEventType = "image_edit.partial_image"
	ImageEditEventTypeCompleted       ImageEventType = "image_edit.completed"
	ImageEditEventTypeError           ImageEventType = "error"
)

type ImageGenerationInput added in v1.3.9 

type ImageGenerationInput struct {
	Prompt string `json:"prompt"`
}

type ImageGenerationParameters added in v1.3.9 

type ImageGenerationParameters struct {
	N                 *int                   `json:"n,omitempty"`                   // Number of images (1-10)
	Background        *string                `json:"background,omitempty"`          // "transparent", "opaque", "auto"
	Moderation        *string                `json:"moderation,omitempty"`          // "low", "auto"
	PartialImages     *int                   `json:"partial_images,omitempty"`      // 0-3
	Size              *string                `json:"size,omitempty"`                // "256x256", "512x512", "1024x1024", "1792x1024", "1024x1792", "1536x1024", "1024x1536", "auto"
	Quality           *string                `json:"quality,omitempty"`             // "auto", "high", "medium", "low", "hd", "standard"
	OutputCompression *int                   `json:"output_compression,omitempty"`  // compression level (0-100%)
	OutputFormat      *string                `json:"output_format,omitempty"`       // "png", "webp", "jpeg"
	Style             *string                `json:"style,omitempty"`               // "natural", "vivid"
	ResponseFormat    *string                `json:"response_format,omitempty"`     // "url", "b64_json"
	Seed              *int                   `json:"seed,omitempty"`                // seed for image generation
	NegativePrompt    *string                `json:"negative_prompt,omitempty"`     // negative prompt for image generation
	NumInferenceSteps *int                   `json:"num_inference_steps,omitempty"` // number of inference steps
	User              *string                `json:"user,omitempty"`
	InputImages       []string               `json:"input_images,omitempty"` // input images for image generation, base64 encoded or URL
	AspectRatio       *string                `json:"aspect_ratio,omitempty"` // aspect ratio of the image
	ExtraParams       map[string]interface{} `json:"-"`
}

type ImageGenerationResponseParameters added in v1.3.9 

type ImageGenerationResponseParameters struct {
	Background    string    `json:"background,omitempty"`
	OutputFormat  string    `json:"output_format,omitempty"`
	Quality       string    `json:"quality,omitempty"`
	Size          string    `json:"size,omitempty"`
	AspectRatio   string    `json:"aspect_ratio,omitempty"`
	FinishReasons []*string `json:"finish_reasons,omitempty"`
	Seeds         []int     `json:"seeds,omitempty"`
}

type ImageInput added in v1.4.0 

type ImageInput struct {
	Image []byte `json:"image"`
}

type ImageTokenDetails added in v1.3.9 

type ImageTokenDetails struct {
	NImages     int `json:"-"` // Number of images generated (used internally for bifrost)
	ImageTokens int `json:"image_tokens,omitempty"`
	TextTokens  int `json:"text_tokens,omitempty"`
}

type ImageUsage added in v1.3.9 

type ImageUsage struct {
	InputTokens         int                `json:"input_tokens,omitempty"` // Always text tokens unless InputTokensDetails is not nil
	InputTokensDetails  *ImageTokenDetails `json:"input_tokens_details,omitempty"`
	TotalTokens         int                `json:"total_tokens,omitempty"`
	OutputTokens        int                `json:"output_tokens,omitempty"` // Always image tokens unless OutputTokensDetails is not nil
	OutputTokensDetails *ImageTokenDetails `json:"output_tokens_details,omitempty"`
	NumInputImages      int                `json:"-"` // Number of input images from the request (populated by Bifrost)
}

func (*ImageUsage) DeepCopy added in v1.5.19 

func (u *ImageUsage) DeepCopy() *ImageUsage

DeepCopy returns an independent copy of u with no shared pointer fields, safe for callers (e.g. cost calculation) that need to derive values without mutating the original response. Returns nil for a nil receiver.

type ImageVariationInput added in v1.4.0 

type ImageVariationInput struct {
	Image ImageInput `json:"image"`
}

type ImageVariationParameters added in v1.4.0 

type ImageVariationParameters struct {
	N              *int                   `json:"n,omitempty"`               // Number of images (1-10)
	ResponseFormat *string                `json:"response_format,omitempty"` // "url", "b64_json"
	Size           *string                `json:"size,omitempty"`            // "256x256", "512x512", "1024x1024", "1792x1024", "1024x1792", "1536x1024", "1024x1536", "auto"
	User           *string                `json:"user,omitempty"`
	ExtraParams    map[string]interface{} `json:"-"`
}

type Interval added in v1.3.12 

type Interval struct {
	// Optional. The start time of the interval.
	StartTime time.Time `json:"start_time,omitempty"`
	// Optional. The end time of the interval.
	EndTime time.Time `json:"end_time,omitempty"`
}

Interval represents a time interval, encoded as a start time (inclusive) and an end time (exclusive). The start time must be less than or equal to the end time. When the start equals the end time, the interval is an empty interval. (matches no time) When both start and end are unspecified, the interval matches any time.

func (*Interval) MarshalJSON added in v1.3.12 

func (i *Interval) MarshalJSON() ([]byte, error)

func (*Interval) UnmarshalJSON added in v1.3.12 

func (i *Interval) UnmarshalJSON(data []byte) error

type JSONKeyOrder added in v1.4.4 

type JSONKeyOrder struct {
	// contains filtered or unexported fields
}

JSONKeyOrder is a lightweight helper that preserves JSON key ordering through struct serialization round-trips. Embed it in any struct with `json:"-"` tag.

LLMs are autoregressive sequence models that are sensitive to JSON key ordering in tool schemas. This helper ensures that when Bifrost deserializes and re-serializes JSON, the original key order from the client is preserved.

Usage: 

type MyStruct struct {
    keyOrder JSONKeyOrder `json:"-"`
    Field1   string      `json:"field1"`
    Field2   string      `json:"field2"`
}

func (s *MyStruct) UnmarshalJSON(data []byte) error {
    type Alias MyStruct
    if err := Unmarshal(data, (*Alias)(s)); err != nil { return err }
    s.keyOrder.Capture(data)
    return nil
}

func (s MyStruct) MarshalJSON() ([]byte, error) {
    type Alias MyStruct
    data, err := Marshal(Alias(s))
    if err != nil { return nil, err }
    return s.keyOrder.Apply(data)
}

func (*JSONKeyOrder) Apply added in v1.4.4 

func (o *JSONKeyOrder) Apply(data []byte) ([]byte, error)

Apply reorders the keys in serialized JSON to match the captured order. If no order was captured (programmatic construction), returns data unchanged. Call this at the end of MarshalJSON.

func (*JSONKeyOrder) Capture added in v1.4.4 

func (o *JSONKeyOrder) Capture(data []byte)

Capture extracts and stores the top-level key order from raw JSON data. Call this at the end of UnmarshalJSON.

type JSONSchemaOrBool added in v1.7.3 

type JSONSchemaOrBool struct {
	SchemaBool *bool
	SchemaMap  *OrderedMap
}

JSONSchemaOrBool holds a JSON Schema value that is either a boolean schema (true/false, valid per JSON Schema draft 6+) or an object schema with key order preserved. Mirrors AdditionalPropertiesStruct.

func (JSONSchemaOrBool) MarshalJSON added in v1.7.3 

func (s JSONSchemaOrBool) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshalling for JSONSchemaOrBool. It marshals either SchemaBool or SchemaMap based on which is set.

func (*JSONSchemaOrBool) UnmarshalJSON added in v1.7.3 

func (s *JSONSchemaOrBool) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshalling for JSONSchemaOrBool. It handles both boolean and object JSON Schemas.

type KVStore added in v1.4.8 

type KVStore interface {
	Get(key string) (any, error)
	SetWithTTL(key string, value any, ttl time.Duration) error
	SetNXWithTTL(key string, value any, ttl time.Duration) (bool, error)
	Delete(key string) (bool, error)
}

KVStore is a minimal interface for a key-value store used by Bifrost internals. The concrete implementation (e.g. framework/kvstore.Store) is injected by the caller and must satisfy this interface. Passing nil disables KV-backed features.

type Key 

type Key struct {
	ID                     string                  `json:"id"`                                  // The unique identifier for the key (used by bifrost to identify the key)
	Name                   string                  `json:"name"`                                // The name of the key (used by users to identify the key, not used by bifrost)
	Value                  SecretVar               `json:"value"`                               // The actual API key value
	Models                 WhiteList               `json:"models"`                              // List of models this key can access
	BlacklistedModels      BlackList               `json:"blacklisted_models"`                  // List of models this key cannot access
	Weight                 float64                 `json:"weight"`                              // Weight for load balancing between multiple keys
	Aliases                KeyAliases              `json:"aliases,omitempty"`                   // Mapping of model identifiers to inference profiles
	AzureKeyConfig         *AzureKeyConfig         `json:"azure_key_config,omitempty"`          // Azure-specific key configuration
	VertexKeyConfig        *VertexKeyConfig        `json:"vertex_key_config,omitempty"`         // Vertex-specific key configuration
	BedrockKeyConfig       *BedrockKeyConfig       `json:"bedrock_key_config,omitempty"`        // AWS Bedrock-specific key configuration
	BedrockMantleKeyConfig *BedrockMantleKeyConfig `json:"bedrock_mantle_key_config,omitempty"` // Bedrock Mantle-specific key configuration
	VLLMKeyConfig          *VLLMKeyConfig          `json:"vllm_key_config,omitempty"`           // vLLM-specific key configuration
	ReplicateKeyConfig     *ReplicateKeyConfig     `json:"replicate_key_config,omitempty"`      // Replicate-specific key configuration
	OllamaKeyConfig        *OllamaKeyConfig        `json:"ollama_key_config,omitempty"`         // Ollama-specific key configuration
	SGLKeyConfig           *SGLKeyConfig           `json:"sgl_key_config,omitempty"`            // SGLang-specific key configuration
	Enabled                *bool                   `json:"enabled,omitempty"`                   // Whether the key is active (default:true)
	UseForBatchAPI         *bool                   `json:"use_for_batch_api,omitempty"`         // Whether this key can be used for batch API operations (default:false for new keys, migrated keys default to true)
	UseAnthropicEndpoints  *bool                   `json:"use_anthropic_endpoints,omitempty"`   // Whether to use anthropic endpoints for this key
	ConfigHash             string                  `json:"config_hash,omitempty"`               // Hash of config.json version, used for change detection
	Status                 KeyStatusType           `json:"status,omitempty"`                    // Status of key
	Description            string                  `json:"description,omitempty"`               // Description of key
}

Key represents an API key and its associated configuration for a provider. It contains the key value, supported models, and a weight for load balancing.

type KeyAliases added in v1.5.1 

type KeyAliases map[string]AliasConfig

KeyAliases maps a user-facing model name to its AliasConfig.

Both the input (UnmarshalJSON) and the output (AliasConfig.MarshalJSON) transparently accept and emit two JSON wire shapes:

  • Legacy: {"my-model": "provider-model-id"} — value is a string
  • New: {"my-model": {"model_id": "provider-model-id", ... }} — value is an object

Legacy entries deserialize to AliasConfig{ModelID: <string>}; an AliasConfig that only has ModelID set serializes back to a plain string. This keeps the wire format byte-for-byte compatible with the pre-refactor flow until ModelName / ModelFamily / provider sub-configs are populated explicitly.

func (KeyAliases) Resolve added in v1.5.1 

func (ka KeyAliases) Resolve(model string) string

Resolve returns the wire model identifier for the given user-facing model name. If no alias matches, the input is returned unchanged. Case-insensitive fallback matches the prior behavior.

This signature is preserved for backward compatibility with existing callers that only need the wire model string. For access to the full AliasConfig (ModelName, ModelFamily, provider overrides), use ResolveConfig.

func (KeyAliases) ResolveConfig added in v1.5.19 

func (ka KeyAliases) ResolveConfig(model string) *AliasConfig

ResolveConfig returns the AliasConfig for the given user-facing model name, or nil if no alias matches. Case-insensitive fallback matches Resolve.

func (*KeyAliases) UnmarshalJSON added in v1.5.19 

func (ka *KeyAliases) UnmarshalJSON(data []byte) error

UnmarshalJSON accepts both the legacy {"k":"v"} and new {"k":{...}} wire shapes for KeyAliases. Legacy string values are promoted to AliasConfig{ModelID: <string>}.

func (KeyAliases) Validate added in v1.5.1 

func (ka KeyAliases) Validate(providerKey ModelProvider) error

Validate checks that every entry in the alias map is well-formed and that any provider-specific sub-configs (AzureAliasCfg, VertexAliasCfg, BedrockAliasCfg, ReplicateAliasCfg) are only set when the owning Key actually belongs to that provider. Catches misconfigurations like an AzureAliasCfg attached to a Bedrock key.

providerKey is the provider this Key is registered under (e.g. schemas.Azure for keys in the azure provider config).

type KeyAttemptRecord added in v1.5.3 

type KeyAttemptRecord struct {
	Attempt           int     `json:"attempt"`
	KeyID             string  `json:"key_id"`
	KeyName           string  `json:"key_name"`
	FailReason        *string `json:"fail_reason,omitempty"`
	TriggeredRotation bool    `json:"triggered_rotation"`
}

KeyAttemptRecord captures the outcome of a single request attempt within executeRequestWithRetries. One record is appended per attempt regardless of whether the key changed between attempts.

FailReason is populated on every failed attempt (retryable or terminal) and is nil only on a successful attempt. Status-derived values are: `rate_limit_error` (429), `authentication_error` (401/403), `billing_error` (402); otherwise the provider's error Type is used, falling back to `unknown`. Use it to inspect what went wrong on a given try.

TriggeredRotation is true iff this attempt's per-key failure caused the next attempt to actually rotate to a *different* key. It is false on:

  • the final (terminal) attempt of a request, regardless of outcome,
  • any successful attempt,
  • same-key retries (transient 5xx / network errors keep the same key),
  • non-retryable failures,
  • fixed-key paths and 429 pool-resets that re-pick the same key (no rotation actually happened).

Use this (not FailReason) to count actual key rotations.

type KeyPoolFilter added in v1.6.0 

type KeyPoolFilter func(ctx *BifrostContext, provider ModelProvider, model string, keys []Key) ([]Key, error)

KeyPoolFilter is an optional hook called before key selection to veto keys from the available pool.

type KeySelector added in v1.2.5 

type KeySelector func(ctx *BifrostContext, keys []Key, providerKey ModelProvider, model string) (Key, error)

type KeyStatus added in v1.4.3 

type KeyStatus struct {
	KeyID    string        `json:"key_id"`   // Empty for keyless providers
	Status   KeyStatusType `json:"status"`   // "success", "failed"
	Provider ModelProvider `json:"provider"` // Always populated
	Error    *BifrostError `json:"error,omitempty"`
}

KeyStatus represents the status of model listing for a specific key

func (KeyStatus) MarshalJSON added in v1.4.7 

func (k KeyStatus) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshaling for KeyStatus to prevent circular reference: KeyStatus.Error → BifrostError.ExtraFields.KeyStatuses → KeyStatus.

type KeyStatusType added in v1.4.3 

type KeyStatusType string
const (
	KeyStatusSuccess          KeyStatusType = "success"
	KeyStatusListModelsFailed KeyStatusType = "list_models_failed"
)

type LLMPlugin added in v1.4.0 

type LLMPlugin interface {
	BasePlugin

	// PreRequestHook is called once per top-level request, after HTTPTransportPreHook and before
	// PreLLMHook. It is the canonical phase for deciding which provider/model/fallbacks the
	// request should be sent to. Plugins are free to mutate any field on req (Provider, Model,
	// Fallbacks, Input, Params, Tools, ...) — unlike PreLLMHook, mutations made here are
	// committed to the request and are observed by all subsequent plugins, the provider call,
	// and every fallback attempt.
	//
	// Error semantics match PreLLMHook: a non-nil error is non-blocking — it is logged as a
	// warning, the request continues, and the pipeline moves on to the next plugin. PreRequestHook
	// CANNOT abort the request via error return. Plugins that need to gate or reject a request
	// (e.g., authorization, content policy) must do so in HTTPTransportPreHook or via a
	// short-circuit response in PreLLMHook — not by returning an error here.
	//
	// Plugins that don't participate in routing should return nil.
	PreRequestHook(ctx *BifrostContext, req *BifrostRequest) error

	PreLLMHook(ctx *BifrostContext, req *BifrostRequest) (*BifrostRequest, *LLMPluginShortCircuit, error)
	PostLLMHook(ctx *BifrostContext, resp *BifrostResponse, bifrostErr *BifrostError) (*BifrostResponse, *BifrostError, error)
}

type LLMPluginShortCircuit added in v1.4.0 

type LLMPluginShortCircuit struct {
	Response *BifrostResponse         // If set, short-circuit with this response (skips provider call)
	Stream   chan *BifrostStreamChunk // If set, short-circuit with this stream (skips provider call)
	Error    *BifrostError            // If set, short-circuit with this error (can set AllowFallbacks field)
}

LLMPluginShortCircuit represents a plugin's decision to short-circuit the normal flow. It can contain either a response (success short-circuit), a stream (streaming short-circuit), or an error (error short-circuit).

type LargePayloadMetadata added in v1.4.8 

type LargePayloadMetadata struct {
	ResponseModalities []string // e.g., ["AUDIO"] for speech, ["IMAGE"] for image generation
	SpeechConfig       bool     // true if generationConfig.speechConfig is present
	Model              string   // model extracted without full body parsing (openai/anthropic multipart/json)
	StreamRequested    *bool    // stream flag when available in request payload metadata
}

LargePayloadMetadata holds routing-relevant metadata selectively extracted from large payloads. This is used when the full request body is too large to parse (e.g., 400MB video upload). Only small routing/observability fields are extracted; the body itself streams through unchanged.

type ListModelsByKeyResult added in v1.2.17 

type ListModelsByKeyResult struct {
	Response *BifrostListModelsResponse
	Err      *BifrostError
	KeyID    string
}

Structure to collect results from goroutines

type LogEventBuilder added in v1.4.1 

type LogEventBuilder interface {
	Str(key, val string) LogEventBuilder
	Int(key string, val int) LogEventBuilder
	Int64(key string, val int64) LogEventBuilder
	Send()
}

LogEventBuilder provides a fluent interface for building structured log entries. 

var NoopLogEvent LogEventBuilder = noopLogEventBuilder{}

NoopLogEvent is a shared singleton no-op LogEventBuilder.

type LogLevel 

type LogLevel string

LogLevel represents the severity level of a log message. Internally it maps to zerolog.Level for interoperability. 

const (
	LogLevelDebug LogLevel = "debug"
	LogLevelInfo  LogLevel = "info"
	LogLevelWarn  LogLevel = "warn"
	LogLevelError LogLevel = "error"
)

LogLevel constants for different severity levels.

type LogProb 

type LogProb struct {
	Bytes   []int   `json:"bytes,omitempty"`
	LogProb float64 `json:"logprob"`
	Token   string  `json:"token"`
}

LogProb represents the log probability of a token.

type Logger 

type Logger interface {
	// Debug logs a debug-level message.
	// This is used for detailed debugging information that is typically only needed
	// during development or troubleshooting.
	Debug(msg string, args ...any)

	// Info logs an info-level message.
	// This is used for general informational messages about normal operation.
	Info(msg string, args ...any)

	// Warn logs a warning-level message.
	// This is used for potentially harmful situations that don't prevent normal operation.
	Warn(msg string, args ...any)

	// Error logs an error-level message.
	// This is used for serious problems that need attention and may prevent normal operation.
	Error(msg string, args ...any)

	// Fatal logs a fatal-level message.
	// This is used for critical situations that require immediate attention and will terminate the program.
	Fatal(msg string, args ...any)

	// SetLevel sets the log level for the logger.
	SetLevel(level LogLevel)

	// SetOutputType sets the output type for the logger.
	SetOutputType(outputType LoggerOutputType)

	// LogHTTPRequest returns a LogEventBuilder for structured HTTP access logging.
	// The level parameter controls the log severity, msg is sent when Send() is called.
	// Use the fluent builder to attach typed fields before calling Send().
	LogHTTPRequest(level LogLevel, msg string) LogEventBuilder
}

Logger defines the interface for logging operations in the Bifrost system. Implementations of this interface should provide methods for logging messages at different severity levels.

type LoggerOutputType added in v1.1.23 

type LoggerOutputType string

LoggerOutputType represents the output type of a logger. 

const (
	LoggerOutputTypeJSON   LoggerOutputType = "json"
	LoggerOutputTypePretty LoggerOutputType = "pretty"
)

LoggerOutputType constants for different output types.

type MCPAuthMode added in v1.5.11 

type MCPAuthMode string

MCPAuthMode describes which identity dimension a per-user OAuth row is keyed by. It is a derived view of context state at the point of token lookup, never stored as a context key. Derived via BifrostContext.MCPAuthMode(). 

const (
	// MCPAuthModeUser: identity is a user id populated by an upstream auth
	// middleware or plugin. Token rows keyed by user_id.
	MCPAuthModeUser MCPAuthMode = "user"
	// MCPAuthModeVK: identity is a virtual key. Token rows keyed by vk_id.
	MCPAuthModeVK MCPAuthMode = "vk"
	// MCPAuthModeSession: identity is a client-issued opaque session ID, asserted
	// via the x-bf-mcp-session-id header. Token rows keyed by session_id.
	// Used when there's no VK or user; the caller owns the session ID and must
	// present it on every subsequent request to use the bound OAuth token.
	MCPAuthModeSession MCPAuthMode = "session"
	// MCPAuthModeNone: no identity dimension is present on the request (no
	// user, no VK, no session header). Lets callers branch on the mode
	// without mistaking an unauthenticated request for a session-mode caller.
	MCPAuthModeNone MCPAuthMode = "none"
)

type MCPAuthRequiredError added in v1.5.14 

type MCPAuthRequiredError struct {
	Kind          string `json:"kind"`
	MCPClientID   string `json:"mcp_client_id"`
	MCPClientName string `json:"mcp_client_name"`
	Message       string `json:"message"`

	// OAuth-specific fields (populated when Kind == "oauth"). SessionID is
	// also populated for Kind == "headers" — see the type-level comment.
	AuthorizeURL string `json:"authorize_url,omitempty"`
	SessionID    string `json:"session_id,omitempty"`

	// Headers-specific fields (populated when Kind == "headers"). SubmitURL is
	// the workspace landing page where the user provides values for
	// RequiredHeaderKeys; AdminHeaderKeys lists the admin-set static headers
	// (names only, no values) for context display.
	SubmitURL          string   `json:"submit_url,omitempty"`
	RequiredHeaderKeys []string `json:"required_header_keys,omitempty"`
	AdminHeaderKeys    []string `json:"admin_header_keys,omitempty"`
}

MCPAuthRequiredError is returned when a per-user MCP credential is missing and the caller must complete an inline auth flow (OAuth dance or headers submission) before tool execution can proceed.

Kind discriminates which set of fields is populated:

  • "oauth": AuthorizeURL, SessionID
  • "headers": SubmitURL, SessionID, RequiredHeaderKeys, AdminHeaderKeys

SessionID is shared by both Kinds: for "oauth" it is the mcp_per_user_oauth_flows row ID, for "headers" the mcp_per_user_header_flows row ID. Either way it lets the caller reference the pending flow row without parsing the URL fragment.

Common fields (MCPClientID, MCPClientName, Message) are always set.

func (*MCPAuthRequiredError) Error added in v1.5.14 

func (e *MCPAuthRequiredError) Error() string

type MCPAuthType added in v1.4.0 

type MCPAuthType string

MCPAuthType defines the authentication type for MCP connections 

const (
	MCPAuthTypeNone           MCPAuthType = "none"             // No authentication
	MCPAuthTypeHeaders        MCPAuthType = "headers"          // Header-based authentication (API keys, etc.)
	MCPAuthTypeOauth          MCPAuthType = "oauth"            // OAuth 2.0 authentication (server-level, admin authenticates once)
	MCPAuthTypePerUserOauth   MCPAuthType = "per_user_oauth"   // Per-user OAuth 2.0 authentication (each user authenticates individually)
	MCPAuthTypePerUserHeaders MCPAuthType = "per_user_headers" // Per-user header authentication (each user submits API keys / signed tokens; admin declares the required key names via PerUserHeaderKeys)
)

type MCPClient added in v1.1.7 

type MCPClient struct {
	Config *MCPClientConfig   `json:"config"` // Tool filtering settings
	Tools  []ChatToolFunction `json:"tools"`  // Available tools
	State  MCPConnectionState `json:"state"`  // Connection state
}

MCPClient represents a connected MCP client with its configuration and tools, and connection information, after it has been initialized. It is returned by GetMCPClients() method in bifrost.

type MCPClientConfig added in v1.1.4 

type MCPClientConfig struct {
	ID                string               `json:"client_id"`                     // Client ID
	Name              string               `json:"name"`                          // Client name
	IsCodeModeClient  bool                 `json:"is_code_mode_client"`           // Whether the client is a code mode client
	ConnectionType    MCPConnectionType    `json:"connection_type"`               // How to connect (HTTP, STDIO, SSE, or InProcess)
	ConnectionString  *SecretVar           `json:"connection_string,omitempty"`   // HTTP or SSE URL (required for HTTP or SSE connections)
	StdioConfig       *MCPStdioConfig      `json:"stdio_config,omitempty"`        // STDIO configuration (required for STDIO connections)
	TLSConfig         *MCPTLSConfig        `json:"tls_config,omitempty"`          // TLS configuration for HTTP/SSE connections
	AuthType          MCPAuthType          `json:"auth_type"`                     // Authentication type (none, headers, or oauth)
	OauthConfigID     *string              `json:"oauth_config_id,omitempty"`     // OAuth config ID (references oauth_configs table)
	OauthClientID     *SecretVar           `json:"oauth_client_id,omitempty"`     // Redacted OAuth client ID (populated on GET, not stored here)
	OauthClientSecret *SecretVar           `json:"oauth_client_secret,omitempty"` // Redacted OAuth client secret (populated on GET, not stored here)
	State             string               `json:"state,omitempty"`               // Connection state (connected, disconnected, error)
	Headers           map[string]SecretVar `json:"headers,omitempty"`             // Headers to send with the request (for headers auth type)
	// PerUserHeaderKeys lists the header *names* each caller must supply for
	// MCPAuthTypePerUserHeaders clients. Admin-declared schema only — the
	// values live per-user in the mcp_per_user_header_credentials table and
	// are resolved at call time. Names in this list are stripped from
	// utils.StaticConfigHeaders so admin-set values in `Headers` with the
	// same name cannot leak through the plugin gate. Required (non-empty)
	// when AuthType == per_user_headers; ignored otherwise.
	PerUserHeaderKeys   []string          `json:"per_user_header_keys,omitempty"`
	AllowedExtraHeaders WhiteList         `json:"allowed_extra_headers,omitempty"` // Allowlist of request-level headers that callers may forward to this MCP server at execution time
	InProcessServer     *server.MCPServer `json:"-"`                               // MCP server instance for in-process connections (Go package only)
	ToolsToExecute      WhiteList         `json:"tools_to_execute,omitempty"`      // Include-only list.
	// ToolsToExecute semantics:
	// - ["*"] => all tools are included
	// - []    => no tools are included (deny-by-default)
	// - nil/omitted => treated as [] (no tools)
	// - ["tool1", "tool2"] => include only the specified tools
	ToolsToAutoExecute WhiteList `json:"tools_to_auto_execute,omitempty"` // Auto-execute list.
	// ToolsToAutoExecute semantics:
	// - ["*"] => all tools are auto-executed
	// - []    => no tools are auto-executed (deny-by-default)
	// - nil/omitted => treated as [] (no tools)
	// - ["tool1", "tool2"] => auto-execute only the specified tools
	// Note: If a tool is in ToolsToAutoExecute but not in ToolsToExecute, it will be skipped.
	IsPingAvailable       *bool              `json:"is_ping_available,omitempty"`      // Whether the MCP server supports ping for health checks (nil/true = ping; false = listTools). Defaults to true.
	ToolSyncInterval      time.Duration      `json:"tool_sync_interval,omitempty"`     // Per-client override for tool sync interval (0 = use global, negative = disabled)
	ToolExecutionTimeout  time.Duration      `json:"tool_execution_timeout,omitempty"` // Per-client override for tool execution timeout (0 = use global from tool_manager_config)
	ToolPricing           map[string]float64 `json:"tool_pricing,omitempty"`           // Tool pricing for each tool (cost per execution)
	Disabled              bool               `json:"disabled"`                         // Whether the client is intentionally disabled (stops connection and workers)
	ConfigHash            string             `json:"-"`                                // Config hash for reconciliation (not serialized)
	AllowOnAllVirtualKeys bool               `json:"allow_on_all_virtual_keys"`        // Whether to allow the MCP client to run on all virtual keys

	// Discovered tools for per-user OAuth clients (persisted so they survive restart)
	DiscoveredTools           map[string]ChatTool `json:"-"` // Discovered tool schemas keyed by prefixed name
	DiscoveredToolNameMapping map[string]string   `json:"-"` // Mapping from sanitized tool names to original MCP names
}

MCPClientConfig defines tool filtering for an MCP client.

func NewMCPClientConfigFromMap added in v1.3.9 

func NewMCPClientConfigFromMap(configMap map[string]any) *MCPClientConfig

NewMCPClientConfigFromMap creates a new MCP client config from a map[string]any.

func (MCPClientConfig) MarshalJSON added in v1.6.3 

func (c MCPClientConfig) MarshalJSON() ([]byte, error)

MarshalJSON emits tool_execution_timeout as a duration string so it round-trips correctly — default time.Duration marshaling emits nanoseconds, but UnmarshalJSON treats bare integers as seconds.

func (*MCPClientConfig) UnmarshalJSON added in v1.5.5 

func (c *MCPClientConfig) UnmarshalJSON(data []byte) error

UnmarshalJSON supports Go duration strings (e.g. "10m") for tool_sync_interval and tool_execution_timeout. Numeric values are treated as raw nanoseconds for tool_sync_interval and as seconds for tool_execution_timeout (matching tool_manager_config behaviour).

type MCPClientConnectionInfo added in v1.3.0 

type MCPClientConnectionInfo struct {
	Type               MCPConnectionType `json:"type"`                           // Connection type (HTTP, STDIO, SSE, or InProcess)
	ConnectionURL      *string           `json:"connection_url,omitempty"`       // HTTP/SSE endpoint URL (for HTTP/SSE connections)
	StdioCommandString *string           `json:"stdio_command_string,omitempty"` // Command string for display (for STDIO connections)
}

MCPClientConnectionInfo stores metadata about how a client is connected.

type MCPClientState added in v1.3.0 

type MCPClientState struct {
	Name            string                   // Unique name for this client
	Conn            *client.Client           // Active MCP client connection
	ExecutionConfig *MCPClientConfig         // Tool filtering settings
	ToolMap         map[string]ChatTool      // Available tools mapped by name
	ToolNameMapping map[string]string        // Maps sanitized_name -> original_mcp_name (e.g., "notion_search" -> "notion-search")
	ConnectionInfo  *MCPClientConnectionInfo `json:"connection_info"` // Connection metadata for management
	CancelFunc      context.CancelFunc       `json:"-"`               // Cancel function for SSE connections (not serialized)
	State           MCPConnectionState       // Connection state (connected, disconnected, error)
}

MCPClientState represents a connected MCP client with its configuration and tools. It is used internally by the MCP manager to track the state of a connected MCP client.

type MCPConfig added in v1.1.4 

type MCPConfig struct {
	ClientConfigs     []*MCPClientConfig    `json:"client_configs,omitempty"`      // Per-client execution configurations
	ToolManagerConfig *MCPToolManagerConfig `json:"tool_manager_config,omitempty"` // MCP tool manager configuration
	ToolSyncInterval  time.Duration         `json:"tool_sync_interval,omitempty"`  // Global default interval for syncing tools from MCP servers (0 = use default 10 min)

	// Function to fetch a new request ID for each tool call result message in agent mode,
	// this is used to ensure that the tool call result messages are unique and can be tracked in plugins or by the user.
	// This id is attached to ctx.Value(schemas.BifrostContextKeyRequestID) in the agent mode.
	// If not provider, same request ID is used for all tool call result messages without any overrides.
	FetchNewRequestIDFunc func(ctx *BifrostContext) string `json:"-"`

	// PluginPipelineProvider returns a plugin pipeline for running MCP plugin hooks.
	// Used when executeCode tool calls nested MCP tools to ensure plugins run for them.
	// The plugin pipeline should be released back to the pool using ReleasePluginPipeline.
	PluginPipelineProvider func() interface{} `json:"-"`

	// ReleasePluginPipeline releases a plugin pipeline back to the pool.
	// This should be called after the plugin pipeline is no longer needed.
	ReleasePluginPipeline func(pipeline interface{}) `json:"-"`
}

MCPConfig represents the configuration for MCP integration in Bifrost. It enables tool auto-discovery and execution from local and external MCP servers.

func (*MCPConfig) UnmarshalJSON added in v1.5.5 

func (c *MCPConfig) UnmarshalJSON(data []byte) error

UnmarshalJSON supports Go duration strings (e.g. "10m") for tool_sync_interval. Numeric values remain supported for backward compatibility (treated as raw nanoseconds).

type MCPConnectionPlugin added in v1.5.10 

type MCPConnectionPlugin interface {
	MCPPlugin

	PreMCPConnectionHook(ctx *BifrostContext, req *BifrostMCPConnectRequest) (*BifrostMCPConnectRequest, *MCPConnectionShortCircuit, error)
	PostMCPConnectionHook(ctx *BifrostContext, resp *BifrostMCPConnectResponse, bifrostErr *BifrostError) (*BifrostMCPConnectResponse, *BifrostError, error)
}

MCPConnectionPlugin is an optional, typed extension interface for handling MCP Connect events. Connect is morally separate from the other MCP lifecycle ops (Ping/ListTools/ExecuteTool) — it establishes the transport before a usable client exists, and carries transport-level inputs (URL, headers, stdio args) that don't apply post-connection. Plugins implementing this interface receive Connect events via the typed methods; their generic PreMCPHook/PostMCPHook (if also implemented) is NOT called for Connect requests.

Plugins registered via MCPPlugins must still satisfy MCPPlugin. To write a plugin that only handles Connect events, embed MCPPluginNoOpHooks for free no-op implementations of the generic Pre/PostMCPHook.

NOTE (backwards compat): keeping the Connect hooks on a separate optional interface — and the MCPPluginNoOpHooks helper — is purely a backwards-compat shim so existing MCPPlugin implementations don't break with the addition of Connect hooks. In a future major release these two methods will move onto MCPPlugin directly and every MCP plugin will be required to implement them.

type MCPConnectionShortCircuit added in v1.5.10 

type MCPConnectionShortCircuit struct {
	Response *BifrostMCPConnectResponse // If set, short-circuit with this synthetic Connect outcome
	Error    *BifrostError              // If set, short-circuit with this error
}

MCPConnectionShortCircuit is the typed short-circuit for MCPConnectionPlugin. It carries a typed Connect response (instead of the generic envelope) so plugin authors don't have to wrap responses in BifrostMCPResponse.

type MCPConnectionState added in v1.1.7 

type MCPConnectionState string
const (
	MCPConnectionStateConnected    MCPConnectionState = "connected"     // Client is connected and ready to use
	MCPConnectionStateDisconnected MCPConnectionState = "disconnected"  // Client is not connected
	MCPConnectionStateError        MCPConnectionState = "error"         // Client is in an error state, and cannot be used
	MCPConnectionStatePendingTools MCPConnectionState = "pending_tools" // Connected but tools not yet populated
	MCPConnectionStateDisabled     MCPConnectionState = "disabled"      // Client is intentionally disabled by the user
)

type MCPConnectionType added in v1.1.4 

type MCPConnectionType string

MCPConnectionType defines the communication protocol for MCP connections 

const (
	MCPConnectionTypeHTTP      MCPConnectionType = "http"      // HTTP-based connection
	MCPConnectionTypeSTDIO     MCPConnectionType = "stdio"     // STDIO-based connection
	MCPConnectionTypeSSE       MCPConnectionType = "sse"       // Server-Sent Events connection
	MCPConnectionTypeInProcess MCPConnectionType = "inprocess" // In-process (in-memory) connection
)

func (MCPConnectionType) OTelNetworkTransport added in v1.7.3 

func (c MCPConnectionType) OTelNetworkTransport() string

OTelNetworkTransport returns the OTel semconv network.transport value: stdio→"pipe", http/sse→"tcp". InProcess has none, so it returns "" and callers omit the attribute.

type MCPCredentialStore added in v1.5.14 

type MCPCredentialStore interface {
	// ConnectionHeaders returns the headers to attach when opening an upstream
	// transport. Called from two sites:
	//
	//  1. At AddClient / Reconnect / UpdateClientConnection for shared-
	//     connection auth types (none, headers, server_oauth). The caller
	//     wraps the Bifrost lifecycle context into a synthetic BifrostContext
	//     with no identity, so the resolver returns admin-level headers
	//     (static config + admin Bearer for server_oauth).
	//
	//  2. Per call inside the ephemeral-transport path for per-user auth
	//     types. The caller passes the real request BifrostContext, and the
	//     resolver returns the caller's full set (static + filtered
	//     context-extras + per-user auth).
	//
	// May return *MCPAuthRequiredError when a per-user credential is missing
	// and the caller must complete an inline auth flow (OAuth dance or
	// headers submission) before retrying.
	ConnectionHeaders(ctx *BifrostContext, config *MCPClientConfig) (http.Header, error)

	// RequestHeaders returns the per-message headers attached to each
	// CallTool / ListTools / Ping that flows over an already-open
	// transport — currently just the filtered context-extras
	// (BifrostContextKeyMCPExtraHeaders, scoped by config.AllowedExtraHeaders).
	//
	// Only meaningful when the connection is shared
	// (RequiresPerCallConnection is false). Per-user types embed all
	// caller-specific headers in the ephemeral transport itself via
	// ConnectionHeaders; the caller skips RequestHeaders in that path.
	RequestHeaders(ctx *BifrostContext, config *MCPClientConfig) (http.Header, error)

	// RequiresPerCallConnection reports whether each tool invocation needs a
	// freshly-built ephemeral upstream connection (rather than reusing a
	// shared persistent one). True for per-user auth types; false for
	// shared (none, headers, oauth-server-level).
	RequiresPerCallConnection(config *MCPClientConfig) bool
}

MCPCredentialStore is the single source of truth for MCP credential resolution. It exposes three predicates that MCPManager consumes uniformly:

  • ConnectionHeaders — headers attached to opening an upstream transport
  • RequestHeaders — per-message headers on an already-open transport
  • RequiresPerCallConnection — whether each call needs an ephemeral transport

Storage lifecycle (orphaning on VK reassignment, cascade on client delete) is NOT part of this interface — those concerns stay in the configstore layer where transactional atomicity is preserved.

type MCPHeadersFlowInitiation added in v1.5.14 

type MCPHeadersFlowInitiation struct {
	FlowID      string    // Flow row primary key
	FrontendURL string    // {base}/workspace/mcp-sessions/auth?flow={id}#t={temp_token}
	ExpiresAt   time.Time // Flow expiration (15 min default; matches OAuth)
}

MCPHeadersFlowInitiation is the response returned by InitiateUserSubmissionFlow. Mirrors OAuth2FlowInitiation structurally so the resolver-side handling on the two per-user-auth surfaces stays uniform: a UUID, an auth-page URL, and an expiry. The "state" field is unused (no PKCE for headers); kept off the struct.

type MCPHeadersProvider added in v1.5.14 

type MCPHeadersProvider interface {
	// GetCredentialByMode returns the persisted credential for a single
	// identity dimension determined by mode. No fallback chain — exactly one
	// identity column is queried. Returns ErrHeadersCredentialNotFound when
	// the row is absent. Both 'active' and 'needs_update' rows are returned;
	// orphaned rows are filtered out at the store layer. The runtime
	// resolver's missing-keys check distinguishes usable from re-submission-
	// required rows, so the caller doesn't need to inspect Status itself.
	GetCredentialByMode(ctx context.Context, mode MCPAuthMode, identity, mcpClientID string) (*MCPHeadersUserCredential, error)

	// UpsertCredential persists a user-submitted set of header values for the
	// (mode, identity, mcp_client_id) triple after a successful verify. The
	// caller is expected to have run VerifyHeadersConnection before invoking
	// this — the provider does not re-test the upstream connection.
	UpsertCredential(ctx context.Context, cred *MCPHeadersUserCredential) error

	// DeleteCredential removes a credential row by its primary-key ID.
	DeleteCredential(ctx context.Context, id string) error

	// InitiateUserSubmissionFlow creates a pending mcp_per_user_header_flows
	// row keyed by (mode, identity, mcp_client_id), mints a mcp_headers_auth
	// temp-token bound to the new row's ID, and returns the auth-page URL
	// with the token embedded as a `#t=<token>` fragment. Mirrors
	// OAuth2Provider.InitiateUserOAuthFlow's role: the resolver calls this
	// when an inline-401 fires, then puts the returned FrontendURL on the
	// MCPAuthRequiredError so the caller can drive the submission flow.
	//
	// baseURL is the bifrost dashboard origin (e.g. "https://host") — the
	// resolver pulls it from BifrostContextKeyMCPCallbackBaseURL and passes
	// it in so the provider can construct the frontend URL without
	// reaching into the BifrostContext itself.
	InitiateUserSubmissionFlow(ctx context.Context, mode MCPAuthMode, identity, mcpClientID, baseURL string) (*MCPHeadersFlowInitiation, error)
}

MCPHeadersProvider is the contract between the per-user-headers CredentialStore resolver and the configstore-backed implementation. Mirrors OAuth2Provider's per-user methods structurally so future provider implementations stay consistent.

type MCPHeadersUserCredential added in v1.5.14 

type MCPHeadersUserCredential struct {
	ID           string
	MCPClientID  string
	AuthMode     MCPAuthMode
	UserID       *string
	VirtualKeyID *string
	SessionID    *string
	Headers      map[string]string // Decrypted header values
	Status       MCPHeadersUserCredentialStatus
	CreatedAt    time.Time
	UpdatedAt    time.Time
}

MCPHeadersUserCredential is the in-memory view of a single per-user header credential row. The transport between core and framework treats Headers as plaintext — encryption at rest is the configstore's responsibility.

type MCPHeadersUserCredentialStatus added in v1.5.14 

type MCPHeadersUserCredentialStatus string

MCPHeadersUserCredentialStatus mirrors the lifecycle states tracked on the mcp_per_user_header_credentials table. Storage-layer concerns; the resolver only cares about "is this row usable right now". 

const (
	MCPHeadersUserCredentialStatusActive      MCPHeadersUserCredentialStatus = "active"       // Row matches the current schema and may be used
	MCPHeadersUserCredentialStatusNeedsUpdate MCPHeadersUserCredentialStatus = "needs_update" // Schema (PerUserHeaderKeys) changed; user must resubmit
	MCPHeadersUserCredentialStatusOrphaned    MCPHeadersUserCredentialStatus = "orphaned"     // Owner (VK / user) was deleted or detached; awaiting cleanup
)

type MCPPlugin added in v1.4.0 

type MCPPlugin interface {
	BasePlugin

	PreMCPHook(ctx *BifrostContext, req *BifrostMCPRequest) (*BifrostMCPRequest, *MCPPluginShortCircuit, error)
	PostMCPHook(ctx *BifrostContext, resp *BifrostMCPResponse, bifrostErr *BifrostError) (*BifrostMCPResponse, *BifrostError, error)
}

type MCPPluginNoOpHooks added in v1.5.10 

type MCPPluginNoOpHooks struct{}

MCPPluginNoOpHooks provides no-op implementations of PreMCPHook and PostMCPHook. Embed this in plugins that only want to implement an extension interface (e.g. MCPConnectionPlugin) and don't need to observe the generic hook surface.

The plugin must still provide its own GetName and Cleanup (from BasePlugin).

func (MCPPluginNoOpHooks) PostMCPHook added in v1.5.10 

func (MCPPluginNoOpHooks) PostMCPHook(_ *BifrostContext, resp *BifrostMCPResponse, bifrostErr *BifrostError) (*BifrostMCPResponse, *BifrostError, error)

PostMCPHook returns the response and error unchanged.

func (MCPPluginNoOpHooks) PreMCPHook added in v1.5.10 

func (MCPPluginNoOpHooks) PreMCPHook(_ *BifrostContext, req *BifrostMCPRequest) (*BifrostMCPRequest, *MCPPluginShortCircuit, error)

PreMCPHook returns the request unchanged with no short-circuit.

type MCPPluginShortCircuit added in v1.4.0 

type MCPPluginShortCircuit struct {
	Response *BifrostMCPResponse // If set, short-circuit with this response (skips MCP call)
	Error    *BifrostError       // If set, short-circuit with this error (can set AllowFallbacks field)
}

MCPPluginShortCircuit represents a plugin's decision to short-circuit the normal flow. It can contain either a response (success short-circuit), or an error (error short-circuit).

type MCPRequestType added in v1.4.0 

type MCPRequestType string
const (
	MCPRequestTypePing      MCPRequestType = "ping"
	MCPRequestTypeListTools MCPRequestType = "list_tools"

	// [DEPRECATED] these will be replaced by MCPRequestTypeExecuteTool in the next major bump, but are kept for backward compatibility for now since some tools still rely on the old fields
	MCPRequestTypeChatToolCall      MCPRequestType = "chat_tool_call"      // Chat API format
	MCPRequestTypeResponsesToolCall MCPRequestType = "responses_tool_call" // Responses API format

	// Will be used in from the next major bump
	MCPRequestTypeExecuteTool MCPRequestType = "execute_tool"
)

func (MCPRequestType) IsExecuteTool added in v1.5.10 

func (t MCPRequestType) IsExecuteTool() bool

IsExecuteTool reports whether this is one of the execute-tool request variants (Chat, Responses, or the future unified ExecuteTool). Used by MCPPlugin pre/post hooks to skip non-tool envelope ops (Ping/ListTools) without sniffing pointer fields on the request/response.

NOTE: this helper exists because three execute-tool request types currently coexist for backwards compat (ChatToolCall + ResponsesToolCall are deprecated). Once callers fully migrate to MCPRequestTypeExecuteTool, this method will be removed and consumers should switch to `t == MCPRequestTypeExecuteTool` directly.

func (MCPRequestType) OTelMethodName added in v1.7.3 

func (t MCPRequestType) OTelMethodName() string

OTelMethodName returns the OTel semconv mcp.method.name for this request type (tools/call, tools/list, ping). Unknown types fall back to the raw string.

type MCPServerCapabilities added in v1.5.10 

type MCPServerCapabilities struct {
	Tools     bool `json:"tools"`     // server supports tools/list + tools/call
	Resources bool `json:"resources"` // server supports resources
	Prompts   bool `json:"prompts"`   // server supports prompts
	Logging   bool `json:"logging"`   // server supports logging
}

MCPServerCapabilities mirrors the high-level capability flags from the MCP initialize handshake. Only the booleans Bifrost cares about today; can grow as needed.

type MCPServerInfo added in v1.5.10 

type MCPServerInfo struct {
	Name    string `json:"name"`
	Version string `json:"version"`
}

MCPServerInfo mirrors the ServerInfo portion of the MCP initialize handshake.

type MCPStdioConfig added in v1.1.4 

type MCPStdioConfig struct {
	Command string   `json:"command"` // Executable command to run
	Args    []string `json:"args"`    // Command line arguments
	Envs    []string `json:"envs"`    // Environment variables required
}

MCPStdioConfig defines how to launch a STDIO-based MCP server.

type MCPTLSConfig added in v1.5.14 

type MCPTLSConfig struct {
	InsecureSkipVerify bool       `json:"insecure_skip_verify,omitempty"` // Disable TLS certificate verification (development only)
	CACertPEM          *SecretVar `json:"ca_cert_pem,omitempty"`          // PEM-encoded CA certificate to trust (supports env.*)
}

MCPTLSConfig holds TLS options for HTTP and SSE MCP connections. InsecureSkipVerify takes priority over CACertPEM when both are set.

func (*MCPTLSConfig) MarshalForStorage added in v1.5.14 

func (t *MCPTLSConfig) MarshalForStorage() ([]byte, error)

MarshalForStorage serializes MCPTLSConfig for DB persistence. ca_cert_pem is stored as a plain string ("env.VAR_NAME" or literal PEM). For HTTP API responses use json.Marshal so clients receive the full SecretVar object.

type MCPToolAnnotations added in v1.4.20 

type MCPToolAnnotations struct {
	Title           string `json:"title,omitempty"`           // Human-readable title for the tool
	ReadOnlyHint    *bool  `json:"readOnlyHint,omitempty"`    // If true, the tool does not modify its environment
	DestructiveHint *bool  `json:"destructiveHint,omitempty"` // If true, the tool may perform destructive updates
	IdempotentHint  *bool  `json:"idempotentHint,omitempty"`  // If true, repeated calls with same args have no additional effect
	OpenWorldHint   *bool  `json:"openWorldHint,omitempty"`   // If true, the tool interacts with external entities
}

type MCPToolManagerConfig added in v1.3.0 

type MCPToolManagerConfig struct {
	// ToolExecutionTimeout accepts a Go duration string (e.g. "30s", "2m") or a
	// bare integer treated as seconds (e.g. 30 → 30s). This intentionally differs
	// from schemas.Duration, which treats bare integers as nanoseconds.
	ToolExecutionTimeout  Duration             `json:"tool_execution_timeout"`
	MaxAgentDepth         int                  `json:"max_agent_depth"`
	CodeModeBindingLevel  CodeModeBindingLevel `json:"code_mode_binding_level,omitempty"`  // How tools are exposed in VFS: "server" or "tool"
	DisableAutoToolInject bool                 `json:"disable_auto_tool_inject,omitempty"` // When true, MCP tools are not injected into requests by default
}

func (*MCPToolManagerConfig) UnmarshalJSON added in v1.5.10 

func (c *MCPToolManagerConfig) UnmarshalJSON(data []byte) error

UnmarshalJSON implements json.Unmarshaler so that tool_execution_timeout treats bare integers as seconds (matching the schema description and user expectation) rather than the nanosecond interpretation used by the underlying Duration type.

type MCPUserOAuthRequiredError deprecated added in v1.5.1 

type MCPUserOAuthRequiredError = MCPAuthRequiredError

MCPUserOAuthRequiredError is an alias retained for backward compatibility with callers that referenced the OAuth-only error type before headers auth was added. New code should use MCPAuthRequiredError directly.

Deprecated: use MCPAuthRequiredError.

type Model added in v1.2.14 

type Model struct {
	ID                  string             `json:"id"`
	CanonicalSlug       *string            `json:"canonical_slug,omitempty"`
	Name                *string            `json:"name,omitempty"`
	NormalizedName      *string            `json:"normalized_name,omitempty"` // Human-readable name derived from the datasheet base_model (e.g. "Claude Sonnet 4.5")
	Alias               *string            `json:"alias,omitempty"`           // Provider API identifier this model alias maps to (e.g. Azure deployment name, Bedrock ARN)
	Created             *int64             `json:"created,omitempty"`
	ContextLength       *int               `json:"context_length,omitempty"`
	MaxInputTokens      *int               `json:"max_input_tokens,omitempty"`
	MaxOutputTokens     *int               `json:"max_output_tokens,omitempty"`
	Architecture        *Architecture      `json:"architecture,omitempty"`
	IsDeprecated        bool               `json:"is_deprecated,omitempty"`
	Pricing             *Pricing           `json:"pricing,omitempty"`
	TopProvider         *TopProvider       `json:"top_provider,omitempty"`
	PerRequestLimits    *PerRequestLimits  `json:"per_request_limits,omitempty"`
	SupportedParameters []string           `json:"supported_parameters,omitempty"`
	DefaultParameters   *DefaultParameters `json:"default_parameters,omitempty"`
	HuggingFaceID       *string            `json:"hugging_face_id,omitempty"`
	Description         *string            `json:"description,omitempty"`

	// AdditionalAttributes carries editorial per-model metadata stored on the
	// governance_model_pricing row (e.g. description, tags). Preserved across
	// the 24-hour pricing sync.
	AdditionalAttributes map[string]string `json:"additional_attributes,omitempty"`

	OwnedBy          *string  `json:"owned_by,omitempty"`
	SupportedMethods []string `json:"supported_methods,omitempty"`

	// ProviderExtra carries opaque provider-specific data (e.g. Anthropic capabilities)
	// through the Bifrost pipeline for integration reverse-conversion. Never serialized.
	ProviderExtra json.RawMessage `json:"-"`
}

type ModelFamily added in v1.5.19 

type ModelFamily string

ModelFamily is a typed enum identifying the underlying model family of an alias target. It enables provider routing decisions (request shape, response parsing, auth headers, URL construction) without substring-sniffing the wire model ID. 

const (
	ModelFamilyAnthropic ModelFamily = "anthropic"
	ModelFamilyOpenAI    ModelFamily = "openai"
	ModelFamilyMistral   ModelFamily = "mistral"
	ModelFamilyCohere    ModelFamily = "cohere"
	ModelFamilyGemini    ModelFamily = "gemini"
	ModelFamilyGemma     ModelFamily = "gemma"
	ModelFamilyLlama     ModelFamily = "llama"
	ModelFamilyImagen    ModelFamily = "imagen"
	ModelFamilyVeo       ModelFamily = "veo"
	ModelFamilyNova      ModelFamily = "nova"
	ModelFamilyTitan     ModelFamily = "titan"
)

func ResolveFamily added in v1.5.19 

func ResolveFamily(ctx *BifrostContext, fallbackModel string) ModelFamily

ResolveFamily returns the model family for the current attempt, walking the precedence: explicit alias ModelFamily → alias ModelName → alias ModelID → alias Key. When no alias matched, falls back to substring matching against fallbackModel (typically request.Model), preserving pre-refactor behavior.

Returns an empty ModelFamily if nothing matches.

func (*ModelFamily) IsValid added in v1.5.19 

func (mf *ModelFamily) IsValid() bool

IsValid reports whether mf is a recognized model family.

type ModelProvider 

type ModelProvider string

ModelProvider represents the different AI model providers supported by Bifrost. 

const (
	OpenAI        ModelProvider = "openai"
	Azure         ModelProvider = "azure"
	Anthropic     ModelProvider = "anthropic"
	Bedrock       ModelProvider = "bedrock"
	BedrockMantle ModelProvider = "bedrock_mantle"
	Cohere        ModelProvider = "cohere"
	Vertex        ModelProvider = "vertex"
	Mistral       ModelProvider = "mistral"
	Ollama        ModelProvider = "ollama"
	OpencodeGo    ModelProvider = "opencode-go"
	OpencodeZen   ModelProvider = "opencode-zen"
	Groq          ModelProvider = "groq"
	SGL           ModelProvider = "sgl"
	Parasail      ModelProvider = "parasail"
	Perplexity    ModelProvider = "perplexity"
	Cerebras      ModelProvider = "cerebras"
	DeepSeek      ModelProvider = "deepseek"
	Gemini        ModelProvider = "gemini"
	OpenRouter    ModelProvider = "openrouter"
	Elevenlabs    ModelProvider = "elevenlabs"
	HuggingFace   ModelProvider = "huggingface"
	Nebius        ModelProvider = "nebius"
	XAI           ModelProvider = "xai"
	Replicate     ModelProvider = "replicate"
	VLLM          ModelProvider = "vllm"
	Runway        ModelProvider = "runway"
	Runware       ModelProvider = "runware"
	Fireworks     ModelProvider = "fireworks"
	Sarvam        ModelProvider = "sarvam"
	Wafer         ModelProvider = "wafer"
)

func ParseModelString added in v1.2.0 

func ParseModelString(model string, defaultProvider ModelProvider) (ModelProvider, string)

ParseModelString extracts provider and model from a model string. For model strings like "anthropic/claude", it returns ("anthropic", "claude"). For model strings like "claude", it returns ("", "claude"). Only splits on "/" when the prefix is a known Bifrost provider, so model namespaces like "meta-llama/Llama-3.1-8B" are preserved as-is.

type NetworkConfig 

type NetworkConfig struct {
	// BaseURL is supported for OpenAI, Anthropic, Cohere, Mistral, and Ollama providers (required for Ollama)
	BaseURL                        string            `json:"base_url,omitempty"`                       // Base URL for the provider (optional)
	ExtraHeaders                   map[string]string `json:"extra_headers,omitempty"`                  // Additional headers to include in requests (optional)
	DefaultRequestTimeoutInSeconds int               `json:"default_request_timeout_in_seconds"`       // Default timeout for requests
	MaxRetries                     int               `json:"max_retries"`                              // Maximum number of retries
	RetryBackoffInitial            time.Duration     `json:"retry_backoff_initial"`                    // Initial backoff duration (stored as nanoseconds, JSON as milliseconds)
	RetryBackoffMax                time.Duration     `json:"retry_backoff_max"`                        // Maximum backoff duration (stored as nanoseconds, JSON as milliseconds)
	InsecureSkipVerify             bool              `json:"insecure_skip_verify,omitempty"`           // Disables TLS certificate verification for provider connections
	CACertPEM                      *SecretVar        `json:"ca_cert_pem,omitempty"`                    // PEM-encoded CA certificate to trust for provider endpoint connections (supports env.*)
	StreamIdleTimeoutInSeconds     int               `json:"stream_idle_timeout_in_seconds,omitempty"` // Idle timeout per stream chunk (0 = use default 60s)
	KeepAliveTimeoutInSeconds      int               `json:"keep_alive_timeout_in_seconds,omitempty"`  // Idle keep-alive for pooled connections; set below the upstream server's keep-alive to avoid reusing connections it has already closed. Default: 30s
	MaxConnsPerHost                int               `json:"max_conns_per_host,omitempty"`             // Max TCP connections per provider host (default: 5000)
	EnforceHTTP2                   bool              `json:"enforce_http2,omitempty"`                  // Force HTTP/2 on provider connections (relevant for net/http-based providers like Bedrock)
	BetaHeaderOverrides            map[string]bool   `json:"beta_header_overrides,omitempty"`          // Override default beta header support per provider (keys are prefixes like "redact-thinking-")
	AllowPrivateNetwork            bool              `json:"allow_private_network,omitempty"`          // Allow connections to RFC 1918 private IPs (for k8s pods, LAN deployments). Link-local (169.254.x.x) is always blocked.
}

NetworkConfig represents the network configuration for provider connections. ExtraHeaders is automatically copied during provider initialization to prevent data races.

RetryBackoffInitial and RetryBackoffMax are stored internally as time.Duration (nanoseconds). They accept two JSON formats for backward compatibility:

  • Duration string: "500ms", "5s", "1m" — parsed via time.ParseDuration (preferred)
  • Integer: treated as milliseconds (legacy format, e.g. 500 means 500ms)

func (NetworkConfig) MarshalJSON added in v1.2.32 

func (nc NetworkConfig) MarshalJSON() ([]byte, error)

MarshalJSON customizes JSON marshaling for NetworkConfig. RetryBackoffInitial and RetryBackoffMax are converted from time.Duration (nanoseconds) to milliseconds (integers) in JSON.

func (*NetworkConfig) Redacted added in v1.4.8 

func (nc *NetworkConfig) Redacted() *NetworkConfig

Redacted returns a redacted copy of the network configuration with CACertPEM masked.

func (*NetworkConfig) UnmarshalJSON added in v1.2.32 

func (nc *NetworkConfig) UnmarshalJSON(data []byte) error

UnmarshalJSON customizes JSON unmarshaling for NetworkConfig.

RetryBackoffInitial and RetryBackoffMax accept two formats:

  • Duration string (preferred): "500ms", "5s", "1m" — parsed via time.ParseDuration
  • Integer (legacy): treated as milliseconds for backward compatibility (e.g. 500 → 500ms, matching the original behavior)

type NoOpTracer added in v1.3.0 

type NoOpTracer struct{}

NoOpTracer is a tracer that does nothing (default when tracing disabled). It satisfies the Tracer interface but performs no actual tracing operations.

func (*NoOpTracer) AddEvent added in v1.3.0 

func (n *NoOpTracer) AddEvent(_ SpanHandle, _ string, _ map[string]any)

AddEvent does nothing.

func (*NoOpTracer) AddStreamingChunk added in v1.3.0 

func (n *NoOpTracer) AddStreamingChunk(_ string, _ *BifrostResponse)

AddStreamingChunk does nothing.

func (*NoOpTracer) AttachPluginLogs added in v1.5.0 

func (n *NoOpTracer) AttachPluginLogs(_ string, _ []PluginLogEntry)

AttachPluginLogs does nothing.

func (*NoOpTracer) CleanupStreamAccumulator added in v1.3.0 

func (n *NoOpTracer) CleanupStreamAccumulator(_ string)

CleanupStreamAccumulator does nothing.

func (*NoOpTracer) ClearDeferredSpan added in v1.3.0 

func (n *NoOpTracer) ClearDeferredSpan(_ string)

ClearDeferredSpan does nothing.

func (*NoOpTracer) CompleteAndFlushTrace added in v1.5.1 

func (n *NoOpTracer) CompleteAndFlushTrace(_ string)

CompleteAndFlushTrace does nothing.

func (*NoOpTracer) CreateStreamAccumulator added in v1.3.0 

func (n *NoOpTracer) CreateStreamAccumulator(_ string, _ time.Time)

CreateStreamAccumulator does nothing.

func (*NoOpTracer) CreateTrace added in v1.3.0 

func (n *NoOpTracer) CreateTrace(_ string, _ ...string) string

CreateTrace returns an empty string (no trace created).

func (*NoOpTracer) EndSpan added in v1.3.0 

func (n *NoOpTracer) EndSpan(_ SpanHandle, _ SpanStatus, _ string)

EndSpan does nothing.

func (*NoOpTracer) EndStream added in v1.6.0 

func (n *NoOpTracer) EndStream(_ string, _ *BifrostError)

EndStream does nothing.

func (*NoOpTracer) EndTrace added in v1.3.0 

func (n *NoOpTracer) EndTrace(_ string) *Trace

EndTrace returns nil (no trace to end).

func (*NoOpTracer) GateSend added in v1.6.0 

func (n *NoOpTracer) GateSend(_ string, chunk *BifrostStreamChunk, _ bool, _ bool, ch chan *BifrostStreamChunk, ctx *BifrostContext) (ok bool)

GateSend forwards the chunk directly to the channel with ctx.Done() guard. NoOpTracer has no gate state, so this is a pure passthrough. Recovers from "send on closed channel" so a closed consumer cannot crash the producer.

func (*NoOpTracer) GetAccumulatedChunks added in v1.3.0 

func (n *NoOpTracer) GetAccumulatedChunks(_ string) (*BifrostResponse, int64, int)

GetAccumulatedChunks returns nil, 0, 0.

func (*NoOpTracer) GetAccumulatedResponse added in v1.6.0 

func (n *NoOpTracer) GetAccumulatedResponse(_ string) *BifrostResponse

GetAccumulatedResponse returns nil — NoOpTracer has no accumulator.

func (*NoOpTracer) GetDeferredSpanHandle added in v1.3.0 

func (n *NoOpTracer) GetDeferredSpanHandle(_ string) SpanHandle

GetDeferredSpanHandle returns nil.

func (*NoOpTracer) GetDeferredSpanID added in v1.3.0 

func (n *NoOpTracer) GetDeferredSpanID(_ string) string

GetDeferredSpanID returns empty string.

func (*NoOpTracer) GetSpanHandleByID added in v1.5.9 

func (n *NoOpTracer) GetSpanHandleByID(_ string, _ *string) SpanHandle

GetSpanHandleByID returns nil.

func (*NoOpTracer) IsStreamEnded added in v1.6.0 

func (n *NoOpTracer) IsStreamEnded(_ string) bool

IsStreamEnded returns false — NoOpTracer has no gate state.

func (*NoOpTracer) IsStreamPaused added in v1.6.0 

func (n *NoOpTracer) IsStreamPaused(_ string) bool

IsStreamPaused returns false — NoOpTracer has no gate state.

func (*NoOpTracer) PauseStream added in v1.6.0 

func (n *NoOpTracer) PauseStream(_ string)

PauseStream does nothing.

func (*NoOpTracer) PopulateLLMRequestAttributes added in v1.3.0 

func (n *NoOpTracer) PopulateLLMRequestAttributes(_ SpanHandle, _ *BifrostRequest)

PopulateLLMRequestAttributes does nothing.

func (*NoOpTracer) PopulateLLMResponseAttributes added in v1.3.0 

func (n *NoOpTracer) PopulateLLMResponseAttributes(_ *BifrostContext, _ SpanHandle, _ *BifrostResponse, _ *BifrostError)

PopulateLLMResponseAttributes does nothing.

func (*NoOpTracer) ProcessStreamingChunk added in v1.3.0 

func (n *NoOpTracer) ProcessStreamingChunk(_ *BifrostContext, _ string, _ bool, _ *BifrostResponse, _ *BifrostError) *StreamAccumulatorResult

ProcessStreamingChunk returns nil.

func (*NoOpTracer) ResumeStream added in v1.6.0 

func (n *NoOpTracer) ResumeStream(_ string)

ResumeStream does nothing.

func (*NoOpTracer) SetAttribute added in v1.3.0 

func (n *NoOpTracer) SetAttribute(_ SpanHandle, _ string, _ any)

SetAttribute does nothing.

func (*NoOpTracer) SetTraceRedactionReplacements added in v1.6.4 

func (n *NoOpTracer) SetTraceRedactionReplacements(_ string, _ RedactionPhase, _ map[string]string)

SetTraceRedactionReplacements does nothing.

func (*NoOpTracer) StartSpan added in v1.3.0 

func (n *NoOpTracer) StartSpan(ctx context.Context, _ string, _ SpanKind) (context.Context, SpanHandle)

StartSpan returns the context unchanged and a nil handle.

func (*NoOpTracer) Stop added in v1.3.0 

func (n *NoOpTracer) Stop()

Stop does nothing.

func (*NoOpTracer) StoreDeferredSpan added in v1.3.0 

func (n *NoOpTracer) StoreDeferredSpan(_ string, _ SpanHandle)

StoreDeferredSpan does nothing.

func (*NoOpTracer) WaitForFlusher added in v1.6.0 

func (n *NoOpTracer) WaitForFlusher(_ string)

WaitForFlusher does nothing — NoOpTracer has no gate or flusher.

type OAuth2Config added in v1.4.0 

type OAuth2Config struct {
	ID              string   `json:"id"`
	ClientID        string   `json:"client_id,omitempty"`        // Optional: Will be obtained via dynamic registration (RFC 7591) if not provided
	ClientSecret    string   `json:"client_secret,omitempty"`    // Optional: For public clients using PKCE, or obtained via dynamic registration
	AuthorizeURL    string   `json:"authorize_url,omitempty"`    // Optional: Will be discovered from ServerURL if not provided
	TokenURL        string   `json:"token_url,omitempty"`        // Optional: Will be discovered from ServerURL if not provided
	RegistrationURL *string  `json:"registration_url,omitempty"` // Optional: For dynamic client registration (RFC 7591), can be discovered
	RedirectURI     string   `json:"redirect_uri"`               // Required
	Scopes          []string `json:"scopes,omitempty"`           // Optional: Can be discovered
	ServerURL       string   `json:"server_url"`                 // MCP server URL for OAuth discovery (required if URLs not provided)
	Resource        string   `json:"resource,omitempty"`         // Optional OAuth resource indicator (RFC 8707); omitted when empty
	UseDiscovery    bool     `json:"use_discovery,omitempty"`    // Deprecated: Discovery now happens automatically when URLs are missing
}

OauthConfig represents OAuth client configuration

type OAuth2FlowInitiation added in v1.4.0 

type OAuth2FlowInitiation struct {
	OauthConfigID string    `json:"oauth_config_id"`
	AuthorizeURL  string    `json:"authorize_url"`
	State         string    `json:"state"`
	ExpiresAt     time.Time `json:"expires_at"`
}

OauthFlowInitiation represents the response when initiating an OAuth flow

type OAuth2Provider added in v1.4.0 

type OAuth2Provider interface {
	// GetAccessToken retrieves the access token for a given oauth_config_id (server-level OAuth)
	GetAccessToken(ctx context.Context, oauthConfigID string) (string, error)

	// RefreshAccessToken refreshes the access token for a given oauth_config_id
	RefreshAccessToken(ctx context.Context, oauthConfigID string) error

	// ValidateToken checks if the token is still valid
	ValidateToken(ctx context.Context, oauthConfigID string) (bool, error)

	// RevokeToken revokes the OAuth token
	RevokeToken(ctx context.Context, oauthConfigID string) error

	// GetUserAccessTokenByMode retrieves the upstream access token for a single
	// identity dimension determined by mode. No fallback chain — exactly one
	// identity column is queried. Filters status='active' so orphaned rows never
	// satisfy a lookup. identity is the user ID for MCPAuthModeUser, the VK row
	// ID for MCPAuthModeVK, and the raw session ID for MCPAuthModeSession.
	GetUserAccessTokenByMode(ctx context.Context, mode MCPAuthMode, identity, mcpClientID string) (string, error)

	// InitiateUserOAuthFlow creates or refreshes the per-user OAuth flow row
	// for a (mode, identity, mcp_client) binding and returns the auth landing
	// URL. flowMode tags the row's flow_mode and decides which identity column
	// gets populated from context (UserID for MCPAuthModeUser, the resolved VK
	// row ID for MCPAuthModeVK, the session ID for MCPAuthModeSession). For
	// MCPAuthModeUser flows where no UserID is available in context yet
	// (external MCP client OAuth init), the column is left NULL and stamped
	// at completion. Returns (flow initiation details, flow row ID, error).
	InitiateUserOAuthFlow(ctx context.Context, oauthConfigID string, mcpClientID string, redirectURI string, flowMode MCPAuthMode) (*OAuth2FlowInitiation, string, error)

	// CompleteUserOAuthFlow handles the OAuth callback for a per-user flow.
	// Returns the SessionID stored on the flow row (populated for session-mode,
	// empty otherwise).
	CompleteUserOAuthFlow(ctx context.Context, state string, code string) (string, error)

	// RefreshUserAccessToken refreshes a per-user OAuth access token, looked up
	// by the token row's primary-key ID.
	RefreshUserAccessToken(ctx context.Context, tokenID string) error
}

OauthProvider interface defines OAuth operations

type OAuth2Token added in v1.4.0 

type OAuth2Token struct {
	ID              string     `json:"id"`
	AccessToken     string     `json:"access_token"`
	RefreshToken    string     `json:"refresh_token"`
	TokenType       string     `json:"token_type"`
	ExpiresAt       time.Time  `json:"expires_at"`
	Scopes          []string   `json:"scopes"`
	LastRefreshedAt *time.Time `json:"last_refreshed_at,omitempty"`
}

OauthToken represents OAuth access and refresh tokens

type OAuth2TokenExchangeRequest added in v1.4.0 

type OAuth2TokenExchangeRequest struct {
	GrantType    string `json:"grant_type"`
	Code         string `json:"code,omitempty"`
	RedirectURI  string `json:"redirect_uri,omitempty"`
	ClientID     string `json:"client_id"`
	ClientSecret string `json:"client_secret,omitempty"`
	RefreshToken string `json:"refresh_token,omitempty"`
	CodeVerifier string `json:"code_verifier,omitempty"` // PKCE verifier for authorization_code grant
	Resource     string `json:"resource,omitempty"`      // OAuth resource indicator (RFC 8707)
}

OAuth2TokenExchangeRequest represents the OAuth token exchange request

type OAuth2TokenExchangeResponse added in v1.4.0 

type OAuth2TokenExchangeResponse struct {
	AccessToken  string `json:"access_token"`
	RefreshToken string `json:"refresh_token,omitempty"`
	TokenType    string `json:"token_type"`
	ExpiresIn    int    `json:"expires_in"`
	Scope        string `json:"scope,omitempty"`
}

OAuth2TokenExchangeResponse represents the OAuth token exchange response

type OCRDocument added in v1.4.18 

type OCRDocument struct {
	Type        OCRDocumentType `json:"type"`
	DocumentURL *string         `json:"document_url,omitempty"`
	ImageURL    *string         `json:"image_url,omitempty"`
}

OCRDocument represents the document input for an OCR request.

type OCRDocumentType added in v1.4.18 

type OCRDocumentType string

OCRDocumentType specifies the type of document input for an OCR request. 

const (
	// OCRDocumentTypeDocumentURL represents a document URL input (e.g., PDF URL or base64 data URL).
	OCRDocumentTypeDocumentURL OCRDocumentType = "document_url"
	// OCRDocumentTypeImageURL represents an image URL input.
	OCRDocumentTypeImageURL OCRDocumentType = "image_url"
)

type OCRPage added in v1.4.18 

type OCRPage struct {
	Index      int                `json:"index"`
	Markdown   string             `json:"markdown"`
	Images     []OCRPageImage     `json:"images,omitempty"`
	Dimensions *OCRPageDimensions `json:"dimensions,omitempty"`
}

OCRPage represents a single processed page from an OCR response.

type OCRPageDimensions added in v1.4.18 

type OCRPageDimensions struct {
	DPI    int `json:"dpi"`
	Height int `json:"height"`
	Width  int `json:"width"`
}

OCRPageDimensions represents the dimensions of an OCR page.

type OCRPageImage added in v1.4.18 

type OCRPageImage struct {
	ID           string  `json:"id"`
	TopLeftX     float64 `json:"top_left_x"`
	TopLeftY     float64 `json:"top_left_y"`
	BottomRightX float64 `json:"bottom_right_x"`
	BottomRightY float64 `json:"bottom_right_y"`
	ImageBase64  *string `json:"image_base64,omitempty"`
}

OCRPageImage represents an extracted image from an OCR page.

type OCRParameters added in v1.4.18 

type OCRParameters struct {
	IncludeImageBase64       *bool                  `json:"include_image_base64,omitempty"`
	Pages                    []int                  `json:"pages,omitempty"`
	ImageLimit               *int                   `json:"image_limit,omitempty"`
	ImageMinSize             *int                   `json:"image_min_size,omitempty"`
	TableFormat              *string                `json:"table_format,omitempty"`
	ExtractHeader            *bool                  `json:"extract_header,omitempty"`
	ExtractFooter            *bool                  `json:"extract_footer,omitempty"`
	BBoxAnnotationFormat     *string                `json:"bbox_annotation_format,omitempty"`
	DocumentAnnotationFormat *string                `json:"document_annotation_format,omitempty"`
	DocumentAnnotationPrompt *string                `json:"document_annotation_prompt,omitempty"`
	ExtraParams              map[string]interface{} `json:"-"`
}

OCRParameters contains optional parameters for an OCR request.

type OCRUsageInfo added in v1.4.18 

type OCRUsageInfo struct {
	PagesProcessed int `json:"pages_processed"`
	DocSizeBytes   int `json:"doc_size_bytes"`
}

OCRUsageInfo represents usage information from an OCR response.

type ObservabilityPlugin added in v1.3.0 

type ObservabilityPlugin interface {
	BasePlugin

	// Inject receives a completed trace for forwarding to observability backends.
	// This method is called asynchronously after the response has been written to the client.
	// The trace contains all spans that were added during request processing.
	//
	// Implementations should:
	// - Convert the trace to their backend's format
	// - Send the trace to the backend (can be async, but see retention note below)
	// - Handle errors gracefully (log and continue)
	//
	// The context passed is a fresh background context, not the request context.
	//
	// Retention: implementations MUST NOT retain the *Trace pointer after Inject
	// returns. The caller releases the underlying trace back to a sync.Pool
	// immediately after Inject completes. If a plugin needs to forward the trace
	// asynchronously, it must copy the data it needs before returning.
	Inject(ctx context.Context, trace *Trace) error
}

ObservabilityPlugin is an interface for plugins that receive completed traces for forwarding to observability backends (e.g., OTEL collectors, Datadog, etc.)

ObservabilityPlugins are called asynchronously after the HTTP response has been written to the wire, ensuring they don't add latency to the client response.

Plugins implementing this interface will: 1. Continue to work as regular plugins via PreLLMHook/PostLLMHook 2. Additionally receive completed traces via the Inject method

Example backends: OpenTelemetry collectors, Datadog, Jaeger, Maxim, etc.

Note: Go type assertion (plugin.(ObservabilityPlugin)) is used to identify plugins implementing this interface - no marker method is needed.

type OllamaKeyConfig added in v1.5.0 

type OllamaKeyConfig struct {
	URL SecretVar `json:"url"` // Ollama server base URL (required, supports env. prefix)
}

OllamaKeyConfig represents the Ollama-specific key configuration. It allows each key to target a different Ollama server URL, enabling per-key routing and round-robin load balancing across multiple Ollama instances.

type OpenAIConfig added in v1.4.14 

type OpenAIConfig struct {
	DisableStore bool `json:"disable_store"` // When true, forces store=false on all outgoing OpenAI requests (default: false)
}

OpenAIConfig holds OpenAI-specific provider configuration.

type OptionalJSON added in v1.5.14 

type OptionalJSON[T any] struct {
	Set   bool
	Null  bool
	Value T
}

OptionalJSON tracks whether a JSON field was omitted, null, or set to a value.

func (OptionalJSON[T]) IsZero added in v1.5.14 

func (o OptionalJSON[T]) IsZero() bool

IsZero reports whether the JSON field was omitted during decoding.

func (OptionalJSON[T]) MarshalJSON added in v1.5.14 

func (o OptionalJSON[T]) MarshalJSON() ([]byte, error)

MarshalJSON encodes null for omitted/null fields and the wrapped value otherwise.

func (*OptionalJSON[T]) UnmarshalJSON added in v1.5.14 

func (o *OptionalJSON[T]) UnmarshalJSON(data []byte) error

UnmarshalJSON records field presence and decodes a non-null field value.

type OrderedMap added in v1.2.31 

type OrderedMap struct {
	// contains filtered or unexported fields
}

OrderedMap is a map that preserves insertion order of keys. It stores key-value pairs and maintains the order in which keys were first inserted. It is NOT safe for concurrent use.

func NewOrderedMap added in v1.4.1 

func NewOrderedMap() *OrderedMap

NewOrderedMap creates a new empty OrderedMap.

func NewOrderedMapFromPairs added in v1.4.1 

func NewOrderedMapFromPairs(pairs ...Pair) *OrderedMap

NewOrderedMapFromPairs creates an OrderedMap from key-value pairs, preserving the given order.

func NewOrderedMapWithCapacity added in v1.4.1 

func NewOrderedMapWithCapacity(cap int) *OrderedMap

NewOrderedMapWithCapacity creates a new empty OrderedMap with preallocated capacity.

func OrderedMapFromMap added in v1.4.1 

func OrderedMapFromMap(m map[string]interface{}) *OrderedMap

OrderedMapFromMap creates an OrderedMap from a plain map. Keys are sorted lexicographically to ensure deterministic ordering.

func SafeExtractOrderedMap added in v1.2.31 

func SafeExtractOrderedMap(value interface{}) (*OrderedMap, bool)

func (*OrderedMap) Clone added in v1.4.1 

func (om *OrderedMap) Clone() *OrderedMap

Clone creates a shallow copy of the OrderedMap (keys and top-level values are copied, but nested values share references).

func (*OrderedMap) Delete added in v1.4.1 

func (om *OrderedMap) Delete(key string)

Delete removes a key and its value. The key is also removed from the ordered keys list.

func (*OrderedMap) Get added in v1.4.1 

func (om *OrderedMap) Get(key string) (interface{}, bool)

Get returns the value associated with the key and whether the key exists.

func (*OrderedMap) Keys added in v1.4.1 

func (om *OrderedMap) Keys() []string

Keys returns the keys in insertion order. The returned slice is a copy.

func (*OrderedMap) Len added in v1.4.1 

func (om *OrderedMap) Len() int

Len returns the number of key-value pairs.

func (OrderedMap) MarshalJSON added in v1.2.31 

func (om OrderedMap) MarshalJSON() ([]byte, error)

MarshalJSON serializes the OrderedMap to JSON, preserving insertion order of keys. Uses a value receiver so that both OrderedMap and *OrderedMap invoke this method (critical for []OrderedMap slices like AnyOf/OneOf/AllOf in ToolFunctionParameters).

func (*OrderedMap) MarshalSorted added in v1.4.1 

func (om *OrderedMap) MarshalSorted() ([]byte, error)

MarshalSorted serializes the OrderedMap to JSON with keys sorted alphabetically. Use this when deterministic output is needed regardless of insertion order (e.g., hashing).

func (*OrderedMap) Range added in v1.4.1 

func (om *OrderedMap) Range(fn func(key string, value interface{}) bool)

Range iterates over key-value pairs in insertion order. If fn returns false, iteration stops.

func (*OrderedMap) Set added in v1.4.1 

func (om *OrderedMap) Set(key string, value interface{})

Set sets the value for a key. If the key is new, it is appended to the end. If the key already exists, its value is updated in place without changing order.

func (*OrderedMap) SortKeys added in v1.4.8 

func (om *OrderedMap) SortKeys()

SortKeys sorts the keys of this OrderedMap using JSON Schema priority ordering (type, description, properties, required first), with remaining keys sorted alphabetically. Nested *OrderedMap values are also sorted recursively.

func (*OrderedMap) SortedCopy added in v1.4.8 

func (om *OrderedMap) SortedCopy() *OrderedMap

SortedCopy returns a new OrderedMap with keys sorted using JSON Schema priority ordering. Nested *OrderedMap values are recursively copied and sorted. Primitive values (strings, numbers, bools) are shared, not cloned. This is much cheaper than a full JSON marshal/unmarshal Clone because it only allocates new key slices and value maps.

func (*OrderedMap) SortedCopyPreservingProperties added in v1.4.9 

func (om *OrderedMap) SortedCopyPreservingProperties() *OrderedMap

SortedCopyPreservingProperties is like SortedCopy but preserves the key order of user-defined property names inside "properties" maps. Structural JSON Schema keys (type, description, properties, required) are still sorted by priority, and all other keys alphabetically. When the key "properties" is encountered, its value (an OrderedMap of user-defined field names) has its top-level key order preserved while each nested schema value is recursively processed with the same property-aware logic.

This ensures deterministic serialization for prompt caching (structural keys are always in the same order) while preserving the client's intended field generation order for LLM structured output.

func (*OrderedMap) ToMap added in v1.4.1 

func (om *OrderedMap) ToMap() map[string]interface{}

ToMap returns a plain map[string]interface{} with the same key-value pairs. The returned map does not preserve insertion order.

func (*OrderedMap) UnmarshalJSON added in v1.4.1 

func (om *OrderedMap) UnmarshalJSON(data []byte) error

UnmarshalJSON deserializes JSON into the OrderedMap, preserving the key order from the JSON document. Nested objects are also deserialized as *OrderedMap. Note: uses encoding/json.Decoder (not sonic) because token-by-token decoding is required to preserve key order from the JSON document.

type Pair added in v1.4.1 

type Pair struct {
	Key   string
	Value interface{}
}

Pair is a key-value pair for constructing OrderedMaps with order preserved.

func KV added in v1.4.1 

func KV(key string, value interface{}) Pair

KV is a shorthand constructor for Pair.

type PassthroughLogParams added in v1.4.8 

type PassthroughLogParams struct {
	Method     string `json:"method"`
	Path       string `json:"path"`      // stripped path, e.g. "/v1/fine-tuning/jobs"
	RawQuery   string `json:"raw_query"` // raw query string, no "?"
	StatusCode int    `json:"status_code"`
	Model      string `json:"model,omitempty"` // model extracted from path or request body
}

type PerRequestLimits added in v1.2.14 

type PerRequestLimits struct {
	PromptTokens     *int `json:"prompt_tokens,omitempty"`
	CompletionTokens *int `json:"completion_tokens,omitempty"`
}

type PluginConfig added in v1.1.23 

type PluginConfig struct {
	Enabled   bool             `json:"enabled"`
	Name      string           `json:"name"`
	Path      *string          `json:"path,omitempty"`
	Version   *int16           `json:"version,omitempty"`
	Config    any              `json:"config,omitempty"`
	Placement *PluginPlacement `json:"placement,omitempty"` // "pre_builtin" or "post_builtin". Default: "post_builtin"
	Order     *int             `json:"order,omitempty"`     // Position within placement group. Lower = earlier. Default: 0
}

PluginConfig is the configuration for a plugin. It contains the name of the plugin, whether it is enabled, and the configuration for the plugin.

type PluginLogEntry added in v1.5.0 

type PluginLogEntry struct {
	PluginName string   `json:"plugin_name"`
	Level      LogLevel `json:"level"`
	Message    string   `json:"message"`
	Timestamp  int64    `json:"timestamp"` // Unix milliseconds
}

PluginLogEntry represents a structured log entry emitted by a plugin via ctx.Log().

type PluginPlacement added in v1.4.9 

type PluginPlacement string

Plugin placement constants control where custom plugins execute relative to built-in plugins. 

const (
	PluginPlacementPostBuiltin PluginPlacement = "post_builtin"
	PluginPlacementPreBuiltin  PluginPlacement = "pre_builtin"
	PluginPlacementBuiltin     PluginPlacement = "builtin"
	PluginPlacementDefault     PluginPlacement = PluginPlacementPostBuiltin
)

type PluginShortCircuit added in v1.1.5 

type PluginShortCircuit = LLMPluginShortCircuit

PluginShortCircuit is the legacy name for LLMPluginShortCircuit (v1.3.x compatibility). Deprecated: Use LLMPluginShortCircuit instead.

type PluginSpanFilter added in v1.5.19 

type PluginSpanFilter struct {
	Mode    PluginSpanFilterMode `json:"mode"`
	Plugins []string             `json:"plugins"`
}

PluginSpanFilter configures which plugin spans an observability connector exports. Mode "include" exports only the listed plugins; mode "exclude" exports everything except them. It is shared by every observability connector (OTEL, Datadog, BigQuery) so the span-name contract and reparenting behavior stay consistent across exporters.

func (*PluginSpanFilter) BuildReparentMap added in v1.5.19 

func (f *PluginSpanFilter) BuildReparentMap(spans []*Span) map[string]string

BuildReparentMap returns a map of filteredSpanID → effective ancestor spanID for all spans that the filter removes. When plugin spans are chained (each span's parent is the previous plugin's span), removing a span from the middle would leave its children with a dangling parent ID. The map lets callers rewrite those parent IDs to the nearest exported ancestor, handling consecutive filtered spans in a chain. Returns nil when the filter is nil or nothing is filtered.

func (*PluginSpanFilter) ShouldExportSpan added in v1.5.19 

func (f *PluginSpanFilter) ShouldExportSpan(span *Span) bool

ShouldExportSpan reports whether a span survives the filter. Non-plugin spans and spans evaluated against a nil filter are always exported. Plugin spans are checked against the filter's plugin list and mode.

func (*PluginSpanFilter) Validate added in v1.5.19 

func (f *PluginSpanFilter) Validate() error

Validate reports whether the filter's mode is one of the two valid modes. A nil filter is valid (it filters nothing).

type PluginSpanFilterMode added in v1.5.19 

type PluginSpanFilterMode string

PluginSpanFilterMode controls whether the plugins list is an allowlist or denylist. 

const (
	// PluginSpanFilterModeInclude exports only the listed plugins' spans.
	PluginSpanFilterModeInclude PluginSpanFilterMode = "include"
	// PluginSpanFilterModeExclude exports everything except the listed plugins' spans.
	PluginSpanFilterModeExclude PluginSpanFilterMode = "exclude"
)

type PluginStatus added in v1.2.17 

type PluginStatus struct {
	Name   string       `json:"name"` // Display name of the plugin
	Status string       `json:"status"`
	Logs   []string     `json:"logs"`
	Types  []PluginType `json:"types"` // Plugin types (LLM, MCP, HTTP)
}

PluginStatus represents the status of a plugin.

type PluginType added in v1.4.0 

type PluginType string

PluginType represents the type of plugin. 

const (
	PluginTypeLLM  PluginType = "llm"
	PluginTypeMCP  PluginType = "mcp"
	PluginTypeHTTP PluginType = "http"
)

type PostHookRunner added in v1.1.8 

type PostHookRunner func(ctx *BifrostContext, result *BifrostResponse, err *BifrostError) (*BifrostResponse, *BifrostError)

type Pricing added in v1.2.14 

type Pricing struct {
	Prompt            *string `json:"prompt,omitempty"`
	Completion        *string `json:"completion,omitempty"`
	Request           *string `json:"request,omitempty"`
	Image             *string `json:"image,omitempty"`
	WebSearch         *string `json:"web_search,omitempty"`
	InternalReasoning *string `json:"internal_reasoning,omitempty"`
	InputCacheRead    *string `json:"input_cache_read,omitempty"`
	InputCacheWrite   *string `json:"input_cache_write,omitempty"`
}

type PromptCacheBreakpoint added in v1.6.4 

type PromptCacheBreakpoint struct {
	Mode *string `json:"mode,omitempty"`
}

PromptCacheBreakpoint marks the end of a cacheable prompt prefix on a content block (OpenAI gpt-5.6+). Only "explicit" is valid for Mode.

type PromptCacheOptions added in v1.6.4 

type PromptCacheOptions struct {
	Mode *string `json:"mode,omitempty"`
	TTL  *string `json:"ttl,omitempty"`
}

PromptCacheOptions is the request-wide prompt-caching configuration OpenAI added with the gpt-5.6 family (echoed back on the response). Mode is "implicit" or "explicit"; TTL is the minimum breakpoint lifetime (currently "30m"). Values are passed through untouched.

type Provider 

type Provider interface {
	// GetProviderKey returns the provider's identifier
	GetProviderKey() ModelProvider
	// ListModels performs a list models request
	ListModels(ctx *BifrostContext, keys []Key, request *BifrostListModelsRequest) (*BifrostListModelsResponse, *BifrostError)
	// TextCompletion performs a text completion request
	TextCompletion(ctx *BifrostContext, key Key, request *BifrostTextCompletionRequest) (*BifrostTextCompletionResponse, *BifrostError)
	// TextCompletionStream performs a text completion stream request.
	// postHookSpanFinalizer is invoked by the provider's stream goroutine on stream completion
	// (or on its panic-recovery defer) to finalize aggregated post-hook spans and release the
	// per-attempt plugin pipeline. Pass nil if the caller does not need finalization.
	TextCompletionStream(ctx *BifrostContext, postHookRunner PostHookRunner, postHookSpanFinalizer func(context.Context), key Key, request *BifrostTextCompletionRequest) (chan *BifrostStreamChunk, *BifrostError)
	// ChatCompletion performs a chat completion request
	ChatCompletion(ctx *BifrostContext, key Key, request *BifrostChatRequest) (*BifrostChatResponse, *BifrostError)
	// ChatCompletionStream performs a chat completion stream request
	ChatCompletionStream(ctx *BifrostContext, postHookRunner PostHookRunner, postHookSpanFinalizer func(context.Context), key Key, request *BifrostChatRequest) (chan *BifrostStreamChunk, *BifrostError)
	// Responses performs a completion request using the Responses API (uses chat completion request internally for non-openai providers)
	Responses(ctx *BifrostContext, key Key, request *BifrostResponsesRequest) (*BifrostResponsesResponse, *BifrostError)
	// ResponsesStream performs a completion request using the Responses API stream (uses chat completion stream request internally for non-openai providers)
	ResponsesStream(ctx *BifrostContext, postHookRunner PostHookRunner, postHookSpanFinalizer func(context.Context), key Key, request *BifrostResponsesRequest) (chan *BifrostStreamChunk, *BifrostError)
	// CountTokens performs a count tokens request
	CountTokens(ctx *BifrostContext, key Key, request *BifrostResponsesRequest) (*BifrostCountTokensResponse, *BifrostError)
	// Compaction compacts a conversation context window (OpenAI-only; other providers return unsupported)
	Compaction(ctx *BifrostContext, key Key, request *BifrostCompactionRequest) (*BifrostCompactionResponse, *BifrostError)
	// Embedding performs an embedding request
	Embedding(ctx *BifrostContext, key Key, request *BifrostEmbeddingRequest) (*BifrostEmbeddingResponse, *BifrostError)
	// Rerank performs a rerank request to reorder documents by relevance to a query
	Rerank(ctx *BifrostContext, key Key, request *BifrostRerankRequest) (*BifrostRerankResponse, *BifrostError)
	// OCR performs an optical character recognition request on a document
	OCR(ctx *BifrostContext, key Key, request *BifrostOCRRequest) (*BifrostOCRResponse, *BifrostError)
	// Speech performs a text to speech request
	Speech(ctx *BifrostContext, key Key, request *BifrostSpeechRequest) (*BifrostSpeechResponse, *BifrostError)
	// SpeechStream performs a text to speech stream request
	SpeechStream(ctx *BifrostContext, postHookRunner PostHookRunner, postHookSpanFinalizer func(context.Context), key Key, request *BifrostSpeechRequest) (chan *BifrostStreamChunk, *BifrostError)
	// Transcription performs a transcription request
	Transcription(ctx *BifrostContext, key Key, request *BifrostTranscriptionRequest) (*BifrostTranscriptionResponse, *BifrostError)
	// TranscriptionStream performs a transcription stream request
	TranscriptionStream(ctx *BifrostContext, postHookRunner PostHookRunner, postHookSpanFinalizer func(context.Context), key Key, request *BifrostTranscriptionRequest) (chan *BifrostStreamChunk, *BifrostError)
	// ImageGeneration performs an image generation request
	ImageGeneration(ctx *BifrostContext, key Key, request *BifrostImageGenerationRequest) (
		*BifrostImageGenerationResponse, *BifrostError)
	// ImageGenerationStream performs an image generation stream request
	ImageGenerationStream(ctx *BifrostContext, postHookRunner PostHookRunner, postHookSpanFinalizer func(context.Context), key Key,
		request *BifrostImageGenerationRequest) (chan *BifrostStreamChunk, *BifrostError)
	// ImageEdit performs an image edit request
	ImageEdit(ctx *BifrostContext, key Key, request *BifrostImageEditRequest) (*BifrostImageGenerationResponse, *BifrostError)
	// ImageEditStream performs an image edit stream request
	ImageEditStream(ctx *BifrostContext, postHookRunner PostHookRunner, postHookSpanFinalizer func(context.Context), key Key,
		request *BifrostImageEditRequest) (chan *BifrostStreamChunk, *BifrostError)
	// ImageVariation performs an image variation request
	ImageVariation(ctx *BifrostContext, key Key, request *BifrostImageVariationRequest) (*BifrostImageGenerationResponse, *BifrostError)
	// VideoGeneration performs a video generation request
	VideoGeneration(ctx *BifrostContext, key Key, request *BifrostVideoGenerationRequest) (*BifrostVideoGenerationResponse, *BifrostError)
	// VideoRetrieve retrieves a video from the provider
	VideoRetrieve(ctx *BifrostContext, key Key, request *BifrostVideoRetrieveRequest) (*BifrostVideoGenerationResponse, *BifrostError)
	// VideoDownload downloads a video from the provider
	VideoDownload(ctx *BifrostContext, key Key, request *BifrostVideoDownloadRequest) (*BifrostVideoDownloadResponse, *BifrostError)
	// VideoDelete deletes a video from the provider
	VideoDelete(ctx *BifrostContext, key Key, request *BifrostVideoDeleteRequest) (*BifrostVideoDeleteResponse, *BifrostError)
	// VideoList lists videos from the provider
	VideoList(ctx *BifrostContext, key Key, request *BifrostVideoListRequest) (*BifrostVideoListResponse, *BifrostError)
	// VideoRemix remixes a video from the provider
	VideoRemix(ctx *BifrostContext, key Key, request *BifrostVideoRemixRequest) (*BifrostVideoGenerationResponse, *BifrostError)
	// BatchCreate creates a new batch job for asynchronous processing
	BatchCreate(ctx *BifrostContext, key Key, request *BifrostBatchCreateRequest) (*BifrostBatchCreateResponse, *BifrostError)
	// BatchList lists batch jobs
	BatchList(ctx *BifrostContext, keys []Key, request *BifrostBatchListRequest) (*BifrostBatchListResponse, *BifrostError)
	// BatchRetrieve retrieves a specific batch job
	BatchRetrieve(ctx *BifrostContext, keys []Key, request *BifrostBatchRetrieveRequest) (*BifrostBatchRetrieveResponse, *BifrostError)
	// BatchCancel cancels a batch job
	BatchCancel(ctx *BifrostContext, keys []Key, request *BifrostBatchCancelRequest) (*BifrostBatchCancelResponse, *BifrostError)
	// BatchDelete deletes a batch job
	BatchDelete(ctx *BifrostContext, keys []Key, request *BifrostBatchDeleteRequest) (*BifrostBatchDeleteResponse, *BifrostError)
	// BatchResults retrieves results from a completed batch job
	BatchResults(ctx *BifrostContext, keys []Key, request *BifrostBatchResultsRequest) (*BifrostBatchResultsResponse, *BifrostError)
	// FileUpload uploads a file to the provider
	FileUpload(ctx *BifrostContext, key Key, request *BifrostFileUploadRequest) (*BifrostFileUploadResponse, *BifrostError)
	// FileList lists files from the provider
	FileList(ctx *BifrostContext, keys []Key, request *BifrostFileListRequest) (*BifrostFileListResponse, *BifrostError)
	// FileRetrieve retrieves file metadata from the provider
	FileRetrieve(ctx *BifrostContext, keys []Key, request *BifrostFileRetrieveRequest) (*BifrostFileRetrieveResponse, *BifrostError)
	// FileDelete deletes a file from the provider
	FileDelete(ctx *BifrostContext, keys []Key, request *BifrostFileDeleteRequest) (*BifrostFileDeleteResponse, *BifrostError)
	// FileContent downloads file content from the provider
	FileContent(ctx *BifrostContext, keys []Key, request *BifrostFileContentRequest) (*BifrostFileContentResponse, *BifrostError)
	// CachedContentCreate creates a new cached content (Gemini / Vertex AI named cache lifecycle)
	CachedContentCreate(ctx *BifrostContext, key Key, request *BifrostCachedContentCreateRequest) (*BifrostCachedContentCreateResponse, *BifrostError)
	// CachedContentList lists cached contents
	CachedContentList(ctx *BifrostContext, keys []Key, request *BifrostCachedContentListRequest) (*BifrostCachedContentListResponse, *BifrostError)
	// CachedContentRetrieve retrieves a single cached content by name
	CachedContentRetrieve(ctx *BifrostContext, keys []Key, request *BifrostCachedContentRetrieveRequest) (*BifrostCachedContentRetrieveResponse, *BifrostError)
	// CachedContentUpdate updates a cached content's expiration (TTL or expireTime)
	CachedContentUpdate(ctx *BifrostContext, keys []Key, request *BifrostCachedContentUpdateRequest) (*BifrostCachedContentUpdateResponse, *BifrostError)
	// CachedContentDelete deletes a cached content by name
	CachedContentDelete(ctx *BifrostContext, keys []Key, request *BifrostCachedContentDeleteRequest) (*BifrostCachedContentDeleteResponse, *BifrostError)
	// ContainerCreate creates a new container
	ContainerCreate(ctx *BifrostContext, key Key, request *BifrostContainerCreateRequest) (*BifrostContainerCreateResponse, *BifrostError)
	// ContainerList lists containers
	ContainerList(ctx *BifrostContext, keys []Key, request *BifrostContainerListRequest) (*BifrostContainerListResponse, *BifrostError)
	// ContainerRetrieve retrieves a specific container
	ContainerRetrieve(ctx *BifrostContext, keys []Key, request *BifrostContainerRetrieveRequest) (*BifrostContainerRetrieveResponse, *BifrostError)
	// ContainerDelete deletes a container
	ContainerDelete(ctx *BifrostContext, keys []Key, request *BifrostContainerDeleteRequest) (*BifrostContainerDeleteResponse, *BifrostError)
	// ContainerFileCreate creates a file in a container
	ContainerFileCreate(ctx *BifrostContext, key Key, request *BifrostContainerFileCreateRequest) (*BifrostContainerFileCreateResponse, *BifrostError)
	// ContainerFileList lists files in a container
	ContainerFileList(ctx *BifrostContext, keys []Key, request *BifrostContainerFileListRequest) (*BifrostContainerFileListResponse, *BifrostError)
	// ContainerFileRetrieve retrieves a file from a container
	ContainerFileRetrieve(ctx *BifrostContext, keys []Key, request *BifrostContainerFileRetrieveRequest) (*BifrostContainerFileRetrieveResponse, *BifrostError)
	// ContainerFileContent retrieves the content of a file from a container
	ContainerFileContent(ctx *BifrostContext, keys []Key, request *BifrostContainerFileContentRequest) (*BifrostContainerFileContentResponse, *BifrostError)
	// ContainerFileDelete deletes a file from a container
	ContainerFileDelete(ctx *BifrostContext, keys []Key, request *BifrostContainerFileDeleteRequest) (*BifrostContainerFileDeleteResponse, *BifrostError)
	// Passthrough executes a non-streaming passthrough; body is fully buffered.
	Passthrough(ctx *BifrostContext, key Key, req *BifrostPassthroughRequest) (*BifrostPassthroughResponse, *BifrostError)
	// PassthroughStream executes a streaming passthrough, forwarding raw response bytes as BifrostStreamChunks.
	PassthroughStream(ctx *BifrostContext, postHookRunner PostHookRunner, postHookSpanFinalizer func(context.Context), key Key, req *BifrostPassthroughRequest) (chan *BifrostStreamChunk, *BifrostError)
}

Provider defines the interface for AI model providers.

type ProviderConfig 

type ProviderConfig struct {
	NetworkConfig            NetworkConfig            `json:"network_config"`              // Network configuration
	ConcurrencyAndBufferSize ConcurrencyAndBufferSize `json:"concurrency_and_buffer_size"` // Concurrency settings
	// Logger instance, can be provided by the user or bifrost default logger is used if not provided
	Logger                  Logger                `json:"-"`
	ProxyConfig             *ProxyConfig          `json:"proxy_config,omitempty"`     // Proxy configuration
	SendBackRawRequest      bool                  `json:"send_back_raw_request"`      // Send raw request back in the bifrost response (default: false)
	SendBackRawResponse     bool                  `json:"send_back_raw_response"`     // Send raw response back in the bifrost response (default: false)
	StoreRawRequestResponse bool                  `json:"store_raw_request_response"` // Capture raw request/response for internal logging only; strip from API responses returned to clients (default: false)
	CustomProviderConfig    *CustomProviderConfig `json:"custom_provider_config,omitempty"`
	OpenAIConfig            *OpenAIConfig         `json:"openai_config,omitempty"`
}

ProviderConfig represents the complete configuration for a provider. An array of ProviderConfig needs to be provided in GetConfigForProvider in your account interface implementation.

func (*ProviderConfig) CheckAndSetDefaults added in v1.0.5 

func (config *ProviderConfig) CheckAndSetDefaults()

type ProxyConfig 

type ProxyConfig struct {
	Type      ProxyType  `json:"type"`        // Type of proxy to use
	URL       *SecretVar `json:"url"`         // URL of the proxy server (supports env.*)
	Username  *SecretVar `json:"username"`    // Username for proxy authentication (supports env.*)
	Password  *SecretVar `json:"password"`    // Password for proxy authentication (supports env.*)
	CACertPEM *SecretVar `json:"ca_cert_pem"` // PEM-encoded CA certificate to trust for TLS connections through the proxy (supports env.*)
}

ProxyConfig holds the configuration for proxy settings.

func (*ProxyConfig) MarshalForStorage added in v1.5.11 

func (pc *ProxyConfig) MarshalForStorage() ([]byte, error)

MarshalForStorage serializes proxy settings for persistence (e.g. proxy_config_json). SecretVar fields are stored as plain strings (env.* token or literal). For HTTP API responses use json.Marshal on *ProxyConfig so clients receive value/env_var/from_env objects.

func (*ProxyConfig) Redacted added in v1.3.13 

func (pc *ProxyConfig) Redacted() *ProxyConfig

Redacted returns a redacted copy of the proxy configuration.

type ProxyType 

type ProxyType string

ProxyType defines the type of proxy to use for connections. 

const (
	// NoProxy indicates no proxy should be used
	NoProxy ProxyType = "none"
	// HTTPProxy indicates an HTTP proxy should be used
	HTTPProxy ProxyType = "http"
	// Socks5Proxy indicates a SOCKS5 proxy should be used
	Socks5Proxy ProxyType = "socks5"
	// EnvProxy indicates the proxy should be read from environment variables
	EnvProxy ProxyType = "environment"
)

type RealtimeDelta added in v1.4.8 

type RealtimeDelta struct {
	Text       string `json:"text,omitempty"`
	Audio      string `json:"audio,omitempty"`
	Transcript string `json:"transcript,omitempty"`
	ItemID     string `json:"item_id,omitempty"`
	OutputIdx  *int   `json:"output_index,omitempty"`
	ContentIdx *int   `json:"content_index,omitempty"`
	ResponseID string `json:"response_id,omitempty"`
}

RealtimeDelta carries incremental content for streaming events.

type RealtimeError added in v1.4.8 

type RealtimeError struct {
	Type        string                     `json:"type,omitempty"`
	Code        string                     `json:"code,omitempty"`
	Message     string                     `json:"message,omitempty"`
	Param       string                     `json:"param,omitempty"`
	ExtraParams map[string]json.RawMessage `json:"extra_params,omitempty"`
}

RealtimeError describes an error from the Realtime API.

type RealtimeEventType added in v1.4.8 

type RealtimeEventType string

RealtimeEventType represents the type of a Bifrost unified Realtime event. 

const (
	RTEventSessionUpdate          RealtimeEventType = "session.update"
	RTEventConversationItemCreate RealtimeEventType = "conversation.item.create"
	RTEventConversationItemDelete RealtimeEventType = "conversation.item.delete"
	RTEventResponseCreate         RealtimeEventType = "response.create"
	RTEventResponseCancel         RealtimeEventType = "response.cancel"
	RTEventInputAudioAppend       RealtimeEventType = "input_audio_buffer.append"
	RTEventInputAudioCommit       RealtimeEventType = "input_audio_buffer.commit"
	RTEventInputAudioClear        RealtimeEventType = "input_audio_buffer.clear"
)

Client-to-server event types (sent by the client through Bifrost) 

const (
	RTEventSessionCreated            RealtimeEventType = "session.created"
	RTEventSessionUpdated            RealtimeEventType = "session.updated"
	RTEventConversationCreated       RealtimeEventType = "conversation.created"
	RTEventConversationItemAdded     RealtimeEventType = "conversation.item.added"
	RTEventConversationItemCreated   RealtimeEventType = "conversation.item.created"
	RTEventConversationItemRetrieved RealtimeEventType = "conversation.item.retrieved"
	RTEventConversationItemDone      RealtimeEventType = "conversation.item.done"
	RTEventResponseCreated           RealtimeEventType = "response.created"
	RTEventResponseDone              RealtimeEventType = "response.done"
	RTEventResponseTextDelta         RealtimeEventType = "response.text.delta"
	RTEventResponseTextDone          RealtimeEventType = "response.text.done"
	RTEventResponseAudioDelta        RealtimeEventType = "response.audio.delta"
	RTEventResponseAudioDone         RealtimeEventType = "response.audio.done"
	RTEventResponseAudioTransDelta   RealtimeEventType = "response.audio_transcript.delta"
	RTEventResponseAudioTransDone    RealtimeEventType = "response.audio_transcript.done"
	RTEventResponseOutputItemAdded   RealtimeEventType = "response.output_item.added"
	RTEventResponseOutputItemDone    RealtimeEventType = "response.output_item.done"
	RTEventResponseContentPartAdded  RealtimeEventType = "response.content_part.added"
	RTEventResponseContentPartDone   RealtimeEventType = "response.content_part.done"
	RTEventRateLimitsUpdated         RealtimeEventType = "rate_limits.updated"
	RTEventInputAudioTransCompleted  RealtimeEventType = "conversation.item.input_audio_transcription.completed"
	RTEventInputAudioTransDelta      RealtimeEventType = "conversation.item.input_audio_transcription.delta"
	RTEventInputAudioTransFailed     RealtimeEventType = "conversation.item.input_audio_transcription.failed"
	RTEventInputAudioBufferCommitted RealtimeEventType = "input_audio_buffer.committed"
	RTEventInputAudioBufferCleared   RealtimeEventType = "input_audio_buffer.cleared"
	RTEventInputAudioSpeechStarted   RealtimeEventType = "input_audio_buffer.speech_started"
	RTEventInputAudioSpeechStopped   RealtimeEventType = "input_audio_buffer.speech_stopped"
	RTEventError                     RealtimeEventType = "error"
)

Server-to-client event types (received from the provider, forwarded to client)

type RealtimeItem added in v1.4.8 

type RealtimeItem struct {
	ID          string                     `json:"id,omitempty"`
	Type        string                     `json:"type,omitempty"`
	Role        string                     `json:"role,omitempty"`
	Status      string                     `json:"status,omitempty"`
	Content     json.RawMessage            `json:"content,omitempty"`
	Name        string                     `json:"name,omitempty"`
	CallID      string                     `json:"call_id,omitempty"`
	Arguments   string                     `json:"arguments,omitempty"`
	Output      string                     `json:"output,omitempty"`
	ExtraParams map[string]json.RawMessage `json:"extra_params,omitempty"`
}

RealtimeItem represents a conversation item in the Realtime protocol.

type RealtimeLegacyWebRTCProvider added in v1.5.1 

type RealtimeLegacyWebRTCProvider interface {
	ExchangeLegacyRealtimeWebRTCSDP(ctx *BifrostContext, key Key, sdp string, session json.RawMessage, model string) (string, *BifrostError)
}

RealtimeLegacyWebRTCProvider is an optional interface for providers that support the beta WebRTC handshake (e.g., OpenAI's /v1/realtime). Only checked for legacy integration routes via type assertion. Takes SDP offer + optional session JSON, same as ExchangeRealtimeWebRTCSDP but targets the provider's legacy/beta endpoint.

type RealtimeProvider added in v1.4.8 

type RealtimeProvider interface {
	SupportsRealtimeAPI() bool
	RealtimeWebSocketURL(key Key, model string) string
	RealtimeHeaders(ctx *BifrostContext, key Key) (map[string]string, *BifrostError)
	// SupportsRealtimeWebRTC reports whether the provider supports WebRTC SDP exchange.
	SupportsRealtimeWebRTC() bool
	// ExchangeRealtimeWebRTCSDP performs the provider-specific SDP signaling exchange.
	// The provider owns the HTTP specifics (URL, headers, body format).
	// session may be nil if the signaling format doesn't include session config.
	ExchangeRealtimeWebRTCSDP(ctx *BifrostContext, key Key, model string, sdp string, session json.RawMessage) (string, *BifrostError)
	ToBifrostRealtimeEvent(providerEvent json.RawMessage) (*BifrostRealtimeEvent, error)
	ToProviderRealtimeEvent(bifrostEvent *BifrostRealtimeEvent) (json.RawMessage, error)
	// ShouldStartRealtimeTurn reports whether the canonical client-side event
	// should start pre-hooks. Providers without an explicit turn-start signal
	// return false and rely on finalize-time fallback hooks.
	ShouldStartRealtimeTurn(event *BifrostRealtimeEvent) bool
	// RealtimeTurnFinalEvent returns the canonical provider event that completes
	// a turn and should trigger post-hooks.
	RealtimeTurnFinalEvent() RealtimeEventType
	RealtimeWebRTCDataChannelLabel() string
	RealtimeWebSocketSubprotocol() string
	ShouldForwardRealtimeEvent(event *BifrostRealtimeEvent) bool
	ShouldAccumulateRealtimeOutput(eventType RealtimeEventType) bool
}

RealtimeProvider is an optional interface that providers can implement to indicate support for bidirectional Realtime API (audio/text streaming). Checked via type assertion: provider.(RealtimeProvider).

type RealtimeSession added in v1.4.8 

type RealtimeSession struct {
	ID               string                     `json:"id,omitempty"`
	Model            string                     `json:"model,omitempty"`
	Modalities       []string                   `json:"modalities,omitempty"`
	Instructions     string                     `json:"instructions,omitempty"`
	Voice            string                     `json:"voice,omitempty"`
	Temperature      *float64                   `json:"temperature,omitempty"`
	MaxOutputTokens  json.RawMessage            `json:"max_output_tokens,omitempty"`
	TurnDetection    json.RawMessage            `json:"turn_detection,omitempty"`
	InputAudioFormat string                     `json:"input_audio_format,omitempty"`
	OutputAudioType  string                     `json:"output_audio_type,omitempty"`
	Tools            json.RawMessage            `json:"tools,omitempty"`
	ExtraParams      map[string]json.RawMessage `json:"extra_params,omitempty"`
}

RealtimeSession describes session configuration for the Realtime connection.

type RealtimeSessionEndpointType added in v1.5.1 

type RealtimeSessionEndpointType string

RealtimeSessionEndpointType identifies the public ephemeral-token endpoint shape the client called so providers can preserve versioned behavior. 

const (
	RealtimeSessionEndpointClientSecrets RealtimeSessionEndpointType = "client_secrets"
	RealtimeSessionEndpointSessions      RealtimeSessionEndpointType = "sessions"
)

type RealtimeSessionProvider added in v1.5.1 

type RealtimeSessionProvider interface {
	CreateRealtimeClientSecret(ctx *BifrostContext, key Key, endpointType RealtimeSessionEndpointType, rawRequest json.RawMessage) (*BifrostPassthroughResponse, *BifrostError)
}

RealtimeSessionProvider is an optional interface for providers that can mint short-lived client secrets for browser/client-side Realtime connections. Checked via type assertion: provider.(RealtimeSessionProvider).

type RealtimeSessionRoute added in v1.5.1 

type RealtimeSessionRoute struct {
	Path            string
	EndpointType    RealtimeSessionEndpointType
	DefaultProvider ModelProvider
}

RealtimeSessionRoute describes a provider-registered public route for ephemeral-token creation.

type RealtimeUsageExtractor added in v1.5.1 

type RealtimeUsageExtractor interface {
	ExtractRealtimeTurnUsage(terminalEventRaw []byte) *BifrostLLMUsage
	ExtractRealtimeTurnOutput(terminalEventRaw []byte) *ChatMessage
}

RealtimeUsageExtractor lets providers parse terminal-turn usage/output from their native wire payloads without coupling handlers to a specific protocol.

type RedactionData added in v1.6.4 

type RedactionData struct {
	LiteralReplacements RedactionMapsByPhase `json:"literal_replacements,omitempty"`
	ReversibleMappings  RedactionMapsByPhase `json:"reversible_mappings,omitempty"`
}

RedactionData carries request-scoped redaction data from guardrails to log and trace sinks.

func RedactionDataFromContext added in v1.6.4 

func RedactionDataFromContext(ctx *BifrostContext) (RedactionData, bool)

RedactionDataFromContext returns the redaction data stored on ctx.

func (RedactionData) Clone added in v1.6.4 

func (d RedactionData) Clone() RedactionData

Clone returns an owned snapshot of the redaction data maps.

Redaction data moves from request context into async log entries. A plain struct copy would still share the underlying Go maps, so the log entry could observe later request-context mutations. Cloning gives the log its own stable data while keeping the copy shallow, which is enough because the maps only contain immutable strings.

func (RedactionData) HasReplacements added in v1.6.4 

func (d RedactionData) HasReplacements() bool

HasReplacements reports whether any reversible or literal redaction data is present.

type RedactionMapsByPhase added in v1.6.4 

type RedactionMapsByPhase struct {
	Input  map[string]string `json:"input,omitempty"`
	Output map[string]string `json:"output,omitempty"`
}

RedactionMapsByPhase stores replacement maps separately for request and response content.

func (RedactionMapsByPhase) Clone added in v1.6.4 

func (m RedactionMapsByPhase) Clone() RedactionMapsByPhase

Clone returns an owned copy of the phase-scoped maps.

func (RedactionMapsByPhase) HasReplacements added in v1.6.4 

func (m RedactionMapsByPhase) HasReplacements() bool

HasReplacements reports whether either phase has replacement entries.

func (*RedactionMapsByPhase) MergePhase added in v1.6.4 

func (m *RedactionMapsByPhase) MergePhase(phase RedactionPhase, replacements map[string]string)

MergePhase merges replacements into one phase, copying entries so callers cannot mutate stored state.

func (RedactionMapsByPhase) MergedForMixedFields added in v1.6.4 

func (m RedactionMapsByPhase) MergedForMixedFields() map[string]string

MergedForMixedFields returns both phase maps for fields that can contain input and output.

type RedactionPhase added in v1.6.4 

type RedactionPhase string

RedactionPhase identifies which request lifecycle phase produced a redaction finding. 

const (
	// RedactionPhaseInput marks redaction findings discovered while inspecting request-side content.
	RedactionPhaseInput RedactionPhase = "input"
	// RedactionPhaseOutput marks redaction findings discovered while inspecting response-side content.
	RedactionPhaseOutput RedactionPhase = "output"
)

type ReplicateAliasCfg added in v1.5.19 

type ReplicateAliasCfg struct {
	UseDeploymentsEndpoint *bool `json:"use_deployments_endpoint,omitempty"`
}

ReplicateAliasCfg holds Replicate-specific overrides that apply to a single alias.

type ReplicateKeyConfig added in v1.4.2 

type ReplicateKeyConfig struct {
	UseDeploymentsEndpoint bool `json:"use_deployments_endpoint"` // Whether to use the deployments endpoint instead of the models endpoint
}

ReplicateKeyConfig represents the Replicate-specific key configuration. It contains Replicate-specific settings required for authentication and service access.

type RequestType added in v1.1.31 

type RequestType string

RequestType represents the type of request being made to a provider. 

const (
	ListModelsRequest            RequestType = "list_models"
	TextCompletionRequest        RequestType = "text_completion"
	TextCompletionStreamRequest  RequestType = "text_completion_stream"
	ChatCompletionRequest        RequestType = "chat_completion"
	ChatCompletionStreamRequest  RequestType = "chat_completion_stream"
	ResponsesRequest             RequestType = "responses"
	ResponsesStreamRequest       RequestType = "responses_stream"
	ResponsesRetrieveRequest     RequestType = "responses_retrieve"
	ResponsesDeleteRequest       RequestType = "responses_delete"
	ResponsesCancelRequest       RequestType = "responses_cancel"
	ResponsesInputItemsRequest   RequestType = "responses_input_items"
	EmbeddingRequest             RequestType = "embedding"
	SpeechRequest                RequestType = "speech"
	SpeechStreamRequest          RequestType = "speech_stream"
	TranscriptionRequest         RequestType = "transcription"
	TranscriptionStreamRequest   RequestType = "transcription_stream"
	ImageGenerationRequest       RequestType = "image_generation"
	ImageGenerationStreamRequest RequestType = "image_generation_stream"
	ImageEditRequest             RequestType = "image_edit"
	ImageEditStreamRequest       RequestType = "image_edit_stream"
	ImageVariationRequest        RequestType = "image_variation"
	VideoGenerationRequest       RequestType = "video_generation"
	VideoRetrieveRequest         RequestType = "video_retrieve"
	VideoDownloadRequest         RequestType = "video_download"
	VideoDeleteRequest           RequestType = "video_delete"
	VideoListRequest             RequestType = "video_list"
	VideoRemixRequest            RequestType = "video_remix"
	BatchCreateRequest           RequestType = "batch_create"
	BatchListRequest             RequestType = "batch_list"
	BatchRetrieveRequest         RequestType = "batch_retrieve"
	BatchCancelRequest           RequestType = "batch_cancel"
	BatchResultsRequest          RequestType = "batch_results"
	BatchDeleteRequest           RequestType = "batch_delete"
	FileUploadRequest            RequestType = "file_upload"
	FileListRequest              RequestType = "file_list"
	FileRetrieveRequest          RequestType = "file_retrieve"
	FileDeleteRequest            RequestType = "file_delete"
	CachedContentCreateRequest   RequestType = "cached_content_create"
	CachedContentListRequest     RequestType = "cached_content_list"
	CachedContentRetrieveRequest RequestType = "cached_content_retrieve"
	CachedContentUpdateRequest   RequestType = "cached_content_update"
	CachedContentDeleteRequest   RequestType = "cached_content_delete"
	FileContentRequest           RequestType = "file_content"
	ContainerCreateRequest       RequestType = "container_create"
	ContainerListRequest         RequestType = "container_list"
	ContainerRetrieveRequest     RequestType = "container_retrieve"
	ContainerDeleteRequest       RequestType = "container_delete"
	ContainerFileCreateRequest   RequestType = "container_file_create"
	ContainerFileListRequest     RequestType = "container_file_list"
	ContainerFileRetrieveRequest RequestType = "container_file_retrieve"
	ContainerFileContentRequest  RequestType = "container_file_content"
	ContainerFileDeleteRequest   RequestType = "container_file_delete"
	RerankRequest                RequestType = "rerank"
	OCRRequest                   RequestType = "ocr"
	CountTokensRequest           RequestType = "count_tokens"
	CompactionRequest            RequestType = "compaction"
	MCPToolExecutionRequest      RequestType = "mcp_tool_execution"
	PassthroughRequest           RequestType = "passthrough"
	PassthroughStreamRequest     RequestType = "passthrough_stream"
	UnknownRequest               RequestType = "unknown"
	WebSocketResponsesRequest    RequestType = "websocket_responses"
	RealtimeRequest              RequestType = "realtime"
)

func (RequestType) Value added in v1.6.3 

func (r RequestType) Value() (driver.Value, error)

Value implements driver.Valuer so database drivers that append typed column values (e.g. clickhouse-go batch inserts) can serialize the type.

type RerankDocument added in v1.4.4 

type RerankDocument struct {
	Text string                 `json:"text"`
	ID   *string                `json:"id,omitempty"`
	Meta map[string]interface{} `json:"meta,omitempty"`
}

RerankDocument represents a document to be reranked.

type RerankParameters added in v1.4.4 

type RerankParameters struct {
	TopN            *int                   `json:"top_n,omitempty"`
	MaxTokensPerDoc *int                   `json:"max_tokens_per_doc,omitempty"`
	Priority        *int                   `json:"priority,omitempty"`
	ReturnDocuments *bool                  `json:"return_documents,omitempty"`
	ExtraParams     map[string]interface{} `json:"-"`
}

RerankParameters contains optional parameters for a rerank request.

type RerankResult added in v1.4.4 

type RerankResult struct {
	Index          int             `json:"index"`
	RelevanceScore float64         `json:"relevance_score"`
	Document       *RerankDocument `json:"document,omitempty"`
}

RerankResult represents a single reranked document with its relevance score.

type ResolvedAlias added in v1.5.19 

type ResolvedAlias struct {
	Key    string
	Config *AliasConfig
}

ResolvedAlias is what core stashes in BifrostContext after key-level alias resolution. Key is the user-facing model name the client sent (LHS of the alias map). Config is the matched AliasConfig.

Carrying the alias key alongside the config lets providers consult it as the lowest-precedence tier for family detection — common case: an admin names their alias "best-claude" but the wire ModelID is an opaque Azure deployment ID, so neither the config fields nor request.Model carry the "claude" substring; the alias key does.

func GetResolvedAlias added in v1.5.19 

func GetResolvedAlias(ctx *BifrostContext) *ResolvedAlias

GetResolvedAlias returns the ResolvedAlias that core stashed in ctx after key-level alias resolution, or nil if no alias matched or ctx is nil.

This is set by bifrost.go alongside req.SetModel(resolved). Plugins must not write to this key directly.

type ResolvedKeyAlias added in v1.5.19 

type ResolvedKeyAlias struct {
	ModelID     string       `json:"model_id"`               // wire model identifier actually sent to the provider
	ModelName   *string      `json:"model_name,omitempty"`   // canonical name (used for pricing/logs)
	ModelFamily *ModelFamily `json:"model_family,omitempty"` // resolved family for routing
}

type ResponsesAdvisorCall added in v1.5.20 

type ResponsesAdvisorCall struct {
	ResultType       string  `json:"result_type,omitempty"`               // "advisor_result" | "advisor_redacted_result" | "advisor_tool_result_error"
	Text             *string `json:"advisor_text,omitempty"`              // advisor_result variant
	EncryptedContent *string `json:"advisor_encrypted_content,omitempty"` // advisor_redacted_result variant
	ErrorCode        *string `json:"advisor_error_code,omitempty"`        // advisor_tool_result_error variant
	StopReason       *string `json:"advisor_stop_reason,omitempty"`       // present when max_tokens is set on the tool
}

ResponsesAdvisorCall carries the Anthropic advisor_tool_result content (a discriminated union) alongside an advisor_call. Anthropic-only.

type ResponsesCodeExecutionCall added in v1.6.1 

type ResponsesCodeExecutionCall struct {
	// ToolName is the sub-tool that produced the call:
	// "code_execution" (legacy Python) | "bash_code_execution" | "text_editor_code_execution".
	ToolName string `json:"code_execution_tool_name,omitempty"`
	// Input is the verbatim server_tool_use input JSON (code / command / path /
	// file_text / old_str / new_str), kept as a string to preserve key ordering.
	Input *string `json:"code_execution_input,omitempty"`
	// ResultType is the inner result-content discriminator, e.g.
	// "bash_code_execution_result" | "code_execution_result" |
	// "text_editor_code_execution_result" | "*_tool_result_error".
	ResultType string `json:"code_execution_result_type,omitempty"`

	// Execution result fields (bash / python variants).
	Stdout          *string `json:"code_execution_stdout,omitempty"`
	Stderr          *string `json:"code_execution_stderr,omitempty"`
	ReturnCode      *int    `json:"code_execution_return_code,omitempty"`
	EncryptedStdout *string `json:"code_execution_encrypted_stdout,omitempty"`

	// File-operation result fields (text_editor variant).
	FileType     *string  `json:"code_execution_file_type,omitempty"`      // view: "text" | "image" | "pdf"
	FileContent  *string  `json:"code_execution_file_content,omitempty"`   // view: file contents
	StartLine    *int     `json:"code_execution_start_line,omitempty"`     // view
	NumLines     *int     `json:"code_execution_num_lines,omitempty"`      // view
	TotalLines   *int     `json:"code_execution_total_lines,omitempty"`    // view
	IsFileUpdate *bool    `json:"code_execution_is_file_update,omitempty"` // create
	OldStart     *int     `json:"code_execution_old_start,omitempty"`      // str_replace
	OldLines     *int     `json:"code_execution_old_lines,omitempty"`      // str_replace
	NewStart     *int     `json:"code_execution_new_start,omitempty"`      // str_replace
	NewLines     *int     `json:"code_execution_new_lines,omitempty"`      // str_replace
	Lines        []string `json:"code_execution_lines,omitempty"`          // str_replace diff

	// ErrorCode is set for *_tool_result_error variants (e.g. "unavailable",
	// "execution_time_exceeded", "container_expired", "file_not_found").
	ErrorCode *string `json:"code_execution_error_code,omitempty"`

	// Files lists outputs created during execution (charts, generated files).
	Files []ResponsesCodeExecutionFileOutput `json:"code_execution_files,omitempty"`

	// ContainerExpiresAt is the sandbox container expiry; its id lives on the
	// neutral ResponsesCodeInterpreterToolCall.ContainerID.
	ContainerExpiresAt *string `json:"code_execution_container_expires_at,omitempty"`

	// Caller links this call to the agentic caller that produced it (programmatic
	// tool calling). Nil for direct top-level calls.
	Caller *ResponsesToolCaller `json:"code_execution_caller,omitempty"`
}

ResponsesCodeExecutionCall carries the Anthropic code-execution fidelity that the neutral ResponsesCodeInterpreterToolCall (code/container_id/outputs) cannot represent, so an Anthropic -> Bifrost -> Anthropic round trip can reconstruct the original server_tool_use + *_code_execution_tool_result blocks exactly. Sibling to ResponsesAdvisorCall; Anthropic-only. The code string and container id live on the neutral ResponsesCodeInterpreterToolCall.

type ResponsesCodeExecutionFileOutput added in v1.6.1 

type ResponsesCodeExecutionFileOutput struct {
	FileID string `json:"file_id"`
}

ResponsesCodeExecutionFileOutput is a file produced during a code execution run, referenced by Files API id. Mirrors Anthropic's *_code_execution_output block.

type ResponsesCodeInterpreterOutput added in v1.2.0 

type ResponsesCodeInterpreterOutput struct {
	*ResponsesCodeInterpreterOutputLogs
	*ResponsesCodeInterpreterOutputImage
}

ResponsesCodeInterpreterOutput represents a code interpreter output

func (ResponsesCodeInterpreterOutput) MarshalJSON added in v1.2.0 

func (o ResponsesCodeInterpreterOutput) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshaling for ResponsesCodeInterpreterOutput

func (*ResponsesCodeInterpreterOutput) UnmarshalJSON added in v1.2.0 

func (o *ResponsesCodeInterpreterOutput) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshaling for ResponsesCodeInterpreterOutput

type ResponsesCodeInterpreterOutputImage added in v1.2.0 

type ResponsesCodeInterpreterOutputImage struct {
	Type string `json:"type"` // always "image"
	URL  string `json:"url"`
}

ResponsesCodeInterpreterOutputImage represents the image output from the code interpreter

type ResponsesCodeInterpreterOutputLogs added in v1.2.0 

type ResponsesCodeInterpreterOutputLogs struct {
	Logs string `json:"logs"`
	Type string `json:"type"` // always "logs"
}

ResponsesCodeInterpreterOutputLogs represents the logs output from the code interpreter

type ResponsesCodeInterpreterToolCall added in v1.2.0 

type ResponsesCodeInterpreterToolCall struct {
	Code        *string                          `json:"code"`         // The code to run, or null if not available
	ContainerID string                           `json:"container_id"` // The ID of the container used to run the code
	Outputs     []ResponsesCodeInterpreterOutput `json:"outputs"`      // The outputs generated by the code interpreter, can be null
}

ResponsesCodeInterpreterToolCall represents a code interpreter tool call

type ResponsesComputerToolCall added in v1.2.0 

type ResponsesComputerToolCall struct {
	PendingSafetyChecks []ResponsesComputerToolCallPendingSafetyCheck `json:"pending_safety_checks,omitempty"`
}

ResponsesComputerToolCall represents a computer tool call

type ResponsesComputerToolCallAcknowledgedSafetyCheck added in v1.2.0 

type ResponsesComputerToolCallAcknowledgedSafetyCheck struct {
	ID      string  `json:"id"`
	Code    *string `json:"code,omitempty"`
	Message *string `json:"message,omitempty"`
}

ResponsesComputerToolCallAcknowledgedSafetyCheck represents a safety check that has been acknowledged by the developer

type ResponsesComputerToolCallAction added in v1.2.0 

type ResponsesComputerToolCallAction struct {
	Type    string                                `json:"type"`             // "click" | "double_click" | "drag" | "keypress" | "move" | "screenshot" | "scroll" | "type" | "wait" | "zoom"
	X       *int                                  `json:"x,omitempty"`      // Common X coordinate field (Click, DoubleClick, Move, Scroll)
	Y       *int                                  `json:"y,omitempty"`      // Common Y coordinate field (Click, DoubleClick, Move, Scroll)
	Button  *string                               `json:"button,omitempty"` // "left" | "right" | "wheel" | "back" | "forward"
	Path    []ResponsesComputerToolCallActionPath `json:"path,omitempty"`
	Keys    []string                              `json:"keys,omitempty"`
	ScrollX *int                                  `json:"scroll_x,omitempty"`
	ScrollY *int                                  `json:"scroll_y,omitempty"`
	Text    *string                               `json:"text,omitempty"`
	Region  []int                                 `json:"region,omitempty"` // [x1, y1, x2, y2] for zoom action (Anthropic Opus 4.5)
}

ResponsesComputerToolCallAction represents the different types of computer actions

type ResponsesComputerToolCallActionPath added in v1.2.0 

type ResponsesComputerToolCallActionPath struct {
	X int `json:"x"`
	Y int `json:"y"`
}

type ResponsesComputerToolCallOutput added in v1.2.0 

type ResponsesComputerToolCallOutput struct {
	AcknowledgedSafetyChecks []ResponsesComputerToolCallAcknowledgedSafetyCheck `json:"acknowledged_safety_checks,omitempty"`
}

ResponsesComputerToolCallOutput represents a computer tool call output

type ResponsesComputerToolCallOutputData added in v1.2.0 

type ResponsesComputerToolCallOutputData struct {
	Type     string  `json:"type"` // always "computer_screenshot"
	FileID   *string `json:"file_id,omitempty"`
	ImageURL *string `json:"image_url,omitempty"`
}

ResponsesComputerToolCallOutputData represents a computer screenshot image used with the computer use tool

type ResponsesComputerToolCallPendingSafetyCheck added in v1.2.0 

type ResponsesComputerToolCallPendingSafetyCheck struct {
	ID      string `json:"id"`
	Code    string `json:"code"`
	Message string `json:"message"`
}

ResponsesComputerToolCallPendingSafetyCheck represents a pending safety check

type ResponsesContextDetails added in v1.5.17 

type ResponsesContextDetails struct {
	InputTokens  int `json:"input_tokens"`
	OutputTokens int `json:"output_tokens"`
}

ResponsesContextDetails holds the per-context token breakdown returned by xAI.

type ResponsesCustomToolCall added in v1.2.0 

type ResponsesCustomToolCall struct {
	Input string `json:"input"` // The input for the custom tool call generated by the model
}

ResponsesCustomToolCall represents a custom tool call

type ResponsesFileSearchToolCall added in v1.2.0 

type ResponsesFileSearchToolCall struct {
	Queries []string                            `json:"queries"`
	Results []ResponsesFileSearchToolCallResult `json:"results,omitempty"`
}

type ResponsesFileSearchToolCallResult added in v1.2.0 

type ResponsesFileSearchToolCallResult struct {
	Attributes *map[string]any `json:"attributes,omitempty"`
	FileID     *string         `json:"file_id,omitempty"`
	Filename   *string         `json:"filename,omitempty"`
	Score      *float64        `json:"score,omitempty"`
	Text       *string         `json:"text,omitempty"`
}

type ResponsesFunctionToolCallOutput added in v1.2.0 

type ResponsesFunctionToolCallOutput struct {
	ResponsesFunctionToolCallOutputStr    *string // A JSON string of the output of the function tool call.
	ResponsesFunctionToolCallOutputBlocks []ResponsesMessageContentBlock
}

ResponsesFunctionToolCallOutput represents a function tool call output

func (ResponsesFunctionToolCallOutput) MarshalJSON added in v1.2.0 

func (rf ResponsesFunctionToolCallOutput) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshalling for ResponsesFunctionToolCallOutput. It marshals either ContentStr or ContentBlocks directly without wrapping.

func (*ResponsesFunctionToolCallOutput) UnmarshalJSON added in v1.2.0 

func (rf *ResponsesFunctionToolCallOutput) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshalling for ResponsesFunctionToolCallOutput. It determines whether "content" is a string or array and assigns to the appropriate field. It also handles direct string/array content without a wrapper object.

type ResponsesImageGenerationCall added in v1.2.0 

type ResponsesImageGenerationCall struct {
	Result string `json:"result"`
}

ResponsesImageGenerationCall represents an image generation tool call

type ResponsesInputMessageContentBlockAudio added in v1.2.0 

type ResponsesInputMessageContentBlockAudio struct {
	Format string `json:"format"` // "mp3" or "wav"
	Data   string `json:"data"`   // base64 encoded audio data
}

type ResponsesInputMessageContentBlockFile added in v1.2.0 

type ResponsesInputMessageContentBlockFile struct {
	FileData *string `json:"file_data,omitempty"` // Base64 encoded file data or plain text
	FileURL  *string `json:"file_url,omitempty"`  // Direct URL to file
	Filename *string `json:"filename,omitempty"`  // Name of the file
	FileType *string `json:"file_type,omitempty"` // MIME type (e.g., "application/pdf", "text/plain")
}

type ResponsesInputMessageContentBlockImage added in v1.2.0 

type ResponsesInputMessageContentBlockImage struct {
	ImageURL *string `json:"image_url,omitempty"`
	Detail   *string `json:"detail,omitempty"` // "low" | "high" | "auto"
}

type ResponsesLifecycleProvider added in v1.6.3 

type ResponsesLifecycleProvider interface {
	ResponsesRetrieve(ctx *BifrostContext, key Key, req *BifrostResponsesRetrieveRequest) (*BifrostResponsesResponse, *BifrostError)
	ResponsesDelete(ctx *BifrostContext, key Key, req *BifrostResponsesDeleteRequest) (*BifrostResponsesDeleteResponse, *BifrostError)
	ResponsesCancel(ctx *BifrostContext, key Key, req *BifrostResponsesCancelRequest) (*BifrostResponsesResponse, *BifrostError)
	ResponsesInputItems(ctx *BifrostContext, key Key, req *BifrostResponsesInputItemsRequest) (*BifrostResponsesInputItemsResponse, *BifrostError)
}

ResponsesLifecycleProvider is an optional interface for OpenAI-style Responses API secondary verbs (retrieve, delete, cancel, list input items). Checked via type assertion in core dispatch; providers that do not implement it return unsupported_operation.

type ResponsesLocalShellToolCallAction added in v1.2.11 

type ResponsesLocalShellToolCallAction struct {
	Command          []string `json:"command"`
	Env              []string `json:"env"`
	Type             string   `json:"type"` // always "exec"
	TimeoutMS        *int     `json:"timeout_ms,omitempty"`
	User             *string  `json:"user,omitempty"`
	WorkingDirectory *string  `json:"working_directory,omitempty"`
}

ResponsesLocalShellCallAction represents the different types of local shell actions

type ResponsesMCPApprovalRequestAction added in v1.2.0 

type ResponsesMCPApprovalRequestAction struct {
	ID          string `json:"id"`
	Type        string `json:"type"` // always "mcp_approval_request"
	Name        string `json:"name"`
	ServerLabel string `json:"server_label"`
	Arguments   string `json:"arguments"`
}

ResponsesMCPApprovalRequestAction represents the different types of MCP approval request actions

type ResponsesMCPApprovalResponse added in v1.2.0 

type ResponsesMCPApprovalResponse struct {
	ApprovalResponseID string  `json:"approval_response_id"`
	Approve            bool    `json:"approve"`
	Reason             *string `json:"reason,omitempty"`
}

ResponsesMCPApprovalResponse represents a MCP approval response

type ResponsesMCPListTools added in v1.2.0 

type ResponsesMCPListTools struct {
	ServerLabel string             `json:"server_label"`
	Tools       []ResponsesMCPTool `json:"tools"`
}

ResponsesMCPListTools represents a list of MCP tools

type ResponsesMCPTool added in v1.2.0 

type ResponsesMCPTool struct {
	Name        string          `json:"name"`
	InputSchema map[string]any  `json:"input_schema"`
	Description *string         `json:"description,omitempty"`
	Annotations *map[string]any `json:"annotations,omitempty"`
}

ResponsesMCPTool represents an MCP tool

type ResponsesMCPToolCall added in v1.2.0 

type ResponsesMCPToolCall struct {
	ServerLabel string `json:"server_label"` // The label of the MCP server running the tool
}

ResponsesMCPToolCall represents a MCP tool call

type ResponsesMessage added in v1.2.0 

type ResponsesMessage struct {
	ID     *string               `json:"id,omitempty"` // Common ID field for most item types
	Type   *ResponsesMessageType `json:"type,omitempty"`
	Status *string               `json:"status,omitempty"` // "in_progress" | "completed" | "incomplete" | "interpreting" | "failed"
	// Phase labels an assistant message as intermediate "commentary" or completed "final_answer".
	// Required on gpt-5.3-codex+ history replay; dropping it causes significant performance degradation.
	// See https://developers.openai.com/api/docs/guides/prompt-guidance
	Phase *string `json:"phase,omitempty"`

	Role    *ResponsesMessageRoleType `json:"role,omitempty"`
	Content *ResponsesMessageContent  `json:"content,omitempty"`

	// Author and Recipient are required on multi-agent collab_tool_call items.
	// Preserved as raw JSON to survive bifrost round-trip without schema coupling.
	Author    json.RawMessage `json:"author,omitempty"`
	Recipient json.RawMessage `json:"recipient,omitempty"`

	*ResponsesToolMessage // For Tool calls and outputs

	CacheControl *CacheControl `json:"cache_control,omitempty"` // Carries cache_control for function_call and function_call_output message types

	// Reasoning
	// gpt-oss models include only reasoning_text content blocks in a message, while other openai models include summaries+encrypted_content
	*ResponsesReasoning
	// contains filtered or unexported fields
}

ResponsesMessage is a union type that can contain different types of input items Only one of the fields should be set at a time

func DeepCopyResponsesMessage added in v1.2.26 

func DeepCopyResponsesMessage(original ResponsesMessage) ResponsesMessage

DeepCopyResponsesMessage creates a deep copy of a ResponsesMessage to prevent shared data mutation between different plugin accumulators

func (ResponsesMessage) MarshalJSON added in v1.2.0 

func (m ResponsesMessage) MarshalJSON() ([]byte, error)

func (*ResponsesMessage) UnmarshalJSON added in v1.2.0 

func (m *ResponsesMessage) UnmarshalJSON(data []byte) error

UnmarshalJSON preserves codex tool_search/additional_tools items verbatim (see rawPreserved) and otherwise normalizes function/tool-call arguments before decoding the rest of the item. OpenAI's Responses API serializes `function_call` `arguments` as a JSON string, but `tool_search_call` items serialize `arguments` as a JSON object — e.g. {} while in_progress and {"query":"...","limit":10} when completed. The embedded ResponsesToolMessage.Arguments field is a *string, so an object value makes a plain decode fail with "Mismatch type string with value object", which silently drops the item mid-stream and hangs streaming clients. We shadow `arguments` as raw JSON, decode everything else as usual, then store the canonical stringified form.

type ResponsesMessageContent added in v1.2.0 

type ResponsesMessageContent struct {
	ContentStr *string // Simple text content

	// Output will ALWAYS be an array of content blocks
	ContentBlocks []ResponsesMessageContentBlock // Rich content with multiple media types
}

ResponsesMessageContent is a union type that can be either a string or array of content blocks

func (ResponsesMessageContent) MarshalJSON added in v1.2.0 

func (rc ResponsesMessageContent) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshalling for ResponsesMessageContent. It marshals either ContentStr or ContentBlocks directly without wrapping.

func (*ResponsesMessageContent) UnmarshalJSON added in v1.2.0 

func (rc *ResponsesMessageContent) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshalling for ResponsesMessageContent. It determines whether "content" is a string or array and assigns to the appropriate field. It also handles direct string/array content without a wrapper object.

type ResponsesMessageContentBlock added in v1.2.0 

type ResponsesMessageContentBlock struct {
	Type      ResponsesMessageContentBlockType `json:"type"`
	FileID    *string                          `json:"file_id,omitempty"` // Reference to uploaded file
	Text      *string                          `json:"text,omitempty"`
	Signature *string                          `json:"signature,omitempty"` // Signature of the content (for reasoning)
	// EncryptedContent is required on reasoning content blocks during history replay.
	// OpenAI returns it alongside summary_text blocks; it must be echoed back verbatim.
	EncryptedContent *string `json:"encrypted_content,omitempty"`

	*ResponsesInputMessageContentBlockImage
	*ResponsesInputMessageContentBlockFile
	Audio *ResponsesInputMessageContentBlockAudio `json:"input_audio,omitempty"`

	*ResponsesOutputMessageContentText            // Normal text output from the model
	*ResponsesOutputMessageContentRefusal         // Model refusal to answer
	*ResponsesOutputMessageContentRenderedContent // Rendered content from search entry point
	*ResponsesOutputMessageContentCompaction      // Compaction content from the model
	*ResponsesOutputMessageContentFallback        // Server-side fallback handoff boundary (from/to model)

	// Not in OpenAI's schemas, but sent by a few providers (Anthropic, Bedrock are some of them)
	CacheControl *CacheControl `json:"cache_control,omitempty"`
	Citations    *Citations    `json:"citations,omitempty"`

	// PromptCacheBreakpoint marks an explicit prompt-cache breakpoint on this block (OpenAI gpt-5.6+).
	PromptCacheBreakpoint *PromptCacheBreakpoint `json:"prompt_cache_breakpoint,omitempty"`
}

ResponsesMessageContentBlock represents different types of content (text, image, file, audio) Only one of the content type fields should be set

type ResponsesMessageContentBlockType added in v1.2.0 

type ResponsesMessageContentBlockType string
const (
	ResponsesInputMessageContentBlockTypeText      ResponsesMessageContentBlockType = "input_text"
	ResponsesInputMessageContentBlockTypeImage     ResponsesMessageContentBlockType = "input_image"
	ResponsesInputMessageContentBlockTypeFile      ResponsesMessageContentBlockType = "input_file"
	ResponsesInputMessageContentBlockTypeAudio     ResponsesMessageContentBlockType = "input_audio"
	ResponsesInputMessageContentBlockTypeContainer ResponsesMessageContentBlockType = "input_container" // Anthropic-only: file staged into the code-execution container input dir

	ResponsesOutputMessageContentTypeText      ResponsesMessageContentBlockType = "output_text"
	ResponsesOutputMessageContentTypeRefusal   ResponsesMessageContentBlockType = "refusal"
	ResponsesOutputMessageContentTypeReasoning ResponsesMessageContentBlockType = "reasoning_text"

	// gemini sends rendered content in google search results
	ResponsesOutputMessageContentTypeRenderedContent ResponsesMessageContentBlockType = "rendered_content"

	ResponsesOutputMessageContentTypeCompaction ResponsesMessageContentBlockType = "compaction"

	// ResponsesOutputMessageContentTypeFallback marks a server-side fallback handoff
	// boundary in the output (Anthropic server-side-fallback-2026-06-01).
	ResponsesOutputMessageContentTypeFallback ResponsesMessageContentBlockType = "fallback"
)

type ResponsesMessageRoleType added in v1.2.0 

type ResponsesMessageRoleType string
const (
	ResponsesInputMessageRoleAssistant ResponsesMessageRoleType = "assistant"
	ResponsesInputMessageRoleUser      ResponsesMessageRoleType = "user"
	ResponsesInputMessageRoleSystem    ResponsesMessageRoleType = "system"
	ResponsesInputMessageRoleDeveloper ResponsesMessageRoleType = "developer"
)

type ResponsesMessageType added in v1.2.0 

type ResponsesMessageType string
const (
	ResponsesMessageTypeMessage              ResponsesMessageType = "message"
	ResponsesMessageTypeFileSearchCall       ResponsesMessageType = "file_search_call"
	ResponsesMessageTypeComputerCall         ResponsesMessageType = "computer_call"
	ResponsesMessageTypeComputerCallOutput   ResponsesMessageType = "computer_call_output"
	ResponsesMessageTypeWebSearchCall        ResponsesMessageType = "web_search_call"
	ResponsesMessageTypeWebFetchCall         ResponsesMessageType = "web_fetch_call"
	ResponsesMessageTypeFunctionCall         ResponsesMessageType = "function_call"
	ResponsesMessageTypeFunctionCallOutput   ResponsesMessageType = "function_call_output"
	ResponsesMessageTypeCodeInterpreterCall  ResponsesMessageType = "code_interpreter_call"
	ResponsesMessageTypeLocalShellCall       ResponsesMessageType = "local_shell_call"
	ResponsesMessageTypeLocalShellCallOutput ResponsesMessageType = "local_shell_call_output"
	ResponsesMessageTypeMCPCall              ResponsesMessageType = "mcp_call"
	ResponsesMessageTypeCustomToolCall       ResponsesMessageType = "custom_tool_call"
	ResponsesMessageTypeCustomToolCallOutput ResponsesMessageType = "custom_tool_call_output"
	ResponsesMessageTypeImageGenerationCall  ResponsesMessageType = "image_generation_call"
	ResponsesMessageTypeMCPListTools         ResponsesMessageType = "mcp_list_tools"
	ResponsesMessageTypeMCPApprovalRequest   ResponsesMessageType = "mcp_approval_request"
	ResponsesMessageTypeMCPApprovalResponses ResponsesMessageType = "mcp_approval_responses"
	ResponsesMessageTypeReasoning            ResponsesMessageType = "reasoning"
	ResponsesMessageTypeItemReference        ResponsesMessageType = "item_reference"
	ResponsesMessageTypeRefusal              ResponsesMessageType = "refusal"
	ResponsesMessageTypeCompaction           ResponsesMessageType = "compaction"
	// Codex deferred-tool discovery (tool_search) and code-mode tool
	// declarations (additional_tools). OpenAI's Responses API supports these
	// item types natively; Bifrost preserves them verbatim because its typed
	// schema doesn't model them (tool_search_call's `arguments` is a JSON
	// object — unlike function_call's string — and tool_search_output /
	// additional_tools carry `tools` arrays whose entries don't fit any typed
	// tool shape). See ResponsesMessage's (Un)MarshalJSON.
	ResponsesMessageTypeToolSearchCall   ResponsesMessageType = "tool_search_call"
	ResponsesMessageTypeToolSearchOutput ResponsesMessageType = "tool_search_output"
	ResponsesMessageTypeAdditionalTools  ResponsesMessageType = "additional_tools"
	ResponsesMessageTypeAdvisorCall      ResponsesMessageType = "advisor_call" // Anthropic advisor server tool (server_tool_use + advisor_tool_result)
)

type ResponsesOutputMessageContentCompaction added in v1.4.1 

type ResponsesOutputMessageContentCompaction struct {
	Summary string `json:"summary,omitempty"` // The compaction summary text
}

type ResponsesOutputMessageContentFallback added in v1.7.3 

type ResponsesOutputMessageContentFallback struct {
	FromModel string `json:"from_model,omitempty"` // model that declined
	ToModel   string `json:"to_model,omitempty"`   // model that continues
	// TriggerType names why the handoff happened (e.g. "refusal"); TriggerCategory
	// is the policy area ("cyber", "bio", ...), absent when unnamed.
	TriggerType     string  `json:"trigger_type,omitempty"`
	TriggerCategory *string `json:"trigger_category,omitempty"`
}

ResponsesOutputMessageContentFallback carries the model boundary of a server-side fallback handoff (Anthropic's fallback content block: from/to model).

type ResponsesOutputMessageContentRefusal added in v1.2.0 

type ResponsesOutputMessageContentRefusal struct {
	Refusal string `json:"refusal"`
}

type ResponsesOutputMessageContentRenderedContent added in v1.3.12 

type ResponsesOutputMessageContentRenderedContent struct {
	RenderedContent string `json:"rendered_content"` // HTML/styled content from search entry point
}

type ResponsesOutputMessageContentText added in v1.2.0 

type ResponsesOutputMessageContentText struct {
	Annotations []ResponsesOutputMessageContentTextAnnotation `json:"annotations"` // Citations and references
	LogProbs    []ResponsesOutputMessageContentTextLogProb    `json:"logprobs"`    // Token log probabilities
}

type ResponsesOutputMessageContentTextAnnotation added in v1.2.0 

type ResponsesOutputMessageContentTextAnnotation struct {
	Type        string  `json:"type"`                  // "file_citation" | "url_citation" | "container_file_citation" | "file_path"
	Index       *int    `json:"index,omitempty"`       // Common index field (FileCitation, FilePath)
	FileID      *string `json:"file_id,omitempty"`     // Common file ID field (FileCitation, ContainerFileCitation, FilePath)
	Text        *string `json:"text,omitempty"`        // Text of the citation
	StartIndex  *int    `json:"start_index,omitempty"` // Common start index field (URLCitation, ContainerFileCitation)
	EndIndex    *int    `json:"end_index,omitempty"`   // Common end index field (URLCitation, ContainerFileCitation)
	Filename    *string `json:"filename,omitempty"`
	Title       *string `json:"title,omitempty"`
	URL         *string `json:"url,omitempty"`
	ContainerID *string `json:"container_id,omitempty"`

	// Anthropic specific fields
	StartCharIndex  *int    `json:"start_char_index,omitempty"`
	EndCharIndex    *int    `json:"end_char_index,omitempty"`
	StartPageNumber *int    `json:"start_page_number,omitempty"`
	EndPageNumber   *int    `json:"end_page_number,omitempty"`
	StartBlockIndex *int    `json:"start_block_index,omitempty"`
	EndBlockIndex   *int    `json:"end_block_index,omitempty"`
	Source          *string `json:"source,omitempty"`
	EncryptedIndex  *string `json:"encrypted_index,omitempty"`
}

type ResponsesOutputMessageContentTextLogProb added in v1.2.0 

type ResponsesOutputMessageContentTextLogProb struct {
	Bytes       []int     `json:"bytes"`
	LogProb     float64   `json:"logprob"`
	Token       string    `json:"token"`
	TopLogProbs []LogProb `json:"top_logprobs"`
}

ResponsesOutputMessageContentTextLogProb represents log probability information for content.

type ResponsesParameters added in v1.2.0 

type ResponsesParameters struct {
	Background           *bool                         `json:"background,omitempty"`
	Conversation         *string                       `json:"conversation,omitempty"`
	Include              []string                      `json:"include,omitempty"` // Supported values: "web_search_call.action.sources", "code_interpreter_call.outputs", "computer_call_output.output.image_url", "file_search_call.results", "message.input_image.image_url", "message.output_text.logprobs", "reasoning.encrypted_content"
	Instructions         *string                       `json:"instructions,omitempty"`
	MaxOutputTokens      *int                          `json:"max_output_tokens,omitempty"`
	MaxToolCalls         *int                          `json:"max_tool_calls,omitempty"`
	Metadata             *map[string]any               `json:"metadata,omitempty"`
	ParallelToolCalls    *bool                         `json:"parallel_tool_calls,omitempty"`
	PreviousResponseID   *string                       `json:"previous_response_id,omitempty"`
	PromptCacheKey       *string                       `json:"prompt_cache_key,omitempty"` // Prompt cache key
	PromptCacheRetention *string                       `json:"prompt_cache_retention,omitempty"`
	PromptCacheOptions   *PromptCacheOptions           `json:"prompt_cache_options,omitempty"` // Request-wide prompt cache options (OpenAI gpt-5.6+)
	Reasoning            *ResponsesParametersReasoning `json:"reasoning,omitempty"`            // Configuration options for reasoning models
	SafetyIdentifier     *string                       `json:"safety_identifier,omitempty"`    // Safety identifier
	ServiceTier          *BifrostServiceTier           `json:"service_tier,omitempty"`
	StreamOptions        *ResponsesStreamOptions       `json:"stream_options,omitempty"`
	Store                *bool                         `json:"store,omitempty"`
	Temperature          *float64                      `json:"temperature,omitempty"`
	Text                 *ResponsesTextConfig          `json:"text,omitempty"`
	TopLogProbs          *int                          `json:"top_logprobs,omitempty"`
	TopP                 *float64                      `json:"top_p,omitempty"`       // Controls diversity via nucleus sampling
	ToolChoice           *ResponsesToolChoice          `json:"tool_choice,omitempty"` // Whether to call a tool
	Tools                []ResponsesTool               `json:"tools,omitempty"`       // Tools to use
	Truncation           *string                       `json:"truncation,omitempty"`
	User                 *string                       `json:"user,omitempty"`
	// Dynamic parameters that can be provider-specific, they are directly
	// added to the request as is.
	ExtraParams map[string]interface{} `json:"-"`
}

type ResponsesParametersReasoning added in v1.2.0 

type ResponsesParametersReasoning struct {
	Context         *string `json:"context,omitempty"`          // "auto" | "current_turn" | "all_turns" (which reasoning items are rendered back to the model on later turns)
	Effort          *string `json:"effort"`                     // "none" | "minimal" | "low" | "medium" | "high" | "xhigh" | "max" (any value other than "none" will enable reasoning)
	GenerateSummary *string `json:"generate_summary,omitempty"` // Deprecated: use summary instead
	Mode            *string `json:"mode,omitempty"`             // "standard" | "pro" (reasoning execution mode)
	Summary         *string `json:"summary"`                    // "auto" | "concise" | "detailed"
	MaxTokens       *int    `json:"max_tokens,omitempty"`       // Maximum number of tokens to generate for the reasoning output (required for anthropic)
}

type ResponsesPrompt added in v1.2.0 

type ResponsesPrompt struct {
	ID        string         `json:"id"`
	Variables map[string]any `json:"variables"`
	Version   *string        `json:"version,omitempty"`
}

type ResponsesReasoning added in v1.2.0 

type ResponsesReasoning struct {
	Summary          []ResponsesReasoningSummary `json:"summary"`
	EncryptedContent *string                     `json:"encrypted_content,omitempty"`
}

ResponsesReasoning represents a reasoning output

type ResponsesReasoningContentBlockType added in v1.2.0 

type ResponsesReasoningContentBlockType string

ResponsesReasoningContentBlockType represents the type of reasoning content 

const (
	ResponsesReasoningContentBlockTypeSummaryText ResponsesReasoningContentBlockType = "summary_text"
)

ResponsesReasoningContentBlockType values

type ResponsesReasoningSummary added in v1.2.37 

type ResponsesReasoningSummary struct {
	Type ResponsesReasoningContentBlockType `json:"type"`
	Text string                             `json:"text"`
}

ResponsesReasoningSummary represents a reasoning content block

type ResponsesResponseContainer added in v1.6.1 

type ResponsesResponseContainer struct {
	ID        string  `json:"id"`
	ExpiresAt *string `json:"expires_at,omitempty"`
}

ResponsesResponseContainer is the code-execution sandbox container returned on a response that used the code execution tool. The id can be passed back to reuse the sandbox across turns.

type ResponsesResponseConversation added in v1.2.0 

type ResponsesResponseConversation struct {
	ResponsesResponseConversationStr    *string
	ResponsesResponseConversationStruct *ResponsesResponseConversationStruct
}

func (ResponsesResponseConversation) MarshalJSON added in v1.2.1 

func (rc ResponsesResponseConversation) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshalling for ResponsesMessageContent. It marshals either ContentStr or ContentBlocks directly without wrapping.

func (*ResponsesResponseConversation) UnmarshalJSON added in v1.2.1 

func (rc *ResponsesResponseConversation) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshalling for ResponsesMessageContent. It determines whether "content" is a string or array and assigns to the appropriate field. It also handles direct string/array content without a wrapper object.

type ResponsesResponseConversationStruct added in v1.2.1 

type ResponsesResponseConversationStruct struct {
	ID string `json:"id"` // The unique ID of the conversation
}

type ResponsesResponseError added in v1.2.0 

type ResponsesResponseError struct {
	Code    string `json:"code"`    // The error code for the response
	Message string `json:"message"` // A human-readable description of the error
}

type ResponsesResponseIncompleteDetails added in v1.2.0 

type ResponsesResponseIncompleteDetails struct {
	Reason string `json:"reason"` // The reason why the response is incomplete
}

type ResponsesResponseInputTokens added in v1.2.0 

type ResponsesResponseInputTokens struct {
	TextTokens  int `json:"text_tokens,omitempty"`  // Tokens for text input
	AudioTokens int `json:"audio_tokens,omitempty"` // Tokens for audio input
	ImageTokens int `json:"image_tokens,omitempty"` // Tokens for image input

	// For Providers which don't separate between cache creation and cache read tokens (like Openai, Gemini, etc), this is the total number of cached tokens read.
	CachedReadTokens        int                          `json:"cached_read_tokens"`
	CachedWriteTokens       int                          `json:"cached_write_tokens"`
	CachedWriteTokenDetails *ChatCachedWriteTokenDetails `json:"cached_write_token_details,omitempty"`
}

func (ResponsesResponseInputTokens) MarshalJSON added in v1.4.5 

func (d ResponsesResponseInputTokens) MarshalJSON() ([]byte, error)

MarshalJSON emits cached_tokens (reads only, per the OpenAI spec and mirroring UnmarshalJSON above) alongside the individual fields. Cache writes are reported separately via cached_write_tokens and are excluded from cached_tokens so that OpenAI-spec consumers do not price cache writes as cache reads.

func (*ResponsesResponseInputTokens) UnmarshalJSON added in v1.4.5 

func (d *ResponsesResponseInputTokens) UnmarshalJSON(data []byte) error

UnmarshalJSON maps OpenAI's cached_tokens into CachedReadTokens for compatibility.

type ResponsesResponseInstructions added in v1.2.1 

type ResponsesResponseInstructions struct {
	ResponsesResponseInstructionsStr   *string
	ResponsesResponseInstructionsArray []ResponsesMessage
}

func (ResponsesResponseInstructions) MarshalJSON added in v1.2.1 

func (rc ResponsesResponseInstructions) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshalling for ResponsesMessageContent. It marshals either ContentStr or ContentBlocks directly without wrapping.

func (*ResponsesResponseInstructions) UnmarshalJSON added in v1.2.1 

func (rc *ResponsesResponseInstructions) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshalling for ResponsesMessageContent. It determines whether "content" is a string or array and assigns to the appropriate field. It also handles direct string/array content without a wrapper object.

type ResponsesResponseOutputTokens added in v1.2.0 

type ResponsesResponseOutputTokens struct {
	TextTokens               int  `json:"text_tokens,omitempty"`
	AcceptedPredictionTokens int  `json:"accepted_prediction_tokens,omitempty"`
	AudioTokens              int  `json:"audio_tokens,omitempty"`
	ImageTokens              *int `json:"image_tokens,omitempty"`
	ReasoningTokens          int  `json:"reasoning_tokens"` // Required for few OpenAI models
	RejectedPredictionTokens int  `json:"rejected_prediction_tokens,omitempty"`
	CitationTokens           *int `json:"citation_tokens,omitempty"`
	NumSearchQueries         *int `json:"num_search_queries,omitempty"`
}

type ResponsesResponseUsage added in v1.2.0 

type ResponsesResponseUsage struct {
	Type                *string                        `json:"type,omitempty"`        // type field is sent by anthropic
	Model               *string                        `json:"model,omitempty"`       // model that produced this (iteration) attempt; sent on iterations[] for Anthropic server-side fallback
	InputTokens         int                            `json:"input_tokens"`          // Number of input tokens (prompt tokens + cached tokens)
	InputTokensDetails  *ResponsesResponseInputTokens  `json:"input_tokens_details"`  // Detailed breakdown of input tokens
	OutputTokens        int                            `json:"output_tokens"`         // Number of output tokens (completion tokens + reasoning tokens)
	OutputTokensDetails *ResponsesResponseOutputTokens `json:"output_tokens_details"` // Detailed breakdown of output tokens	TotalTokens int `json:"total_tokens"` // Total number of tokens used
	TotalTokens         int                            `json:"total_tokens"`          // Total number of tokens used
	Cost                *BifrostCost                   `json:"cost,omitempty"`        // Only for the providers which support cost calculation
	Iterations          []ResponsesResponseUsage       `json:"iterations,omitempty"`  // iterations field is sent by anthropic

	// xAI-specific usage fields
	NumSourcesUsed             *int                                 `json:"num_sources_used,omitempty"`
	NumServerSideToolsUsed     *int                                 `json:"num_server_side_tools_used,omitempty"`
	CostInUsdTicks             *int64                               `json:"cost_in_usd_ticks,omitempty"`
	ServerSideToolUsageDetails *ResponsesServerSideToolUsageDetails `json:"server_side_tool_usage_details,omitempty"`
	ContextDetails             *ResponsesContextDetails             `json:"context_details,omitempty"`
}

func (*ResponsesResponseUsage) ToBifrostLLMUsage added in v1.2.13 

func (ru *ResponsesResponseUsage) ToBifrostLLMUsage() *BifrostLLMUsage

type ResponsesServerSideToolUsageDetails added in v1.5.17 

type ResponsesServerSideToolUsageDetails struct {
	WebSearchCalls       int `json:"web_search_calls"`
	XSearchCalls         int `json:"x_search_calls"`
	CodeInterpreterCalls int `json:"code_interpreter_calls"`
	FileSearchCalls      int `json:"file_search_calls"`
	MCPCalls             int `json:"mcp_calls"`
	DocumentSearchCalls  int `json:"document_search_calls"`
}

ResponsesServerSideToolUsageDetails holds per-tool call counts returned by xAI.

type ResponsesStopDetails added in v1.7.3 

type ResponsesStopDetails struct {
	Type             string  `json:"type"`
	Category         *string `json:"category,omitempty"`
	Explanation      *string `json:"explanation,omitempty"`
	RecommendedModel *string `json:"recommended_model,omitempty"`
	// FallbackCreditToken is the one-time credit redeemable on a manual retry to
	// avoid re-paying cache-write rates; null when no credit was minted.
	FallbackCreditToken *string `json:"fallback_credit_token,omitempty"`
	// FallbackHasPrefillClaim selects the retry body shape; absent means "unknown",
	// which callers must not collapse to false.
	FallbackHasPrefillClaim *bool `json:"fallback_has_prefill_claim,omitempty"`
}

ResponsesStopDetails carries Anthropic's stop_details for a "refusal" stop_reason. Category and Explanation are null when the refusal maps to no named category; RecommendedModel names a model to retry directly when a fallback attempt was skipped.

type ResponsesStreamOptions added in v1.2.0 

type ResponsesStreamOptions struct {
	IncludeObfuscation *bool `json:"include_obfuscation,omitempty"`
}

type ResponsesStreamResponseType added in v1.2.6 

type ResponsesStreamResponseType string
const (
	// Ping events are just keepalive (sent by very few providers, Anthropic is one of them)
	ResponsesStreamResponseTypePing ResponsesStreamResponseType = "response.ping"

	ResponsesStreamResponseTypeCreated    ResponsesStreamResponseType = "response.created"
	ResponsesStreamResponseTypeInProgress ResponsesStreamResponseType = "response.in_progress"
	ResponsesStreamResponseTypeCompleted  ResponsesStreamResponseType = "response.completed"
	ResponsesStreamResponseTypeFailed     ResponsesStreamResponseType = "response.failed"
	ResponsesStreamResponseTypeIncomplete ResponsesStreamResponseType = "response.incomplete"

	ResponsesStreamResponseTypeOutputItemAdded ResponsesStreamResponseType = "response.output_item.added"
	ResponsesStreamResponseTypeOutputItemDone  ResponsesStreamResponseType = "response.output_item.done"

	ResponsesStreamResponseTypeContentPartAdded ResponsesStreamResponseType = "response.content_part.added"
	ResponsesStreamResponseTypeContentPartDone  ResponsesStreamResponseType = "response.content_part.done"

	ResponsesStreamResponseTypeOutputTextDelta ResponsesStreamResponseType = "response.output_text.delta"
	ResponsesStreamResponseTypeOutputTextDone  ResponsesStreamResponseType = "response.output_text.done"

	ResponsesStreamResponseTypeRefusalDelta ResponsesStreamResponseType = "response.refusal.delta"
	ResponsesStreamResponseTypeRefusalDone  ResponsesStreamResponseType = "response.refusal.done"

	ResponsesStreamResponseTypeFunctionCallArgumentsDelta     ResponsesStreamResponseType = "response.function_call_arguments.delta"
	ResponsesStreamResponseTypeFunctionCallArgumentsDone      ResponsesStreamResponseType = "response.function_call_arguments.done"
	ResponsesStreamResponseTypeFileSearchCallInProgress       ResponsesStreamResponseType = "response.file_search_call.in_progress"
	ResponsesStreamResponseTypeFileSearchCallSearching        ResponsesStreamResponseType = "response.file_search_call.searching"
	ResponsesStreamResponseTypeFileSearchCallResultsAdded     ResponsesStreamResponseType = "response.file_search_call.results.added"
	ResponsesStreamResponseTypeFileSearchCallResultsCompleted ResponsesStreamResponseType = "response.file_search_call.results.completed"
	ResponsesStreamResponseTypeWebSearchCallInProgress        ResponsesStreamResponseType = "response.web_search_call.in_progress"
	ResponsesStreamResponseTypeWebSearchCallSearching         ResponsesStreamResponseType = "response.web_search_call.searching"
	ResponsesStreamResponseTypeWebSearchCallCompleted         ResponsesStreamResponseType = "response.web_search_call.completed"
	ResponsesStreamResponseTypeWebSearchCallResultsAdded      ResponsesStreamResponseType = "response.web_search_call.results.added"
	ResponsesStreamResponseTypeWebSearchCallResultsCompleted  ResponsesStreamResponseType = "response.web_search_call.results.completed"

	ResponsesStreamResponseTypeWebFetchCallInProgress ResponsesStreamResponseType = "response.web_fetch_call.in_progress"
	ResponsesStreamResponseTypeWebFetchCallFetching   ResponsesStreamResponseType = "response.web_fetch_call.fetching"
	ResponsesStreamResponseTypeWebFetchCallCompleted  ResponsesStreamResponseType = "response.web_fetch_call.completed"

	ResponsesStreamResponseTypeReasoningSummaryPartAdded ResponsesStreamResponseType = "response.reasoning_summary_part.added"
	ResponsesStreamResponseTypeReasoningSummaryPartDone  ResponsesStreamResponseType = "response.reasoning_summary_part.done"
	ResponsesStreamResponseTypeReasoningSummaryTextDelta ResponsesStreamResponseType = "response.reasoning_summary_text.delta"
	ResponsesStreamResponseTypeReasoningSummaryTextDone  ResponsesStreamResponseType = "response.reasoning_summary_text.done"

	ResponsesStreamResponseTypeImageGenerationCallCompleted    ResponsesStreamResponseType = "response.image_generation_call.completed"
	ResponsesStreamResponseTypeImageGenerationCallGenerating   ResponsesStreamResponseType = "response.image_generation_call.generating"
	ResponsesStreamResponseTypeImageGenerationCallInProgress   ResponsesStreamResponseType = "response.image_generation_call.in_progress"
	ResponsesStreamResponseTypeImageGenerationCallPartialImage ResponsesStreamResponseType = "response.image_generation_call.partial_image"

	ResponsesStreamResponseTypeMCPCallArgumentsDelta  ResponsesStreamResponseType = "response.mcp_call_arguments.delta"
	ResponsesStreamResponseTypeMCPCallArgumentsDone   ResponsesStreamResponseType = "response.mcp_call_arguments.done"
	ResponsesStreamResponseTypeMCPCallCompleted       ResponsesStreamResponseType = "response.mcp_call.completed"
	ResponsesStreamResponseTypeMCPCallFailed          ResponsesStreamResponseType = "response.mcp_call.failed"
	ResponsesStreamResponseTypeMCPCallInProgress      ResponsesStreamResponseType = "response.mcp_call.in_progress"
	ResponsesStreamResponseTypeMCPListToolsCompleted  ResponsesStreamResponseType = "response.mcp_list_tools.completed"
	ResponsesStreamResponseTypeMCPListToolsFailed     ResponsesStreamResponseType = "response.mcp_list_tools.failed"
	ResponsesStreamResponseTypeMCPListToolsInProgress ResponsesStreamResponseType = "response.mcp_list_tools.in_progress"

	ResponsesStreamResponseTypeCodeInterpreterCallInProgress   ResponsesStreamResponseType = "response.code_interpreter_call.in_progress"
	ResponsesStreamResponseTypeCodeInterpreterCallInterpreting ResponsesStreamResponseType = "response.code_interpreter_call.interpreting"
	ResponsesStreamResponseTypeCodeInterpreterCallCompleted    ResponsesStreamResponseType = "response.code_interpreter_call.completed"
	ResponsesStreamResponseTypeCodeInterpreterCallCodeDelta    ResponsesStreamResponseType = "response.code_interpreter_call_code.delta"
	ResponsesStreamResponseTypeCodeInterpreterCallCodeDone     ResponsesStreamResponseType = "response.code_interpreter_call_code.done"

	ResponsesStreamResponseTypeOutputTextAnnotationAdded ResponsesStreamResponseType = "response.output_text.annotation.added"
	ResponsesStreamResponseTypeOutputTextAnnotationDone  ResponsesStreamResponseType = "response.output_text.annotation.done"

	ResponsesStreamResponseTypeQueued ResponsesStreamResponseType = "response.queued"

	ResponsesStreamResponseTypeCustomToolCallInputDelta ResponsesStreamResponseType = "response.custom_tool_call_input.delta"
	ResponsesStreamResponseTypeCustomToolCallInputDone  ResponsesStreamResponseType = "response.custom_tool_call_input.done"

	ResponsesStreamResponseTypeError ResponsesStreamResponseType = "error"
)

type ResponsesTextConfig added in v1.2.0 

type ResponsesTextConfig struct {
	Format    *ResponsesTextConfigFormat `json:"format,omitempty"`    // An object specifying the format that the model must output
	Verbosity *string                    `json:"verbosity,omitempty"` // "low" | "medium" | "high" or null
}

type ResponsesTextConfigFormat added in v1.2.0 

type ResponsesTextConfigFormat struct {
	Type        string                               `json:"type"`                  // "text" | "json_schema" | "json_object"
	Name        *string                              `json:"name,omitempty"`        // Name of the format
	Description *string                              `json:"description,omitempty"` // Description of the schema
	JSONSchema  *ResponsesTextConfigFormatJSONSchema `json:"schema,omitempty"`      // when type == "json_schema"
	Strict      *bool                                `json:"strict,omitempty"`
}

type ResponsesTextConfigFormatJSONSchema added in v1.2.0 

type ResponsesTextConfigFormatJSONSchema struct {
	Name                 *string                     `json:"name,omitempty"`
	Schema               *JSONSchemaOrBool           `json:"schema,omitempty"`
	Description          *string                     `json:"description,omitempty"`
	Strict               *bool                       `json:"strict,omitempty"`
	AdditionalProperties *AdditionalPropertiesStruct `json:"additionalProperties,omitempty"`
	Properties           *OrderedMap                 `json:"properties,omitempty"`
	Required             []string                    `json:"required,omitempty"`
	Type                 *string                     `json:"type,omitempty"`

	// JSON Schema definition fields
	Defs        *OrderedMap `json:"$defs,omitempty"`       // JSON Schema draft 2019-09+ definitions
	Definitions *OrderedMap `json:"definitions,omitempty"` // Legacy JSON Schema draft-07 definitions
	Ref         *string     `json:"$ref,omitempty"`        // Reference to definition

	// Array schema fields
	Items    *OrderedMap `json:"items,omitempty"`    // Array element schema
	MinItems *int64      `json:"minItems,omitempty"` // Minimum array length
	MaxItems *int64      `json:"maxItems,omitempty"` // Maximum array length

	// Composition fields (union types)
	AnyOf []OrderedMap `json:"anyOf,omitempty"` // Union types (any of these schemas)
	OneOf []OrderedMap `json:"oneOf,omitempty"` // Exclusive union types (exactly one of these)
	AllOf []OrderedMap `json:"allOf,omitempty"` // Schema intersection (all of these)

	// String validation fields
	Format    *string `json:"format,omitempty"`    // String format (email, date, uri, etc.)
	Pattern   *string `json:"pattern,omitempty"`   // Regex pattern for strings
	MinLength *int64  `json:"minLength,omitempty"` // Minimum string length
	MaxLength *int64  `json:"maxLength,omitempty"` // Maximum string length

	// Number validation fields
	Minimum *float64 `json:"minimum,omitempty"` // Minimum number value
	Maximum *float64 `json:"maximum,omitempty"` // Maximum number value

	// Misc fields
	Title            *string     `json:"title,omitempty"`            // Schema title
	Default          interface{} `json:"default,omitempty"`          // Default value
	Nullable         *bool       `json:"nullable,omitempty"`         // Nullable indicator (OpenAPI 3.0 style)
	Enum             []string    `json:"enum,omitempty"`             // Enum values
	PropertyOrdering []string    `json:"propertyOrdering,omitempty"` // Ordering of properties, specific to Gemini
}

ResponsesTextConfigFormatJSONSchema represents a JSON schema specification It supports JSON Schema fields used by various providers for structured outputs. Schema-bearing fields use OrderedMap (mirroring ToolFunctionParameters) because structured-output generation is sensitive to JSON schema property order: providers like OpenAI follow the literal key order of the schema, so decoding into plain Go maps (and re-marshaling them sorted) degrades output quality.

func JSONSchemaFromMap added in v1.5.11 

func JSONSchemaFromMap(v interface{}) *ResponsesTextConfigFormatJSONSchema

JSONSchemaFromMap builds a ResponsesTextConfigFormatJSONSchema from a raw interface{}

func (*ResponsesTextConfigFormatJSONSchema) CompositeSchema added in v1.7.3 

func (s *ResponsesTextConfigFormatJSONSchema) CompositeSchema() (*OrderedMap, bool, error)

CompositeSchema resolves the composite Schema field (the wrapped `format.schema.schema` position). Returns (schemaMap, acceptAll, err):

  • schemaMap non-nil: an object schema was provided; it takes precedence over the decomposed typed fields
  • acceptAll true: the boolean schema `true` (accept any value); providers that must re-encode the schema should emit their widest representable form
  • err non-nil: the boolean schema `false` (ErrUnsatisfiableSchema)

The zero return (nil, false, nil) means no composite schema is set; callers should build from the decomposed typed fields (Type, Properties, ...).

func (*ResponsesTextConfigFormatJSONSchema) ToMap added in v1.5.11 

func (s *ResponsesTextConfigFormatJSONSchema) ToMap() interface{}

ToMap reconstructs the raw schema map from a ResponsesTextConfigFormatJSONSchema.

type ResponsesTool added in v1.2.0 

type ResponsesTool struct {
	Type        ResponsesToolType `json:"type"`                  // "function" | "file_search" | "computer_use_preview" | "web_search" | "web_search_2025_08_26" | "mcp" | "code_interpreter" | "image_generation" | "local_shell" | "custom" | "web_search_preview" | "web_search_preview_2025_03_11" | "x_search"
	Name        *string           `json:"name,omitempty"`        // Common name field (Function, Custom tools)
	Description *string           `json:"description,omitempty"` // Common description field (Function, Custom tools)

	// Not in OpenAI's schemas, but sent by a few providers (Anthropic, Bedrock are some of them)
	CacheControl *CacheControl `json:"cache_control,omitempty"`

	// Anthropic-native tool flags promoted to the neutral layer. All optional;
	// ignored by providers that don't support them. Gated per ProviderFeatures
	// in core/providers/anthropic/types.go.
	DeferLoading        *bool                  `json:"defer_loading,omitempty"`         // Anthropic advanced-tool-use: defer loading of tool definition
	AllowedCallers      []string               `json:"allowed_callers,omitempty"`       // Anthropic advanced-tool-use: which callers can invoke this tool
	InputExamples       []ChatToolInputExample `json:"input_examples,omitempty"`        // Anthropic tool-examples-2025-10-29: example inputs for the tool
	EagerInputStreaming *bool                  `json:"eager_input_streaming,omitempty"` // Anthropic fine-grained-tool-streaming-2025-05-14

	*ResponsesToolFunction
	*ResponsesToolFileSearch
	*ResponsesToolComputerUsePreview
	*ResponsesToolWebSearch
	*ResponsesToolWebFetch
	*ResponsesToolMCP
	*ResponsesToolCodeInterpreter
	*ResponsesToolImageGeneration
	*ResponsesToolLocalShell
	*ResponsesToolCustom
	*ResponsesToolWebSearchPreview
	*ResponsesToolToolSearch
	*ResponsesToolNamespace
	*ResponsesToolXSearch
	*ResponsesToolAdvisor
}

ResponsesTool represents a tool

func (ResponsesTool) MarshalJSON added in v1.3.9 

func (t ResponsesTool) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshaling for ResponsesTool. It merges common fields with the appropriate embedded struct based on type. Uses sjson to build JSON bytes incrementally, ensuring deterministic key ordering critical for prompt caching (OpenAI caches based on request prefix).

func (*ResponsesTool) ToChatTool added in v1.2.0 

func (rt *ResponsesTool) ToChatTool() *ChatTool

ToChatTool converts a ResponsesTool to ChatTool format

func (*ResponsesTool) UnmarshalJSON added in v1.3.9 

func (t *ResponsesTool) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshaling for ResponsesTool It unmarshals common fields first, then the appropriate embedded struct based on type

type ResponsesToolAdvisor added in v1.5.20 

type ResponsesToolAdvisor struct {
	Model     string                       `json:"model,omitempty"`      // advisor model id (required by Anthropic)
	MaxUses   *int                         `json:"max_uses,omitempty"`   // per-request cap on advisor calls
	MaxTokens *int                         `json:"max_tokens,omitempty"` // caps advisor output per call; minimum 1024
	Caching   *ResponsesToolAdvisorCaching `json:"caching,omitempty"`    // advisor-side prompt caching toggle
}

ResponsesToolAdvisor carries the Anthropic advisor_20260301 server-tool config. Anthropic-only; ignored by providers that don't support it.

type ResponsesToolAdvisorCaching added in v1.5.20 

type ResponsesToolAdvisorCaching struct {
	Type string `json:"type"`          // "ephemeral"
	TTL  string `json:"ttl,omitempty"` // "5m" | "1h"
}

ResponsesToolAdvisorCaching toggles advisor-side prompt caching.

type ResponsesToolCaller added in v1.6.1 

type ResponsesToolCaller struct {
	Type   string  `json:"type"`              // "direct" | "code_execution_20250825" | "code_execution_20260120"
	ToolID *string `json:"tool_id,omitempty"` // required for code_execution_* caller types
}

ResponsesToolCaller is the neutral form of Anthropic's "caller" union on server_tool_use / *_tool_result blocks. It links a tool call to the agentic caller that produced it (e.g. programmatic tool calling from inside the code execution sandbox). Nil for direct top-level calls.

type ResponsesToolChoice added in v1.2.0 

type ResponsesToolChoice struct {
	ResponsesToolChoiceStr    *string
	ResponsesToolChoiceStruct *ResponsesToolChoiceStruct
}

ResponsesToolChoice represents a tool choice

func (ResponsesToolChoice) MarshalJSON added in v1.2.0 

func (tc ResponsesToolChoice) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshalling for ChatMessageContent. It marshals either ContentStr or ContentBlocks directly without wrapping.

func (*ResponsesToolChoice) ToChatToolChoice added in v1.2.0 

func (tc *ResponsesToolChoice) ToChatToolChoice() *ChatToolChoice

ToChatToolChoice converts a ResponsesToolChoice to ChatToolChoice format

func (*ResponsesToolChoice) UnmarshalJSON added in v1.2.0 

func (tc *ResponsesToolChoice) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshalling for ChatMessageContent. It determines whether "content" is a string or array and assigns to the appropriate field. It also handles direct string/array content without a wrapper object.

type ResponsesToolChoiceAllowedToolDef added in v1.2.0 

type ResponsesToolChoiceAllowedToolDef struct {
	Type        string  `json:"type"`                   // "function" | "mcp" | "image_generation"
	Name        *string `json:"name,omitempty"`         // for function tools
	ServerLabel *string `json:"server_label,omitempty"` // for MCP tools
}

ResponsesToolChoiceAllowedToolDef represents a tool choice allowed tool definition

type ResponsesToolChoiceStruct added in v1.2.0 

type ResponsesToolChoiceStruct struct {
	Type        ResponsesToolChoiceType             `json:"type"`                   // Type of tool choice
	Mode        *string                             `json:"mode,omitempty"`         //"none" | "auto" | "required"
	Name        *string                             `json:"name,omitempty"`         // Common name field for function/MCP/custom tools
	ServerLabel *string                             `json:"server_label,omitempty"` // Common server label field for MCP tools
	Tools       []ResponsesToolChoiceAllowedToolDef `json:"tools,omitempty"`
}

ResponsesToolChoiceStruct represents a tool choice struct

type ResponsesToolChoiceType added in v1.2.0 

type ResponsesToolChoiceType string

ResponsesToolChoiceType represents the type of tool choice 

const (
	// ResponsesToolChoiceTypeNone means no tool should be called
	ResponsesToolChoiceTypeNone ResponsesToolChoiceType = "none"
	// ResponsesToolChoiceTypeAuto means an automatic tool should be called
	ResponsesToolChoiceTypeAuto ResponsesToolChoiceType = "auto"
	// ResponsesToolChoiceTypeAny means any tool can be called
	ResponsesToolChoiceTypeAny ResponsesToolChoiceType = "any"
	// ResponsesToolChoiceTypeRequired means a specific tool must be called
	ResponsesToolChoiceTypeRequired ResponsesToolChoiceType = "required"
	// ResponsesToolChoiceTypeFunction means a specific tool must be called
	ResponsesToolChoiceTypeFunction ResponsesToolChoiceType = "function"
	// ResponsesToolChoiceTypeAllowedTools means a specific tool must be called
	ResponsesToolChoiceTypeAllowedTools ResponsesToolChoiceType = "allowed_tools"
	// ResponsesToolChoiceTypeFileSearch means a file search tool must be called
	ResponsesToolChoiceTypeFileSearch ResponsesToolChoiceType = "file_search"
	// ResponsesToolChoiceTypeWebSearchPreview means a web search preview tool must be called
	ResponsesToolChoiceTypeWebSearchPreview ResponsesToolChoiceType = "web_search_preview"
	// ResponsesToolChoiceTypeComputerUsePreview means a computer use preview tool must be called
	ResponsesToolChoiceTypeComputerUsePreview ResponsesToolChoiceType = "computer_use_preview"
	// ResponsesToolChoiceTypeCodeInterpreter means a code interpreter tool must be called
	ResponsesToolChoiceTypeCodeInterpreter ResponsesToolChoiceType = "code_interpreter"
	// ResponsesToolChoiceTypeImageGeneration means an image generation tool must be called
	ResponsesToolChoiceTypeImageGeneration ResponsesToolChoiceType = "image_generation"
	// ResponsesToolChoiceTypeMCP means an MCP tool must be called
	ResponsesToolChoiceTypeMCP ResponsesToolChoiceType = "mcp"
	// ResponsesToolChoiceTypeCustom means a custom tool must be called
	ResponsesToolChoiceTypeCustom ResponsesToolChoiceType = "custom"
)

ResponsesToolChoiceType values

type ResponsesToolCodeInterpreter added in v1.2.0 

type ResponsesToolCodeInterpreter struct {
	Container interface{} `json:"container"` // Container ID or object with file IDs
	// Anthropic code_execution tool version (code_execution_20250825 |
	// _20260120 | _20260521 | legacy _20250522). Preserved verbatim so the
	// requested capability tier round-trips; ignored by other providers and
	// stripped before any OpenAI-compatible request (see openai/types.go).
	Version *string `json:"code_execution_version,omitempty"`
}

ResponsesToolCodeInterpreter represents a tool code interpreter

type ResponsesToolComputerUsePreview added in v1.2.0 

type ResponsesToolComputerUsePreview struct {
	DisplayHeight int    `json:"display_height"` // The height of the computer display
	DisplayWidth  int    `json:"display_width"`  // The width of the computer display
	Environment   string `json:"environment"`    // The type of computer environment to control

	EnableZoom *bool `json:"enable_zoom,omitempty"` // for computer tool in anthropic only
}

ResponsesToolComputerUsePreview represents a tool computer use preview

type ResponsesToolCustom added in v1.2.0 

type ResponsesToolCustom struct {
	Format *ResponsesToolCustomFormat `json:"format,omitempty"` // The input format
}

ResponsesToolCustom represents a custom tool

type ResponsesToolCustomFormat added in v1.2.0 

type ResponsesToolCustomFormat struct {
	Type string `json:"type"` // always "text"

	// For Grammar
	Definition *string `json:"definition,omitempty"` // The grammar definition
	Syntax     *string `json:"syntax,omitempty"`     // "lark" | "regex"
}

ResponsesToolCustomFormat represents the input format for the custom tool

type ResponsesToolFileSearch added in v1.2.0 

type ResponsesToolFileSearch struct {
	VectorStoreIDs []string                               `json:"vector_store_ids"`          // The IDs of the vector stores to search
	Filters        *ResponsesToolFileSearchFilter         `json:"filters,omitempty"`         // A filter to apply
	MaxNumResults  *int                                   `json:"max_num_results,omitempty"` // Maximum results (1-50)
	RankingOptions *ResponsesToolFileSearchRankingOptions `json:"ranking_options,omitempty"` // Ranking options for search
}

ResponsesToolFileSearch represents a tool file search

type ResponsesToolFileSearchComparisonFilter added in v1.2.0 

type ResponsesToolFileSearchComparisonFilter struct {
	Key   string      `json:"key"`   // The key to compare against the value
	Type  string      `json:"type"`  //
	Value interface{} `json:"value"` // The value to compare (string, number, or boolean)
}

ResponsesToolFileSearchComparisonFilter represents a file search comparison filter

type ResponsesToolFileSearchCompoundFilter added in v1.2.0 

type ResponsesToolFileSearchCompoundFilter struct {
	Filters []ResponsesToolFileSearchFilter `json:"filters"` // Array of filters to combine
}

ResponsesToolFileSearchCompoundFilter represents a file search compound filter

type ResponsesToolFileSearchFilter added in v1.2.0 

type ResponsesToolFileSearchFilter struct {
	Type string `json:"type"` // "eq" | "ne" | "gt" | "gte" | "lt" | "lte" | "and" | "or"

	// Filter types - only one should be set
	*ResponsesToolFileSearchComparisonFilter
	*ResponsesToolFileSearchCompoundFilter
}

ResponsesToolFileSearchFilter represents a file search filter

func (*ResponsesToolFileSearchFilter) MarshalJSON added in v1.2.0 

func (f *ResponsesToolFileSearchFilter) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshaling for ResponsesToolFileSearchFilter

func (*ResponsesToolFileSearchFilter) UnmarshalJSON added in v1.2.0 

func (f *ResponsesToolFileSearchFilter) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshaling for ResponsesToolFileSearchFilter

type ResponsesToolFileSearchRankingOptions added in v1.2.0 

type ResponsesToolFileSearchRankingOptions struct {
	Ranker         *string  `json:"ranker,omitempty"`          // The ranker to use
	ScoreThreshold *float64 `json:"score_threshold,omitempty"` // Score threshold (0-1)
}

ResponsesToolFileSearchRankingOptions represents a file search ranking options

type ResponsesToolFunction added in v1.2.0 

type ResponsesToolFunction struct {
	Parameters *ToolFunctionParameters `json:"parameters,omitempty"` // A JSON schema object describing the parameters
	Strict     *bool                   `json:"strict"`               // Whether to enforce strict parameter validation
}

ResponsesToolFunction represents a tool function

type ResponsesToolImageGeneration added in v1.2.0 

type ResponsesToolImageGeneration struct {
	Background        *string                                     `json:"background,omitempty"`         // "transparent" | "opaque" | "auto"
	InputFidelity     *string                                     `json:"input_fidelity,omitempty"`     // "high" | "low"
	InputImageMask    *ResponsesToolImageGenerationInputImageMask `json:"input_image_mask,omitempty"`   // Optional mask for inpainting
	Model             *string                                     `json:"model,omitempty"`              // Image generation model
	Moderation        *string                                     `json:"moderation,omitempty"`         // Moderation level
	OutputCompression *int                                        `json:"output_compression,omitempty"` // Compression level (0-100)
	OutputFormat      *string                                     `json:"output_format,omitempty"`      // "png" | "webp" | "jpeg"
	PartialImages     *int                                        `json:"partial_images,omitempty"`     // Number of partial images (0-3)
	Quality           *string                                     `json:"quality,omitempty"`            // "low" | "medium" | "high" | "auto"
	Size              *string                                     `json:"size,omitempty"`               // Image size
}

ResponsesToolImageGeneration represents a tool image generation

type ResponsesToolImageGenerationInputImageMask added in v1.2.0 

type ResponsesToolImageGenerationInputImageMask struct {
	FileID   *string `json:"file_id,omitempty"`   // File ID for the mask image
	ImageURL *string `json:"image_url,omitempty"` // Base64-encoded mask image
}

ResponsesToolImageGenerationInputImageMask represents a image generation input image mask

type ResponsesToolLocalShell added in v1.2.0 

type ResponsesToolLocalShell struct {
}

ResponsesToolLocalShell represents a tool local shell

type ResponsesToolMCP added in v1.2.0 

type ResponsesToolMCP struct {
	ServerLabel       string                                       `json:"server_label"`                 // A label for this MCP server
	AllowedTools      *ResponsesToolMCPAllowedTools                `json:"allowed_tools,omitempty"`      // List of allowed tool names or filter
	Authorization     *string                                      `json:"authorization,omitempty"`      // OAuth access token
	ConnectorID       *string                                      `json:"connector_id,omitempty"`       // Service connector ID
	Headers           *map[string]string                           `json:"headers,omitempty"`            // Optional HTTP headers
	RequireApproval   *ResponsesToolMCPAllowedToolsApprovalSetting `json:"require_approval,omitempty"`   // Tool approval settings
	ServerDescription *string                                      `json:"server_description,omitempty"` // Optional server description
	ServerURL         *string                                      `json:"server_url,omitempty"`         // The URL for the MCP server
}

ResponsesToolMCP - Give the model access to additional tools via remote MCP servers

type ResponsesToolMCPAllowedTools added in v1.2.0 

type ResponsesToolMCPAllowedTools struct {
	// Either a simple array of tool names or a filter object
	ToolNames []string                            `json:",omitempty"`
	Filter    *ResponsesToolMCPAllowedToolsFilter `json:",omitempty"`
}

ResponsesToolMCPAllowedTools - List of allowed tool names or a filter object

type ResponsesToolMCPAllowedToolsApprovalFilter added in v1.2.0 

type ResponsesToolMCPAllowedToolsApprovalFilter struct {
	ReadOnly  *bool    `json:"read_only,omitempty"`  // Whether tool is read-only
	ToolNames []string `json:"tool_names,omitempty"` // List of tool names
}

ResponsesToolMCPAllowedToolsApprovalFilter - Filter for approval settings

type ResponsesToolMCPAllowedToolsApprovalSetting added in v1.2.0 

type ResponsesToolMCPAllowedToolsApprovalSetting struct {
	// Either a string setting or filter objects
	Setting *string                                     `json:",omitempty"` // "always" | "never"
	Always  *ResponsesToolMCPAllowedToolsApprovalFilter `json:"always,omitempty"`
	Never   *ResponsesToolMCPAllowedToolsApprovalFilter `json:"never,omitempty"`
}

ResponsesToolMCPAllowedToolsApprovalSetting - Specify which tools require approval

func (ResponsesToolMCPAllowedToolsApprovalSetting) MarshalJSON added in v1.2.17 

func (as ResponsesToolMCPAllowedToolsApprovalSetting) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshalling for ResponsesToolMCPAllowedToolsApprovalSetting

func (*ResponsesToolMCPAllowedToolsApprovalSetting) UnmarshalJSON added in v1.2.17 

func (as *ResponsesToolMCPAllowedToolsApprovalSetting) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshalling for ResponsesToolMCPAllowedToolsApprovalSetting

type ResponsesToolMCPAllowedToolsFilter added in v1.2.0 

type ResponsesToolMCPAllowedToolsFilter struct {
	ReadOnly  *bool    `json:"read_only,omitempty"`  // Whether tool is read-only
	ToolNames []string `json:"tool_names,omitempty"` // List of allowed tool names
}

ResponsesToolMCPAllowedToolsFilter - A filter object to specify which tools are allowed

type ResponsesToolMessage added in v1.2.0 

type ResponsesToolMessage struct {
	CallID    *string                           `json:"call_id,omitempty"`   // Common call ID for tool calls and outputs
	Name      *string                           `json:"name,omitempty"`      // Common name field for tool calls
	Namespace *string                           `json:"namespace,omitempty"` // Namespace for function_call items (set by OpenAI when namespace tools are used)
	Arguments *string                           `json:"arguments,omitempty"`
	Output    *ResponsesToolMessageOutputStruct `json:"output,omitempty"`
	Action    *ResponsesToolMessageActionStruct `json:"action,omitempty"`
	Error     *string                           `json:"error,omitempty"`
	// Caller is the neutral form of Anthropic's "caller" union on server-tool blocks
	Caller *ResponsesToolCaller `json:"tool_caller,omitempty"`

	// Tool calls and outputs
	*ResponsesFileSearchToolCall
	*ResponsesComputerToolCall
	*ResponsesComputerToolCallOutput
	*ResponsesCodeInterpreterToolCall
	*ResponsesMCPToolCall
	*ResponsesCustomToolCall
	*ResponsesImageGenerationCall

	// MCP-specific
	*ResponsesMCPListTools
	*ResponsesMCPApprovalResponse

	// Anthropic advisor-specific (advisor_call): carries the advisor_tool_result payload
	*ResponsesAdvisorCall

	// Anthropic tool_search-specific (tool_search_call): carries the discovered tool references
	*ResponsesToolSearchCall

	// Anthropic web-fetch-specific (web_fetch_call): carries the web_fetch_tool_result payload
	*ResponsesWebFetchCall

	// Anthropic code-execution-specific (code_interpreter_call): carries the
	// server_tool_use input + *_code_execution_tool_result payload that the
	// neutral ResponsesCodeInterpreterToolCall cannot represent.
	*ResponsesCodeExecutionCall
}

func (*ResponsesToolMessage) ToChatAssistantMessageToolCall added in v1.3.0 

func (rtm *ResponsesToolMessage) ToChatAssistantMessageToolCall() *ChatAssistantMessageToolCall

ToChatAssistantMessageToolCall converts a ResponsesToolMessage to ChatAssistantMessageToolCall format. This is useful for executing Responses API tool calls using the Chat API tool executor.

Returns:

  • *ChatAssistantMessageToolCall: The converted tool call in Chat API format

Example: 

responsesToolMsg := &ResponsesToolMessage{
    CallID:    Ptr("call-123"),
    Name:      Ptr("calculate"),
    Arguments: Ptr("{\"x\": 10, \"y\": 20}"),
}
chatToolCall := responsesToolMsg.ToChatAssistantMessageToolCall()

type ResponsesToolMessageActionStruct added in v1.2.11 

type ResponsesToolMessageActionStruct struct {
	ResponsesComputerToolCallAction   *ResponsesComputerToolCallAction
	ResponsesWebSearchToolCallAction  *ResponsesWebSearchToolCallAction
	ResponsesWebFetchToolCallAction   *ResponsesWebFetchToolCallAction
	ResponsesLocalShellToolCallAction *ResponsesLocalShellToolCallAction
	ResponsesMCPApprovalRequestAction *ResponsesMCPApprovalRequestAction
}

func (ResponsesToolMessageActionStruct) MarshalJSON added in v1.2.11 

func (action ResponsesToolMessageActionStruct) MarshalJSON() ([]byte, error)

func (*ResponsesToolMessageActionStruct) UnmarshalJSON added in v1.2.11 

func (action *ResponsesToolMessageActionStruct) UnmarshalJSON(data []byte) error

type ResponsesToolMessageOutputStruct added in v1.2.11 

type ResponsesToolMessageOutputStruct struct {
	ResponsesToolCallOutputStr            *string // Common output string for tool calls and outputs (used by function, custom and local shell tool calls)
	ResponsesFunctionToolCallOutputBlocks []ResponsesMessageContentBlock
	ResponsesComputerToolCallOutput       *ResponsesComputerToolCallOutputData
}

func (ResponsesToolMessageOutputStruct) MarshalJSON added in v1.2.11 

func (output ResponsesToolMessageOutputStruct) MarshalJSON() ([]byte, error)

func (*ResponsesToolMessageOutputStruct) UnmarshalJSON added in v1.2.11 

func (output *ResponsesToolMessageOutputStruct) UnmarshalJSON(data []byte) error

type ResponsesToolNamespace added in v1.5.5 

type ResponsesToolNamespace struct {
	Tools []ResponsesTool `json:"tools,omitempty"`
}

ResponsesToolNamespace represents a namespace tool that groups related function tools.

type ResponsesToolSearchCall added in v1.6.4 

type ResponsesToolSearchCall struct {
	ToolReferences []string `json:"tool_references,omitempty"` // names of discovered (deferred) tools
}

ResponsesToolSearchCall carries the payload of an Anthropic server-side tool_search (server_tool_use + tool_search_tool_result). ToolReferences holds the names of the deferred tools the search discovered (from the result block's tool_references); the model then emits a normal tool_use to call one of them.

type ResponsesToolToolSearch added in v1.5.6 

type ResponsesToolToolSearch struct {
	Execution  *string                 `json:"execution,omitempty"`
	Parameters *ToolFunctionParameters `json:"parameters,omitempty"`
}

ResponsesToolToolSearch represents a Responses API tool_search tool.

type ResponsesToolType added in v1.2.17 

type ResponsesToolType string
const (
	ResponsesToolTypeFunction           ResponsesToolType = "function"
	ResponsesToolTypeFileSearch         ResponsesToolType = "file_search"
	ResponsesToolTypeComputerUsePreview ResponsesToolType = "computer_use_preview"
	ResponsesToolTypeWebSearch          ResponsesToolType = "web_search"
	ResponsesToolTypeWebFetch           ResponsesToolType = "web_fetch"
	ResponsesToolTypeMCP                ResponsesToolType = "mcp"
	ResponsesToolTypeCodeInterpreter    ResponsesToolType = "code_interpreter"
	ResponsesToolTypeImageGeneration    ResponsesToolType = "image_generation"
	ResponsesToolTypeLocalShell         ResponsesToolType = "local_shell"
	ResponsesToolTypeCustom             ResponsesToolType = "custom"
	ResponsesToolTypeWebSearchPreview   ResponsesToolType = "web_search_preview"
	ResponsesToolTypeMemory             ResponsesToolType = "memory"
	ResponsesToolTypeToolSearch         ResponsesToolType = "tool_search"
	ResponsesToolTypeNamespace          ResponsesToolType = "namespace"
	ResponsesToolTypeXSearch            ResponsesToolType = "x_search"
	ResponsesToolTypeAdvisor            ResponsesToolType = "advisor"
)

type ResponsesToolWebFetch added in v1.4.9 

type ResponsesToolWebFetch struct {
	MaxUses           *int                           `json:"max_uses,omitempty"`
	Filters           *ResponsesToolWebSearchFilters `json:"filters,omitempty"`
	MaxContentTokens  *int                           `json:"max_content_tokens,omitempty"`
	UseCache          *bool                          `json:"use_cache,omitempty"`
	ResponseInclusion *string                        `json:"response_inclusion,omitempty"` // "full" | "excluded" (web_fetch_20260318+)
}

ResponsesToolWebFetch represents a web fetch tool

type ResponsesToolWebSearch added in v1.2.0 

type ResponsesToolWebSearch struct {
	ExternalWebAccess  *bool                               `json:"external_web_access,omitempty"`
	Filters            *ResponsesToolWebSearchFilters      `json:"filters,omitempty"` // Filters for the search
	SearchContentTypes []string                            `json:"search_content_types,omitempty"`
	SearchContextSize  *string                             `json:"search_context_size,omitempty"` // "low" | "medium" | "high"
	UserLocation       *ResponsesToolWebSearchUserLocation `json:"user_location,omitempty"`       // The approximate location of the user

	// Anthropic only
	MaxUses *int `json:"max_uses,omitempty"` // Maximum number of uses for the search
}

ResponsesToolWebSearch represents a tool web search

type ResponsesToolWebSearchFilters added in v1.2.0 

type ResponsesToolWebSearchFilters struct {
	AllowedDomains []string `json:"allowed_domains,omitempty"` // Allowed domains for the search
	BlockedDomains []string `json:"blocked_domains,omitempty"` // Blocked domains for the search, only used in anthropic

	// Gemini only
	// Filter search results to a specific time range.
	// If users set a start time, they must set an end time (and vice versa).
	TimeRangeFilter *Interval `json:"time_range_filter,omitempty"`
}

ResponsesToolWebSearchFilters represents filters for web search

type ResponsesToolWebSearchPreview added in v1.2.0 

type ResponsesToolWebSearchPreview struct {
	SearchContextSize *string                             `json:"search_context_size,omitempty"` // "low" | "medium" | "high"
	UserLocation      *ResponsesToolWebSearchUserLocation `json:"user_location,omitempty"`       // The user's location
}

ResponsesToolWebSearchPreview represents a web search preview

type ResponsesToolWebSearchUserLocation added in v1.2.0 

type ResponsesToolWebSearchUserLocation struct {
	City     *string `json:"city,omitempty"`     // Free text input for the city
	Country  *string `json:"country,omitempty"`  // Two-letter ISO country code
	Region   *string `json:"region,omitempty"`   // Free text input for the region
	Timezone *string `json:"timezone,omitempty"` // IANA timezone
	Type     *string `json:"type,omitempty"`     // always "approximate"
}

ResponsesToolWebSearchUserLocation - The approximate location of the user

type ResponsesToolXSearch added in v1.5.17 

type ResponsesToolXSearch struct {
	// AllowedXHandles restricts search to posts from these X accounts (max 10).
	// Mutually exclusive with ExcludedXHandles.
	AllowedXHandles []string `json:"allowed_x_handles,omitempty"`
	// ExcludedXHandles excludes posts from these X accounts from results.
	// Mutually exclusive with AllowedXHandles.
	ExcludedXHandles []string `json:"excluded_x_handles,omitempty"`
	// FromDate is the start date for tweet search (ISO 8601 date or datetime string).
	FromDate *string `json:"from_date,omitempty"`
	// ToDate is the end date for tweet search (ISO 8601 date or datetime string).
	ToDate *string `json:"to_date,omitempty"`
	// EnableImageUnderstanding controls whether images in tweets are analyzed.
	EnableImageUnderstanding *bool `json:"enable_image_understanding,omitempty"`
	// EnableVideoUnderstanding controls whether videos in tweets are analyzed.
	EnableVideoUnderstanding *bool `json:"enable_video_understanding,omitempty"`
}

ResponsesToolXSearch represents the xAI-native x_search server-side tool. All fields are optional; when omitted xAI searches without restrictions. See https://docs.x.ai/developers/tools/x-search#x-search-parameters

type ResponsesWebFetchCall added in v1.6.3 

type ResponsesWebFetchCall struct {
	ResultType  string                     `json:"web_fetch_result_type,omitempty"` // "web_fetch_result" | "web_fetch_tool_result_error"
	URL         *string                    `json:"web_fetch_result_url,omitempty"`
	RetrievedAt *string                    `json:"web_fetch_retrieved_at,omitempty"`
	Document    *ResponsesWebFetchDocument `json:"web_fetch_document,omitempty"`
	ErrorCode   *string                    `json:"web_fetch_error_code,omitempty"`
}

ResponsesWebFetchCall carries the Anthropic web_fetch_tool_result payload alongside a web_fetch_call. Anthropic-only; the request URL lives on ResponsesWebFetchToolCallAction.

type ResponsesWebFetchDocument added in v1.6.3 

type ResponsesWebFetchDocument struct {
	Type      string                   `json:"type,omitempty"` // "document"
	Text      *string                  `json:"text,omitempty"`
	Title     *string                  `json:"title,omitempty"`
	Source    *ResponsesWebFetchSource `json:"source,omitempty"`
	Citations *Citations               `json:"citations,omitempty"`
	Context   *string                  `json:"context,omitempty"`
}

type ResponsesWebFetchSource added in v1.6.3 

type ResponsesWebFetchSource struct {
	Type      string  `json:"type,omitempty"` // "text" | "base64" | "url" | "file"
	MediaType *string `json:"media_type,omitempty"`
	Data      *string `json:"data,omitempty"`
	URL       *string `json:"url,omitempty"`
	FileID    *string `json:"file_id,omitempty"`
}

type ResponsesWebFetchToolCallAction added in v1.4.9 

type ResponsesWebFetchToolCallAction struct {
	Type string `json:"type,omitempty"` // "fetch"
	URL  string `json:"url"`
}

ResponsesWebFetchToolCallAction represents a web fetch action

type ResponsesWebSearchToolCallAction added in v1.2.11 

type ResponsesWebSearchToolCallAction struct {
	Type    string                                         `json:"type"`          // "search" | "open_page" | "find"
	URL     *string                                        `json:"url,omitempty"` // Common URL field (OpenPage, Find)
	Query   *string                                        `json:"query,omitempty"`
	Queries []string                                       `json:"queries,omitempty"`
	Sources []ResponsesWebSearchToolCallActionSearchSource `json:"sources,omitempty"`
	Pattern *string                                        `json:"pattern,omitempty"`
}

ResponsesWebSearchToolCallAction represents the different types of web search actions

type ResponsesWebSearchToolCallActionSearchSource added in v1.2.11 

type ResponsesWebSearchToolCallActionSearchSource struct {
	Type string `json:"type"` // always "url"
	URL  string `json:"url"`

	// Anthropic specific fields
	Title            *string `json:"title,omitempty"`
	EncryptedContent *string `json:"encrypted_content,omitempty"`
	PageAge          *string `json:"page_age,omitempty"`
}

ResponsesWebSearchToolCallActionSearchSource represents a web search action search source

type RoutingEngineLogEntry added in v1.4.3 

type RoutingEngineLogEntry struct {
	Engine    string   `json:"engine"` // e.g., "governance", "routing-rule", "openrouter"
	Level     LogLevel `json:"level"`
	Message   string   `json:"message"`   // Human-readable decision/action message
	Timestamp int64    `json:"timestamp"` // Unix milliseconds
}

RoutingEngineLogEntry represents a log entry from a routing engine format: [timestamp] [engine] - message

type RoutingInfo added in v1.5.19 

type RoutingInfo struct {
	// What actually handled this attempt
	Provider ModelProvider `json:"provider,omitempty"`
	Model    string        `json:"model,omitempty"` // model name passed to this attempt's key
	Key      string        `json:"key,omitempty"`   // KeyName of the key used

	// Populated only when Model matched an entry in this key's Aliases map
	ResolvedKeyAlias *ResolvedKeyAlias `json:"resolved_key_alias,omitempty"`

	IsFallback bool `json:"is_fallback,omitempty"`

	// What the caller asked for, before any fallback resolution (populated only when fallback resolution occurred)
	PrimaryProvider *ModelProvider `json:"primary_provider,omitempty"`
	PrimaryModel    *string        `json:"primary_model,omitempty"`

	// ServerSideFallbackModel names the model that actually produced the response
	// when the provider swapped models *inside* a single upstream call — today only
	// Anthropic's server-side fallback (server-side-fallback-2026-06-01). Model
	// still names what the caller asked for, since routing never saw the swap.
	//
	// Unlike every other field here this one is provider-owned, not written by the
	// orchestrator: only the provider can see a handoff that happened within its own
	// response. PopulateRoutingInfo preserves it across core's overwrite. Nil on
	// every ordinary response, so pricing behaviour is unchanged when it is absent.
	ServerSideFallbackModel *string `json:"server_side_fallback_model,omitempty"`
}

func BuildRoutingInfo added in v1.5.19 

func BuildRoutingInfo(ctx *BifrostContext, attemptProvider ModelProvider, attemptModel string, attemptKey Key) RoutingInfo

BuildRoutingInfo constructs a RoutingInfo for the current attempt from this attempt's chosen provider/model/key and the resolved alias stashed in ctx.

Populates only the per-attempt fields (Provider, Model, Key, ResolvedKeyAlias). IsFallback and PrimaryProvider/PrimaryModel are layered on later by the orchestrator (handleRequest / handleStreamRequest) via SetFallbackRoutingInfo on the final response/error, since those signals belong to the orchestrator scope rather than the per-attempt one.

ResolvedKeyAlias.ModelFamily reflects the family explicitly configured on the alias (nil when the admin didn't set one) — not the substring-resolved family used for routing.

type RoutingScopeMatch added in v1.5.11 

type RoutingScopeMatch struct {
	Scope   string
	ScopeID string
}

RoutingScopeMatch is a single (scope, scope_id) pair the caller is allowed to see in routing_rules. The "global" scope is always implicitly allowed and does not need to be enumerated.

type S3BucketConfig added in v1.2.39 

type S3BucketConfig struct {
	BucketName string `json:"bucket_name"`          // S3 bucket name
	Prefix     string `json:"prefix,omitempty"`     // S3 key prefix for batch files
	IsDefault  bool   `json:"is_default,omitempty"` // Whether this is the default bucket for batch operations
}

S3BucketConfig represents a single S3 bucket configuration for batch operations.

type S3StorageConfig added in v1.2.38 

type S3StorageConfig struct {
	Bucket string `json:"bucket,omitempty"`
	Region string `json:"region,omitempty"`
	Prefix string `json:"prefix,omitempty"`
}

S3StorageConfig represents AWS S3 storage configuration.

type SGLKeyConfig added in v1.5.0 

type SGLKeyConfig struct {
	URL SecretVar `json:"url"` // SGLang server base URL (required, supports env. prefix)
}

SGLKeyConfig represents the SGLang-specific key configuration. It allows each key to target a different SGLang server URL, enabling per-key routing and round-robin load balancing across multiple SGLang instances.

type SearchResult added in v1.2.19 

type SearchResult struct {
	Title       string  `json:"title"`
	URL         string  `json:"url"`
	Date        *string `json:"date,omitempty"`
	LastUpdated *string `json:"last_updated,omitempty"`
	Snippet     *string `json:"snippet,omitempty"`
	Source      *string `json:"source,omitempty"`
}

type SecretType added in v1.6.0 

type SecretType string

SecretType identifies the source of a SecretVar's value. 

const (
	SecretTypePlainText SecretType = "plain_text"
	SecretTypeEnv       SecretType = "env"
	SecretTypeVault     SecretType = "vault"
)

type SecretVar added in v1.6.0 

type SecretVar struct {
	Val string `json:"value"`

	SecretType SecretType `json:"type,omitempty"`
	// contains filtered or unexported fields
}

SecretVar is a wrapper around a value that can be sourced from an environment variable or an external vault (e.g. AWS Secrets Manager, GCP Secret Manager, HashiCorp Vault). Three reference forms are accepted: plain text, "env.VAR_NAME", and "vault.path/to/secret".

func NewSecretVar added in v1.6.0 

func NewSecretVar(value string) *SecretVar

NewSecretVar creates a new SecretVar from a string.

func (*SecretVar) CoerceBool added in v1.6.0 

func (e *SecretVar) CoerceBool(defaultValue bool) bool

CoerceBool coerces value to bool

func (*SecretVar) CoerceInt added in v1.6.0 

func (e *SecretVar) CoerceInt(defaultValue int) int

CoerceInt coerces value to int

func (*SecretVar) EnvKey added in v1.6.0 

func (e *SecretVar) EnvKey() string

EnvKey returns the environment variable name without the "env." prefix. Returns an empty string if the SecretVar is not env-backed.

func (*SecretVar) Equals added in v1.6.0 

func (e *SecretVar) Equals(other *SecretVar) bool

Equals checks if two SecretVars are equal.

func (*SecretVar) FullyRedacted added in v1.6.0 

func (e *SecretVar) FullyRedacted() *SecretVar

FullyRedacted returns a copy of the SecretVar with Val replaced by a fixed placeholder so no substring of the original value is exposed. Use for API responses where Redacted is unsafe (e.g. literal proxy passwords). secretRef and secretType are preserved so references remain visible.

func (*SecretVar) GetRawRef added in v1.6.0 

func (e *SecretVar) GetRawRef() string

GetRawRef returns the full secret reference string including prefix (e.g. "env.MY_VAR" or "vault.path/to/secret"). Returns an empty string for plain-value SecretVars.

func (*SecretVar) GetRef added in v1.6.0 

func (e *SecretVar) GetRef() string

GetRef returns the secret reference without its type prefix. For env secrets it strips "env." (returning "MY_VAR"), for vault it strips "vault." (returning "path/to/secret"), and for plain values it returns the ref as-is.

func (*SecretVar) GetValue added in v1.6.0 

func (e *SecretVar) GetValue() string

GetValue returns the resolved value.

func (*SecretVar) GetValuePtr added in v1.6.0 

func (e *SecretVar) GetValuePtr() *string

GetValuePtr returns a pointer to the value.

func (*SecretVar) IsFromEnv added in v1.6.0 

func (e *SecretVar) IsFromEnv() bool

IsFromEnv returns true if the value is sourced from an environment variable.

func (*SecretVar) IsFromSecret added in v1.6.0 

func (e *SecretVar) IsFromSecret() bool

IsFromSecret returns true if the value is sourced from an external secret (env var or vault).

func (*SecretVar) IsFromVault added in v1.6.0 

func (e *SecretVar) IsFromVault() bool

IsFromVault returns true if the value is sourced from a vault path.

func (*SecretVar) IsMaskedPlaceholder added in v1.6.4 

func (e *SecretVar) IsMaskedPlaceholder() bool

IsMaskedPlaceholder reports whether the value is a client-side redaction placeholder that must not overwrite a stored credential. Secret references are intentional updates and are never treated as placeholders.

func (*SecretVar) IsRedacted added in v1.6.0 

func (e *SecretVar) IsRedacted() bool

IsRedacted returns true if the value is redacted.

func (*SecretVar) IsSet added in v1.6.0 

func (e *SecretVar) IsSet() bool

IsSet returns true if the SecretVar has a resolved value or a secret reference. Use instead of GetValue() != "" when checking whether a field was configured, because references may have an empty Val before resolution.

func (SecretVar) MarshalJSON added in v1.6.0 

func (e SecretVar) MarshalJSON() ([]byte, error)

MarshalJSON serializes the SecretVar, emitting ref and type fields.

func (*SecretVar) Redacted added in v1.6.0 

func (e *SecretVar) Redacted() *SecretVar

Redacted returns a new SecretVar with the value redacted.

func (*SecretVar) Scan added in v1.6.0 

func (e *SecretVar) Scan(value any) error

Scan scans the value from the database.

func (*SecretVar) ShouldPreserveStored added in v1.6.0 

func (e *SecretVar) ShouldPreserveStored() bool

ShouldPreserveStored returns true when the SecretVar is a client-side placeholder that should not overwrite the stored credential. Returns true for a nil receiver, an empty non-secret value, or a redacted non-secret value. Returns false for secret references (always intentional) and plain non-empty values.

func (*SecretVar) String added in v1.6.0 

func (e *SecretVar) String() string

String returns the value as a string.

func (*SecretVar) Type added in v1.6.0 

func (e *SecretVar) Type() SecretType

Type returns the SecretType of this SecretVar.

func (*SecretVar) UnmarshalJSON added in v1.6.0 

func (e *SecretVar) UnmarshalJSON(data []byte) error

UnmarshalJSON unmarshals the value from JSON.

func (SecretVar) Value added in v1.6.0 

func (e SecretVar) Value() (driver.Value, error)

Value implements driver.Valuer for database storage. It stores the secret reference (e.g., "env.API_KEY" or "vault.path/to/secret") if the type is env or vault, otherwise the raw value.

func (*SecretVar) VaultPath added in v1.6.0 

func (e *SecretVar) VaultPath() string

VaultPath returns the vault path without the "vault." prefix. Returns an empty string if the SecretVar is not vault-backed.

type SerialCursor added in v1.2.39 

type SerialCursor struct {
	Version  int    `json:"v"` // Version for compatibility
	KeyIndex int    `json:"i"` // Current key index in sorted keys array
	Cursor   string `json:"c"` // Native cursor for current key (empty = start fresh)
}

SerialCursor tracks pagination state for serial key exhaustion. When paginating across multiple keys, we exhaust all pages from one key before moving to the next, ensuring only one API call per pagination request.

func DecodeSerialCursor added in v1.2.39 

func DecodeSerialCursor(encoded string) (*SerialCursor, error)

DecodeSerialCursor decodes a base64 string back to a SerialCursor. Returns (nil, nil) if the encoded string is empty; returns an error for invalid data.

func NewSerialCursor added in v1.2.39 

func NewSerialCursor(keyIndex int, cursor string) *SerialCursor

NewSerialCursor creates a new SerialCursor with version 1.

type SkippedMCPTool added in v1.5.10 

type SkippedMCPTool struct {
	OriginalName string `json:"original_name"`
	Reason       string `json:"reason"`
}

SkippedMCPTool describes a tool that the MCP server returned but Bifrost did not include in the final tool map (e.g. invalid normalized name).

type Span added in v1.3.0 

type Span struct {
	SpanID     string         // Unique identifier for this span
	ParentID   string         // Parent span ID (empty for root span)
	TraceID    string         // The trace this span belongs to
	Name       string         // Name of the operation
	Kind       SpanKind       // Type of span (LLM call, plugin, etc.)
	StartTime  time.Time      // When the span started
	EndTime    time.Time      // When the span completed
	Status     SpanStatus     // Status of the operation
	StatusMsg  string         // Optional status message (for errors)
	Attributes map[string]any // Additional attributes for the span
	Events     []SpanEvent    // Events that occurred during the span
	// contains filtered or unexported fields
}

Span represents a single operation within a trace

func (*Span) AddEvent added in v1.3.0 

func (s *Span) AddEvent(event SpanEvent)

AddEvent adds an event to the span in a thread-safe manner

func (*Span) End added in v1.3.0 

func (s *Span) End(status SpanStatus, statusMsg string)

End marks the span as complete with the given status

func (*Span) Reset added in v1.3.0 

func (s *Span) Reset()

Reset clears the span for reuse from pool. It holds s.mu — like every other Span mutator — so a straggling writer (e.g. streaming finalization) that races pool release can't trigger a fatal concurrent map access on s.Attributes.

func (*Span) SetAttribute added in v1.3.0 

func (s *Span) SetAttribute(key string, value any)

SetAttribute sets an attribute on the span in a thread-safe manner

type SpanEvent added in v1.3.0 

type SpanEvent struct {
	Name       string         // Name of the event
	Timestamp  time.Time      // When the event occurred
	Attributes map[string]any // Additional attributes for the event
}

SpanEvent represents a time-stamped event within a span

type SpanHandle added in v1.3.0 

type SpanHandle interface{}

SpanHandle is an opaque handle to a span, implementation-specific. Different Tracer implementations can use their own concrete types.

type SpanKind added in v1.3.0 

type SpanKind string

SpanKind represents the type of operation a span represents These are LLM-specific kinds designed for AI gateway observability 

const (
	// SpanKindUnspecified is the default span kind
	SpanKindUnspecified SpanKind = ""
	// SpanKindLLMCall represents a call to an LLM provider
	SpanKindLLMCall SpanKind = "llm.call"
	// SpanKindPlugin represents plugin execution (PreLLMHook/PostLLMHook)
	SpanKindPlugin SpanKind = "plugin"
	// SpanKindMCPTool represents an MCP tool invocation
	SpanKindMCPTool SpanKind = "mcp.tool"
	// SpanKindMCPClient represents an MCP client lifecycle operation (connect/ping/list_tools).
	// These run in the background per-client and are not part of an LLM request flow.
	SpanKindMCPClient SpanKind = "mcp.client"
	// SpanKindRetry represents a retry attempt
	SpanKindRetry SpanKind = "retry"
	// SpanKindFallback represents a fallback to another provider
	SpanKindFallback SpanKind = "fallback"
	// SpanKindHTTPRequest represents the root HTTP request span
	SpanKindHTTPRequest SpanKind = "http.request"
	// SpanKindEmbedding represents an embedding request
	SpanKindEmbedding SpanKind = "embedding"
	// SpanKindSpeech represents a text-to-speech request
	SpanKindSpeech SpanKind = "speech"
	// SpanKindTranscription represents a speech-to-text request
	SpanKindTranscription SpanKind = "transcription"
	// SpanKindInternal represents internal operations (key selection, etc.)
	SpanKindInternal SpanKind = "internal"
)

type SpanStatus added in v1.3.0 

type SpanStatus string

SpanStatus represents the status of a span's operation 

const (
	// SpanStatusUnset indicates status has not been set
	SpanStatusUnset SpanStatus = "unset"
	// SpanStatusOk indicates the operation completed successfully
	SpanStatusOk SpanStatus = "ok"
	// SpanStatusError indicates the operation failed
	SpanStatusError SpanStatus = "error"
)

type SpeechAlignment added in v1.2.24 

type SpeechAlignment struct {
	CharStartTimesMs []float64 `json:"char_start_times_ms"` // Start time in milliseconds for each character
	CharEndTimesMs   []float64 `json:"char_end_times_ms"`   // End time in milliseconds for each character
	Characters       []string  `json:"characters"`          // Characters corresponding to timing info
}

SpeechAlignment represents character-level timing information for audio-text synchronization

type SpeechInput added in v1.1.11 

type SpeechInput struct {
	Input string `json:"input"`
}

SpeechInput represents the input for a speech request.

type SpeechParameters added in v1.2.0 

type SpeechParameters struct {
	VoiceConfig    *SpeechVoiceInput `json:"voice"`
	Instructions   string            `json:"instructions,omitempty"`
	ResponseFormat string            `json:"response_format,omitempty"` // Default is "mp3"
	Speed          *float64          `json:"speed,omitempty"`

	LanguageCode                    *string                                `json:"language_code,omitempty"`
	PronunciationDictionaryLocators []SpeechPronunciationDictionaryLocator `json:"pronunciation_dictionary_locators,omitempty"`
	EnableLogging                   *bool                                  `json:"enable_logging,omitempty"`
	OptimizeStreamingLatency        *bool                                  `json:"optimize_streaming_latency,omitempty"`
	WithTimestamps                  *bool                                  `json:"with_timestamps,omitempty"` // Returns character-level timing information

	// Dynamic parameters that can be provider-specific, they are directly
	// added to the request as is.
	ExtraParams map[string]interface{} `json:"-"`
}

type SpeechPronunciationDictionaryLocator added in v1.2.24 

type SpeechPronunciationDictionaryLocator struct {
	PronunciationDictionaryID string  `json:"pronunciation_dictionary_id"`
	VersionID                 *string `json:"version_id,omitempty"`
}

type SpeechStreamResponseType added in v1.2.7 

type SpeechStreamResponseType string
const (
	SpeechStreamResponseTypeDelta SpeechStreamResponseType = "speech.audio.delta"
	SpeechStreamResponseTypeDone  SpeechStreamResponseType = "speech.audio.done"
)

type SpeechUsage added in v1.2.7 

type SpeechUsage struct {
	InputTokens       int                           `json:"input_tokens"`
	InputChars        int                           `json:"input_chars,omitempty"`
	InputTokenDetails *SpeechUsageInputTokenDetails `json:"input_token_details,omitempty"`
	OutputTokens      int                           `json:"output_tokens"`
	TotalTokens       int                           `json:"total_tokens"`
	// AudioSeconds is the generated audio duration in seconds. Populated for
	// duration-based audio models (e.g. ElevenLabs sound effects) so per-second
	// pricing can be applied; zero for token/character-billed TTS.
	AudioSeconds int `json:"audio_seconds,omitempty"`
}

type SpeechUsageInputTokenDetails added in v1.4.1 

type SpeechUsageInputTokenDetails struct {
	TextTokens  int `json:"text_tokens,omitempty"`
	AudioTokens int `json:"audio_tokens,omitempty"`
}

type SpeechVoiceInput added in v1.1.11 

type SpeechVoiceInput struct {
	Voice            *string
	MultiVoiceConfig []VoiceConfig
}

func (*SpeechVoiceInput) MarshalJSON added in v1.1.11 

func (vi *SpeechVoiceInput) MarshalJSON() ([]byte, error)

MarshalJSON implements custom JSON marshalling for SpeechVoiceInput. It marshals either Voice or MultiVoiceConfig directly without wrapping.

func (*SpeechVoiceInput) UnmarshalJSON added in v1.1.11 

func (vi *SpeechVoiceInput) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshalling for SpeechVoiceInput. It determines whether "voice" is a string or a VoiceConfig object/array and assigns to the appropriate field. It also handles direct string/array content without a wrapper object.

type StreamAccumulatorResult added in v1.3.0 

type StreamAccumulatorResult struct {
	RequestID             string                          // Request ID
	RequestedModel        string                          // Original model requested by the caller
	ResolvedModel         string                          // Actual model used by the provider (equals RequestedModel when no alias mapping exists)
	Provider              ModelProvider                   // Provider used
	Status                string                          // Status of the stream
	Latency               int64                           // Latency in milliseconds
	TimeToFirstToken      int64                           // Time to first token in milliseconds
	OutputMessage         *ChatMessage                    // Accumulated output message
	OutputMessages        []ResponsesMessage              // For responses API
	TokenUsage            *BifrostLLMUsage                // Token usage
	Cost                  *float64                        // Cost in dollars
	CacheDebug            *BifrostCacheDebug              // Semantic cache debug info if available
	ErrorDetails          *BifrostError                   // Error details if any
	AudioOutput           *BifrostSpeechResponse          // For speech streaming
	TranscriptionOutput   *BifrostTranscriptionResponse   // For transcription streaming
	ImageGenerationOutput *BifrostImageGenerationResponse // For image generation streaming
	PassthroughOutput     *BifrostPassthroughResponse     // For passthrough streaming
	FinishReason          *string                         // Finish reason
	RawResponse           *string                         // Raw response
	RawRequest            interface{}                     // Raw request
}

StreamAccumulatorResult contains the accumulated data from streaming chunks. This is the return type for tracer's streaming accumulation methods.

type StreamControl added in v1.1.15 

type StreamControl struct {
	LogError   *bool `json:"log_error,omitempty"`   // Optional: Controls logging of error
	SkipStream *bool `json:"skip_stream,omitempty"` // Optional: Controls skipping of stream chunk
}

StreamControl represents stream control options.

type StreamInterceptionError added in v1.6.0 

type StreamInterceptionError struct {
	BifrostError *BifrostError
}

StreamInterceptionError carries a structured client error when an HTTP stream plugin terminates a stream.

func (*StreamInterceptionError) Error added in v1.6.0 

func (e *StreamInterceptionError) Error() string

Error returns the best available client message for callers that only understand Go errors.

type TextCompletionInput added in v1.2.0 

type TextCompletionInput struct {
	PromptStr   *string
	PromptArray []string
}

func (*TextCompletionInput) MarshalJSON added in v1.2.0 

func (t *TextCompletionInput) MarshalJSON() ([]byte, error)

func (*TextCompletionInput) UnmarshalJSON added in v1.2.0 

func (t *TextCompletionInput) UnmarshalJSON(data []byte) error

type TextCompletionLogProb 

type TextCompletionLogProb struct {
	TextOffset    []int                `json:"text_offset"`
	TokenLogProbs []float64            `json:"token_logprobs"`
	Tokens        []string             `json:"tokens"`
	TopLogProbs   []map[string]float64 `json:"top_logprobs"`
}

TextCompletionLogProb represents log probability information for text completion.

type TextCompletionParameters added in v1.2.0 

type TextCompletionParameters struct {
	BestOf           *int                `json:"best_of,omitempty"`
	Echo             *bool               `json:"echo,omitempty"`
	FrequencyPenalty *float64            `json:"frequency_penalty,omitempty"`
	LogitBias        *map[string]float64 `json:"logit_bias,omitempty"`
	LogProbs         *int                `json:"logprobs,omitempty"`
	MaxTokens        *int                `json:"max_tokens,omitempty"`
	N                *int                `json:"n,omitempty"`
	PresencePenalty  *float64            `json:"presence_penalty,omitempty"`
	Seed             *int                `json:"seed,omitempty"`
	Stop             []string            `json:"stop,omitempty"`
	Suffix           *string             `json:"suffix,omitempty"`
	StreamOptions    *ChatStreamOptions  `json:"stream_options,omitempty"`
	Temperature      *float64            `json:"temperature,omitempty"`
	TopP             *float64            `json:"top_p,omitempty"`
	User             *string             `json:"user,omitempty"`

	// Dynamic parameters that can be provider-specific, they are directly
	// added to the request as is.
	ExtraParams map[string]any `json:"-"`
}

type TextCompletionResponseChoice added in v1.2.7 

type TextCompletionResponseChoice struct {
	Text *string `json:"text,omitempty"`
}

type ToolFunctionParameters added in v1.2.0 

type ToolFunctionParameters struct {
	Type                 string                      `json:"type"`                           // Type of the parameters
	Description          *string                     `json:"description,omitempty"`          // Description of the parameters
	Properties           *OrderedMap                 `json:"properties"`                     // Parameter properties - always include even if empty (required by JSON Schema and some providers like OpenAI)
	Required             []string                    `json:"required,omitempty"`             // Required parameter names
	AdditionalProperties *AdditionalPropertiesStruct `json:"additionalProperties,omitempty"` // Whether to allow additional properties
	Enum                 []string                    `json:"enum,omitempty"`                 // Enum values for the parameters

	// JSON Schema definition fields
	Defs        *OrderedMap `json:"$defs,omitempty"`       // JSON Schema draft 2019-09+ definitions
	Definitions *OrderedMap `json:"definitions,omitempty"` // Legacy JSON Schema draft-07 definitions
	Ref         *string     `json:"$ref,omitempty"`        // Reference to definition

	// Array schema fields
	Items    *OrderedMap `json:"items,omitempty"`    // Array element schema
	MinItems *int64      `json:"minItems,omitempty"` // Minimum array length
	MaxItems *int64      `json:"maxItems,omitempty"` // Maximum array length

	// Composition fields (union types)
	AnyOf []OrderedMap `json:"anyOf,omitempty"` // Union types (any of these schemas)
	OneOf []OrderedMap `json:"oneOf,omitempty"` // Exclusive union types (exactly one of these)
	AllOf []OrderedMap `json:"allOf,omitempty"` // Schema intersection (all of these)

	// String validation fields
	Format    *string `json:"format,omitempty"`    // String format (email, date, uri, etc.)
	Pattern   *string `json:"pattern,omitempty"`   // Regex pattern for strings
	MinLength *int64  `json:"minLength,omitempty"` // Minimum string length
	MaxLength *int64  `json:"maxLength,omitempty"` // Maximum string length

	// Number validation fields
	Minimum *float64 `json:"minimum,omitempty"` // Minimum number value
	Maximum *float64 `json:"maximum,omitempty"` // Maximum number value

	// Misc fields
	Title    *string     `json:"title,omitempty"`    // Schema title
	Default  interface{} `json:"default,omitempty"`  // Default value
	Nullable *bool       `json:"nullable,omitempty"` // Nullable indicator (OpenAPI 3.0 style)
	// contains filtered or unexported fields
}

ToolFunctionParameters represents the parameters for a function definition. It supports JSON Schema fields used by various providers (OpenAI, Anthropic, Gemini, etc.). Field order follows JSON Schema / OpenAI conventions for consistent serialization.

IMPORTANT: When marshalling to JSON, key order is preserved from the original input (captured during UnmarshalJSON). When constructing programmatically, the default struct field declaration order is used. This is critical because LLMs are sensitive to JSON key ordering in tool schemas.

func DeepCopyToolFunctionParameters added in v1.5.14 

func DeepCopyToolFunctionParameters(original *ToolFunctionParameters) *ToolFunctionParameters

DeepCopyToolFunctionParameters creates a deep copy of ToolFunctionParameters, preserving all JSON Schema fields so references and validation metadata are not dropped during tool cloning.

func (ToolFunctionParameters) MarshalJSON added in v1.4.4 

func (t ToolFunctionParameters) MarshalJSON() ([]byte, error)

MarshalJSON serializes ToolFunctionParameters while preserving the original top-level key order when available. A client-supplied raw `{}` stays `{}`; otherwise object schemas always emit `properties` as an object, never null.

func (*ToolFunctionParameters) Normalized added in v1.4.8 

func (t *ToolFunctionParameters) Normalized() *ToolFunctionParameters

Normalized returns a shallow copy of the ToolFunctionParameters with JSON Schema structural keys sorted by priority (type, description, properties, required first, then alphabetically), while preserving the client's original ordering of user-defined property names inside "properties" maps. The copy shares primitive values with the original but has independent key slices, so sorting does not mutate the caller's data.

User-defined property names (e.g., "chain_of_thought", "answer") are kept in their original order because LLMs generate structured output fields in schema-declared order. Reordering them alphabetically can degrade output quality (e.g., forcing the model to write an answer before its reasoning).

The captured keyOrder is cleared so the struct field declaration order is used for the top-level keys. This produces deterministic JSON serialization regardless of the client's original structural key ordering, which is critical for Anthropic's prefix-based prompt caching.

func (*ToolFunctionParameters) UnmarshalJSON added in v1.2.47 

func (t *ToolFunctionParameters) UnmarshalJSON(data []byte) error

UnmarshalJSON implements custom JSON unmarshalling for ToolFunctionParameters. It handles both JSON object format (standard) and JSON string format (used by some providers like xAI). It captures the original key order for order-preserving re-serialization and records whether the client provided an explicit empty object schema.

type TopProvider added in v1.2.14 

type TopProvider struct {
	IsModerated         *bool `json:"is_moderated,omitempty"`
	ContextLength       *int  `json:"context_length,omitempty"`
	MaxCompletionTokens *int  `json:"max_completion_tokens,omitempty"`
}

type Trace added in v1.3.0 

type Trace struct {
	RequestID      string            // Request ID for the trace
	TraceID        string            // Unique identifier for this trace
	ParentID       string            // Parent trace ID from incoming W3C traceparent header
	RootSpan       *Span             // The root span of this trace
	Spans          []*Span           // All spans in this trace
	StartTime      time.Time         // When the trace started
	EndTime        time.Time         // When the trace completed
	Attributes     map[string]any    // Additional attributes for the trace
	RequestHeaders map[string]string // Lowercased request headers, populated only when a connector opts in
	PluginLogs     []PluginLogEntry  // Plugin log entries accumulated during request processing
	// contains filtered or unexported fields
}

Trace represents a distributed trace that captures the full lifecycle of a request

func (*Trace) AddSpan added in v1.3.0 

func (t *Trace) AddSpan(span *Span)

AddSpan adds a span to the trace in a thread-safe manner

func (*Trace) AppendPluginLogs added in v1.5.0 

func (t *Trace) AppendPluginLogs(logs []PluginLogEntry)

AppendPluginLogs appends plugin log entries to the trace in a thread-safe manner.

func (*Trace) ApplyRedactionReplacements added in v1.6.4 

func (t *Trace) ApplyRedactionReplacements()

ApplyRedactionReplacements redacts content attributes on every span in the trace and clears the replacement map.

func (*Trace) GetAttribute added in v1.5.19 

func (t *Trace) GetAttribute(key string) (any, bool)

GetAttribute retrieves a trace-level attribute in a thread-safe manner. The second return value reports whether the key was present.

func (*Trace) GetRequestID added in v1.5.0 

func (t *Trace) GetRequestID() string

GetRequestID retrieves the request ID from the trace

func (*Trace) GetSpan added in v1.3.0 

func (t *Trace) GetSpan(spanID string) *Span

GetSpan retrieves a span by ID

func (*Trace) Reset added in v1.3.0 

func (t *Trace) Reset()

Reset clears the trace for reuse from pool

func (*Trace) SetAttribute added in v1.5.19 

func (t *Trace) SetAttribute(key string, value any)

SetAttribute sets a trace-level attribute in a thread-safe manner

func (*Trace) SetRedactionReplacements added in v1.6.4 

func (t *Trace) SetRedactionReplacements(phase RedactionPhase, replacements map[string]string)

SetRedactionReplacements merges connector-facing raw-to-placeholder replacements on the trace.

func (*Trace) SetRequestHeaders added in v1.5.17 

func (t *Trace) SetRequestHeaders(headers map[string]string)

SetRequestHeaders sets the captured request headers for the trace.

func (*Trace) SetRequestID added in v1.5.0 

func (t *Trace) SetRequestID(requestID string)

SetRequestID sets the request ID for the trace

func (*Trace) SnapshotForExport added in v1.6.4 

func (t *Trace) SnapshotForExport() *Trace

SnapshotForExport returns a copy of the trace that is safe for concurrent read-only use by observability exporters (Datadog, OTEL, ...) while the original trace's spans may still be mutated.

Trace- and span-level attribute maps are cloned under their respective locks, so an exporter iterating the returned maps can never race a late writer — e.g. streaming span finalization (completeDeferredSpan) or redaction replacement — that legitimately holds the span lock. Without this, an exporter iterating the live span.Attributes map (which cannot take the unexported span lock) triggers a fatal "concurrent map iteration and map write" that recover() cannot catch.

Span pointer identity is preserved *within* the returned trace: RootSpan and the entries of Spans refer to the same copied *Span values, so pointer-equality checks (e.g. span == finalAttempt) still work against the snapshot's own spans. Attribute values are copied by reference and must be treated as read-only.

type Tracer added in v1.3.0 

type Tracer interface {
	// CreateTrace creates a new trace with optional parent ID and returns the trace ID.
	// The parentID can be extracted from W3C traceparent headers for distributed tracing.
	// The requestID is optional and can be used to identify the request.
	CreateTrace(parentID string, requestID ...string) string

	// EndTrace completes a trace and returns the trace data for observation/export.
	// After this call, the trace is removed from active tracking and returned for cleanup.
	// Returns nil if trace not found.
	EndTrace(traceID string) *Trace

	// StartSpan creates a new span as a child of the current span in context.
	// Returns updated context with new span and a handle for the span.
	// The context should be used for subsequent operations to maintain span hierarchy.
	StartSpan(ctx context.Context, name string, kind SpanKind) (context.Context, SpanHandle)

	// EndSpan completes a span with status and optional message.
	// Should be called when the operation represented by the span is complete.
	EndSpan(handle SpanHandle, status SpanStatus, statusMsg string)

	// SetAttribute sets an attribute on the span.
	// Attributes provide additional context about the operation.
	SetAttribute(handle SpanHandle, key string, value any)

	// GetSpanHandleByID retrieves a span handle for the given trace and span ID.
	// If spanID is nil, returns a handle for the trace's root span.
	GetSpanHandleByID(traceID string, spanID *string) SpanHandle

	// AddEvent adds a timestamped event to the span.
	// Events represent discrete occurrences during the span's lifetime.
	AddEvent(handle SpanHandle, name string, attrs map[string]any)

	// PopulateLLMRequestAttributes populates all LLM-specific request attributes on the span.
	// This includes model parameters, input messages, temperature, max tokens, etc.
	PopulateLLMRequestAttributes(handle SpanHandle, req *BifrostRequest)

	// PopulateLLMResponseAttributes populates all LLM-specific response attributes on the span.
	// This includes output messages, tokens, usage stats, and error information if present.
	PopulateLLMResponseAttributes(ctx *BifrostContext, handle SpanHandle, resp *BifrostResponse, err *BifrostError)

	// StoreDeferredSpan stores a span handle for later completion (used for streaming requests).
	// The span handle is stored keyed by trace ID so it can be retrieved when the stream completes.
	StoreDeferredSpan(traceID string, handle SpanHandle)

	// GetDeferredSpanHandle retrieves a deferred span handle by trace ID.
	// Returns nil if no deferred span exists for the given trace ID.
	GetDeferredSpanHandle(traceID string) SpanHandle

	// ClearDeferredSpan removes the deferred span handle for a trace ID.
	// Should be called after the deferred span has been completed.
	ClearDeferredSpan(traceID string)

	// GetDeferredSpanID returns the span ID for the deferred span.
	// Returns empty string if no deferred span exists.
	GetDeferredSpanID(traceID string) string

	// AddStreamingChunk accumulates a streaming chunk for the deferred span.
	// Pass the full BifrostResponse to capture content, tool calls, reasoning, etc.
	// This is called for each streaming chunk to build up the complete response.
	AddStreamingChunk(traceID string, response *BifrostResponse)

	// GetAccumulatedChunks returns the accumulated response, TTFT, and chunk count for a deferred span.
	// The response is built from the streaming accumulator during the final ProcessStreamingChunk call.
	// Returns nil response if no plugin has called ProcessStreamingChunk (callers should nil-check).
	// Returns nil, 0, 0 if no accumulated data exists.
	GetAccumulatedChunks(traceID string) (response *BifrostResponse, ttftNs int64, chunkCount int)

	// CreateStreamAccumulator creates a new stream accumulator for the given trace ID.
	// This should be called at the start of a streaming request.
	CreateStreamAccumulator(traceID string, startTime time.Time)

	// CleanupStreamAccumulator removes the stream accumulator for the given trace ID.
	// This should be called after the streaming request is complete.
	CleanupStreamAccumulator(traceID string)

	// ProcessStreamingChunk processes a streaming chunk and accumulates it.
	// Returns the accumulated result. IsFinal will be true when the stream is complete.
	// This method is used by plugins to access accumulated streaming data.
	// The ctx parameter must contain the stream end indicator for proper final chunk detection.
	ProcessStreamingChunk(ctx *BifrostContext, traceID string, isFinalChunk bool, result *BifrostResponse, err *BifrostError) *StreamAccumulatorResult

	// PauseStream marks the streaming response identified by traceID as paused.
	// While paused, post-processed chunks are buffered (not delivered) but plugin
	// hooks continue to fire. Idempotent. No-op if no active stream is found.
	PauseStream(traceID string)

	// ResumeStream resumes a previously paused stream. Buffered chunks are flushed
	// to the client in order, then live streaming continues. Idempotent.
	ResumeStream(traceID string)

	// EndStream terminates the stream. If err is non-nil, it is delivered to the
	// client as a final error chunk after any buffered chunks are flushed. After
	// EndStream, all further chunks for this stream are dropped (post-hooks still
	// run but no client delivery happens). Idempotent.
	EndStream(traceID string, err *BifrostError)

	// WaitForFlusher blocks until the gate flusher goroutine for traceID has
	// fully drained and exited. Provider stream goroutines call this from
	// their deferred close so the consumer-facing channel is not closed while
	// the gate still has buffered chunks pending delivery (e.g. a stream that
	// was paused by a plugin and not yet resumed). Returns immediately when no
	// flusher is active.
	WaitForFlusher(traceID string)

	// IsStreamEnded reports whether the gate for traceID is in the Ended
	// state. Returns false if no accumulator exists for traceID.
	IsStreamEnded(traceID string) bool

	// IsStreamPaused reports whether the gate for traceID is currently
	// Paused. Returns false if no accumulator exists for traceID.
	IsStreamPaused(traceID string) bool

	// GetAccumulatedResponse returns the *current* accumulated response
	// snapshot for traceID — built on demand from chunks accumulated so far,
	// not from the post-final-chunk stored copy. Useful for plugins that need
	// to inspect the assembled output mid-stream (e.g. while paused). Returns
	// nil if no accumulator exists, no chunks have been accumulated yet, or
	// the stream type cannot be determined.
	GetAccumulatedResponse(traceID string) *BifrostResponse

	// GateSend is called by stream producers (provider helpers) instead of writing
	// directly to the response channel. It implements the pause/resume/end gate:
	//   - Active state: chunk is forwarded to ch (with ctx.Done() guard)
	//   - Paused state: chunk is buffered for later replay
	//   - Ended state:  chunk is dropped
	// Final chunks (isFinal) and hard provider errors (isHardErr) bypass the gate
	// and force-flush + transition to Ended.
	// Returns true if the chunk was handled (delivered or buffered), false if the
	// caller should stop sending (ctx done or stream ended).
	GateSend(traceID string, chunk *BifrostStreamChunk, isFinal, isHardErr bool, ch chan *BifrostStreamChunk, ctx *BifrostContext) bool

	// AttachPluginLogs appends plugin log entries to the trace identified by traceID.
	// Thread-safe. Should be called after plugin hooks complete, before trace completion.
	AttachPluginLogs(traceID string, logs []PluginLogEntry)

	// SetTraceRedactionReplacements stores phase-scoped connector-facing replacements on a trace.
	SetTraceRedactionReplacements(traceID string, phase RedactionPhase, replacements map[string]string)

	// CompleteAndFlushTrace ends a trace, exports it to observability plugins, and
	// releases the trace resources. Used by transports that bypass normal HTTP trace completion.
	CompleteAndFlushTrace(traceID string)

	// Stop releases resources associated with the tracer.
	// Should be called during shutdown to stop background goroutines.
	Stop()
}

Tracer defines the interface for distributed tracing in Bifrost. Implementations can be injected via BifrostConfig to enable automatic instrumentation. The interface is designed to be minimal and implementation-agnostic.

func DefaultTracer added in v1.3.0 

func DefaultTracer() Tracer

DefaultTracer returns a no-op tracer for use when tracing is disabled.

All Tracer-mediated features are inert under DefaultTracer: trace/span creation, accumulator, deferred spans, AND the streaming pause/resume/end gate. Callers who need real gate behavior (chunk buffering on pause, in-order replay on resume, terminal-error delivery on end) MUST inject a real Tracer via the Bifrost config — typically `framework/streaming/Accumulator`, which is what production deployments wire in. `core/schemas` cannot import `framework/streaming` (would be a circular dep), so the gate impl cannot live here. This is the same fall-back contract every other Tracer feature follows when tracing is disabled.

type TranscriptionAdditionalFormat added in v1.2.24 

type TranscriptionAdditionalFormat struct {
	Format                      TranscriptionExportOptions `json:"format"`
	IncludeSpeakers             *bool                      `json:"include_speakers,omitempty"`
	IncludeTimestamps           *bool                      `json:"include_timestamps,omitempty"`
	SegmentOnSilenceLongerThanS *float64                   `json:"segment_on_silence_longer_than_s,omitempty"`
	MaxSegmentDurationS         *float64                   `json:"max_segment_duration_s,omitempty"`
	MaxSegmentChars             *int                       `json:"max_segment_chars,omitempty"`
	MaxCharactersPerLine        *int                       `json:"max_characters_per_line,omitempty"`
}

type TranscriptionDiarizedSegment added in v1.6.4 

type TranscriptionDiarizedSegment struct {
	ID      string  `json:"id"`
	Type    string  `json:"type"` // Always "transcript.text.segment"
	Speaker string  `json:"speaker"`
	Start   float64 `json:"start"`
	End     float64 `json:"end"`
	Text    string  `json:"text"`
}

TranscriptionDiarizedSegment represents a speaker-diarized segment of transcript text, as returned by response_format=diarized_json (e.g. OpenAI's gpt-4o-transcribe-diarize). Unlike TranscriptionSegment, the segment id here is a string (e.g. "seg_154") and there is no seek/tokens/logprob data.

type TranscriptionExportOptions added in v1.2.24 

type TranscriptionExportOptions string
const (
	TranscriptionExportOptionsSegmentedJson TranscriptionExportOptions = "segmented_json"
	TranscriptionExportOptionsDocx          TranscriptionExportOptions = "docx"
	TranscriptionExportOptionsPdf           TranscriptionExportOptions = "pdf"
	TranscriptionExportOptionsTxt           TranscriptionExportOptions = "txt"
	TranscriptionExportOptionsHtml          TranscriptionExportOptions = "html"
	TranscriptionExportOptionsSrt           TranscriptionExportOptions = "srt"
)

type TranscriptionInput added in v1.1.11 

type TranscriptionInput struct {
	File     []byte `json:"file"`
	Filename string `json:"filename,omitempty"` // Original filename, used to preserve file format extension
}

type TranscriptionLogProb added in v1.1.11 

type TranscriptionLogProb struct {
	Token   string  `json:"token"`
	LogProb float64 `json:"logprob"`
	Bytes   []int   `json:"bytes"`
}

TranscriptionLogProb represents log probability information for transcription

type TranscriptionParameters added in v1.2.0 

type TranscriptionParameters struct {
	Language               *string  `json:"language,omitempty"`
	Prompt                 *string  `json:"prompt,omitempty"`
	ResponseFormat         *string  `json:"response_format,omitempty"`         // Default is "json"
	Temperature            *float64 `json:"temperature,omitempty"`             // Sampling temperature (0.0-1.0)
	TimestampGranularities []string `json:"timestamp_granularities,omitempty"` // "word" and/or "segment"; requires response_format=verbose_json
	Include                []string `json:"include,omitempty"`                 // Additional response info (e.g., logprobs)
	Format                 *string  `json:"file_format,omitempty"`             // Type of file, not required in openai, but required in gemini
	MaxLength              *int     `json:"max_length,omitempty"`              // Maximum length of the transcription used by HuggingFace
	MinLength              *int     `json:"min_length,omitempty"`              // Minimum length of the transcription used by HuggingFace
	MaxNewTokens           *int     `json:"max_new_tokens,omitempty"`          // Maximum new tokens to generate used by HuggingFace
	MinNewTokens           *int     `json:"min_new_tokens,omitempty"`          // Minimum new tokens to generate used by HuggingFace

	// Elevenlabs-specific fields
	AdditionalFormats []TranscriptionAdditionalFormat `json:"additional_formats,omitempty"`
	WebhookMetadata   interface{}                     `json:"webhook_metadata,omitempty"`

	// Dynamic parameters that can be provider-specific, they are directly
	// added to the request as is.
	ExtraParams map[string]interface{} `json:"-"`
}

type TranscriptionSegment added in v1.1.11 

type TranscriptionSegment struct {
	ID               int     `json:"id"`
	Seek             int     `json:"seek"`
	Start            float64 `json:"start"`
	End              float64 `json:"end"`
	Text             string  `json:"text"`
	Tokens           []int   `json:"tokens"`
	Temperature      float64 `json:"temperature"`
	AvgLogProb       float64 `json:"avg_logprob"`
	CompressionRatio float64 `json:"compression_ratio"`
	NoSpeechProb     float64 `json:"no_speech_prob"`
}

TranscriptionSegment represents segment-level transcription information

type TranscriptionStreamResponseType added in v1.2.7 

type TranscriptionStreamResponseType string
const (
	TranscriptionStreamResponseTypeDelta TranscriptionStreamResponseType = "transcript.text.delta"
	TranscriptionStreamResponseTypeDone  TranscriptionStreamResponseType = "transcript.text.done"
)

type TranscriptionUsage added in v1.1.11 

type TranscriptionUsage struct {
	Type              string                               `json:"type"` // "tokens" or "duration"
	InputTokens       *int                                 `json:"input_tokens,omitempty"`
	InputTokenDetails *TranscriptionUsageInputTokenDetails `json:"input_token_details,omitempty"`
	OutputTokens      *int                                 `json:"output_tokens,omitempty"`
	TotalTokens       *int                                 `json:"total_tokens,omitempty"`
	Seconds           *float64                             `json:"seconds,omitempty"` // For duration-based usage (fractional, e.g. 523.5)
}

TranscriptionUsage represents usage information for transcription

type TranscriptionUsageInputTokenDetails added in v1.2.7 

type TranscriptionUsageInputTokenDetails struct {
	TextTokens  int `json:"text_tokens"`
	AudioTokens int `json:"audio_tokens"`
}

type TranscriptionWord added in v1.1.11 

type TranscriptionWord struct {
	Word    string  `json:"word"`
	Start   float64 `json:"start"`
	End     float64 `json:"end"`
	Speaker *string `json:"speaker,omitempty"` // Speaker label/id when diarization is enabled (e.g. ElevenLabs' speaker_id)
}

TranscriptionWord represents word-level timing information

type URLTypeInfo added in v1.2.0 

type URLTypeInfo struct {
	Type                 ImageContentType
	MediaType            *string
	DataURLWithoutPrefix *string // URL without the prefix (eg data:image/png;base64,iVBORw0KGgo...)
}

URLTypeInfo contains extracted information about a URL

func ExtractURLTypeInfo added in v1.2.0 

func ExtractURLTypeInfo(sanitizedURL string) URLTypeInfo

ExtractURLTypeInfo extracts type and media type information from a sanitized URL. For data URLs, it parses the media type and encoding. For regular URLs, it attempts to infer the media type from the file extension.

type UserAgentIdentifiers added in v1.5.3 

type UserAgentIdentifiers []string

UserAgentIdentifiers lists substrings that may appear in User-Agent for a given integration. Versions of the same client may use different strings; Matches checks any of them.

func (UserAgentIdentifiers) Matches added in v1.5.3 

func (ids UserAgentIdentifiers) Matches(userAgent string) bool

Matches reports whether userAgent contains any identifier (case-insensitive substring match).

func (UserAgentIdentifiers) String added in v1.5.3 

func (ids UserAgentIdentifiers) String() string

String returns the first identifier for logging and tests that need a canonical sample value.

type VLLMKeyConfig added in v1.4.4 

type VLLMKeyConfig struct {
	URL       SecretVar `json:"url"`        // VLLM server base URL (required, supports env. prefix)
	ModelName string    `json:"model_name"` // Exact model name served on this VLLM instance (used for key selection)
}

VLLMKeyConfig represents the vLLM-specific key configuration. It allows each key to target a different vLLM server URL and model name, enabling per-key routing and round-robin load balancing across multiple vLLM instances.

type VaultPathKeyer added in v1.6.0 

type VaultPathKeyer interface {
	VaultPathKey() string
}

VaultPathKeyer is implemented by GORM models that own vault secrets. The global vault callback uses VaultPathKey() (together with the table name) to build the base path for auto-store and auto-remove, so individual models do not need to wire StoreOwnedVaultSecretVars / RemoveOwnedVaultSecretVars manually.

type VertexAliasCfg added in v1.5.19 

type VertexAliasCfg struct {
	ProjectID         *SecretVar `json:"-"` // superseded by AliasConfig.ProjectID; not (de)serialized to avoid colliding with the top-level project_id
	ProjectNumber     *SecretVar `json:"project_number,omitempty"`
	ForceSingleRegion *bool      `json:"force_single_region,omitempty"`
}

VertexAliasCfg holds Vertex-specific overrides that apply to a single alias.

Deprecated for ProjectID: the per-alias project override now lives on the shared top-level AliasConfig.ProjectID field (see below), so one alias key can scope any provider (Vertex, Bedrock, Bedrock Mantle) to a project without a JSON field-name collision between the embedded sub-configs. ProjectID is kept here only so Go code that constructs VertexAliasCfg directly keeps compiling; the Vertex resolver reads AliasConfig.ProjectID first and falls back to this.

type VertexKeyConfig added in v1.1.9 

type VertexKeyConfig struct {
	ProjectID       SecretVar `json:"project_id"`
	ProjectNumber   SecretVar `json:"project_number"`
	Region          SecretVar `json:"region"`
	AuthCredentials SecretVar `json:"auth_credentials"`
	// ForceSingleRegion pins requests to the configured region and disables automatic promotion of
	// multi-region-only models to a multi-region pool endpoint (e.g. for provisioned throughput).
	ForceSingleRegion bool `json:"force_single_region,omitempty"`
}

VertexKeyConfig represents the Vertex-specific configuration. It contains Vertex-specific settings required for authentication and service access.

type VideoCreateError added in v1.4.4 

type VideoCreateError struct {
	Code    string `json:"code,omitempty"`
	Message string `json:"message,omitempty"`
}

VideoCreateError is the error payload when video generation fails.

type VideoDownloadVariant added in v1.4.4 

type VideoDownloadVariant string
const (
	VideoDownloadVariantVideo       VideoDownloadVariant = "video"
	VideoDownloadVariantThumbnail   VideoDownloadVariant = "thumbnail"
	VideoDownloadVariantSpriteSheet VideoDownloadVariant = "sprite_sheet"
)

type VideoGenerationInput added in v1.4.4 

type VideoGenerationInput struct {
	Prompt         string  `json:"prompt"`
	InputReference *string `json:"input_reference,omitempty"` // Primary image for image-to-video (OpenAI-compatible)
}

type VideoGenerationParameters added in v1.4.4 

type VideoGenerationParameters struct {
	Seconds *string `json:"seconds,omitempty"`
	Size    string  `json:"size,omitempty"`

	NegativePrompt *string        `json:"negative_prompt,omitempty"`
	Seed           *int           `json:"seed,omitempty"`
	VideoURI       *string        `json:"video_uri,omitempty"` // for video to video generation
	Audio          *bool          `json:"audio,omitempty"`
	ExtraParams    map[string]any `json:"-"`
}

type VideoLogParams added in v1.4.4 

type VideoLogParams struct {
	VideoID string `json:"video_id"`
}

type VideoObject added in v1.4.4 

type VideoObject struct {
	ID                 string            `json:"id"`
	Object             string            `json:"object"` // always "video"
	Model              string            `json:"model"`
	Status             VideoStatus       `json:"status"`
	CreatedAt          int64             `json:"created_at"`
	CompletedAt        *int64            `json:"completed_at,omitempty"`
	ExpiresAt          *int64            `json:"expires_at,omitempty"`
	Progress           *float64          `json:"progress,omitempty"`
	Prompt             string            `json:"prompt"`
	RemixedFromVideoID *string           `json:"remixed_from_video_id,omitempty"`
	Seconds            *string           `json:"seconds"`
	Size               string            `json:"size"`
	Error              *VideoCreateError `json:"error,omitempty"`
}

type VideoOutput added in v1.4.4 

type VideoOutput struct {
	Type        VideoOutputType `json:"type"` // "url" | "base64"
	URL         *string         `json:"url,omitempty"`
	Base64Data  *string         `json:"base64,omitempty"`
	ContentType string          `json:"content_type"`
}

type VideoOutputType added in v1.4.4 

type VideoOutputType string
const (
	VideoOutputTypeBase64 VideoOutputType = "base64"
	VideoOutputTypeURL    VideoOutputType = "url"
)

type VideoReferenceInput added in v1.4.4 

type VideoReferenceInput struct {
	Image         []byte `json:"image"`                    // Image bytes
	ReferenceType string `json:"reference_type,omitempty"` // "style" or "asset" (Gemini: "REFERENCE_TYPE_STYLE" or "REFERENCE_TYPE_ASSET")
}

VideoReferenceInput represents a reference image for video generation

type VideoResult added in v1.2.19 

type VideoResult struct {
	URL             string   `json:"url"`
	ThumbnailURL    *string  `json:"thumbnail_url,omitempty"`
	ThumbnailWidth  *int     `json:"thumbnail_width,omitempty"`
	ThumbnailHeight *int     `json:"thumbnail_height,omitempty"`
	Duration        *float64 `json:"duration,omitempty"`
}

type VideoStatus added in v1.4.4 

type VideoStatus string

VideoStatus is the lifecycle status of a video job. 

const (
	VideoStatusQueued     VideoStatus = "queued"
	VideoStatusInProgress VideoStatus = "in_progress"
	VideoStatusCompleted  VideoStatus = "completed"
	VideoStatusFailed     VideoStatus = "failed"
)

type VisibilityFilter added in v1.5.11 

type VisibilityFilter struct {
	UserIDs         *[]string
	TeamIDs         *[]string
	OwnTeamIDs      *[]string
	VirtualKeyIDs   *[]string
	RoutingScopes   *[]RoutingScopeMatch
	RoleIDs         *[]uint
	BusinessUnitIDs *[]string
	CustomerIDs     *[]string
}

VisibilityFilter restricts list and read queries to rows the caller is allowed to see.

Semantics: a row is visible iff ANY non-nil dimension matches its corresponding column. A nil dimension means that dimension imposes no restriction — but other non-nil dimensions still apply. An empty slice means the dimension matches no rows (callers that need "everything except this dimension" should use a nil pointer instead). A filter with all dimensions nil is equivalent to "no filter" (full visibility).

A VisibilityFilter is the response shape of the per-request provider closure (see VisibilityFilterProvider). The provider returns a filter with only the dimensions relevant to the requested Entity populated; every other dimension is nil.

Each dimension lines up with a real column on the target table:

  • UserIDs → table.user_id OR governance_users.id (when scoping the users table itself)
  • TeamIDs → table.team_id (logs / prompts / VKs)
  • OwnTeamIDs → governance_teams.id (when scoping the teams table itself; populated from the principal's own team membership for both own-data and team-data)
  • VirtualKeyIDs → governance_virtual_keys.id (when scoping the virtual_keys table itself) or table.virtual_key_id (when scoping logs)
  • RoutingScopes → routing_rules.(scope, scope_id) tuples
  • RoleIDs → enterprise_governance_roles.id
  • BusinessUnitIDs → governance_business_units.id
  • CustomerIDs → governance_customers.id

func FilterForEntity added in v1.5.11 

func FilterForEntity(ctx context.Context, entity Entity) *VisibilityFilter

FilterForEntity is a convenience that fetches the provider from ctx and resolves the filter for entity in one call. Returns nil when no provider is present (full visibility) or when the provider returns nil for the requested entity (e.g. all-data scope).

func (*VisibilityFilter) IsUnrestricted added in v1.5.11 

func (f *VisibilityFilter) IsUnrestricted() bool

IsUnrestricted reports whether the filter has no dimensions set, in which case query builders may skip applying any WHERE clause. A nil receiver is also unrestricted, so a nil filter on context is equivalent to no filter.

type VisibilityFilterProvider added in v1.5.11 

type VisibilityFilterProvider func(Entity) *VisibilityFilter

VisibilityFilterProvider returns the visibility filter to apply for a given Entity. Returning nil signals full visibility (no filter applied).

The provider is set on the request context by the upstream (enterprise) middleware. Each call returns only the dimensions of the VisibilityFilter that the requested Entity's table can consume — for example, EntityCustomer returns only CustomerIDs populated; every other field is nil. This lets OSS query helpers stay agnostic of how the filter is computed and keeps per-request allocations minimal.

func VisibilityFilterProviderFromContext added in v1.5.11 

func VisibilityFilterProviderFromContext(ctx context.Context) VisibilityFilterProvider

VisibilityFilterProviderFromContext returns the provider closure stashed on the context, or nil when none is present (background jobs, internal queries, requests that bypassed the upstream middleware, OSS-only deployments without DAC). A nil provider is equivalent to full visibility — query builders apply no WHERE clause.

type VoiceConfig added in v1.1.11 

type VoiceConfig struct {
	Speaker string `json:"speaker"`
	Voice   string `json:"voice"`
}

type WSPoolConfig added in v1.4.8 

type WSPoolConfig struct {
	MaxIdlePerKey                int `json:"max_idle_per_key"`
	MaxTotalConnections          int `json:"max_total_connections"`
	IdleTimeoutSeconds           int `json:"idle_timeout_seconds"`
	MaxConnectionLifetimeSeconds int `json:"max_connection_lifetime_seconds"`
}

WSPoolConfig configures the upstream WebSocket connection pool.

func (*WSPoolConfig) CheckAndSetDefaults added in v1.4.8 

func (c *WSPoolConfig) CheckAndSetDefaults()

CheckAndSetDefaults fills in default values for WSPoolConfig.

type WebSocketCapableProvider added in v1.4.8 

type WebSocketCapableProvider interface {
	// SupportsWebSocketMode returns true if the provider supports the Responses API WebSocket Mode.
	SupportsWebSocketMode() bool
	// WebSocketResponsesURL returns the WebSocket URL for the Responses API.
	WebSocketResponsesURL(key Key) string
	// WebSocketHeaders returns the headers required for the upstream WebSocket connection.
	WebSocketHeaders(key Key) map[string]string
}

WebSocketCapableProvider is an optional interface that providers can implement to indicate support for the OpenAI Responses API WebSocket Mode. Checked via type assertion: provider.(WebSocketCapableProvider). Providers that implement this interface will have native WS upstream connections instead of the HTTP bridge fallback for Responses WS mode.

type WebSocketConfig added in v1.4.8 

type WebSocketConfig struct {
	MaxConnections       int           `json:"max_connections_per_user"`
	TranscriptBufferSize int           `json:"transcript_buffer_size"`
	Pool                 *WSPoolConfig `json:"pool,omitempty"`
}

WebSocketConfig provides optional tuning for WebSocket gateway features. WebSocket is always enabled. These fields allow overriding the high defaults.

func (*WebSocketConfig) CheckAndSetDefaults added in v1.4.8 

func (c *WebSocketConfig) CheckAndSetDefaults()

CheckAndSetDefaults fills in default values for WebSocketConfig.

type WebSocketErrorBody added in v1.4.8 

type WebSocketErrorBody struct {
	Code    string `json:"code,omitempty"`
	Message string `json:"message,omitempty"`
	Param   string `json:"param,omitempty"`
}

WebSocketErrorBody is the error detail within a WebSocketErrorEvent.

type WebSocketErrorEvent added in v1.4.8 

type WebSocketErrorEvent struct {
	Type   WebSocketEventType  `json:"type"`
	Status int                 `json:"status,omitempty"`
	Error  *WebSocketErrorBody `json:"error,omitempty"`
}

WebSocketErrorEvent represents a server-sent error event over WebSocket.

type WebSocketEventType added in v1.4.8 

type WebSocketEventType string

WebSocketEventType represents event types in the Responses API WebSocket protocol. 

const (
	WSEventResponseCreate WebSocketEventType = "response.create"
	WSEventError          WebSocketEventType = "error"
)

type WebSocketResponsesEvent added in v1.4.8 

type WebSocketResponsesEvent struct {
	Type               WebSocketEventType `json:"type"`
	Model              string             `json:"model,omitempty"`
	Store              *bool              `json:"store,omitempty"`
	Input              json.RawMessage    `json:"input,omitempty"`
	Instructions       string             `json:"instructions,omitempty"`
	PreviousResponseID string             `json:"previous_response_id,omitempty"`
	Generate           *bool              `json:"generate,omitempty"`
	Tools              json.RawMessage    `json:"tools,omitempty"`
	ToolChoice         json.RawMessage    `json:"tool_choice,omitempty"`
	Temperature        *float64           `json:"temperature,omitempty"`
	TopP               *float64           `json:"top_p,omitempty"`
	MaxOutputTokens    *int               `json:"max_output_tokens,omitempty"`
	Reasoning          json.RawMessage    `json:"reasoning,omitempty"`
	Metadata           json.RawMessage    `json:"metadata,omitempty"`
	Text               json.RawMessage    `json:"text,omitempty"`
	Truncation         string             `json:"truncation,omitempty"`
}

WebSocketResponsesEvent represents a client-sent event over the Responses WebSocket connection. The payload mirrors the Responses API create body, with transport-specific fields (stream, background) omitted since they're implicit in the WebSocket context.

type WhiteList added in v1.5.0 

type WhiteList []string

WhiteList is a list of values that are allowed to be used. Semantics:

  • "*" (alone) means all values are allowed.
  • Empty list means nothing is allowed.
  • Non-empty list (without "*") means only the listed values are allowed.

This type is used generically for any field that needs whitelist behavior (e.g., allowed models, allowed tools).

func (WhiteList) Contains added in v1.5.0 

func (wl WhiteList) Contains(value string) bool

Contains reports whether value is in the whitelist. Returns true if value is in the list.

func (WhiteList) IsAllowed added in v1.5.0 

func (wl WhiteList) IsAllowed(value string) bool

IsAllowed reports whether value is in the whitelist. Returns true if value is in the list.

func (WhiteList) IsEmpty added in v1.5.0 

func (wl WhiteList) IsEmpty() bool

IsEmpty reports whether the whitelist has no entries.

func (WhiteList) IsRestricted added in v1.5.0 

func (wl WhiteList) IsRestricted() bool

IsRestricted reports whether the whitelist contains entries other than "*", meaning only the listed values are allowed.

func (WhiteList) IsUnrestricted added in v1.5.0 

func (wl WhiteList) IsUnrestricted() bool

IsUnrestricted reports whether the whitelist contains only "*", meaning all values are allowed.

func (WhiteList) Validate added in v1.5.0 

func (wl WhiteList) Validate() error

Validate checks that the whitelist is well-formed. Returns an error if "*" is present alongside other values, or if there are duplicate entries.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.