utils

package
v1.4.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 12, 2026 License: Apache-2.0 Imports: 25 Imported by: 0

Documentation

Overview

Package utils provides common utility functions used across different provider implementations. This file contains audio-related utility functions for format conversion.

Package providers implements various LLM providers and their utility functions. This file contains common utility functions used across different provider implementations.

Index

Constants

This section is empty.

Variables

View Source 
var UnsupportedSpeechStreamModels = []string{"tts-1", "tts-1-hd"}

Functions

func CheckAndDecodeBody 

func CheckAndDecodeBody(resp *fasthttp.Response) ([]byte, error)

CheckAndDecodeBody checks the content encoding and decodes the body accordingly. It returns a copy of the body to avoid race conditions when the response is released back to fasthttp's buffer pool.

func CheckAndGetRawRequestBody 

func CheckAndGetRawRequestBody(ctx context.Context, request RequestBodyGetter) ([]byte, bool)

CheckAndGetRawRequestBody checks if the raw request body should be used, and returns it if it exists.

func CheckAndSetDefaultProvider added in v1.3.12 

func CheckAndSetDefaultProvider(ctx *schemas.BifrostContext, defaultProvider schemas.ModelProvider) schemas.ModelProvider

CheckAndSetDefaultProvider checks if the default provider should be used based on the context. It returns the default provider if it should be used, otherwise it returns an empty string. Checks if the direct key is set in the context, or if key selection is skipped. Or if the available providers are set in the context and the default provider is in the list.

func CheckContextAndGetRequestBody 

func CheckContextAndGetRequestBody(ctx context.Context, request RequestBodyGetter, requestConverter RequestBodyConverter, providerType schemas.ModelProvider) ([]byte, *schemas.BifrostError)

CheckContextAndGetRequestBody checks if the raw request body should be used, and returns it if it exists.

func CheckOperationAllowed 

func CheckOperationAllowed(defaultProvider schemas.ModelProvider, config *schemas.CustomProviderConfig, operation schemas.RequestType) *schemas.BifrostError

CheckOperationAllowed enforces per-op gating using schemas.Operation. Behavior: - If no gating is configured (config == nil or AllowedRequests == nil), the operation is allowed. - If gating is configured, returns an error when the operation is not explicitly allowed.

func ConfigureProxy 

func ConfigureProxy(client *fasthttp.Client, proxyConfig *schemas.ProxyConfig, logger schemas.Logger) *fasthttp.Client

ConfigureProxy sets up a proxy for the fasthttp client based on the provided configuration. It supports HTTP, SOCKS5, and environment-based proxy configurations. Returns the configured client or the original client if proxy configuration is invalid.

func ConvertPCMToWAV added in v1.2.31 

func ConvertPCMToWAV(pcmData []byte, config PCMConfig) ([]byte, error)

ConvertPCMToWAV converts raw PCM audio data to WAV format The PCM data is expected to be in signed little-endian format (s16le for 16-bit)

func CreateBifrostChatCompletionChunkResponse 

func CreateBifrostChatCompletionChunkResponse(
	id string,
	usage *schemas.BifrostLLMUsage,
	finishReason *string,
	currentChunkIndex int,
	requestType schemas.RequestType,
	providerName schemas.ModelProvider,
	model string,
) *schemas.BifrostChatResponse

CreateBifrostChatCompletionChunkResponse creates a bifrost chat completion chunk response.

func CreateBifrostTextCompletionChunkResponse 

func CreateBifrostTextCompletionChunkResponse(
	id string,
	usage *schemas.BifrostLLMUsage,
	finishReason *string,
	currentChunkIndex int,
	requestType schemas.RequestType,
	providerName schemas.ModelProvider,
	model string,
) *schemas.BifrostTextCompletionResponse

CreateBifrostTextCompletionChunkResponse creates a bifrost text completion chunk response.

func DetectAudioMimeType added in v1.2.40 

func DetectAudioMimeType(audioData []byte) string

DetectAudioMimeType attempts to detect the MIME type from audio file headers. Supports detection of: WAV, MP3, AIFF, AAC, OGG Vorbis, and FLAC formats.

func EnrichError added in v1.3.9 

func EnrichError(
	ctx *schemas.BifrostContext,
	bifrostErr *schemas.BifrostError,
	requestBody []byte,
	responseBody []byte,
	sendBackRawRequest bool,
	sendBackRawResponse bool,
) *schemas.BifrostError

EnrichError attaches the raw request and response to a BifrostError. Returns the request and response from provider embedded in BifrostError.ExtraFields.

func ExtractHTMLErrorMessage added in v1.2.39 

func ExtractHTMLErrorMessage(body []byte) string

ExtractHTMLErrorMessage extracts meaningful error information from an HTML response. It attempts to find error messages from title tags, headers, and visible text. UNUSED for now but could be useful in the future

func FileBytesToBase64DataURL added in v1.4.2 

func FileBytesToBase64DataURL(fileBytes []byte) string

FileBytesToBase64DataURL converts raw file bytes to base64 data URL format

func GetBifrostResponseForStreamResponse 

func GetBifrostResponseForStreamResponse(
	textCompletionResponse *schemas.BifrostTextCompletionResponse,
	chatResponse *schemas.BifrostChatResponse,
	responsesStreamResponse *schemas.BifrostResponsesStreamResponse,
	speechStreamResponse *schemas.BifrostSpeechStreamResponse,
	transcriptionStreamResponse *schemas.BifrostTranscriptionStreamResponse,
	imageGenerationStreamResponse *schemas.BifrostImageGenerationStreamResponse,
) *schemas.BifrostResponse

GetBifrostResponseForStreamResponse converts the provided responses to a bifrost response.

func GetBudgetTokensFromReasoningEffort added in v1.2.37 

func GetBudgetTokensFromReasoningEffort(
	effort string,
	minBudgetTokens int,
	maxTokens int,
) (int, error)

GetBudgetTokensFromReasoningEffort converts OpenAI reasoning effort into a reasoning token budget. effort ∈ {"none", "minimal", "low", "medium", "high"}

func GetPathFromContext 

func GetPathFromContext(ctx context.Context, defaultPath string) string

GetPathFromContext gets the path from the context, if it exists, otherwise returns the default path.

func GetProviderName 

func GetProviderName(defaultProvider schemas.ModelProvider, customConfig *schemas.CustomProviderConfig) schemas.ModelProvider

GetProviderName extracts the provider name from custom provider configuration. If a custom provider key is specified, it returns that; otherwise, it returns the default provider. Note: CustomProviderKey is internally set by Bifrost and should always match the provider name.

func GetRandomString added in v1.2.37 

func GetRandomString(length int) string

GetRandomString generates a random alphanumeric string of the given length.

func GetReasoningEffortFromBudgetTokens added in v1.2.37 

func GetReasoningEffortFromBudgetTokens(
	budgetTokens int,
	minBudgetTokens int,
	maxTokens int,
) string

GetReasoningEffortFromBudgetTokens maps a reasoning token budget to OpenAI reasoning effort. Valid values: none, low, medium, high

func GetRequestPath added in v1.2.21 

func GetRequestPath(ctx context.Context, defaultPath string, customProviderConfig *schemas.CustomProviderConfig, requestType schemas.RequestType) (string, bool)

GetRequestPath gets the request path from the context, if it exists, checking for path overrides in the custom provider config. It returns the resolved value and a boolean indicating whether the value is a full absolute URL. If the boolean is false, the returned string is a path (leading slash ensured).

func HandleMultipleListModelsRequests 

func HandleMultipleListModelsRequests(
	ctx *schemas.BifrostContext,
	keys []schemas.Key,
	request *schemas.BifrostListModelsRequest,
	listModelsByKey func(ctx *schemas.BifrostContext, key schemas.Key, request *schemas.BifrostListModelsRequest) (*schemas.BifrostListModelsResponse, *schemas.BifrostError),
) (*schemas.BifrostListModelsResponse, *schemas.BifrostError)

HandleMultipleListModelsRequests handles multiple list models requests concurrently for different keys. It launches concurrent requests for all keys and waits for all goroutines to complete. It returns the aggregated response or an error if the request fails.

func HandleProviderAPIError 

func HandleProviderAPIError(resp *fasthttp.Response, errorResp any) *schemas.BifrostError

HandleProviderAPIError processes error responses from provider APIs. It attempts to unmarshal the error response and returns a BifrostError with the appropriate status code and error information. HTML detection only runs if JSON parsing fails to avoid expensive regex operations on responses that are almost certainly valid JSON. errorResp must be a pointer to the target struct for unmarshaling.

func HandleProviderResponse 

func HandleProviderResponse[T any](responseBody []byte, response *T, requestBody []byte, sendBackRawRequest bool, sendBackRawResponse bool) (rawRequest interface{}, rawResponse interface{}, bifrostErr *schemas.BifrostError)

HandleProviderResponse handles common response parsing logic for provider responses. It attempts to parse the response body into the provided response type and returns either the parsed response or a BifrostError if parsing fails. If sendBackRawResponse is true, it returns the raw response interface, otherwise nil. HTML detection only runs if JSON parsing fails to avoid expensive regex operations on responses that are almost certainly valid JSON.

func HandleStreamCancellation added in v1.3.8 

func HandleStreamCancellation(
	ctx *schemas.BifrostContext,
	postHookRunner schemas.PostHookRunner,
	responseChan chan *schemas.BifrostStreamChunk,
	provider schemas.ModelProvider,
	model string,
	requestType schemas.RequestType,
	logger schemas.Logger,
)

HandleStreamCancellation should be called when a streaming goroutine exits due to context cancellation. It ensures proper cleanup by: 1. Checking if StreamEndIndicator was already set (to avoid duplicate handling) 2. Setting StreamEndIndicator to true 3. Sending a cancellation error through PostHook chain

This is critical for the logging plugin to update log status from "processing" to "error" when a client disconnects mid-stream.

func HandleStreamControlSkip 

func HandleStreamControlSkip(bifrostErr *schemas.BifrostError) bool

HandleStreamControlSkip checks if the stream control should be skipped.

func HandleStreamTimeout added in v1.3.8 

func HandleStreamTimeout(
	ctx *schemas.BifrostContext,
	postHookRunner schemas.PostHookRunner,
	responseChan chan *schemas.BifrostStreamChunk,
	provider schemas.ModelProvider,
	model string,
	requestType schemas.RequestType,
	logger schemas.Logger,
)

HandleStreamTimeout should be called when a streaming goroutine exits due to context deadline exceeded. It ensures proper cleanup by: 1. Checking if StreamEndIndicator was already set (to avoid duplicate handling) 2. Setting StreamEndIndicator to true 3. Sending a timeout error through PostHook chain

This is critical for the logging plugin to update log status from "processing" to "error" when a request times out mid-stream.

func IsHTMLResponse added in v1.2.39 

func IsHTMLResponse(resp *fasthttp.Response, body []byte) bool

IsHTMLResponse checks if the response is HTML by examining the Content-Type header and/or the response body for HTML indicators.

func MakeRequestWithContext 

func MakeRequestWithContext(ctx context.Context, client *fasthttp.Client, req *fasthttp.Request, resp *fasthttp.Response) (time.Duration, *schemas.BifrostError)

MakeRequestWithContext makes a request with a context and returns the latency and error. IMPORTANT: This function does NOT truly cancel the underlying fasthttp network request if the context is done. The fasthttp client call will continue in its goroutine until it completes or times out based on its own settings. This function merely stops *waiting* for the fasthttp call and returns an error related to the context. Returns the request latency and any error that occurred.

func MergeExtraParams added in v1.4.0 

func MergeExtraParams(jsonMap map[string]interface{}, extraParams map[string]interface{})

MergeExtraParams merges extraParams into jsonMap, handling nested maps recursively.

func NewBifrostOperationError 

func NewBifrostOperationError(message string, err error, providerType schemas.ModelProvider) *schemas.BifrostError

NewBifrostOperationError creates a standardized error for bifrost operation errors. This helper reduces code duplication across providers that have bifrost operation errors.

func NewConfigurationError 

func NewConfigurationError(message string, providerType schemas.ModelProvider) *schemas.BifrostError

NewConfigurationError creates a standardized error for configuration errors. This helper reduces code duplication across providers that have configuration errors.

func NewProviderAPIError 

func NewProviderAPIError(message string, err error, statusCode int, providerType schemas.ModelProvider, errorType *string, eventID *string) *schemas.BifrostError

NewProviderAPIError creates a standardized error for provider API errors. This helper reduces code duplication across providers that have provider API errors.

func NewUnsupportedOperationError 

func NewUnsupportedOperationError(requestType schemas.RequestType, providerName schemas.ModelProvider) *schemas.BifrostError

NewUnsupportedOperationError creates a standardized error for unsupported operations. This helper reduces code duplication across providers that don't support certain operations.

func ParseAndSetRawRequest added in v1.2.37 

func ParseAndSetRawRequest(extraFields *schemas.BifrostResponseExtraFields, jsonBody []byte)

ParseAndSetRawRequest parses the raw request body and sets it in the extra fields.

func ProcessAndSendBifrostError 

func ProcessAndSendBifrostError(
	ctx *schemas.BifrostContext,
	postHookRunner schemas.PostHookRunner,
	bifrostErr *schemas.BifrostError,
	responseChan chan *schemas.BifrostStreamChunk,
	logger schemas.Logger,
)

ProcessAndSendBifrostError handles post-hook processing and sends the bifrost error to the channel. This utility reduces code duplication across streaming implementations by encapsulating the common pattern of running post hooks, handling errors, and sending responses with proper context cancellation handling. It also completes the deferred LLM span when the final chunk is sent (StreamEndIndicator is true).

func ProcessAndSendError 

func ProcessAndSendError(
	ctx *schemas.BifrostContext,
	postHookRunner schemas.PostHookRunner,
	err error,
	responseChan chan *schemas.BifrostStreamChunk,
	requestType schemas.RequestType,
	providerName schemas.ModelProvider,
	model string,
	logger schemas.Logger,
)

ProcessAndSendError handles post-hook processing and sends the error to the channel. This utility reduces code duplication across streaming implementations by encapsulating the common pattern of running post hooks, handling errors, and sending responses with proper context cancellation handling.

func ProcessAndSendResponse 

func ProcessAndSendResponse(
	ctx *schemas.BifrostContext,
	postHookRunner schemas.PostHookRunner,
	response *schemas.BifrostResponse,
	responseChan chan *schemas.BifrostStreamChunk,
)

ProcessAndSendResponse handles post-hook processing and sends the response to the channel. This utility reduces code duplication across streaming implementations by encapsulating the common pattern of running post hooks, handling errors, and sending responses with proper context cancellation handling. It also completes the deferred LLM span when the final chunk is sent (StreamEndIndicator is true).

func ProviderIsResponsesAPINative added in v1.2.19 

func ProviderIsResponsesAPINative(providerName schemas.ModelProvider) bool

func ProviderSendsDoneMarker added in v1.2.18 

func ProviderSendsDoneMarker(providerName schemas.ModelProvider) bool

ProviderSendsDoneMarker returns true if the provider sends the [DONE] marker in streaming responses. Some OpenAI-compatible providers (like Cerebras) don't send [DONE] and instead end the stream after sending the finish_reason. This function helps determine the correct stream termination logic.

func ReleaseStreamingResponse 

func ReleaseStreamingResponse(resp *fasthttp.Response)

ReleaseStreamingResponse releases a streaming response by draining the body stream and releasing the response.

func SendCreatedEventResponsesChunk 

func SendCreatedEventResponsesChunk(ctx *schemas.BifrostContext, postHookRunner schemas.PostHookRunner, provider schemas.ModelProvider, model string, startTime time.Time, responseChan chan *schemas.BifrostStreamChunk)

SendCreatedEventResponsesChunk sends a ResponsesStreamResponseTypeCreated event.

func SendInProgressEventResponsesChunk 

func SendInProgressEventResponsesChunk(ctx *schemas.BifrostContext, postHookRunner schemas.PostHookRunner, provider schemas.ModelProvider, model string, startTime time.Time, responseChan chan *schemas.BifrostStreamChunk)

SendInProgressEventResponsesChunk sends a ResponsesStreamResponseTypeInProgress event

func SetExtraHeaders 

func SetExtraHeaders(ctx context.Context, req *fasthttp.Request, extraHeaders map[string]string, skipHeaders []string)

SetExtraHeaders sets additional headers from NetworkConfig to the fasthttp request. This allows users to configure custom headers for their provider requests. Header keys are canonicalized using textproto.CanonicalMIMEHeaderKey to avoid duplicates. It accepts a list of headers (all canonicalized) to skip for security reasons. Headers are only set if they don't already exist on the request to avoid overwriting important headers.

func SetExtraHeadersHTTP 

func SetExtraHeadersHTTP(ctx context.Context, req *http.Request, extraHeaders map[string]string, skipHeaders []string)

SetExtraHeadersHTTP sets additional headers from NetworkConfig to the standard HTTP request. This allows users to configure custom headers for their provider requests. Header keys are canonicalized using textproto.CanonicalMIMEHeaderKey to avoid duplicates. It accepts a list of headers (all canonicalized) to skip for security reasons. Headers are only set if they don't already exist on the request to avoid overwriting important headers.

func SetLogger 

func SetLogger(l schemas.Logger)

SetLogger sets the logger for the provider utils (thread-safe).

func SetupStreamCancellation added in v1.3.8 

func SetupStreamCancellation(ctx context.Context, bodyStream io.Reader, logger schemas.Logger) (cleanup func())

SetupStreamCancellation spawns a goroutine that closes the body stream when the context is cancelled or deadline exceeded, unblocking any blocked Read/Scan operations. Returns a cleanup function that MUST be called when streaming is done to prevent the goroutine from closing the stream during normal operation. Works with both fasthttp's BodyStream() (io.Reader) and net/http's resp.Body (io.ReadCloser).

func ShouldSendBackRawRequest added in v1.2.37 

func ShouldSendBackRawRequest(ctx context.Context, defaultSendBackRawRequest bool) bool

ShouldSendBackRawRequest checks if the raw request should be sent back. Context overrides are intentionally restricted to asymmetric behavior: a context value can only promote false→true and will not override a true config to false, avoiding accidental suppression.

func ShouldSendBackRawResponse 

func ShouldSendBackRawResponse(ctx context.Context, defaultSendBackRawResponse bool) bool

ShouldSendBackRawResponse checks if the raw response should be sent back, and returns it if it exists.

Types

type JSONLParseResult added in v1.2.38 

type JSONLParseResult struct {
	Errors []schemas.BatchError
}

JSONLParseResult holds parsed items and any line-level errors encountered during parsing.

func ParseJSONL added in v1.2.38 

func ParseJSONL(data []byte, parseLine func(line []byte) error) JSONLParseResult

ParseJSONL parses JSONL data line by line, calling the provided callback for each line. It collects parse errors with line numbers rather than silently skipping failed lines. The callback receives the line bytes and returns an error if parsing fails. This function operates directly on byte slices to avoid unnecessary string conversions.

type PCMConfig added in v1.2.31 

type PCMConfig struct {
	SampleRate    int // Sample rate in Hz (e.g., 24000)
	NumChannels   int // Number of audio channels (1 = mono, 2 = stereo)
	BitsPerSample int // Bits per sample (e.g., 16)
}

PCMConfig holds the configuration for PCM audio data

func DefaultGeminiPCMConfig added in v1.2.31 

func DefaultGeminiPCMConfig() PCMConfig

DefaultGeminiPCMConfig returns the default PCM configuration for Gemini TTS Gemini TTS returns audio in PCM format with the following specs: - Format: signed 16-bit little-endian (s16le) - Sample rate: 24000 Hz - Channels: 1 (mono)

type RequestBodyConverter 

type RequestBodyConverter func() (RequestBodyWithExtraParams, error)

type RequestBodyGetter 

type RequestBodyGetter interface {
	GetRawRequestBody() []byte
}

type RequestBodyWithExtraParams added in v1.4.0 

type RequestBodyWithExtraParams interface {
	GetExtraParams() map[string]interface{}
}

type RequestMetadata added in v1.2.38 

type RequestMetadata struct {
	Provider    schemas.ModelProvider
	Model       string
	RequestType schemas.RequestType
}

RequestMetadata contains metadata about a request for error reporting. This struct is used to pass request context to parseError functions.

type SerialListHelper added in v1.2.39 

type SerialListHelper struct {
	Keys   []schemas.Key
	Cursor *schemas.SerialCursor
	Logger schemas.Logger
}

SerialListHelper manages serial key pagination for list operations. It ensures that all pages from one key are exhausted before moving to the next, guaranteeing only one API call per pagination request regardless of key count.

func NewSerialListHelper added in v1.2.39 

func NewSerialListHelper(keys []schemas.Key, encodedCursor *string, logger schemas.Logger) (*SerialListHelper, error)

NewSerialListHelper creates a new SerialListHelper from the provided keys and encoded cursor. If the cursor is empty or nil, pagination starts from the first key. If the cursor is invalid, an error is returned.

func (*SerialListHelper) BuildNextCursor added in v1.2.39 

func (h *SerialListHelper) BuildNextCursor(hasMore bool, nativeCursor string) (string, bool)

BuildNextCursor creates the cursor for the next pagination request. Parameters:

  • hasMore: whether the current key has more pages
  • nativeCursor: the native cursor returned by the current key's API

Returns:

  • encodedCursor: the encoded cursor for the next request (empty if all keys exhausted)
  • moreAvailable: true if there are more results available (either from current key or remaining keys)

func (*SerialListHelper) GetCurrentKey added in v1.2.39 

func (h *SerialListHelper) GetCurrentKey() (schemas.Key, string, bool)

GetCurrentKey returns the key to query and its native cursor. Returns (key, nativeCursor, true) if there's a key to query. Returns (Key{}, "", false) if all keys are exhausted.

func (*SerialListHelper) GetCurrentKeyIndex added in v1.2.39 

func (h *SerialListHelper) GetCurrentKeyIndex() int

GetCurrentKeyIndex returns the current key index being processed.

func (*SerialListHelper) HasMoreKeys added in v1.2.39 

func (h *SerialListHelper) HasMoreKeys() bool

HasMoreKeys returns true if there are more keys after the current one.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.