bifrost

package module
v1.4.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 12, 2026 License: Apache-2.0 Imports: 42 Imported by: 21

Documentation

Overview

Package bifrost provides the core implementation of the Bifrost system. Bifrost is a unified interface for interacting with various AI model providers, managing concurrent requests, and handling provider-specific configurations.

Package bifrost provides the core implementation of the Bifrost system.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func GetErrorMessage added in v1.2.20 

func GetErrorMessage(err *schemas.BifrostError) string

func GetIntFromContext added in v1.2.21 

func GetIntFromContext(ctx context.Context, key any) int

GetIntFromContext safely extracts an int value from context

func GetResponseFields added in v1.2.7 

func GetResponseFields(result *schemas.BifrostResponse, err *schemas.BifrostError) (requestType schemas.RequestType, provider schemas.ModelProvider, model string)

GetResponseFields extracts the request type, provider, and model from the result or error

func GetStringFromContext added in v1.2.21 

func GetStringFromContext(ctx context.Context, key any) string

GetStringFromContext safely extracts a string value from context

func GetTracerFromContext added in v1.3.4 

func GetTracerFromContext(ctx *schemas.BifrostContext) (schemas.Tracer, string, error)

func IsCodemodeTool added in v1.4.0 

func IsCodemodeTool(toolName string) bool

IsCodemodeTool returns true if the given tool name is a codemode tool.

func IsFinalChunk added in v1.1.31 

func IsFinalChunk(ctx *schemas.BifrostContext) bool

IsFinalChunk returns true if the given context is a final chunk.

func IsRateLimitErrorMessage added in v1.2.18 

func IsRateLimitErrorMessage(errorMessage string) bool

IsRateLimitErrorMessage checks if an error message indicates a rate limit issue

func IsStandardProvider added in v1.1.26 

func IsStandardProvider(providerKey schemas.ModelProvider) bool

IsStandardProvider reports whether providerKey is a built-in (non-custom) provider.

func IsStreamRequestType added in v1.1.31 

func IsStreamRequestType(reqType schemas.RequestType) bool

IsStreamRequestType returns true if the given request type is a stream request.

func IsSupportedBaseProvider added in v1.1.26 

func IsSupportedBaseProvider(providerKey schemas.ModelProvider) bool

IsSupportedBaseProvider reports whether providerKey is allowed as a base provider for custom providers.

func MarshalUnsafe added in v1.2.17 

func MarshalUnsafe(v any) string

MarshalUnsafe marshals the given value to a JSON string without escaping HTML characters. Returns empty string if marshaling fails.

func NewNoOpLogger added in v1.4.1 

func NewNoOpLogger() schemas.Logger

NewNoOpLogger creates a new NoOpLogger instance.

func Ptr added in v1.0.4 

func Ptr[T any](v T) *T

Ptr returns a pointer to the given value.

func RedactSensitiveString added in v1.2.30 

func RedactSensitiveString(s string) string

RedactSensitiveString redacts sensitive information in a string

func ValidateExternalURL added in v1.2.30 

func ValidateExternalURL(urlStr string) error

ValidateExternalURL validates a URL for security concerns (SSRF protection)

func WeightedRandomKeySelector added in v1.2.5 

func WeightedRandomKeySelector(ctx *schemas.BifrostContext, keys []schemas.Key, providerKey schemas.ModelProvider, model string) (schemas.Key, error)

Types

type Bifrost 

type Bifrost struct {
	MCPManager mcp.MCPManagerInterface // MCP integration manager (nil if MCP not configured)
	// contains filtered or unexported fields
}

Bifrost manages providers and maintains specified open channels for concurrent processing. It handles request routing, provider management, and response processing.

func Init 

func Init(ctx context.Context, config schemas.BifrostConfig) (*Bifrost, error)

Init initializes a new Bifrost instance with the given configuration. It sets up the account, plugins, object pools, and initializes providers. Returns an error if initialization fails. Initial Memory Allocations happens here as per the initial pool size.

func (*Bifrost) AddMCPClient added in v1.1.7 

func (bifrost *Bifrost) AddMCPClient(config *schemas.MCPClientConfig) error

AddMCPClient adds a new MCP client to the Bifrost instance. This allows for dynamic MCP client management at runtime.

Parameters:

  • config: MCP client configuration

Returns:

  • error: Any registration error

Example: 

err := bifrost.AddMCPClient(schemas.MCPClientConfig{
    Name: "my-mcp-client",
    ConnectionType: schemas.MCPConnectionTypeHTTP,
    ConnectionString: &url,
})

func (*Bifrost) BatchCancelRequest added in v1.2.38 

func (bifrost *Bifrost) BatchCancelRequest(ctx *schemas.BifrostContext, req *schemas.BifrostBatchCancelRequest) (*schemas.BifrostBatchCancelResponse, *schemas.BifrostError)

BatchCancelRequest cancels a batch job.

func (*Bifrost) BatchCreateRequest added in v1.2.38 

func (bifrost *Bifrost) BatchCreateRequest(ctx *schemas.BifrostContext, req *schemas.BifrostBatchCreateRequest) (*schemas.BifrostBatchCreateResponse, *schemas.BifrostError)

BatchCreateRequest creates a new batch job for asynchronous processing.

func (*Bifrost) BatchListRequest added in v1.2.38 

func (bifrost *Bifrost) BatchListRequest(ctx *schemas.BifrostContext, req *schemas.BifrostBatchListRequest) (*schemas.BifrostBatchListResponse, *schemas.BifrostError)

BatchListRequest lists batch jobs for the specified provider.

func (*Bifrost) BatchResultsRequest added in v1.2.38 

func (bifrost *Bifrost) BatchResultsRequest(ctx *schemas.BifrostContext, req *schemas.BifrostBatchResultsRequest) (*schemas.BifrostBatchResultsResponse, *schemas.BifrostError)

BatchResultsRequest retrieves results from a completed batch job.

func (*Bifrost) BatchRetrieveRequest added in v1.2.38 

func (bifrost *Bifrost) BatchRetrieveRequest(ctx *schemas.BifrostContext, req *schemas.BifrostBatchRetrieveRequest) (*schemas.BifrostBatchRetrieveResponse, *schemas.BifrostError)

BatchRetrieveRequest retrieves a specific batch job.

func (*Bifrost) ChatCompletionRequest 

func (bifrost *Bifrost) ChatCompletionRequest(ctx *schemas.BifrostContext, req *schemas.BifrostChatRequest) (*schemas.BifrostChatResponse, *schemas.BifrostError)

ChatCompletionRequest sends a chat completion request to the specified provider.

func (*Bifrost) ChatCompletionStreamRequest added in v1.1.8 

func (bifrost *Bifrost) ChatCompletionStreamRequest(ctx *schemas.BifrostContext, req *schemas.BifrostChatRequest) (chan *schemas.BifrostStreamChunk, *schemas.BifrostError)

ChatCompletionStreamRequest sends a chat completion stream request to the specified provider.

func (*Bifrost) ContainerCreateRequest added in v1.3.12 

func (bifrost *Bifrost) ContainerCreateRequest(ctx *schemas.BifrostContext, req *schemas.BifrostContainerCreateRequest) (*schemas.BifrostContainerCreateResponse, *schemas.BifrostError)

ContainerCreateRequest creates a new container.

func (*Bifrost) ContainerDeleteRequest added in v1.3.12 

func (bifrost *Bifrost) ContainerDeleteRequest(ctx *schemas.BifrostContext, req *schemas.BifrostContainerDeleteRequest) (*schemas.BifrostContainerDeleteResponse, *schemas.BifrostError)

ContainerDeleteRequest deletes a container.

func (*Bifrost) ContainerFileContentRequest added in v1.3.12 

func (bifrost *Bifrost) ContainerFileContentRequest(ctx *schemas.BifrostContext, req *schemas.BifrostContainerFileContentRequest) (*schemas.BifrostContainerFileContentResponse, *schemas.BifrostError)

ContainerFileContentRequest retrieves the content of a file from a container.

func (*Bifrost) ContainerFileCreateRequest added in v1.3.12 

func (bifrost *Bifrost) ContainerFileCreateRequest(ctx *schemas.BifrostContext, req *schemas.BifrostContainerFileCreateRequest) (*schemas.BifrostContainerFileCreateResponse, *schemas.BifrostError)

ContainerFileCreateRequest creates a file in a container.

func (*Bifrost) ContainerFileDeleteRequest added in v1.3.12 

func (bifrost *Bifrost) ContainerFileDeleteRequest(ctx *schemas.BifrostContext, req *schemas.BifrostContainerFileDeleteRequest) (*schemas.BifrostContainerFileDeleteResponse, *schemas.BifrostError)

ContainerFileDeleteRequest deletes a file from a container.

func (*Bifrost) ContainerFileListRequest added in v1.3.12 

func (bifrost *Bifrost) ContainerFileListRequest(ctx *schemas.BifrostContext, req *schemas.BifrostContainerFileListRequest) (*schemas.BifrostContainerFileListResponse, *schemas.BifrostError)

ContainerFileListRequest lists files in a container.

func (*Bifrost) ContainerFileRetrieveRequest added in v1.3.12 

func (bifrost *Bifrost) ContainerFileRetrieveRequest(ctx *schemas.BifrostContext, req *schemas.BifrostContainerFileRetrieveRequest) (*schemas.BifrostContainerFileRetrieveResponse, *schemas.BifrostError)

ContainerFileRetrieveRequest retrieves a file from a container.

func (*Bifrost) ContainerListRequest added in v1.3.12 

func (bifrost *Bifrost) ContainerListRequest(ctx *schemas.BifrostContext, req *schemas.BifrostContainerListRequest) (*schemas.BifrostContainerListResponse, *schemas.BifrostError)

ContainerListRequest lists containers.

func (*Bifrost) ContainerRetrieveRequest added in v1.3.12 

func (bifrost *Bifrost) ContainerRetrieveRequest(ctx *schemas.BifrostContext, req *schemas.BifrostContainerRetrieveRequest) (*schemas.BifrostContainerRetrieveResponse, *schemas.BifrostError)

ContainerRetrieveRequest retrieves a specific container.

func (*Bifrost) CountTokensRequest added in v1.2.43 

func (bifrost *Bifrost) CountTokensRequest(ctx *schemas.BifrostContext, req *schemas.BifrostResponsesRequest) (*schemas.BifrostCountTokensResponse, *schemas.BifrostError)

CountTokensRequest sends a count tokens request to the specified provider.

func (*Bifrost) EmbeddingRequest added in v1.1.7 

func (bifrost *Bifrost) EmbeddingRequest(ctx *schemas.BifrostContext, req *schemas.BifrostEmbeddingRequest) (*schemas.BifrostEmbeddingResponse, *schemas.BifrostError)

EmbeddingRequest sends an embedding request to the specified provider.

func (*Bifrost) ExecuteChatMCPTool added in v1.3.0 

func (bifrost *Bifrost) ExecuteChatMCPTool(ctx *schemas.BifrostContext, toolCall *schemas.ChatAssistantMessageToolCall) (*schemas.ChatMessage, *schemas.BifrostError)

ExecuteChatMCPTool executes an MCP tool call and returns the result as a chat message. This is the main public API for manual MCP tool execution in Chat format.

Parameters:

  • ctx: Execution context
  • toolCall: The tool call to execute (from assistant message)

Returns:

  • *schemas.ChatMessage: Tool message with execution result
  • *schemas.BifrostError: Any execution error

func (*Bifrost) ExecuteResponsesMCPTool added in v1.3.0 

func (bifrost *Bifrost) ExecuteResponsesMCPTool(ctx *schemas.BifrostContext, toolCall *schemas.ResponsesToolMessage) (*schemas.ResponsesMessage, *schemas.BifrostError)

ExecuteResponsesMCPTool executes an MCP tool call and returns the result as a responses message. This is the main public API for manual MCP tool execution in Responses format.

Parameters:

  • ctx: Execution context
  • toolCall: The tool call to execute (from assistant message)

Returns:

  • *schemas.ResponsesMessage: Tool message with execution result
  • *schemas.BifrostError: Any execution error

func (*Bifrost) FileContentRequest added in v1.2.38 

func (bifrost *Bifrost) FileContentRequest(ctx *schemas.BifrostContext, req *schemas.BifrostFileContentRequest) (*schemas.BifrostFileContentResponse, *schemas.BifrostError)

FileContentRequest downloads file content from the specified provider.

func (*Bifrost) FileDeleteRequest added in v1.2.38 

func (bifrost *Bifrost) FileDeleteRequest(ctx *schemas.BifrostContext, req *schemas.BifrostFileDeleteRequest) (*schemas.BifrostFileDeleteResponse, *schemas.BifrostError)

FileDeleteRequest deletes a file from the specified provider.

func (*Bifrost) FileListRequest added in v1.2.38 

func (bifrost *Bifrost) FileListRequest(ctx *schemas.BifrostContext, req *schemas.BifrostFileListRequest) (*schemas.BifrostFileListResponse, *schemas.BifrostError)

FileListRequest lists files from the specified provider.

func (*Bifrost) FileRetrieveRequest added in v1.2.38 

func (bifrost *Bifrost) FileRetrieveRequest(ctx *schemas.BifrostContext, req *schemas.BifrostFileRetrieveRequest) (*schemas.BifrostFileRetrieveResponse, *schemas.BifrostError)

FileRetrieveRequest retrieves file metadata from the specified provider.

func (*Bifrost) FileUploadRequest added in v1.2.38 

func (bifrost *Bifrost) FileUploadRequest(ctx *schemas.BifrostContext, req *schemas.BifrostFileUploadRequest) (*schemas.BifrostFileUploadResponse, *schemas.BifrostError)

FileUploadRequest uploads a file to the specified provider.

func (*Bifrost) GetAvailableMCPTools added in v1.3.0 

func (bifrost *Bifrost) GetAvailableMCPTools(ctx context.Context) []schemas.ChatTool

GetAvailableTools returns the available tools for the given context.

Returns:

  • []schemas.ChatTool: List of available tools

func (*Bifrost) GetConfiguredProviders added in v1.2.16 

func (bifrost *Bifrost) GetConfiguredProviders() ([]schemas.ModelProvider, error)

GetConfiguredProviders returns the configured providers.

Returns:

  • []schemas.ModelProvider: List of configured providers
  • error: Any error that occurred during the retrieval process

Example: 

providers, err := bifrost.GetConfiguredProviders()
if err != nil {
	return nil, err
}
fmt.Println(providers)

func (*Bifrost) GetDropExcessRequests added in v1.1.7 

func (bifrost *Bifrost) GetDropExcessRequests() bool

GetDropExcessRequests returns the current value of DropExcessRequests

func (*Bifrost) GetMCPClients added in v1.1.7 

func (bifrost *Bifrost) GetMCPClients() ([]schemas.MCPClient, error)

GetMCPClients returns all MCP clients managed by the Bifrost instance.

Returns:

  • []schemas.MCPClient: List of all MCP clients
  • error: Any retrieval error

func (*Bifrost) ImageEditRequest added in v1.4.0 

func (bifrost *Bifrost) ImageEditRequest(ctx *schemas.BifrostContext, req *schemas.BifrostImageEditRequest) (*schemas.BifrostImageGenerationResponse, *schemas.BifrostError)

ImageEditRequest sends an image edit request to the specified provider.

func (*Bifrost) ImageEditStreamRequest added in v1.4.0 

func (bifrost *Bifrost) ImageEditStreamRequest(ctx *schemas.BifrostContext, req *schemas.BifrostImageEditRequest) (chan *schemas.BifrostStreamChunk, *schemas.BifrostError)

ImageEditStreamRequest sends an image edit stream request to the specified provider.

func (*Bifrost) ImageGenerationRequest added in v1.3.9 

func (bifrost *Bifrost) ImageGenerationRequest(ctx *schemas.BifrostContext,
	req *schemas.BifrostImageGenerationRequest) (*schemas.BifrostImageGenerationResponse, *schemas.BifrostError)

ImageGenerationRequest sends an image generation request to the specified provider.

func (*Bifrost) ImageGenerationStreamRequest added in v1.3.9 

func (bifrost *Bifrost) ImageGenerationStreamRequest(ctx *schemas.BifrostContext,
	req *schemas.BifrostImageGenerationRequest) (chan *schemas.BifrostStreamChunk, *schemas.BifrostError)

ImageGenerationStreamRequest sends an image generation stream request to the specified provider.

func (*Bifrost) ImageVariationRequest added in v1.4.0 

func (bifrost *Bifrost) ImageVariationRequest(ctx *schemas.BifrostContext, req *schemas.BifrostImageVariationRequest) (*schemas.BifrostImageGenerationResponse, *schemas.BifrostError)

ImageVariationRequest sends an image variation request to the specified provider.

func (*Bifrost) ListAllModels added in v1.2.14 

func (bifrost *Bifrost) ListAllModels(ctx *schemas.BifrostContext, request *schemas.BifrostListModelsRequest) (*schemas.BifrostListModelsResponse, *schemas.BifrostError)

ListAllModels lists all models from all configured providers. It accumulates responses from all providers with a limit of 1000 per provider to get all results.

func (*Bifrost) ListModelsRequest added in v1.2.14 

func (bifrost *Bifrost) ListModelsRequest(ctx *schemas.BifrostContext, req *schemas.BifrostListModelsRequest) (*schemas.BifrostListModelsResponse, *schemas.BifrostError)

ListModelsRequest sends a list models request to the specified provider.

func (*Bifrost) ReconnectMCPClient added in v1.1.7 

func (bifrost *Bifrost) ReconnectMCPClient(id string) error

ReconnectMCPClient attempts to reconnect an MCP client if it is disconnected.

Parameters:

  • id: ID of the client to reconnect

Returns:

  • error: Any reconnection error

func (*Bifrost) RegisterMCPTool added in v1.1.4 

func (bifrost *Bifrost) RegisterMCPTool(name, description string, handler func(args any) (string, error), toolSchema schemas.ChatTool) error

RegisterMCPTool registers a typed tool handler with the MCP integration. This allows developers to easily add custom tools that will be available to all LLM requests processed by this Bifrost instance.

Parameters:

  • name: Unique tool name
  • description: Human-readable tool description
  • handler: Function that handles tool execution
  • toolSchema: Bifrost tool schema for function calling

Returns:

  • error: Any registration error

Example: 

type EchoArgs struct {
    Message string `json:"message"`
}

err := bifrost.RegisterMCPTool("echo", "Echo a message",
    func(args EchoArgs) (string, error) {
        return args.Message, nil
    }, toolSchema)

func (*Bifrost) ReloadConfig added in v1.2.3 

func (bifrost *Bifrost) ReloadConfig(config schemas.BifrostConfig) error

ReloadConfig reloads the config from DB Currently we update account, drop excess requests, and plugin lists We will keep on adding other aspects as required

func (*Bifrost) ReloadPlugin added in v1.2.0 

func (bifrost *Bifrost) ReloadPlugin(plugin schemas.BasePlugin, pluginTypes []schemas.PluginType) error

ReloadPlugin reloads a plugin with new instance During the reload - it's stop the world phase where we take a global lock on the plugin mutex

func (*Bifrost) RemoveMCPClient added in v1.1.7 

func (bifrost *Bifrost) RemoveMCPClient(id string) error

RemoveMCPClient removes an MCP client from the Bifrost instance. This allows for dynamic MCP client management at runtime.

Parameters:

  • id: ID of the client to remove

Returns:

  • error: Any removal error

Example: 

err := bifrost.RemoveMCPClient("my-mcp-client-id")
if err != nil {
    log.Fatalf("Failed to remove MCP client: %v", err)
}

func (*Bifrost) RemovePlugin added in v1.2.0 

func (bifrost *Bifrost) RemovePlugin(name string, pluginTypes []schemas.PluginType) error

RemovePlugin removes a plugin from the server.

func (*Bifrost) RemoveProvider added in v1.3.13 

func (bifrost *Bifrost) RemoveProvider(providerKey schemas.ModelProvider) error

RemoveProvider removes a provider from the server. This method gracefully stops all workers for the provider, closes the request queue, and removes the provider from the providers slice.

Parameters:

  • providerKey: The provider to remove

Returns:

  • error: Any error that occurred during the removal process

func (*Bifrost) ResponsesRequest added in v1.2.0 

func (bifrost *Bifrost) ResponsesRequest(ctx *schemas.BifrostContext, req *schemas.BifrostResponsesRequest) (*schemas.BifrostResponsesResponse, *schemas.BifrostError)

ResponsesRequest sends a responses request to the specified provider.

func (*Bifrost) ResponsesStreamRequest added in v1.2.0 

func (bifrost *Bifrost) ResponsesStreamRequest(ctx *schemas.BifrostContext, req *schemas.BifrostResponsesRequest) (chan *schemas.BifrostStreamChunk, *schemas.BifrostError)

ResponsesStreamRequest sends a responses stream request to the specified provider.

func (*Bifrost) SetMCPManager added in v1.4.0 

func (bifrost *Bifrost) SetMCPManager(manager mcp.MCPManagerInterface)

SetMCPManager sets the MCP manager for this Bifrost instance. This allows injecting a custom MCP manager implementation (e.g., for enterprise features).

Parameters:

  • manager: The MCP manager to set (must implement MCPManagerInterface)

func (*Bifrost) SetTracer added in v1.3.0 

func (bifrost *Bifrost) SetTracer(tracer schemas.Tracer)

SetTracer sets the tracer for the Bifrost instance.

func (*Bifrost) Shutdown 

func (bifrost *Bifrost) Shutdown()

Shutdown gracefully stops all workers when triggered. It closes all request channels and waits for workers to exit.

func (*Bifrost) SpeechRequest added in v1.1.11 

func (bifrost *Bifrost) SpeechRequest(ctx *schemas.BifrostContext, req *schemas.BifrostSpeechRequest) (*schemas.BifrostSpeechResponse, *schemas.BifrostError)

SpeechRequest sends a speech request to the specified provider.

func (*Bifrost) SpeechStreamRequest added in v1.1.11 

func (bifrost *Bifrost) SpeechStreamRequest(ctx *schemas.BifrostContext, req *schemas.BifrostSpeechRequest) (chan *schemas.BifrostStreamChunk, *schemas.BifrostError)

SpeechStreamRequest sends a speech stream request to the specified provider.

func (*Bifrost) TextCompletionRequest 

func (bifrost *Bifrost) TextCompletionRequest(ctx *schemas.BifrostContext, req *schemas.BifrostTextCompletionRequest) (*schemas.BifrostTextCompletionResponse, *schemas.BifrostError)

TextCompletionRequest sends a text completion request to the specified provider.

func (*Bifrost) TextCompletionStreamRequest added in v1.2.1 

func (bifrost *Bifrost) TextCompletionStreamRequest(ctx *schemas.BifrostContext, req *schemas.BifrostTextCompletionRequest) (chan *schemas.BifrostStreamChunk, *schemas.BifrostError)

TextCompletionStreamRequest sends a streaming text completion request to the specified provider.

func (*Bifrost) TranscriptionRequest added in v1.1.11 

func (bifrost *Bifrost) TranscriptionRequest(ctx *schemas.BifrostContext, req *schemas.BifrostTranscriptionRequest) (*schemas.BifrostTranscriptionResponse, *schemas.BifrostError)

TranscriptionRequest sends a transcription request to the specified provider.

func (*Bifrost) TranscriptionStreamRequest added in v1.1.11 

func (bifrost *Bifrost) TranscriptionStreamRequest(ctx *schemas.BifrostContext, req *schemas.BifrostTranscriptionRequest) (chan *schemas.BifrostStreamChunk, *schemas.BifrostError)

TranscriptionStreamRequest sends a transcription stream request to the specified provider.

func (*Bifrost) UpdateDropExcessRequests added in v1.1.7 

func (bifrost *Bifrost) UpdateDropExcessRequests(value bool)

UpdateDropExcessRequests updates the DropExcessRequests setting at runtime. This allows for hot-reloading of this configuration value.

func (*Bifrost) UpdateMCPClient added in v1.4.0 

func (bifrost *Bifrost) UpdateMCPClient(id string, updatedConfig *schemas.MCPClientConfig) error

UpdateMCPClient updates the MCP client. This allows for dynamic MCP client tool management at runtime.

Parameters:

  • id: ID of the client to edit
  • updatedConfig: Updated MCP client configuration

Returns:

  • error: Any edit error

Example: 

err := bifrost.UpdateMCPClient("my-mcp-client-id", schemas.MCPClientConfig{
    Name:           "my-mcp-client-name",
    ToolsToExecute: []string{"tool1", "tool2"},
})

func (*Bifrost) UpdateProvider added in v1.2.16 

func (bifrost *Bifrost) UpdateProvider(providerKey schemas.ModelProvider) error

UpdateProvider dynamically updates a provider with new configuration. This method gracefully recreates the provider instance with updated settings, stops existing workers, creates a new queue with updated settings, and starts new workers with the updated provider and concurrency configuration.

Parameters:

  • providerKey: The provider to update

Returns:

  • error: Any error that occurred during the update process

Note: This operation will temporarily pause request processing for the specified provider while the transition occurs. In-flight requests will complete before workers are stopped. Buffered requests in the old queue will be transferred to the new queue to prevent loss.

func (*Bifrost) UpdateToolManagerConfig added in v1.3.0 

func (bifrost *Bifrost) UpdateToolManagerConfig(maxAgentDepth int, toolExecutionTimeoutInSeconds int, codeModeBindingLevel string) error

UpdateToolManagerConfig updates the tool manager config for the MCP manager. This allows for hot-reloading of the tool manager config at runtime.

type ChannelMessage 

type ChannelMessage struct {
	schemas.BifrostRequest
	Context        *schemas.BifrostContext
	Response       chan *schemas.BifrostResponse
	ResponseStream chan chan *schemas.BifrostStreamChunk
	Err            chan schemas.BifrostError
}

ChannelMessage represents a message passed through the request channel. It contains the request, response and error channels, and the request type.

type DefaultLogger 

type DefaultLogger struct {
	// contains filtered or unexported fields
}

DefaultLogger implements the Logger interface with stdout/stderr printing. It provides a simple logging implementation that writes to standard output and error streams with formatted timestamps and log levels. It is used as the default logger if no logger is provided in the BifrostConfig.

func NewDefaultLogger 

func NewDefaultLogger(level schemas.LogLevel) *DefaultLogger

NewDefaultLogger creates a new DefaultLogger instance with the specified log level. The log level determines which messages will be output based on their severity.

func (*DefaultLogger) Debug 

func (logger *DefaultLogger) Debug(msg string, args ...any)

Debug logs a debug level message to stdout. Messages are only output if the logger's level is set to LogLevelDebug.

func (*DefaultLogger) Error 

func (logger *DefaultLogger) Error(msg string, args ...any)

Error logs an error level message to stderr. Error messages are always output regardless of the logger's level.

func (*DefaultLogger) Fatal added in v1.1.16 

func (logger *DefaultLogger) Fatal(msg string, args ...any)

Fatal logs a fatal-level message to stderr. Fatal messages are always output regardless of the logger's level.

func (*DefaultLogger) Info 

func (logger *DefaultLogger) Info(msg string, args ...any)

Info logs an info level message to stdout. Messages are output if the logger's level is LogLevelDebug or LogLevelInfo.

func (*DefaultLogger) LogHTTPRequest added in v1.4.1 

func (logger *DefaultLogger) LogHTTPRequest(level schemas.LogLevel, msg string) schemas.LogEventBuilder

LogHTTPRequest returns a LogEventBuilder for structured HTTP access logging. We are exposing the zerolog loggers directly to allow for more flexibility in logging and also to reduce the number of allocations we do in the logger.

func (*DefaultLogger) SetLevel 

func (logger *DefaultLogger) SetLevel(level schemas.LogLevel)

SetLevel sets the logging level for the logger. This determines which messages will be output based on their severity.

func (*DefaultLogger) SetOutputType added in v1.1.16 

func (logger *DefaultLogger) SetOutputType(outputType schemas.LoggerOutputType)

SetOutputType sets the output type for the logger. This determines the format of the log output. If the output type is unknown, it defaults to JSON

func (*DefaultLogger) Warn 

func (logger *DefaultLogger) Warn(msg string, args ...any)

Warn logs a warning level message to stdout. Messages are output if the logger's level is LogLevelDebug, LogLevelInfo, or LogLevelWarn.

type NoOpLogger added in v1.4.1 

type NoOpLogger struct{}

NoOpLogger is a no-op implementation of schemas.Logger.

func (*NoOpLogger) Debug added in v1.4.1 

func (l *NoOpLogger) Debug(string, ...any)

func (*NoOpLogger) Error added in v1.4.1 

func (l *NoOpLogger) Error(string, ...any)

func (*NoOpLogger) Fatal added in v1.4.1 

func (l *NoOpLogger) Fatal(string, ...any)

func (*NoOpLogger) Info added in v1.4.1 

func (l *NoOpLogger) Info(string, ...any)

func (*NoOpLogger) LogHTTPRequest added in v1.4.1 

func (l *NoOpLogger) LogHTTPRequest(schemas.LogLevel, string) schemas.LogEventBuilder

func (*NoOpLogger) SetLevel added in v1.4.1 

func (l *NoOpLogger) SetLevel(schemas.LogLevel)

func (*NoOpLogger) SetOutputType added in v1.4.1 

func (l *NoOpLogger) SetOutputType(schemas.LoggerOutputType)

func (*NoOpLogger) Warn added in v1.4.1 

func (l *NoOpLogger) Warn(string, ...any)

type PluginPipeline added in v1.1.2 

type PluginPipeline struct {
	// contains filtered or unexported fields
}

PluginPipeline encapsulates the execution of plugin PreHooks and PostHooks, tracks how many plugins ran, and manages short-circuiting and error aggregation.

func (*PluginPipeline) FinalizeStreamingPostHookSpans added in v1.3.0 

func (p *PluginPipeline) FinalizeStreamingPostHookSpans(ctx context.Context)

FinalizeStreamingPostHookSpans creates aggregated spans for each plugin after streaming completes. This should be called once at the end of streaming to create one span per plugin with average timing. Spans are nested to mirror the pre-hook hierarchy (each post-hook is a child of the previous one).

func (*PluginPipeline) GetChunkCount added in v1.3.0 

func (p *PluginPipeline) GetChunkCount() int

GetChunkCount returns the number of chunks processed during streaming

func (*PluginPipeline) RunLLMPreHooks added in v1.4.0 

func (p *PluginPipeline) RunLLMPreHooks(ctx *schemas.BifrostContext, req *schemas.BifrostRequest) (*schemas.BifrostRequest, *schemas.LLMPluginShortCircuit, int)

RunPreHooks executes PreHooks in order, tracks how many ran, and returns the final request, any short-circuit decision, and the count.

func (*PluginPipeline) RunMCPPostHooks added in v1.4.0 

func (p *PluginPipeline) RunMCPPostHooks(ctx *schemas.BifrostContext, mcpResp *schemas.BifrostMCPResponse, bifrostErr *schemas.BifrostError, runFrom int) (*schemas.BifrostMCPResponse, *schemas.BifrostError)

RunMCPPostHooks executes MCP PostHooks in reverse order for the plugins whose PreMCPHook ran. Accepts the MCP response and error, and allows plugins to transform either (e.g., recover from error, or invalidate a response). Returns the final MCP response and error after all hooks. If both are set, error takes precedence unless error is nil. runFrom is the count of plugins whose PreHooks ran; PostHooks will run in reverse from index (runFrom - 1) down to 0

func (*PluginPipeline) RunMCPPreHooks added in v1.4.0 

func (p *PluginPipeline) RunMCPPreHooks(ctx *schemas.BifrostContext, req *schemas.BifrostMCPRequest) (*schemas.BifrostMCPRequest, *schemas.MCPPluginShortCircuit, int)

RunMCPPreHooks executes MCP PreHooks in order for all registered MCP plugins. Returns the modified request, any short-circuit decision, and the count of hooks that ran. If a plugin short-circuits, only PostHooks for plugins up to and including that plugin will run.

func (*PluginPipeline) RunPostHooks added in v1.1.2 

func (p *PluginPipeline) RunPostHooks(ctx *schemas.BifrostContext, resp *schemas.BifrostResponse, bifrostErr *schemas.BifrostError, runFrom int) (*schemas.BifrostResponse, *schemas.BifrostError)

RunPostHooks executes PostHooks in reverse order for the plugins whose PreLLMHook ran. Accepts the response and error, and allows plugins to transform either (e.g., recover from error, or invalidate a response). Returns the final response and error after all hooks. If both are set, error takes precedence unless error is nil. runFrom is the count of plugins whose PreHooks ran; PostHooks will run in reverse from index (runFrom - 1) down to 0 For streaming requests, it accumulates timing per plugin instead of creating individual spans per chunk.

type ProviderQueue added in v1.3.13 

type ProviderQueue struct {
	// contains filtered or unexported fields
}

ProviderQueue wraps a provider's request channel with lifecycle management to prevent "send on closed channel" panics during provider removal/update. Producers must check the closing flag or select on the done channel before sending.

Source Files

View all Source files

Directories

Path Synopsis
internal
llmtests
Package llmtests provides comprehensive test account and configuration management for the Bifrost system.
 Package llmtests provides comprehensive test account and configuration management for the Bifrost system.
mcptests
mcp
codemode/starlark
Package starlark provides a Starlark-based implementation of the CodeMode interface.
 Package starlark provides a Starlark-based implementation of the CodeMode interface.
Package network provides centralized HTTP client management with proxy support.
 Package network provides centralized HTTP client management with proxy support.
providers
anthropic
Package anthropic implements the Anthropic provider for the Bifrost API.
 Package anthropic implements the Anthropic provider for the Bifrost API.
azure
Package azure implements the Azure provider.
 Package azure implements the Azure provider.
bedrock
cerebras
Package cerebras implements the Cerebras LLM provider.
 Package cerebras implements the Cerebras LLM provider.
cohere
elevenlabs
gemini
Package gemini provides types and structures for interacting with Google's Gemini API.
 Package gemini provides types and structures for interacting with Google's Gemini API.
groq
Package groq implements the Groq provider and its utility functions.
 Package groq implements the Groq provider and its utility functions.
huggingface
Package huggingface provides a HuggingFace chat provider.
 Package huggingface provides a HuggingFace chat provider.
mistral
Package mistral implements the Mistral provider.
 Package mistral implements the Mistral provider.
nebius
Package nebius implements the Nebius LLM provider.
 Package nebius implements the Nebius LLM provider.
ollama
Package providers implements various LLM providers and their utility functions.
 Package providers implements various LLM providers and their utility functions.
openai
Package openai provides the OpenAI provider implementation for the Bifrost framework.
 Package openai provides the OpenAI provider implementation for the Bifrost framework.
openrouter
Package openrouter implements the OpenRouter LLM provider.
 Package openrouter implements the OpenRouter LLM provider.
parasail
Package providers implements various LLM providers and their utility functions.
 Package providers implements various LLM providers and their utility functions.
perplexity
Package providers implements various LLM providers and their utility functions.
 Package providers implements various LLM providers and their utility functions.
replicate
Package providers implements various LLM providers and their utility functions.
 Package providers implements various LLM providers and their utility functions.
sgl
Package providers implements various LLM providers and their utility functions.
 Package providers implements various LLM providers and their utility functions.
utils
Package utils provides common utility functions used across different provider implementations.
 Package utils provides common utility functions used across different provider implementations.
vertex
xai
Package providers implements various LLM providers and their utility functions.
 Package providers implements various LLM providers and their utility functions.
Package schemas defines the core schemas and types used by the Bifrost system.
 Package schemas defines the core schemas and types used by the Bifrost system.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.