virtualmodel

package
v0.26041.1000-preview Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 11, 2026 License: MPL-2.0 Imports: 14 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AnthropicContent 

type AnthropicContent struct {
	Type  string          `json:"type"`
	Text  string          `json:"text,omitempty"`
	ID    string          `json:"id,omitempty"`
	Name  string          `json:"name,omitempty"`
	Input json.RawMessage `json:"input,omitempty"`
}

AnthropicContent represents content block in Anthropic response

type AnthropicDelta 

type AnthropicDelta struct {
	Type       string `json:"type,omitempty"`
	Text       string `json:"text,omitempty"`
	StopReason string `json:"stop_reason,omitempty"`
}

AnthropicDelta represents a delta in streaming response

type AnthropicMessage 

type AnthropicMessage struct {
	Role    string `json:"role"`
	Content string `json:"content"`
}

AnthropicMessage represents a message in Anthropic format

type AnthropicMessageRequest 

type AnthropicMessageRequest struct {
	Model     string             `json:"model"`
	Messages  []AnthropicMessage `json:"messages"`
	MaxTokens int                `json:"max_tokens,omitempty"`
	Stream    bool               `json:"stream,omitempty"`
	Tools     []AnthropicTool    `json:"tools,omitempty"`
}

AnthropicMessageRequest represents an Anthropic-compatible messages request

type AnthropicMessageResponse 

type AnthropicMessageResponse struct {
	ID         string             `json:"id"`
	Type       string             `json:"type"`
	Role       string             `json:"role"`
	Model      string             `json:"model"`
	Content    []AnthropicContent `json:"content"`
	StopReason string             `json:"stop_reason"`
	Usage      AnthropicUsage     `json:"usage"`
}

AnthropicMessageResponse represents an Anthropic-compatible message response

type AnthropicStreamEvent 

type AnthropicStreamEvent struct {
	Type    string                    `json:"type"`
	Message *AnthropicMessageResponse `json:"message,omitempty"`
	Index   int                       `json:"index,omitempty"`
	Delta   *AnthropicDelta           `json:"delta,omitempty"`
	Usage   *AnthropicUsage           `json:"usage,omitempty"`
}

AnthropicStreamEvent represents a streaming event in Anthropic format

type AnthropicTool 

type AnthropicTool struct {
	Name        string          `json:"name"`
	Description string          `json:"description"`
	InputSchema json.RawMessage `json:"input_schema"`
}

AnthropicTool represents a tool definition in Anthropic format

type AnthropicUsage 

type AnthropicUsage struct {
	InputTokens  int `json:"input_tokens"`
	OutputTokens int `json:"output_tokens"`
}

AnthropicUsage represents token usage in Anthropic format

type ChatCompletionRequest 

type ChatCompletionRequest struct {
	Messages    []Message `json:"messages"`
	Model       string    `json:"model"`
	Temperature *float64  `json:"temperature,omitempty"`
	MaxTokens   *int      `json:"max_tokens,omitempty"`
	Stream      bool      `json:"stream,omitempty"`
}

ChatCompletionRequest represents an OpenAI-compatible chat completion request

type ChatCompletionResponse 

type ChatCompletionResponse struct {
	ID      string   `json:"id"`
	Object  string   `json:"object"`
	Created int64    `json:"created"`
	Model   string   `json:"model"`
	Choices []Choice `json:"choices"`
	Usage   Usage    `json:"usage"`
}

ChatCompletionResponse represents an OpenAI-compatible chat completion response

type ChatCompletionStreamResponse 

type ChatCompletionStreamResponse struct {
	ID      string         `json:"id"`
	Object  string         `json:"object"`
	Created int64          `json:"created"`
	Model   string         `json:"model"`
	Choices []StreamChoice `json:"choices"`
}

ChatCompletionStreamResponse represents a streaming chunk in OpenAI format

type Choice 

type Choice struct {
	Index        int        `json:"index"`
	Message      Message    `json:"message"`
	FinishReason string     `json:"finish_reason"`
	Delta        Delta      `json:"delta,omitempty"`
	ToolCalls    []ToolCall `json:"tool_calls,omitempty"`
}

Choice represents a choice in the response

type DelayConfig 

type DelayConfig struct {
	// Delay before the first SSE chunk is emitted (controls TTFT).
	MinFirstTokenDelayMs int
	MaxFirstTokenDelayMs int

	// Delay between the last content chunk and [DONE] (controls stream duration / TPS).
	MinEndDelayMs int
	MaxEndDelayMs int
}

DelayConfig configures the random delay ranges for a DelayProvider.

func DefaultDelayConfig 

func DefaultDelayConfig() DelayConfig

DefaultDelayConfig returns sensible defaults: 50–500ms TTFT, 100–1000ms end delay.

type DelayProvider 

type DelayProvider struct {
	// contains filtered or unexported fields
}

DelayProvider is an embedded OpenAI-compatible HTTP server with configurable random delays. It accepts any chat completions request, streams the fixed response "I am testing and random sleep", and produces realistic TTFT / TPS / latency values for metrics E2E testing.

Use it as a typ.Provider in routing rules so requests flow through the full proxy pipeline (routing → OpenAI client → metrics tracking).

func NewDelayProvider 

func NewDelayProvider() *DelayProvider

NewDelayProvider creates and starts a DelayProvider with default config.

func NewDelayProviderWithConfig 

func NewDelayProviderWithConfig(cfg DelayConfig) *DelayProvider

NewDelayProviderWithConfig creates and starts a DelayProvider with a custom config.

func (*DelayProvider) Close 

func (dp *DelayProvider) Close()

Close shuts down the embedded HTTP server.

func (*DelayProvider) Provider 

func (dp *DelayProvider) Provider(name string) *typ.Provider

Provider returns a *typ.Provider configured to route to this delay server. name is used as both UUID and Name; it must be unique within the test config.

func (*DelayProvider) URL 

func (dp *DelayProvider) URL() string

URL returns the base URL of the embedded server.

type Delta 

type Delta struct {
	Role      string     `json:"role,omitempty"`
	Content   string     `json:"content,omitempty"`
	ToolCalls []ToolCall `json:"tool_calls,omitempty"`
}

Delta represents the delta in a streaming response

type FunctionCall 

type FunctionCall struct {
	Name      string `json:"name"`
	Arguments string `json:"arguments"`
}

FunctionCall represents a function call

type Handler 

type Handler struct {
	// contains filtered or unexported fields
}

Handler handles HTTP requests for virtual models

func NewHandler 

func NewHandler(registry *Registry) *Handler

NewHandler creates a new virtual model handler

func (*Handler) ChatCompletions 

func (h *Handler) ChatCompletions(c *gin.Context)

ChatCompletions handles the POST /virtual/v1/chat/completions endpoint

func (*Handler) ListModels 

func (h *Handler) ListModels(c *gin.Context)

ListModels handles the GET /virtual/v1/models endpoint

func (*Handler) Messages 

func (h *Handler) Messages(c *gin.Context)

Messages handles the POST /virtual/v1/messages endpoint (Anthropic format)

type Message 

type Message struct {
	Role       string     `json:"role"`
	Content    string     `json:"content"`
	ToolCalls  []ToolCall `json:"tool_calls,omitempty"`
	ToolCallID string     `json:"tool_call_id,omitempty"`
}

Message represents a chat message

type Model 

type Model struct {
	ID      string `json:"id"`
	Object  string `json:"object"`
	Created int64  `json:"created"`
	OwnedBy string `json:"owned_by"`
}

Model represents a virtual model in the models list (OpenAI-compatible format)

type OpenAIModelsResponse 

type OpenAIModelsResponse struct {
	Object string  `json:"object"`
	Data   []Model `json:"data"`
}

OpenAIModelsResponse represents OpenAI's models API response format

type Registry 

type Registry struct {
	// contains filtered or unexported fields
}

Registry manages virtual models

func NewRegistry 

func NewRegistry() *Registry

NewRegistry creates a new virtual model registry

func (*Registry) Clear 

func (r *Registry) Clear()

Clear clears all registered models

func (*Registry) Get 

func (r *Registry) Get(id string) *VirtualModel

Get retrieves a virtual model by ID

func (*Registry) List 

func (r *Registry) List() []*VirtualModel

List returns all registered virtual models

func (*Registry) ListModels 

func (r *Registry) ListModels() []Model

ListModels returns all registered models as Model slices

func (*Registry) Register 

func (r *Registry) Register(vm *VirtualModel) error

Register registers a virtual model

func (*Registry) RegisterDefaults 

func (r *Registry) RegisterDefaults()

RegisterDefaults registers default virtual models

func (*Registry) Unregister 

func (r *Registry) Unregister(id string)

Unregister unregisters a virtual model

type Service 

type Service struct {
	// contains filtered or unexported fields
}

Service manages the virtual model service

func NewService 

func NewService() *Service

NewService creates a new virtual model service

func (*Service) GetHandler 

func (s *Service) GetHandler() *Handler

GetHandler returns the HTTP handler

func (*Service) GetModel 

func (s *Service) GetModel(id string) *VirtualModel

GetModel retrieves a virtual model by ID

func (*Service) GetRegistry 

func (s *Service) GetRegistry() *Registry

GetRegistry returns the model registry

func (*Service) ListModels 

func (s *Service) ListModels() []Model

ListModels returns all registered models

func (*Service) RegisterModel 

func (s *Service) RegisterModel(vm *VirtualModel) error

RegisterModel registers a new virtual model

func (*Service) SetupRoutes 

func (s *Service) SetupRoutes(group *gin.RouterGroup)

SetupRoutes sets up the virtual model routes on a Gin router group

func (*Service) UnregisterModel 

func (s *Service) UnregisterModel(id string)

UnregisterModel unregisters a virtual model

type StreamChoice 

type StreamChoice struct {
	Index        int        `json:"index"`
	Delta        Delta      `json:"delta"`
	FinishReason *string    `json:"finish_reason"`
	ToolCalls    []ToolCall `json:"tool_calls,omitempty"`
}

StreamChoice represents a choice in a streaming response

type ToolCall 

type ToolCall struct {
	ID       string       `json:"id"`
	Type     string       `json:"type"`
	Function FunctionCall `json:"function"`
}

ToolCall represents a tool call in the response

type ToolCallConfig 

type ToolCallConfig struct {
	// Tool name (e.g., "ask_user_question", "web_search", "code_interpreter")
	Name string `json:"name" yaml:"name"`
	// Tool arguments as a map (will be serialized to JSON)
	// For ask_user_question: {"question": "...", "options": [...]}
	// For web_search: {"query": "..."}
	Arguments map[string]interface{} `json:"arguments" yaml:"arguments"`
}

ToolCallConfig defines a tool call to be returned by the virtual model This is a generic configuration that can represent any tool call

type Usage 

type Usage struct {
	PromptTokens     int `json:"prompt_tokens"`
	CompletionTokens int `json:"completion_tokens"`
	TotalTokens      int `json:"total_tokens"`
}

Usage represents token usage information

type VirtualModel 

type VirtualModel struct {
	// contains filtered or unexported fields
}

VirtualModel represents a registered virtual model

func NewVirtualModel 

func NewVirtualModel(cfg *VirtualModelConfig) *VirtualModel

NewVirtualModel creates a new virtual model

func (*VirtualModel) GetContent 

func (vm *VirtualModel) GetContent() string

GetContent returns the response content

func (*VirtualModel) GetDelay 

func (vm *VirtualModel) GetDelay() time.Duration

GetDelay returns the response delay

func (*VirtualModel) GetDelegateModel 

func (vm *VirtualModel) GetDelegateModel() string

GetDelegateModel returns the delegate model for proxy mode

func (*VirtualModel) GetID 

func (vm *VirtualModel) GetID() string

GetID returns the model ID

func (*VirtualModel) GetName 

func (vm *VirtualModel) GetName() string

GetName returns the model name

func (*VirtualModel) GetStreamChunks 

func (vm *VirtualModel) GetStreamChunks() []string

GetStreamChunks returns the streaming chunks

func (*VirtualModel) GetToolCall 

func (vm *VirtualModel) GetToolCall() *ToolCallConfig

GetToolCall returns the tool call configuration

func (*VirtualModel) GetTransformer 

func (vm *VirtualModel) GetTransformer() protocol.Transformer

GetTransformer returns the transformer for proxy mode

func (*VirtualModel) GetType 

func (vm *VirtualModel) GetType() VirtualModelType

GetType returns the model type

func (*VirtualModel) IsProxy 

func (vm *VirtualModel) IsProxy() bool

IsProxy returns true if this is a proxy model

func (*VirtualModel) IsStatic 

func (vm *VirtualModel) IsStatic() bool

IsStatic returns true if this is a static model

func (*VirtualModel) IsTool 

func (vm *VirtualModel) IsTool() bool

IsTool returns true if this is a tool-type model

func (*VirtualModel) ToModel 

func (vm *VirtualModel) ToModel() Model

ToModel converts to Model type for API response

type VirtualModelConfig 

type VirtualModelConfig struct {
	ID           string
	Name         string
	Description  string
	Type         VirtualModelType // "static", "tool", or "proxy"
	Content      string           // For static models
	Role         string
	FinishReason string
	Delay        time.Duration
	StreamChunks []string // For streaming: chunks to send

	// For tool-type models
	ToolCall *ToolCallConfig

	// For proxy-type models
	DelegateModel string               // Real model to delegate to (e.g., "claude-3-5-sonnet-20241022")
	Transformer   protocol.Transformer // Optional transformer for proxy mode
}

VirtualModelConfig holds the configuration for a virtual model

type VirtualModelType 

type VirtualModelType string

VirtualModelType represents the type of virtual model 

const (
	VirtualModelTypeStatic VirtualModelType = "static" // Original: fixed response
	VirtualModelTypeTool   VirtualModelType = "tool"   // Returns tool calls
	VirtualModelTypeProxy  VirtualModelType = "proxy"  // Proxy mode with transformer
)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.