lmsproxy

package
v0.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 15, 2026 License: MIT Imports: 12 Imported by: 0

Documentation

Index

Constants

View Source 
const MinContextLength = 32768

MinContextLength is the minimum context length required for Claude Code's tools + system prompt. Claude Code sends ~20K tokens of tools definitions alone.

Variables

This section is empty.

Functions

func DetectLMSBaseURL 

func DetectLMSBaseURL() string

DetectLMSBaseURL runs "lms status" to discover the LM Studio server port. Returns "http://localhost:1234" as fallback.

func EnsureModelContext 

func EnsureModelContext(logger *slog.Logger)

EnsureModelContext checks loaded models via "lms ps" and reloads any model whose context length is below MinContextLength. This is necessary because Claude Code sends large tool definitions that exceed small context windows.

Types

type AccumulatedMessage 

type AccumulatedMessage struct {
	ResponseID string
	Message    map[string]interface{}
}

AccumulatedMessage holds the result of consuming an OAI SSE stream and converting it into a single Anthropic Messages API response.

func AccumulateResponse 

func AccumulateResponse(body io.Reader, model string) (*AccumulatedMessage, error)

AccumulateResponse reads an OAI Responses SSE stream to completion and returns a single Anthropic Messages API Message object. Used when the client sends stream=false (e.g. Claude CLI's non-streaming fallback).

type AnthropicContentBlock 

type AnthropicContentBlock struct {
	Type      string          `json:"type"` // "text" | "tool_use" | "tool_result" | "thinking"
	Text      string          `json:"text,omitempty"`
	ID        string          `json:"id,omitempty"`          // tool_use id
	Name      string          `json:"name,omitempty"`        // tool_use function name
	Input     json.RawMessage `json:"input,omitempty"`       // tool_use arguments
	ToolUseID string          `json:"tool_use_id,omitempty"` // tool_result reference
	Content   json.RawMessage `json:"content,omitempty"`     // tool_result content
	IsError   bool            `json:"is_error,omitempty"`    // tool_result error flag
}

AnthropicContentBlock represents a typed content block inside a message.

type AnthropicMessage 

type AnthropicMessage struct {
	Role    string          `json:"role"` // "user" | "assistant"
	Content json.RawMessage `json:"content"`
}

AnthropicMessage is a single message in the messages array.

type AnthropicRequest 

type AnthropicRequest struct {
	Model     string             `json:"model"`
	MaxTokens int                `json:"max_tokens,omitempty"`
	System    json.RawMessage    `json:"system,omitempty"` // string or []SystemBlock
	Messages  []AnthropicMessage `json:"messages"`
	Tools     []AnthropicTool    `json:"tools,omitempty"`
	Stream    bool               `json:"stream"`
	// Claude-specific fields we accept but ignore.
	Thinking json.RawMessage `json:"thinking,omitempty"`
	Metadata json.RawMessage `json:"metadata,omitempty"`
}

AnthropicRequest is the POST /v1/messages request body.

type AnthropicTool 

type AnthropicTool struct {
	Name        string          `json:"name"`
	Description string          `json:"description,omitempty"`
	InputSchema json.RawMessage `json:"input_schema"`
}

AnthropicTool defines a tool the model can call.

type OAIContentPart 

type OAIContentPart struct {
	Type string `json:"type"` // "input_text" | "output_text"
	Text string `json:"text"`
}

OAIContentPart is a typed part inside a message's content array.

type OAIContentPartAdded 

type OAIContentPartAdded struct {
	Type         string `json:"type"`
	OutputIndex  int    `json:"output_index"`
	ContentIndex int    `json:"content_index"`
	Part         struct {
		Type string `json:"type"` // "output_text"
		Text string `json:"text"`
	} `json:"part"`
}

OAIContentPartAdded is the payload for response.content_part.added.

type OAIFuncCallArgsDelta 

type OAIFuncCallArgsDelta struct {
	Type        string `json:"type"`
	OutputIndex int    `json:"output_index"`
	ItemID      string `json:"item_id"`
	Delta       string `json:"delta"`
}

OAIFuncCallArgsDelta is the payload for response.function_call_arguments.delta.

type OAIFuncCallArgsDone 

type OAIFuncCallArgsDone struct {
	Type        string `json:"type"`
	OutputIndex int    `json:"output_index"`
	ItemID      string `json:"item_id"`
	Name        string `json:"name"`
	CallID      string `json:"call_id"`
	Arguments   string `json:"arguments"`
}

OAIFuncCallArgsDone is the payload for response.function_call_arguments.done.

type OAIInputItem 

type OAIInputItem struct {
	Type    string           `json:"type"` // "message" | "function_call" | "function_call_output"
	Role    string           `json:"role,omitempty"`
	Content []OAIContentPart `json:"content,omitempty"`
	// function_call fields
	Name      string `json:"name,omitempty"`
	Arguments string `json:"arguments,omitempty"`
	// function_call / function_call_output fields
	CallID string `json:"call_id,omitempty"`
	Output string `json:"output"` // required for function_call_output; LM Studio rejects items without it
}

OAIInputItem is one element in the input array.

type OAIOutputItem 

type OAIOutputItem struct {
	Type   string `json:"type"` // "message" | "function_call"
	ID     string `json:"id"`
	Name   string `json:"name,omitempty"` // function_call name
	CallID string `json:"call_id,omitempty"`
}

OAIOutputItem represents an output item (message or function_call).

type OAIOutputItemAdded 

type OAIOutputItemAdded struct {
	Type        string        `json:"type"`
	OutputIndex int           `json:"output_index"`
	Item        OAIOutputItem `json:"item"`
}

OAIOutputItemAdded is the payload for response.output_item.added.

type OAIOutputItemDone 

type OAIOutputItemDone struct {
	Type        string        `json:"type"`
	OutputIndex int           `json:"output_index"`
	Item        OAIOutputItem `json:"item"`
}

OAIOutputItemDone is the payload for response.output_item.done.

type OAIOutputTextDelta 

type OAIOutputTextDelta struct {
	Type         string `json:"type"`
	OutputIndex  int    `json:"output_index"`
	ContentIndex int    `json:"content_index"`
	Delta        string `json:"delta"`
}

OAIOutputTextDelta is the payload for response.output_text.delta.

type OAIRequest 

type OAIRequest struct {
	Model              string          `json:"model"`
	Input              json.RawMessage `json:"input"` // string or []InputItem
	Instructions       string          `json:"instructions,omitempty"`
	PreviousResponseID string          `json:"previous_response_id,omitempty"`
	MaxOutputTokens    int             `json:"max_output_tokens,omitempty"`
	Tools              []OAITool       `json:"tools,omitempty"`
	Stream             bool            `json:"stream"`
	Store              bool            `json:"store"`
}

OAIRequest is the POST /v1/responses request body.

func BuildOAIRequest 

func BuildOAIRequest(req *AnthropicRequest, prevID string, newMsgs []AnthropicMessage, allowedTools map[string]bool) (*OAIRequest, error)

BuildOAIRequest converts an Anthropic Messages request into an OpenAI Responses request. allowedTools filters which tools are forwarded (nil/empty = all tools).

type OAIResponseCompleted 

type OAIResponseCompleted struct {
	Type     string            `json:"type"`
	Response OAIResponseObject `json:"response"`
}

OAIResponseCompleted is the payload for response.completed.

type OAIResponseCreated 

type OAIResponseCreated struct {
	Type     string            `json:"type"`
	Response OAIResponseObject `json:"response"`
}

OAIResponseCreated is the payload for response.created.

type OAIResponseObject 

type OAIResponseObject struct {
	ID     string    `json:"id"`
	Model  string    `json:"model"`
	Status string    `json:"status,omitempty"`
	Usage  *OAIUsage `json:"usage,omitempty"`
}

OAIResponseObject is the top-level response envelope.

type OAITool 

type OAITool struct {
	Type        string          `json:"type"` // "function"
	Name        string          `json:"name"`
	Description string          `json:"description,omitempty"`
	Parameters  json.RawMessage `json:"parameters"`
}

OAITool defines a function-type tool for the Responses API. Unlike Chat Completions, Responses API uses a flat structure without a nested "function" wrapper.

type OAIUsage 

type OAIUsage struct {
	InputTokens  int `json:"input_tokens"`
	OutputTokens int `json:"output_tokens"`
}

OAIUsage tracks token counts.

type Proxy 

type Proxy struct {
	// contains filtered or unexported fields
}

Proxy translates Anthropic Messages API requests into OpenAI Responses API requests for LM Studio, maintaining stateful sessions via previous_response_id.

func New 

func New(lmsBaseURL string, logger *slog.Logger) *Proxy

New creates a new proxy targeting the given LM Studio base URL.

func (*Proxy) Start 

func (p *Proxy) Start(ctx context.Context, preferredPort int) (int, error)

Start listens on preferredPort (with fallback) and serves in the background. Returns the actual port. The server stops when ctx is cancelled.

type SessionConfig 

type SessionConfig struct {
	ModelOverride string
	AllowedTools  map[string]bool
}

SessionStore tracks the last response_id per session+model so that subsequent requests can use previous_response_id for LM Studio KV cache reuse.

Claude CLI rewrites message content between turns (injecting system-reminders etc.), so hash-based prefix matching doesn't work. Instead we track the latest response_id per session key and always send only the trailing new messages (user + tool_result) as delta. SessionConfig holds per-session settings.

type SessionStore 

type SessionStore struct {
	// contains filtered or unexported fields
}

func NewSessionStore 

func NewSessionStore() *SessionStore

NewSessionStore creates a new empty session store.

func (*SessionStore) GetConfig 

func (s *SessionStore) GetConfig(sessionID string) (SessionConfig, bool)

GetConfig returns configuration for a session.

func (*SessionStore) Len 

func (s *SessionStore) Len() int

Len returns the number of stored sessions.

func (*SessionStore) Lookup 

func (s *SessionStore) Lookup(sessionID, model string, msgs []AnthropicMessage) (prevID string, newMsgs []AnthropicMessage)

Lookup returns the previous_response_id for the model (if any) and extracts only the new messages to send as delta. If no session exists, all messages are returned and prevID is empty.

func (*SessionStore) Reset 

func (s *SessionStore) Reset()

Reset clears all stored sessions.

func (*SessionStore) SetConfig 

func (s *SessionStore) SetConfig(sessionID string, cfg SessionConfig)

SetConfig stores configuration for a session.

func (*SessionStore) Store 

func (s *SessionStore) Store(sessionID, model, responseID string)

Store saves a response_id for a model.

type StreamConverter 

type StreamConverter struct {
	// contains filtered or unexported fields
}

StreamConverter reads OAI Responses SSE events from an io.Reader and writes Anthropic Messages SSE events to an http.ResponseWriter.

func NewStreamConverter 

func NewStreamConverter(w http.ResponseWriter, model string) *StreamConverter

NewStreamConverter creates a converter that writes Anthropic SSE to w.

func (*StreamConverter) Process 

func (c *StreamConverter) Process(body io.Reader) error

Process reads the OAI SSE stream and writes Anthropic SSE events. Returns an error only for I/O failures; LM Studio errors are forwarded as SSE.

func (*StreamConverter) ResponseID 

func (c *StreamConverter) ResponseID() string

ResponseID returns the response ID from LM Studio after processing completes.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.