openai

OpenAI Compatibility Package

This package provides types and utilities for building OpenAI-compatible APIs that use MCP tools. It bridges the gap between MCP's tool format and OpenAI's function calling format.

Installation

import "github.com/paularlott/mcp/openai"

Features

• OpenAI Types: Complete request/response types for chat completions
• Tool Conversion: Convert MCP tools to OpenAI function format
• Response Extraction: Extract string results from MCP tool responses
• Streaming Support: Iterator and accumulator for streaming responses
• Tool Handlers: Event notifications during tool execution

Quick Start

Converting MCP Tools to OpenAI Format

import (
    "github.com/paularlott/mcp"
    "github.com/paularlott/mcp/openai"
)

// Create your MCP server
server := mcp.NewServer("my-server", "1.0.0")
server.RegisterTool(/* ... */)

// Convert MCP tools to OpenAI format
mcpTools := server.ListTools()
openAITools := openai.MCPToolsToOpenAI(mcpTools)

// Or filter tools by name
filteredTools := openai.MCPToolsToOpenAIFiltered(mcpTools, openai.ToolsByName("search", "calculate"))

// Or exclude certain tools
excludedTools := openai.MCPToolsToOpenAIFiltered(mcpTools, openai.ExcludeTools("dangerous_tool"))

Extracting Tool Results

// After executing an MCP tool
response, err := server.CallTool(ctx, toolName, args)
if err != nil {
    return err
}

// Extract the result as a string for OpenAI
result, err := openai.ExtractToolResult(response)
if err != nil {
    return err
}

// Create tool result message
toolResultMessage := openai.Message{
    Role:       "tool",
    ToolCallID: toolCall.ID,
}
toolResultMessage.SetContentAsString(result)

Streaming with ChatStream

// Assuming you have response and error channels from your streaming implementation
stream := openai.NewChatStream(ctx, responseChan, errorChan)

for stream.Next() {
    chunk := stream.Current()

    // Process each chunk
    for _, choice := range chunk.Choices {
        if choice.Delta.Content != "" {
            fmt.Print(choice.Delta.Content)
        }
    }
}

if err := stream.Err(); err != nil {
    log.Printf("Stream error: %v", err)
}

Accumulating Streaming Responses

acc := &openai.CompletionAccumulator{}

for stream.Next() {
    chunk := stream.Current()
    acc.AddChunk(chunk)
}

// Check what we got
if content, ok := acc.FinishedContent(); ok {
    fmt.Println("Content:", content)
}

if toolCalls, ok := acc.FinishedToolCalls(); ok {
    for _, tc := range toolCalls {
        fmt.Printf("Tool call: %s(%v)\n", tc.Function.Name, tc.Function.Arguments)
    }
}

if refusal, ok := acc.FinishedRefusal(); ok {
    fmt.Println("Refusal:", refusal)
}

Using Tool Handlers

Tool handlers receive events during tool execution, useful for sending SSE events or logging.

Important: The expected call order is:

1. OnToolCall() - called BEFORE executing the tool
2. Execute the tool
3. OnToolResult() - called AFTER the tool completes

type MyToolHandler struct{}

func (h *MyToolHandler) OnToolCall(toolCall openai.ToolCall) error {
    log.Printf("Starting tool: %s", toolCall.Function.Name)
    return nil
}

func (h *MyToolHandler) OnToolResult(toolCallID, toolName, result string) error {
    log.Printf("Tool %s completed with result: %s", toolName, result)
    return nil
}

// Attach handler to context
ctx = openai.WithToolHandler(ctx, &MyToolHandler{})

// Later, retrieve and use it during tool processing
if handler := openai.ToolHandlerFromContext(ctx); handler != nil {
    // 1. Notify BEFORE execution
    handler.OnToolCall(toolCall)

    // 2. Execute the tool
    result, err := mcpServer.CallTool(ctx, toolCall.Function.Name, toolCall.Function.Arguments)

    // 3. Notify AFTER execution with result
    handler.OnToolResult(toolCall.ID, toolCall.Function.Name, result)
}

Types Reference

Request/Response Types

Tool Types

Message Content Helpers

// Get content as string (handles both string and array formats)
content := message.GetContentAsString()

// Set content as string
message.SetContentAsString("Hello, world!")

Tool Call JSON Handling

The ToolCallFunction type includes custom JSON marshaling/unmarshaling to handle OpenAI's format where arguments are a JSON string rather than an object:

// When marshaling to JSON, arguments become a string:
// {"name": "search", "arguments": "{\"query\": \"hello\"}"}

// When unmarshaling, the string is parsed back to map[string]any

Tool Filtering

Several filter helpers are provided:

// Include all tools
openai.AllTools()

// Include only specific tools
openai.ToolsByName("tool1", "tool2")

// Exclude specific tools
openai.ExcludeTools("admin_tool", "debug_tool")

// Custom filter
customFilter := func(name string) bool {
    return strings.HasPrefix(name, "public_")
}

Generating Tool Call IDs

When streaming, some LLMs don't provide tool call IDs. Generate one:

id := openai.GenerateToolCallID(index)
// Returns something like "call_a1b2c3d4e5f6..."

SSE Tool Status Events

When streaming chat completions with server-side tool processing, you can send tool execution status events to clients. These events are sent as SSE comments (prefixed with :) so standard SSE clients ignore them, but custom clients can parse them to show tool execution progress in the UI.

Event Format

Tool events are sent as SSE comments in the format :eventType:jsonData:

:tool_start:{"tool_call_id":"call_abc123","tool_name":"search","status":"running"}

:tool_end:{"tool_call_id":"call_abc123","tool_name":"search","status":"complete","result":"Search found 5 results..."}

The tool_end event includes the tool's result, allowing clients to display what the tool returned.

Using SSEToolHandler

The SSEToolHandler implements ToolHandler to automatically send tool events during execution:

// Create an SSE event writer (adapt to your HTTP framework)
sseWriter := openai.NewSimpleSSEWriter(responseWriter, func() {
    if f, ok := responseWriter.(http.Flusher); ok {
        f.Flush()
    }
})

// Create the tool handler with optional error logging
toolHandler := openai.NewSSEToolHandler(sseWriter, func(err error, eventType, toolName string) {
    log.Printf("Failed to write %s event for %s: %v", eventType, toolName, err)
})

// Attach to context for use during tool execution
ctx = openai.WithToolHandler(ctx, toolHandler)

// During tool processing, call in order:
// 1. toolHandler.OnToolCall(toolCall)  <- sends :tool_start: event
// 2. Execute the tool
// 3. toolHandler.OnToolResult(...)     <- sends :tool_end: event with result

Custom SSEEventWriter

For production use with your HTTP framework, implement the SSEEventWriter interface:

type SSEEventWriter interface {
    WriteEvent(eventType string, data any) error
}

Complete Stream Example

A streaming response with tool execution looks like:

data: {"id":"chatcmpl-xxx","choices":[{"delta":{"role":"assistant","content":"Let me look that up..."}}]}

:tool_start:{"tool_call_id":"call_abc","tool_name":"search","status":"running"}

:tool_end:{"tool_call_id":"call_abc","tool_name":"search","status":"complete","result":"Found 3 results for your query..."}

data: {"id":"chatcmpl-xxx","choices":[{"delta":{"content":"Based on the search results..."}}]}

data: {"id":"chatcmpl-xxx","choices":[{"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Parsing Tool Events (JavaScript)

const response = await fetch(url, options);
const reader = response.body.getReader();
const decoder = new TextDecoder();

while (true) {
  const { done, value } = await reader.read();
  if (done) break;

  const text = decoder.decode(value);
  for (const line of text.split('\n')) {
    if (line.startsWith(':tool_start:')) {
      const event = JSON.parse(line.slice(':tool_start:'.length));
      showSpinner(`Running ${event.tool_name}...`);
    } else if (line.startsWith(':tool_end:')) {
      const event = JSON.parse(line.slice(':tool_end:'.length));
      hideSpinner(event.tool_name);
      // Optionally display the result
      if (event.result) {
        showToolResult(event.tool_name, event.result);
      }
    } else if (line.startsWith('data: ') && line !== 'data: [DONE]') {
      const chunk = JSON.parse(line.slice(6));
      // Handle OpenAI chunk...
    }
  }
}

Complete Example

Here's a complete example of an OpenAI-compatible endpoint using MCP tools:

func handleChatCompletion(w http.ResponseWriter, r *http.Request) {
    var req openai.ChatCompletionRequest
    if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    // Add MCP tools to the request
    mcpTools := mcpServer.ListTools()
    req.Tools = append(req.Tools, openai.MCPToolsToOpenAI(mcpTools)...)

    // Forward to upstream LLM...
    response := callUpstreamLLM(req)

    // Process tool calls if any
    for _, choice := range response.Choices {
        for _, toolCall := range choice.Message.ToolCalls {
            // Execute MCP tool
            mcpResponse, err := mcpServer.CallTool(r.Context(),
                toolCall.Function.Name,
                toolCall.Function.Arguments)
            if err != nil {
                // Handle error
                continue
            }

            // Extract result
            result, _ := openai.ExtractToolResult(mcpResponse)

            // Add to messages for next iteration...
        }
    }

    json.NewEncoder(w).Encode(response)
}

License

This package is part of the MCP library and is licensed under the same terms.

Token Estimation

When the upstream LLM doesn't provide token usage in its response, you can use the TokenCounter to estimate tokens:

// Initialize counter and estimate prompt tokens from initial messages
tokenCounter := openai.NewTokenCounter()
tokenCounter.AddPromptTokensFromMessages(req.Messages)

// After receiving streaming deltas, track completion tokens
tokenCounter.AddCompletionTokensFromDelta(&delta)

// Or for non-streaming, track from the full message
tokenCounter.AddCompletionTokensFromMessage(&response.Choices[0].Message)

// When tools are used, track tool result tokens (they become prompt tokens for next iteration)
toolResultMsg := openai.BuildToolResultMessage(toolCall.ID, result)
tokenCounter.AddPromptTokensFromMessages([]openai.Message{toolResultMsg})

// Get the estimated usage
usage := tokenCounter.GetUsage()
// usage.PromptTokens, usage.CompletionTokens, usage.TotalTokens

// Or inject into a response if it's missing
tokenCounter.InjectUsageIfMissing(response)

Token Estimation Algorithm

The EstimateTokens function provides a fast, reproducible approximation based on:

• Word boundaries (using strings.Fields)
• Punctuation counting (each punctuation mark = 1 token)
• Chat template overhead (~4 tokens for conversation structure)
• Per-message overhead (~3 tokens for role markers and special tokens)

This is suitable for billing estimates and UI display, but not for exact token limit calculations.