llm-mux

module

v1.0.34 Latest Latest Go to latest Published: Dec 25, 2025 License: MIT

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/nghyane/llm-mux

Links

README ¶

llm-mux

Turn your existing AI subscriptions into standard API endpoints.
A local gateway that enables your Claude Pro, GitHub Copilot, and Google Gemini subscriptions to work with any AI tool.

Universal Compatibility: Supports OpenAI, Anthropic (Claude), Google (Gemini), and Ollama API formats. Zero Overhead: No separate API billing. No per-token charges. Runs locally.

⚡️ Quick Install

No dependencies required. These scripts download the pre-compiled binary for your architecture.

macOS / Linux

curl -fsSL https://raw.githubusercontent.com/nghyane/llm-mux/main/install.sh | bash

Windows (PowerShell)

irm https://raw.githubusercontent.com/nghyane/llm-mux/main/install.ps1 | iex

🚀 Getting Started

1. Initialize

Run this once to set up the configuration directory:

llm-mux --init

2. Authenticate Providers

Login to the services you have subscriptions for. This opens your browser to cache OAuth tokens locally.

# For Google Gemini (Free Tier or Advanced)
llm-mux --antigravity-login

# For Claude Pro / Max
llm-mux --claude-login

# For GitHub Copilot
llm-mux --copilot-login

View other login commands (OpenAI Codex, Qwen, Amazon Q)

Provider	Command	Description
OpenAI Codex	`llm-mux --codex-login`	Access GPT-5 series (if eligible)
Qwen	`llm-mux --qwen-login`	Alibaba Cloud Qwen models
Amazon Q	`llm-mux --kiro-login`	AWS/Amazon Q Developer
Cline	`llm-mux --cline-login`	Cline API integration
iFlow	`llm-mux --iflow-login`	iFlow integration

3. Verify

Check if the server is running and models are available:

curl http://localhost:8317/v1/models

🛠 Integration Guide

Point your tools to the local proxy. Base URL: http://localhost:8317/v1
API Key: any-string (unused, but required by some clients)

Cursor

Go to Settings > Models.
Toggle "OpenAI API Base URL" to ON.
Enter: http://localhost:8317/v1

VS Code (Cline / Roo Code)

API Provider: OpenAI Compatible
Base URL: http://localhost:8317/v1
Model ID: claude-sonnet-4-20250514 (or any available model)

Aider / Claude Code

# Using Claude Sonnet via llm-mux
aider --openai-api-base http://localhost:8317/v1 --model claude-sonnet-4-20250514

Python (OpenAI SDK)

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8317/v1",
    api_key="unused"
)

response = client.chat.completions.create(
    model="gemini-2.5-pro",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

🤖 Supported Models

llm-mux automatically maps your subscription access to these model identifiers.

Provider	Top Models
Google	`gemini-2.5-pro`, `gemini-2.5-flash`, `gemini-3-pro-preview`
Anthropic	`claude-sonnet-4-20250514`, `claude-opus-4-5-20251101`, `claude-3-5-sonnet`
GitHub	`gpt-4.1`, `gpt-4o`, `gpt-5`, `gpt-5-mini`, `gpt-5.1`, `gpt-5.2`

Note: Run curl http://localhost:8317/v1/models to see the exact list available to your account.

🧩 Architecture

llm-mux acts as a universal adapter, accepting requests in multiple formats (OpenAI, Claude, Gemini, Ollama) and routing them to your active subscriptions.

graph LR
    subgraph "Inbound Protocols"
        OpenAI[OpenAI / Compatible]
        Anthropic[Anthropic SDK]
        Google[Google AI SDK]
        Ollama[Ollama Clients]
    end

    Mux[llm-mux :8317]
    
    subgraph "Outbound Subscriptions"
        Gemini[Google Gemini]
        Claude[Claude Pro]
        Copilot[GitHub Copilot]
    end

    OpenAI --> Mux
    Anthropic --> Mux
    Google --> Mux
    Ollama --> Mux

    Mux -->|Translation Layer| Gemini
    Mux -->|Translation Layer| Claude
    Mux -->|Translation Layer| Copilot

🔌 Supported API Protocols

You can use llm-mux as a drop-in replacement for these providers without changing your client code structure.

Protocol	Endpoint	Use Case
OpenAI	`POST /v1/chat/completions`	Cursor, VS Code, LangChain
Anthropic	`POST /v1/messages`	Native Anthropic SDKs, Claude-native tools
Gemini	`POST /v1beta/models/...`	Google AI Studio tools, Vertex AI SDKs
Ollama	`POST /api/chat`	WebUIs built for Ollama

⚙️ Advanced Configuration

Load Balancing & Multiple Accounts

You can login multiple times with different accounts to distribute load and increase quotas.

llm-mux --login              # Account 1
llm-mux --login              # Account 2

The system will automatically rotate requests and handle failovers.

Docker Usage

If you prefer containerization:

docker run -p 8317:8317 -v ~/.config/llm-mux:/root/.config/llm-mux nghyane/llm-mux

Custom Proxies

Edit ~/.config/llm-mux/config.yaml to add upstream proxies:

proxy-url: "socks5://user:pass@proxy.example.com:1080"

License

MIT License — see LICENSE

Directories ¶

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL

Path	Synopsis
cmd
server command Package main provides the entry point for the CLI Proxy API server.	Package main provides the entry point for the CLI Proxy API server.
examples
custom-provider command Package main demonstrates creating a custom AI provider executor and integrating it with the llm-mux server.	Package main demonstrates creating a custom AI provider executor and integrating it with the llm-mux server.
internal
access
access/config_access
api Package api provides the HTTP API server implementation for the CLI Proxy API.	Package api provides the HTTP API server implementation for the CLI Proxy API.
api/handlers/management Package management provides the management API handlers and middleware for configuring the server and managing auth files.	Package management provides the management API handlers and middleware for configuring the server and managing auth files.
api/middleware Package middleware provides HTTP middleware components for the CLI Proxy API server.	Package middleware provides HTTP middleware components for the CLI Proxy API server.
api/modules Package modules provides a pluggable routing module system for extending the API server with optional features without modifying core routing logic.	Package modules provides a pluggable routing module system for extending the API server with optional features without modifying core routing logic.
api/modules/amp Package amp implements the Amp CLI routing module, providing OAuth-based integration with Amp CLI for ChatGPT and Anthropic subscriptions.	Package amp implements the Amp CLI routing module, providing OAuth-based integration with Amp CLI for ChatGPT and Anthropic subscriptions.
auth Package auth provides authentication functionality for various AI service providers.	Package auth provides authentication functionality for various AI service providers.
auth/claude Package claude provides OAuth2 authentication functionality for Anthropic's Claude API.	Package claude provides OAuth2 authentication functionality for Anthropic's Claude API.
auth/cline
auth/codex Package codex provides authentication and token management for OpenAI's Codex API.	Package codex provides authentication and token management for OpenAI's Codex API.
auth/copilot Package copilot provides authentication for GitHub Copilot API.	Package copilot provides authentication for GitHub Copilot API.
auth/gemini Package gemini provides authentication and token management functionality for Google's Gemini AI services.	Package gemini provides authentication and token management functionality for Google's Gemini AI services.
auth/iflow
auth/kiro
auth/qwen Package qwen provides authentication and token management functionality for Alibaba's Qwen AI services.	Package qwen provides authentication and token management functionality for Alibaba's Qwen AI services.
auth/vertex Package vertex provides token storage for Google Vertex AI Gemini via service account credentials.	Package vertex provides token storage for Google Vertex AI Gemini via service account credentials.
browser Package browser provides cross-platform functionality for opening URLs in the default web browser.	Package browser provides cross-platform functionality for opening URLs in the default web browser.
buildinfo Package buildinfo exposes compile-time metadata shared across the server.	Package buildinfo exposes compile-time metadata shared across the server.
cmd Package cmd provides command-line interface functionality for the CLI Proxy API server.	Package cmd provides command-line interface functionality for the CLI Proxy API server.
config Package config provides configuration management for the CLI Proxy API server.	Package config provides configuration management for the CLI Proxy API server.
constant Package constant defines provider name constants used throughout the CLI Proxy API.	Package constant defines provider name constants used throughout the CLI Proxy API.
embedded Package embedded provides access to generated default configuration.	Package embedded provides access to generated default configuration.
interfaces Package interfaces defines the core interfaces and shared structures for the CLI Proxy API server.	Package interfaces defines the core interfaces and shared structures for the CLI Proxy API server.
logging Package logging provides Gin middleware for HTTP request logging and panic recovery.	Package logging provides Gin middleware for HTTP request logging and panic recovery.
misc Package misc provides miscellaneous utility functions and embedded data for the CLI Proxy API.	Package misc provides miscellaneous utility functions and embedded data for the CLI Proxy API.
oauth Package oauth provides OAuth HTML response generation.	Package oauth provides OAuth HTML response generation.
registry Package registry provides shared Gemini model metadata for all Google providers.	Package registry provides shared Gemini model metadata for all Google providers.
runtime/executor Package executor provides common functionality for provider executors.	Package executor provides common functionality for provider executors.
runtime/geminicli
store
translator/from_ir Package from_ir converts unified request format to provider-specific formats.	Package from_ir converts unified request format to provider-specific formats.
translator/ir Package ir provides memory pools for the translator layer.	Package ir provides memory pools for the translator layer.
translator/to_ir Package to_ir converts provider-specific API formats into unified format.	Package to_ir converts provider-specific API formats into unified format.
usage Package usage provides usage tracking and logging functionality for the CLI Proxy API server.	Package usage provides usage tracking and logging functionality for the CLI Proxy API server.
util Package util provides utility functions for the llm-mux project.	Package util provides utility functions for the llm-mux project.
watcher Package watcher provides file system monitoring functionality for the CLI Proxy API.	Package watcher provides file system monitoring functionality for the CLI Proxy API.
wsrelay
sdk
access
api/handlers
api/handlers/claude
api/handlers/gemini
api/handlers/ollama
api/handlers/openai
auth
cliproxy Package cliproxy provides the core service implementation for the CLI Proxy API.	Package cliproxy provides the core service implementation for the CLI Proxy API.
cliproxy/auth
cliproxy/executor
cliproxy/pipeline
cliproxy/usage
config Package config provides configuration management for the CLI Proxy API server.	Package config provides configuration management for the CLI Proxy API server.
translator Package translator provides format definitions for request/response translation.	Package translator provides format definitions for request/response translation.