llm-router

command module
v0.8.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 6, 2026 License: MIT Imports: 2 Imported by: 0

README

llm-router

Single-binary OpenAI-compatible routing gateway. Go backend, Svelte dashboard, SQLite storage, one binary to run.

Problems it solves

One central endpoint for every model. Seamlessly route through multiple providers and accounts. Never hit rate limits. Bypass region blocks with automatic proxying.

What makes it different

Go plus Svelte, low memory footprint. One self-contained binary: no runtime, no external services, quick installation.

Free providers via Lua plugins. Install them from the catalog and use immediately.

Speaks OpenAI- and Anthropic-compatible formats. Connect your harness to the router and you're done.

Install from releases

Download the latest release: https://github.com/TheSlopMachine/llm-router/releases/latest

Install from source

make install

Configure

Configure via .env file or environment variables.

llm-router config init
llm-router set-dashboard-password
llm-router daemon start

This starts one public listener on PORT (default 8080) for the dashboard and /v1. The dashboard asks for the password only: it has one operator and no accounts. The server does not start until the password is set.

llm-router help lists every command. The main ones:

Command Purpose
serve Run in the foreground (systemd, Docker).
daemon start|stop|restart|status Run in the background (Linux, macOS, FreeBSD).
config Interactive settings menu; config get|set|unset|list|init for scripts.
set-dashboard-password Set the dashboard password.
logs Print server events (-f, --subsystem, --severity, ...).
status, doctor Inspect the running server, check the database.
update Install the latest release.

Only one instance runs per database. A background server does not start again after a reboot; for that, run serve under a supervisor.

Provider keys are encrypted in the database. The first start creates the key file ~/.config/llm-router/credentials.key; back it up together with the database, because credentials cannot be decrypted without it. CREDENTIALS_KEY (base64, 32 bytes) or CREDENTIALS_KEY_FILE replace it, for example with a container secret.

Output is Russian or English: LLM_ROUTER_LANG (ru or en), else LC_ALL, LC_MESSAGES, LANG or the Windows user locale; anything but Russian gives English. Event messages, --json, config values and command names stay English.

systemd and Docker

Run llm-router serve in the foreground; the supervisor restarts it. Set the password before serve, for example in the container entrypoint. --if-absent keeps a password changed in the dashboard across restarts:

llm-router config init
llm-router set-dashboard-password --password-stdin --if-absent < /run/secrets/dashboard_password
exec llm-router serve

Usage

  1. Install plugins.
  2. Add credentials.
  3. Create a new token.
  4. Pick a model.
  5. Connect your harness to the router. Set the base URL to http://HOST:PORT/v1 and the API key to the token you've just created.

Documentation

The Go Gopher

There is no documentation for this package.

Directories

Path Synopsis
Package cmd defines the llm-router command line using cobra.
Package cmd defines the llm-router command line using cobra.
internal
adapters/generic
Package generic provides a generic OpenAI-compatible backend for custom providers.
Package generic provides a generic OpenAI-compatible backend for custom providers.
api/v1
Package v1 implements the OpenAI-compatible /v1/...
Package v1 implements the OpenAI-compatible /v1/...
cli
Package cli holds the pure helpers of the command line: argument parsing and text formatting with no dependency on the server.
Package cli holds the pure helpers of the command line: argument parsing and text formatting with no dependency on the server.
config
Package config holds the runtime configuration populated from environment variables with optional .env file support.
Package config holds the runtime configuration populated from environment variables with optional .env file support.
credentialcipher
Package credentialcipher encrypts stored credential data at rest with AES-256-GCM.
Package credentialcipher encrypts stored credential data at rest with AES-256-GCM.
dashboard
Package dashboard serves the admin SPA and its backing JSON API.
Package dashboard serves the admin SPA and its backing JSON API.
db
Package db manages the embedded database handles.
Package db manages the embedded database handles.
db/migrate
Package migrate owns the forward-only SQLite migration registry and runner.
Package migrate owns the forward-only SQLite migration registry and runner.
db/migrations
Package migrations owns numbered SQLite schema changes.
Package migrations owns numbered SQLite schema changes.
db/record
Package record owns GORM persistence structs.
Package record owns GORM persistence structs.
errors
Package errors defines standard error types used across llm-router services.
Package errors defines standard error types used across llm-router services.
httpapi
Package httpapi owns the HTTP route tables shared by both listeners.
Package httpapi owns the HTTP route tables shared by both listeners.
httpkit
Package httpkit holds shared HTTP transport helpers for the API surfaces.
Package httpkit holds shared HTTP transport helpers for the API surfaces.
i18n
Package i18n translates the text the command line prints for a person: help, errors and status labels.
Package i18n translates the text the command line prints for a person: help, errors and status labels.
instance
Package instance guards one llm-router process per database.
Package instance guards one llm-router process per database.
logging
Package logging provides the bracket log handler for llm-router plus the events handler that persists records at or above the configured level to the events database while rendering them to stderr.
Package logging provides the bracket log handler for llm-router plus the events handler that persists records at or above the configured level to the events database while rendering them to stderr.
models
Package models compatibility additions: OpenAI prior-generation and Anthropic-compatible wire types plus the router-side async stores backing responses, conversations, assistants, threads, runs and message batches.
Package models compatibility additions: OpenAI prior-generation and Anthropic-compatible wire types plus the router-side async stores backing responses, conversations, assistants, threads, runs and message batches.
netguard
Package netguard owns the outbound-address policy: which destinations the router may dial on behalf of untrusted input (client-supplied URLs, plugin requests, third-party proxy feeds).
Package netguard owns the outbound-address policy: which destinations the router may dial on behalf of untrusted input (client-supplied URLs, plugin requests, third-party proxy feeds).
requestsource
Package requestsource identifies which listener served a request.
Package requestsource identifies which listener served a request.
server
Package server wires all services together and starts the HTTP servers.
Package server wires all services together and starts the HTTP servers.
services/batches
SQLiteService persists batches in SQLite: one header row in message_batches plus ordered child rows in message_batch_entries and message_batch_results.
SQLiteService persists batches in SQLite: one header row in message_batches plus ordered child rows in message_batch_entries and message_batch_results.
services/config
Package config manages RouterConfiguration persisted in the instance_settings row.
Package config manages RouterConfiguration persisted in the instance_settings row.
services/datamanagement
GORM-backed data management.
GORM-backed data management.
services/doctor
GORM-backed database inspection and repair.
GORM-backed database inspection and repair.
services/events
Package events owns operator telemetry: append-only event records with free-form metadata keys, retention pruning, and filtered queries for GET /api/events.
Package events owns operator telemetry: append-only event records with free-form metadata keys, retention pruning, and filtered queries for GET /api/events.
services/healthcheck
Package healthcheck owns failure-triggered credential health verification.
Package healthcheck owns failure-triggered credential health verification.
services/luaplugin/statepool
Package statepool holds idle Lua states in per-key buckets.
Package statepool holds idle Lua states in per-key buckets.
services/maintenance
GORM-backed maintenance scheduling.
GORM-backed maintenance scheduling.
services/metrics
Package metrics implements the Metrics Service for tracking API usage.
Package metrics implements the Metrics Service for tracking API usage.
services/pluginrepo
Package pluginrepo manages external plugin repositories (the plugin store).
Package pluginrepo manages external plugin repositories (the plugin store).
services/proxypool
Package proxypool adapts the proxypool library to router policy and storage.
Package proxypool adapts the proxypool library to router policy and storage.
services/router
Package router implements the Router Service.
Package router implements the Router Service.
services/token
Package token implements the Internal Token Service.
Package token implements the Internal Token Service.
services/videojobs
Package videojobs persists router-side video generation jobs.
Package videojobs persists router-side video generation jobs.
testutil
Package testutil provides shared test utilities for llm-router tests.
Package testutil provides shared test utilities for llm-router tests.
testutil/mockplugin
Package mockplugin owns deterministic Lua plugin fixtures for Go tests.
Package mockplugin owns deterministic Lua plugin fixtures for Go tests.
updater
Package updater checks GitHub releases and replaces the running binary.
Package updater checks GitHub releases and replaces the running binary.
util
Package util provides shared utility functions for the llm-router framework.
Package util provides shared utility functions for the llm-router framework.
version
Package version owns the Version struct plus Parse/String helpers.
Package version owns the Version struct plus Parse/String helpers.
providers
virtual
Package virtual implements the virtual-model provider backend: a virtual model routes each request to its models in list order (fall-through) and fails only when every model failed.
Package virtual implements the virtual-model provider backend: a virtual model routes each request to its models in list order (fall-through) and fails only when every model failed.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL