Documentation
¶
Overview ¶
Package proxy provides an LLM inference proxy that stores conversations in a Merkle DAG.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
// ListenAddr is the address to listen on (e.g., ":8080")
ListenAddr string
// UpstreamURL is the upstream LLM provider URL (e.g., "http://localhost:11434")
UpstreamURL string
// ProviderType specifies the LLM provider type (e.g., "anthropic", "openai", "ollama")
// This determines how requests and responses are parsed.
ProviderType string
// VectorDriver is an optional vector store for storing embeddings.
// If nil, vector storage is disabled.
VectorDriver vector.Driver
// Embedder is an optional embedder for generating embeddings.
// Required if VectorDriver is set.
Embedder embeddings.Embedder
}
Config is the proxy server configuration.
type Proxy ¶
type Proxy struct {
// contains filtered or unexported fields
}
Proxy is a client, LLM inference proxy that instruments storing sessions as Merkle DAGs. The proxy is transparent: it forwards requests to the upstream LLM provider and enqueues conversation turns for async storage via its worker pool.
func New ¶
New creates a new Proxy. The storer is injected to handle async persistence of conversation turns. Returns an error if the configured provider type is not recognized.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package header provides header filtering for the tapes proxy.
|
Package header provides header filtering for the tapes proxy. |
|
Package worker provides an asynchronous worker pool and utils for persisting conversation turns using the provided storage.Driver and generating embeddings using the provided embeddings.Embedder.
|
Package worker provides an asynchronous worker pool and utils for persisting conversation turns using the provided storage.Driver and generating embeddings using the provided embeddings.Embedder. |
Click to show internal directories.
Click to hide internal directories.