proxy

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 26, 2026 License: AGPL-3.0 Imports: 23 Imported by: 0

Documentation

Overview

Package proxy provides an LLM inference proxy that stores conversations in a Merkle DAG.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AgentRoute

type AgentRoute struct {
	ProviderType string
	UpstreamURL  string
}

AgentRoute defines proxy routing for a specific agent.

type Config

type Config struct {
	// ListenAddr is the address to listen on (e.g., ":8080")
	ListenAddr string

	// UpstreamURL is the upstream LLM provider URL (e.g., "http://localhost:11434")
	UpstreamURL string

	// ProviderType specifies the LLM provider type (e.g., "anthropic", "openai", "ollama")
	// This determines how requests and responses are parsed.
	ProviderType string

	// AgentRoutes maps agent names to provider routing configuration.
	AgentRoutes map[string]AgentRoute

	// ProviderUpstreams optionally overrides upstream URLs per provider.
	ProviderUpstreams map[string]string

	// VectorDriver is an optional vector store for storing embeddings.
	// If nil, vector storage is disabled.
	VectorDriver vector.Driver

	// Embedder is an optional embedder for generating embeddings.
	// Required if VectorDriver is set.
	Embedder embeddings.Embedder

	// Publisher is an optional event publisher for new DAG nodes.
	// If nil, publishing is disabled.
	Publisher publisher.Publisher

	// Project is the git repository or project name to tag on stored nodes.
	Project string
}

Config is the proxy server configuration.

type Proxy

type Proxy struct {
	// contains filtered or unexported fields
}

Proxy is a client, LLM inference proxy that instruments storing sessions as Merkle DAGs. The proxy is transparent: it forwards requests to the upstream LLM provider and enqueues conversation turns for async storage via its worker pool.

func New

func New(config Config, driver storage.Driver, log *slog.Logger) (*Proxy, error)

New creates a new Proxy. The storer is injected to handle async persistence of conversation turns. Returns an error if the configured provider type is not recognized.

func (*Proxy) Close

func (p *Proxy) Close() error

Close gracefully shuts down the proxy and waits for the worker pool to drain

func (*Proxy) Run

func (p *Proxy) Run() error

Run starts the proxy server on the given listening address

func (*Proxy) RunWithListener

func (p *Proxy) RunWithListener(listener net.Listener) error

RunWithListener starts the proxy server using the provided listener.

Directories

Path Synopsis
Package header provides header filtering for the tapes proxy.
Package header provides header filtering for the tapes proxy.
Package worker provides an asynchronous worker pool and utils for persisting conversation turns using the provided storage.Driver and generating embeddings using the provided embeddings.Embedder.
Package worker provides an asynchronous worker pool and utils for persisting conversation turns using the provided storage.Driver and generating embeddings using the provided embeddings.Embedder.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL