extproc

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 21, 2025 License: Apache-2.0 Imports: 35 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func StartConfigWatcher

func StartConfigWatcher(ctx context.Context, path string, rcv ConfigReceiver, l *slog.Logger, tick time.Duration) error

StartConfigWatcher starts a watcher for the given path and Receiver. Periodically checks the file for changes and calls the Receiver's UpdateConfig method.

Types

type ConfigReceiver

type ConfigReceiver interface {
	// LoadConfig updates the configuration.
	LoadConfig(ctx context.Context, config *filterapi.Config) error
}

ConfigReceiver is an interface that can receive *filterapi.Config updates. This is mostly for decoupling and testing purposes.

type Processor

type Processor interface {
	// ProcessRequestHeaders processes the request headers message.
	ProcessRequestHeaders(context.Context, *corev3.HeaderMap) (*extprocv3.ProcessingResponse, error)
	// ProcessRequestBody processes the request body message.
	ProcessRequestBody(context.Context, *extprocv3.HttpBody) (*extprocv3.ProcessingResponse, error)
	// ProcessResponseHeaders processes the response headers message.
	ProcessResponseHeaders(context.Context, *corev3.HeaderMap) (*extprocv3.ProcessingResponse, error)
	// ProcessResponseBody processes the response body message.
	ProcessResponseBody(context.Context, *extprocv3.HttpBody) (*extprocv3.ProcessingResponse, error)
	// SetBackend instructs the processor to set the backend to use for the request. This is only called
	// when the processor is used in the upstream filter.
	//
	// routerProcessor is the processor that is the "parent" which was used to determine the route at the
	// router level. It holds the additional state that can be used to determine the backend to use.
	SetBackend(ctx context.Context, backend *filterapi.Backend, handler backendauth.Handler, routerProcessor Processor) error
}

Processor is the interface for the processor which corresponds to a single gRPC stream per the external processor filter. This decouples the processor implementation detail from the server implementation.

This can be either a router filter level processor or an upstream filter level processor.

func NewModelsProcessor

func NewModelsProcessor(config *processorConfig, _ map[string]string, logger *slog.Logger, _ tracing.Tracing, isUpstreamFilter bool) (Processor, error)

NewModelsProcessor creates a new processor that returns the list of declared models.

type ProcessorFactory

type ProcessorFactory func(_ *processorConfig, _ map[string]string, _ *slog.Logger, _ tracing.Tracing, isUpstreamFilter bool) (Processor, error)

ProcessorFactory is the factory function used to create new instances of a processor.

func ChatCompletionProcessorFactory added in v0.1.3

func ChatCompletionProcessorFactory(ccm metrics.ChatCompletionMetrics) ProcessorFactory

ChatCompletionProcessorFactory returns a factory method to instantiate the chat completion processor.

func EmbeddingsProcessorFactory added in v0.3.0

func EmbeddingsProcessorFactory(em metrics.EmbeddingsMetrics) ProcessorFactory

EmbeddingsProcessorFactory returns a factory method to instantiate the embeddings processor.

func MessagesProcessorFactory added in v0.3.0

func MessagesProcessorFactory(ccm metrics.ChatCompletionMetrics) ProcessorFactory

MessagesProcessorFactory returns a factory for the Anthropic /v1/messages endpoint.

Requests: Only accepts Anthropic format requests. Responses: Returns Anthropic format responses.

type Server

type Server struct {
	// contains filtered or unexported fields
}

Server implements the external processor server.

func NewServer

func NewServer(logger *slog.Logger, tracing tracing.Tracing) (*Server, error)

NewServer creates a new external processor server.

func (*Server) List added in v0.2.0

List implements grpc_health_v1.HealthServer.

func (*Server) LoadConfig

func (s *Server) LoadConfig(ctx context.Context, config *filterapi.Config) error

LoadConfig updates the configuration of the external processor.

func (*Server) Process

Process implements extprocv3.ExternalProcessorServer.

func (*Server) Register

func (s *Server) Register(path string, newProcessor ProcessorFactory)

Register a new processor for the given request path.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL