worker

package
v0.0.1-rc.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 6, 2026 License: Apache-2.0 Imports: 10 Imported by: 0

Documentation

Overview

Package worker provides an asynchronous worker pool and utils for persisting conversation turns using the provided storage.Driver and generating embeddings using the provided embeddings.Embedder.

The pool decouples storage operations from the proxy's HTTP hot path so that the client-proxy-upstream interaction is fully transparent.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Config

type Config struct {
	// Driver is the storage backend for persisting nodes.
	Driver storage.Driver

	// VectorDriver is the optional vector store driver for embeddings.
	VectorDriver vector.Driver

	// Embedder generates optional text embeddings.
	// A configured Embedder is required if VectorDriver is set.
	Embedder embeddings.Embedder

	// NumWorkers is the number of background workers in the pool.
	NumWorkers uint

	// QueueSize is the capacity of the buffered job channel (defaults to 256).
	QueueSize uint

	// Logger is the provided zap logger
	Logger *zap.Logger
}

Config is the configuration options for the worker pool.

type Job

type Job struct {
	Provider string
	Req      *llm.ChatRequest
	Resp     *llm.ChatResponse
}

Job is a unit of work for the worker pool to execute against.

type Pool

type Pool struct {
	// contains filtered or unexported fields
}

Pool processes storage jobs asynchronously via a worker pool.

func NewPool

func NewPool(c *Config) (*Pool, error)

NewPool creates a new Storer and starts its worker goroutines.

func (*Pool) Close

func (p *Pool) Close()

Close signals workers to stop and waits for in-flight jobs to drain. Call this during graceful shutdown after the proxy HTTP server has stopped.

func (*Pool) Enqueue

func (p *Pool) Enqueue(job Job) bool

Enqueue submits a job for processing by the worker pool. Returns true if enqueued, false if the queue is full, resulting in the job being dropped

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL