Documentation
¶
Index ¶
- Constants
- func ApplyTokenizerEndpoint(config *kvcache.Config)
- func ConfigureInternalTokenizer(config *kvcache.Config, modelName string) errordeprecated
- func SetupEventsPool(ctx context.Context, kvBlockIndex kvblock.Index) (*kvevents.Pool, error)
- func SetupKVCacheIndexer(ctx context.Context) (*kvcache.Indexer, error)
- func SimulateProduceEvent(ctx context.Context, publisher *Publisher) error
- func SimulateRemoveEvent(ctx context.Context, publisher *Publisher) error
- type Publisher
Constants ¶
View Source
const ( // EnvTokenizerEndpoint is the env var for the UDS tokenizer socket path or TCP address. // Use a path (e.g. /tmp/tokenizer/tokenizer-uds.socket) for UDS mode, // or host:port (e.g. localhost:50051) for TCP mode. EnvTokenizerEndpoint = "TOKENIZER_ENDPOINT" //nolint:gosec // env var name, not a credential )
Variables ¶
This section is empty.
Functions ¶
func ApplyTokenizerEndpoint ¶ added in v0.8.0
ApplyTokenizerEndpoint reads TOKENIZER_ENDPOINT and overrides the UDS config on the given indexer config. No-op when TokenizersPoolConfig is nil.
func ConfigureInternalTokenizer
deprecated
added in
v0.8.0
func SetupEventsPool ¶
func SimulateProduceEvent ¶
Types ¶
type Publisher ¶
type Publisher struct {
// contains filtered or unexported fields
}
Publisher sends KV cache events to a ZMQ endpoint.
func NewPublisher ¶
NewPublisher creates a new ZMQ publisher. endpoint is the ZMQ address to connect to (e.g., "tcp://localhost:5557").
Source Files
Click to show internal directories.
Click to hide internal directories.