requestcontrol

Package probabilisticadmitter implements a binary-tier probabilistic admission control plugin.

Package multimodal provides a data producer for multimodal encoder-cache affinity.

Package preciseprefixcache hashes TokenizedPrompt into KV-block keys, looks them up in the index, and writes per-endpoint PrefixCacheMatchInfo.

Package requestcontrol contains helpers to decouple latency-predictor logic.

Package sessionid provides a DataProducer that extracts a session identifier from a configured request header or named cookie and publishes it as a SessionID attribute on the InferenceRequest attribute store, so that affinity-aware scorers and filters can read it without knowing how the session was carried on the wire.

Package tokenizer provides a DataProducer plugin that tokenizes the request prompt and publishes the result on InferenceRequestBody.TokenizedPrompt for downstream consumers (scorers, filters, other data producers).

Package agentidentity provides a PreAdmitter plugin that resolves agent identity from provider-specific headers into the FairnessID field.