Documentation
¶
Overview ¶
Package builtin holds the package-level middleware registry that concrete middleware packages register themselves into via init(). Server boot anonymous-imports each middleware sub-package; the resolver attached to the middleware Manager pulls factories out of this registry.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Configure ¶
Configure stores the per-process FactoryContext. Concrete factories reach for it via Context(). mgmt may be nil on tests / standalone builds with no management server; consumers must guard.
func DefaultRegistry ¶
func DefaultRegistry() *middleware.Registry
DefaultRegistry returns the shared registry. The proxy server constructs the Resolver from it at boot.
func Register ¶
func Register(f middleware.Factory)
Register adds a factory to the default registry. Called from init() blocks of concrete middleware packages. Panics on collision so duplicate IDs surface at startup.
Types ¶
type FactoryContext ¶
type FactoryContext struct {
Context context.Context
Meter metric.Meter
Logger *log.Logger
MgmtClient MgmtClient
}
FactoryContext is the per-process bag that concrete factories may consult during construction. It carries the proxy-lifetime context, the OTel meter, and the proxy logger.
Configure must be called once at boot before any chain build calls Resolve. Calling it twice overwrites the prior value; tests may rely on this to reset state.
func Context ¶
func Context() FactoryContext
Context returns the stored FactoryContext. Returns a zero value when Configure was never called; consumers must guard against nil Context/Meter/Logger if they care.
type MgmtClient ¶
type MgmtClient interface {
CheckLLMPolicyLimits(ctx context.Context, in *proto.CheckLLMPolicyLimitsRequest, opts ...grpc.CallOption) (*proto.CheckLLMPolicyLimitsResponse, error)
RecordLLMUsage(ctx context.Context, in *proto.RecordLLMUsageRequest, opts ...grpc.CallOption) (*proto.RecordLLMUsageResponse, error)
}
MgmtClient is the narrow slice of proto.ProxyServiceClient that builtin middlewares may use during request / response handling. Only the agent-network limit pair (llm_limit_check + llm_limit_record) uses this today; declaring the surface here keeps the dependency explicit at boot time.
proto.ProxyServiceClient already satisfies this interface so server boot just forwards its existing client.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package cost_meter implements the SlotOnResponse middleware that converts token-usage metadata emitted by llm_response_parser into a per-request USD cost estimate.
|
Package cost_meter implements the SlotOnResponse middleware that converts token-usage metadata emitted by llm_response_parser into a per-request USD cost estimate. |
|
Package llm_guardrail implements the SlotOnRequest middleware that enforces the per-target LLM guardrail policy: a model allowlist check and an opt-in prompt-capture step that may run a PII redactor before emitting the prompt into the metadata bag.
|
Package llm_guardrail implements the SlotOnRequest middleware that enforces the per-target LLM guardrail policy: a model allowlist check and an opt-in prompt-capture step that may run a PII redactor before emitting the prompt into the metadata bag. |
|
Package llm_identity_inject implements the SlotOnRequest middleware that stamps the caller's NetBird identity onto upstream LLM-gateway requests.
|
Package llm_identity_inject implements the SlotOnRequest middleware that stamps the caller's NetBird identity onto upstream LLM-gateway requests. |
|
Package llm_limit_check is the SlotOnRequest middleware that asks management which agent-network policy "pays" for the current LLM request.
|
Package llm_limit_check is the SlotOnRequest middleware that asks management which agent-network policy "pays" for the current LLM request. |
|
Package llm_limit_record is the SlotOnResponse middleware that posts the served request's token + cost deltas back to management so the per-(user, group, window) consumption counters tick.
|
Package llm_limit_record is the SlotOnResponse middleware that posts the served request's token + cost deltas back to management so the per-(user, group, window) consumption counters tick. |
|
Package llm_request_parser implements the SlotOnRequest middleware that detects the LLM provider from the request URL, parses the JSON request body for model and streaming flags, and extracts the user prompt text.
|
Package llm_request_parser implements the SlotOnRequest middleware that detects the LLM provider from the request URL, parses the JSON request body for model and streaming flags, and extracts the user prompt text. |
|
Package llm_response_parser implements the SlotOnResponse middleware that decodes OpenAI- and Anthropic-shaped LLM responses (buffered or streaming) and emits token usage and completion metadata.
|
Package llm_response_parser implements the SlotOnResponse middleware that decodes OpenAI- and Anthropic-shaped LLM responses (buffered or streaming) and emits token usage and completion metadata. |
|
Package llm_router implements the SlotOnRequest middleware that routes a request to an upstream LLM provider based on the model name emitted upstream by llm_request_parser.
|
Package llm_router implements the SlotOnRequest middleware that routes a request to an upstream LLM provider based on the model name emitted upstream by llm_request_parser. |