Documentation
¶
Overview ¶
Package builtin holds the package-level middleware registry that concrete middleware packages register themselves into via init(). Server boot anonymous-imports each middleware sub-package; the resolver attached to the middleware Manager pulls factories out of this registry.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Configure ¶
func Configure(ctx context.Context, dataDir string, meter metric.Meter, logger *log.Logger, mgmt MgmtClient)
Configure stores the per-process FactoryContext. Concrete factories reach for it via Context(). mgmt may be nil on tests / standalone builds with no management server; consumers must guard.
func DefaultRegistry ¶
func DefaultRegistry() *middleware.Registry
DefaultRegistry returns the shared registry. The proxy server constructs the Resolver from it at boot.
func Register ¶
func Register(f middleware.Factory)
Register adds a factory to the default registry. Called from init() blocks of concrete middleware packages. Panics on collision so duplicate IDs surface at startup.
Types ¶
type FactoryContext ¶
type FactoryContext struct {
Context context.Context
DataDir string
Meter metric.Meter
Logger *log.Logger
MgmtClient MgmtClient
}
FactoryContext is the per-process bag that concrete factories may consult during construction. It carries the proxy-lifetime context, the data directory used for static config files (pricing tables, allowlists), the OTel meter, and the proxy logger.
Configure must be called once at boot before any chain build calls Resolve. Calling it twice overwrites the prior value; tests may rely on this to reset state.
func Context ¶
func Context() FactoryContext
Context returns the stored FactoryContext. Returns a zero value when Configure was never called; consumers must guard against nil Context/Meter/Logger if they care.
type MgmtClient ¶
type MgmtClient interface {
CheckLLMPolicyLimits(ctx context.Context, in *proto.CheckLLMPolicyLimitsRequest, opts ...grpc.CallOption) (*proto.CheckLLMPolicyLimitsResponse, error)
RecordLLMUsage(ctx context.Context, in *proto.RecordLLMUsageRequest, opts ...grpc.CallOption) (*proto.RecordLLMUsageResponse, error)
}
MgmtClient is the narrow slice of proto.ProxyServiceClient that builtin middlewares may use during request / response handling. Only the agent-network limit pair (llm_limit_check + llm_limit_record) uses this today; declaring the surface here keeps the dependency explicit at boot time.
proto.ProxyServiceClient already satisfies this interface so server boot just forwards its existing client.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package cost_meter implements the SlotOnResponse middleware that converts token-usage metadata emitted by llm_response_parser into a per-request USD cost estimate.
|
Package cost_meter implements the SlotOnResponse middleware that converts token-usage metadata emitted by llm_response_parser into a per-request USD cost estimate. |
|
Package llm_guardrail implements the SlotOnRequest middleware that enforces the per-target LLM guardrail policy: a model allowlist check and an opt-in prompt-capture step that may run a PII redactor before emitting the prompt into the metadata bag.
|
Package llm_guardrail implements the SlotOnRequest middleware that enforces the per-target LLM guardrail policy: a model allowlist check and an opt-in prompt-capture step that may run a PII redactor before emitting the prompt into the metadata bag. |
|
Package llm_identity_inject implements the SlotOnRequest middleware that stamps the caller's NetBird identity onto upstream LLM-gateway requests.
|
Package llm_identity_inject implements the SlotOnRequest middleware that stamps the caller's NetBird identity onto upstream LLM-gateway requests. |
|
Package llm_limit_check is the SlotOnRequest middleware that asks management which agent-network policy "pays" for the current LLM request.
|
Package llm_limit_check is the SlotOnRequest middleware that asks management which agent-network policy "pays" for the current LLM request. |
|
Package llm_limit_record is the SlotOnResponse middleware that posts the served request's token + cost deltas back to management so the per-(user, group, window) consumption counters tick.
|
Package llm_limit_record is the SlotOnResponse middleware that posts the served request's token + cost deltas back to management so the per-(user, group, window) consumption counters tick. |
|
Package llm_request_parser implements the SlotOnRequest middleware that detects the LLM provider from the request URL, parses the JSON request body for model and streaming flags, and extracts the user prompt text.
|
Package llm_request_parser implements the SlotOnRequest middleware that detects the LLM provider from the request URL, parses the JSON request body for model and streaming flags, and extracts the user prompt text. |
|
Package llm_response_parser implements the SlotOnResponse middleware that decodes OpenAI- and Anthropic-shaped LLM responses (buffered or streaming) and emits token usage and completion metadata.
|
Package llm_response_parser implements the SlotOnResponse middleware that decodes OpenAI- and Anthropic-shaped LLM responses (buffered or streaming) and emits token usage and completion metadata. |
|
Package llm_router implements the SlotOnRequest middleware that routes a request to an upstream LLM provider based on the model name emitted upstream by llm_request_parser.
|
Package llm_router implements the SlotOnRequest middleware that routes a request to an upstream LLM provider based on the model name emitted upstream by llm_request_parser. |