Documentation
¶
Overview ¶
Package toolratelimit wires the per-user tool-call rate limiter (#929) into the platform: an MCP receiving middleware that refuses authenticated tools/call requests exceeding a generous per-identity token-bucket limit before they reach the handler, audit pipeline, or upstream.
A script principal is the exception (#1534): a platform run is a loop by construction, its calls are serial, and it runs under roles captured at the save through the persona filter, so an over-limit call from it is held until the bucket admits it rather than refused. The sustained rate still governs the run's throughput; what differs is that the call is delayed, not failed.
It lives in its own package so the platform facade stays within its field/method budget and so the ratelimit dependency stays localized to a cohesive, independently-testable seam (mirroring reflexivecapture and the other internal/platform middleware seams). Platform builds a Handle inside the middleware-chain registration and registers Close on the lifecycle, so no Platform field or method is added.
The limiter is a safety net, not a throughput throttle: with the generous platform defaults, ordinary interactive and agent use never touches it, but a runaway agent loop or a compromised account is bounded before it can saturate shared resources. See pkg/platform.RateLimitConfig for the keying and per-replica rationale.
Index ¶
Constants ¶
const CodeRateLimited = "rate_limited"
CodeRateLimited is the stable machine-readable error code a refused call carries, the one an agent branches on to back off and retry rather than treat the refusal as a caller-input fault or a platform outage, and the one the script engine (internal/platform/scriptrun) absorbs by pacing the call. It mirrors the taxonomy in pkg/middleware and lives with its producer, as the search/session gate categories do in their own files.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Handle ¶
type Handle struct {
// contains filtered or unexported fields
}
Handle is the per-user tool-call rate limiter. It wraps a token bucket keyed on the authenticated principal (middleware.PlatformContext.RateLimitKey) and refuses over-limit tools/call requests with the self-describing error contract before they reach the handler, audit pipeline, or upstream.
func New ¶
func New(requestsPerMinute, burst int, extraExempt []string, metrics *observability.Metrics) *Handle
New builds a Handle enforcing requestsPerMinute (sustained) with a bucket depth of burst per user. platform_info is always exempt; the tools in extraExempt are exempt in addition to it. metrics may be nil (the refusal counter is then a no-op). Call Close to stop the background eviction goroutine.
func (*Handle) Close ¶
func (h *Handle) Close()
Close stops the underlying limiter's background eviction goroutine. Idempotent and nil-safe.
func (*Handle) Middleware ¶
func (h *Handle) Middleware() mcp.Middleware
Middleware returns the MCP receiving middleware enforcing the per-user limit. On a refusal it short-circuits with a RATE_LIMITED error result and never invokes the underlying tool handler, so the throttled call consumes no handler, audit, or upstream work. A script principal's over-limit call is held here until a token is available and then handed on; a wait the request context ends first is returned as that context's error, so a run canceled mid-wait (its context is the call's, and the client forwards its cancellation to this side) ends as a canceled run rather than with a refusal, and the handler never runs for that call.
The middleware must be positioned INNER to MCPToolCallMiddleware so the PlatformContext (identity, tool name) is available, and OUTER to audit, metrics, tracing, and enrichment so a refused call never reaches those layers. It is positioned INNER to the session and workflow gates so it only meters calls that pass those gates and would actually execute — a gate-refused call is already cheap and self-limiting and should not consume a rate-limit token.
Source Files