Documentation
¶
Overview ¶
Package sched provides the per-binding request scheduler: the admission control that decides which of a binding's requests may be in flight on its connection at once.
One MCP connection may be shared — a Session-scoped binding serves every Loop allowed to use it — so the requests on it come from callers that do not know about each other. This package is what makes that safe without making it serial by accident.
What it guarantees ¶
- Tool calls are serialized by default. A tool call has effects, and a server that was never written to expect two at once is the common case, not the exception. Parallelism is opt-in per binding (Config.AllowParallel) because only the application knows whether a particular server tolerates it. A server's own tool annotations may inform that decision, but they can never make it: annotations are untrusted input, and a server that could widen its own concurrency by claiming to be idempotent would be deciding the host's policy for it.
- Everything is bounded. MaxConcurrent caps in-flight requests of every class, whether or not parallel calls are allowed — enabling parallelism raises a serialization constraint, it does not remove a budget.
- Ordering-sensitive work is serialized among itself. Lifecycle, auth and catalog-refresh operations take one class and run one at a time, so two refreshes can never interleave into one candidate.
- Cancelling one request cancels one request. Each Begin returns its own derived context and its own cancel; nothing here is shared between two live requests but the counter.
- Shutdown rejects, then cancels. New work is refused before in-flight work is torn down, so nothing can slip in behind the teardown.
What it is not ¶
It is not a queue and not a fair scheduler. Waiters take their turn in whatever order the runtime hands them the channel, and nothing here promises FIFO. The bounds are the contract; the order in which two callers pass a bound is not one, because no caller can depend on it — a shared connection has no principled ordering between the requests of two unrelated Loops.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrShutdown = errors.New("sched: scheduler is shut down")
ErrShutdown reports that the scheduler is shut down: no further work is admitted. Callers classify it into their own taxonomy; this package has none.
Functions ¶
This section is empty.
Types ¶
type Class ¶
type Class uint8
Class is the kind of work a request is, which is what decides its serialization. The zero value is not a valid class.
const ( // ClassCall is a tool call: it may have effects, so it is serialized // against other tool calls unless the binding allows parallelism. ClassCall Class = iota + 1 // ClassRequest is any other server request — reading a resource, getting a // prompt. These are reads: they are bounded by the concurrency budget, but // not serialized against each other or against tool calls, because nothing // about their ordering is load-bearing. ClassRequest // ClassControl is an operation whose state transitions require ordering: // lifecycle, auth, catalog refresh. Control operations run one at a time. ClassControl // ClassReentrantCall is a tool call issued from within a sampling handler, // while the tool call that provoked the sampling is still in flight and still // holding the ClassCall permit. It is a call — it may have effects — but it // must NOT take that permit: the outer call it descends from holds it and will // not release it until the sampling round-trip this call is part of returns, // so queueing behind it is a self-deadlock that only the deadline breaks. // // It is therefore admitted without the tool-call serializer, and bounded // instead by the sampling depth and concurrency caps that already govern the // chain (the client's sampleGate). It still counts against the concurrency // budget like every other in-flight request: raising a serialization // constraint is not the same as removing a budget. ClassReentrantCall )
The request classes.
type Config ¶
type Config struct {
// MaxConcurrent caps in-flight requests of every class combined. Values
// below 1 are clamped to 1; see New.
MaxConcurrent int
// AllowParallel opts in to parallel tool calls, up to MaxConcurrent. It is
// the application's decision and never a server's.
AllowParallel bool
}
Config is a scheduler's policy.
type Scheduler ¶
type Scheduler struct {
// contains filtered or unexported fields
}
Scheduler admits a binding's requests. It is safe for concurrent use, and its zero value is not usable — call New.
func New ¶
New returns a Scheduler for cfg.
A MaxConcurrent below 1 is clamped to 1 rather than rejected or honored. This module's rule is that a non-positive bound fails closed rather than meaning "unbounded" — but a budget of zero does not refuse work, it blocks forever, which is a deadlock wearing a bound's clothes. One is the most restrictive budget that can still make progress, so it is the fail-closed value here. The client normalizes its limits before this is reached, so the clamp is defence in depth rather than a configuration path.
func (*Scheduler) Begin ¶
Begin admits one request of the given class, blocking until the binding's bounds allow it.
It returns a context for the request and a release func the caller must call exactly once, whatever the outcome of the request — the canonical use is a defer on the line after the error check. Releasing is what returns the budget and the serialization lock; a caller that forgets one wedges the binding.
The returned context is derived from ctx and is additionally cancelled by Shutdown. It is per-request: cancelling it, or cancelling the ctx behind it, affects this request and no other. That is the design's isolation rule, and it is structural here — there is no shared cancel for a caller to reach.
Waiting respects ctx: a caller that gives up while queued gets ctx's error and consumes nothing. Time spent waiting counts against the caller's own deadline, which is the point of passing an already-bounded context — a request that queues past its deadline has missed it, whether it waited on a bound or on a server.
It fails closed: an undeclared class is refused rather than admitted unserialized, and a shutdown scheduler admits nothing.
func (*Scheduler) InFlight ¶
InFlight reports how many admitted requests have not been released.
It exists for tests and diagnostics: the count is a fact about a moment that has passed by the time a caller reads it, so nothing may branch on it. It is not a substitute for release.
func (*Scheduler) Shutdown ¶
func (s *Scheduler) Shutdown()
Shutdown refuses all further work and cancels everything in flight, in that order. It is idempotent and safe to call concurrently.
The order is the point. Cancelling first and refusing second would leave a gap in which a request admitted between the two steps survives the shutdown that was supposed to cancel it — on a connection that is about to close.
It does not wait for the cancelled requests to return. What a cancel guarantees is that they stop soon and on their own terms; the caller (the client's Close) is what bounds the wait for them, because only it knows what it is willing to wait for.
Source Files