Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func PreparePayload ¶
func PreparePayload(routingCtx *types.RoutingContext, pod *v1.Pod, llmEngine string, handler engine.EngineHandler) ([]byte, error)
PreparePayload transforms routingCtx.ReqBody into a prefill-specific payload ready to be POSTed to the prefill pod. It delegates all engine-specific field injection to handler.AugmentPrefillRequest, then applies common constraints: max_tokens=1, stream=false, no stream_options, no min_tokens. TRT-LLM does not accept max_completion_tokens, so that field is omitted for that engine.
This function is exported so pdRouter can expose a thin backward-compat wrapper for tests that call preparePrefillPayload directly.
Types ¶
type DefaultExecutor ¶
type DefaultExecutor struct {
// contains filtered or unexported fields
}
DefaultExecutor is the standard PrefillExecutor. It owns the HTTP client and the prefill-request tracker so that prefill logic can be tested in isolation from the routing/scoring concerns in pdRouter.
func (*DefaultExecutor) Execute ¶
func (e *DefaultExecutor) Execute(routingCtx *types.RoutingContext, prefillPod *v1.Pod, llmEngine string, logCtx LogContext) error
Execute implements PrefillExecutor.
type LogContext ¶
LogContext carries router-resolved metadata included in prefill structured logs.
type PrefillExecutor ¶
type PrefillExecutor interface {
Execute(routingCtx *types.RoutingContext, prefillPod *v1.Pod, llmEngine string, logCtx LogContext) error
}
PrefillExecutor executes the prefill phase of a disaggregated-inference request. Implementations are responsible for payload preparation, HTTP dispatch (async or sync depending on the engine), and tracker lifecycle.
func NewDefaultExecutor ¶
func NewDefaultExecutor(httpClient *http.Client, tracker *pd.PrefillRequestTracker, requestTimeout int) PrefillExecutor
NewDefaultExecutor constructs a DefaultExecutor. httpClient and tracker are shared with the router; requestTimeout is in seconds.
Source Files