Documentation
¶
Overview ¶
Package sessionkit holds the small backend-neutral helpers shared by the modeld transport.Session adapters (llama.cpp in modeld/llama/llamasession and OpenVINO in modeld/openvino). Both adapters drive the same EnsurePrefix/PrefillSuffix/Decode contract over genuinely different engines, so their KV mechanics stay separate — but the surrounding plumbing (cancel-safe stream sends, longest-common-prefix reuse, the chat-template role vocabulary) is identical and lives here once instead of drifting as per-adapter copies.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ChatRole ¶
ChatRole maps a manifest segment kind to a chat-template role, or "" for a control segment the model's own template renders itself (BOS, the assistant generation cue). Centralizing the role vocabulary keeps the two adapters from recognizing different role sets.
func CommonPrefixLen ¶
CommonPrefixLen returns the length of the longest shared prefix of a and b. The adapters use it to find how many already-resident tokens a new prefix can reuse before the divergent tail must be (re)prefilled.
func ResidencyReport ¶
func ResidencyReport(plan residency.Plan, errMsg string, caps residency.Capabilities) *transport.ResidencyReport
ResidencyReport maps a residency.Plan (plus any planning error and the backend's capabilities) onto the backend-neutral transport.ResidencyReport for explain-context observability, so the two adapters surface residency the same way. It returns nil when there is no plan to report (empty session).
Types ¶
This section is empty.
Source Files