Documentation
¶
Overview ¶
Package corrections is Harbor's provider correction layer ( — RFC §6.5). It sits BETWEEN the runtime and the `llm.LLMClient` driver, rewriting `CompleteRequest`s per a per-model `CorrectionsProfile` before delegating, and optionally backfilling `CompleteResponse.Usage` from the request's byte length.
Compose order is settled:
Open() → corrections.Wrap(safety.New(driver))
— the corrections wrapper is the OUTERMOST layer so the safety pass sees the POST-correction request (the final outgoing payload reaching the driver). Materialization, leak-detection, and the token-budget guard all run against the corrected payload, so any future correction that grows token count is caught by the safety pass.
Five quirks:
- Message reordering — NIM and some OpenAI-compatible proxies reject mid-thread `system` messages. `OrderingSystemFirstStrict` collapses all system messages to the front.
- Schema sanitization — OpenAI's structured-output mode requires `additionalProperties:false`+`strict:true` at every nested object schema; other providers reject those keys. `SchemaSanitizationMode` flips between adding / stripping them.
- Reasoning-effort routing — thinking-class models (`o1`, `o3`, `deepseek-reasoner`) consume the effort hint via a provider-specific path. `ReasoningRouteThinking` moves the hint from the top-level field into `req.Extra["reasoning_effort"]`.
- Response-format envelope translation — `ResponseFormatJSONOnly` downgrades `FormatJSONSchema` to `FormatJSONObject` for providers that reject `json_schema`; `ResponseFormatAnthropic` packages the schema into Anthropic's tool-schema envelope inside `req.Extra`.
- Usage backfill — some streaming proxies report `0/0` tokens. `UsageBackfillEnabled` substitutes an estimate computed from request input length + response content length when the driver returns an all-zeros `Usage`.
Scope is structured-output and message-shape correctness only — NEVER provider-native tool dispatch (RFC §6.4). The smoke static guard extends to this package; the banned provider-native tool-call API symbols MUST NOT appear here. The runtime owns tool dispatch entirely.
Concurrent-reuse: the wrapper is stateless across calls. `Wrap` returns a value that holds an inner `LLMClient` and a `ConfigSnapshot`; both are read-only after construction. The per- call work allocates fresh slices/maps so concurrent callers never share mutable state. The package's `corrections_test.go` pins this with N=128 invocations under `-race`.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func DefaultOutputModeFor ¶
func DefaultOutputModeFor(model string) llm.OutputMode
DefaultOutputModeFor returns the per-known-provider default `OutputMode` for the supplied model name. Harbor reads this when the operator's profile leaves `OutputMode` unset.
Defaults:
- `openai/*` (non-reasoning) → Native (OpenAI supports native json_schema).
- `openai/o1*`, `openai/o3*`, `*deepseek-reasoner*` → Prompted (thinking-class models behave better with prompted JSON).
- `anthropic/*` → Native (Anthropic supports structured outputs).
- `google/*` → Native.
- `nim/*` and any other prefix → Prompted (the conservative fallback that works for OpenAI-compatible endpoints without native schema support).
The result is purely a fallback — `ProfileFor` returns the operator-supplied `OutputMode` when present.
func ProfileFor ¶
func ProfileFor(cfg llm.ConfigSnapshot, model string) llm.CorrectionsProfile
ProfileFor returns the `CorrectionsProfile` for the given model name. The lookup is operator-config-first; when the operator hasn't declared a profile, a per-known-provider default applies based on the model name prefix.
The defaults table is intentionally small and conservative: the per-provider quirks catalogued in the MCP research notes are encoded as defaults so an operator who omits the corrections block still gets sensible behaviour when running against a known model. An operator who overrides the profile in `harbor.yaml` wins.
This function is exported for tests; production code reads `cfg.ModelProfiles[req.Model].Corrections` directly. The function exists to demonstrate the defaults table and let smoke tests prove the wiring works.
func Wrap ¶
Wrap composes the corrections layer on top of `inner`. The returned `llm.LLMClient` rewrites requests per the profile in `cfg.ModelProfiles[req.Model].Corrections` (zero-valued profile = no-op) and optionally backfills `Usage` on the response.
Production callers wire this through `llm.Open()`; tests that need to exercise the wrapper in isolation can call `Wrap` directly.
A nil `inner` panics — composition error caught at boot.
Types ¶
This section is empty.
Source Files