providerkit

package module
v0.6.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 11, 2026 License: MIT Imports: 25 Imported by: 0

README

CLIProxy provider kit

Shared Go building blocks for first-class CLIProxyAPI provider plugins. This repository is a library, not an installable provider or a scheduler. The echo plugin is a loopback-only example for tests.

  • SDK: github.com/router-for-me/CLIProxyAPI/v8 v8.0.63
  • Native C ABI 1, RPC schema 10. Minimum host: CLIProxyAPI v8.0.49 (schema 10); newer hosts are accepted, older hosts (schema 9 or below, v8.0.48 and earlier) are refused. Go 1.26.11+, CGO and a C compiler. Credential-scoped upstream errors (account-wide 429s) additionally require host v8.0.62 or newer; older schema-10 hosts may ignore the scope field.
  • Native libraries built with -buildmode=c-shared, not Go's plugin package
  • Linux is the CI-tested target. Match the deployment architecture and libc; build provider release artifacts in golang:1.26-bookworm for a Debian-bookworm host. The exact host/plugin Go patch versions need not match across the C ABI.

Provider repository usage

Add the kit at a reviewed commit (or a released version when available), and pin the SDK to v8.0.63. No kit release/tag is required to use a commit through Go's pseudo-version resolution.

go get github.com/TogetherWeOwn/cliproxy-provider-kit@<reviewed-commit>
go get github.com/router-for-me/CLIProxyAPI/v8@v8.0.63

Implement the SDK interfaces for your provider and pass them to providerkit.New. In the provider's package main, export a runtime factory from init. A c-shared library does not run main.

package main

import (
    providerkit "github.com/TogetherWeOwn/cliproxy-provider-kit"
    "github.com/TogetherWeOwn/cliproxy-provider-kit/cabi"
    "github.com/router-for-me/CLIProxyAPI/v8/sdk/pluginapi"
)

func main() {}

func init() {
    if err := cabi.Export(func(host providerkit.Caller) (*providerkit.Runtime, error) {
        return providerkit.New(providerkit.Definition{
            Metadata: pluginapi.Metadata{
                Name: "example-provider", Version: "0.1.0",
                Author: "Example Organization",
                GitHubRepository: "https://github.com/example/provider",
            },
            Capabilities: pluginapi.Capabilities{
                // API-key providers must omit AuthProvider: there is no login
                // flow (StartLogin/PollLogin fail with api_key_only), so a
                // registered provider would advertise a login card that can
                // only fail. Auth files load through the host's generic file
                // path instead; it does not call APIKeyAuth.ParseAuth or
                // Validate. Check provider-specific fields in the executor.
                // Add ModelRegistrar / ModelProvider, Executor, QuotaProvider,
                // and optionally UsagePlugin implemented by your provider.
            },
        }, host)
    }); err != nil {
        panic("cannot install provider factory")
    }
}

Use examples/echo/main.go for a complete executable example, including lifecycle configuration, static/per-auth models, auth headers, host HTTP execution and a cached quota provider. Its only permitted upstream is a literal loopback address supplied by the test harness. It has no production endpoint or real credentials.

Provider plugin standards

Owner rule (2026-11-01): any provider that serves models made by another vendor must namespace its model ids, so they never clash with the official provider's ids when it is added later. This is enforced by the kit, not left as a convention:

  • Declare the fixed namespace once in the Definition, as a bare segment: ModelNamespace: "synthetic". A definition that registers or serves models (ModelRegistrar or ModelProvider set) with an empty, missing or malformed namespace fails in New — a startup error, never a silent fallback.
  • The kit refuses to register any model whose id is not namespaced exactly once (<namespace>/…). That covers static, discovered and dynamic lists alike, including multi-provider Registrations: the offending model is dropped with a logged error and never served bare. Bare ids, foreign namespaces and doubled namespaces (nvidia/nvidia/…, usually a plugin prefixing an already-prefixed upstream id) are dropped; a second layer past the namespace (cloudflare/@cf/…) passes.
  • The executor keeps stripping exactly one leading <namespace>/ before the upstream request (plugin-side, as today). The host auth-file prefix is an outer host clone and does not replace the fixed model namespace.
  • Prove it in the plugin's own test suite with the reusable conformance check, called with every id the plugin can emit (static fallback, discovery fixtures and dynamic refresh paths):
providerkit.AssertModelNamespace(t, "synthetic", models)

CheckModelNamespace returns the same verdict as an error for harnesses without testing.T. ValidateModelNamespace checks a bare segment on its own.

Public Go API
API Provider responsibility / behavior
cabi.Export(cabi.Factory) error Install once during main.init; the factory receives the host callback boundary and constructs a fresh runtime per native initialization.
New(Definition, Caller) (*Runtime, error) Supply validated SDK metadata, capabilities, and optional Configure, OnQuiesce, OnShutdown hooks and DrainTimeout (quiesce drain bound; non-positive uses DefaultDrainTimeout).
Runtime.Call(ctx, method, raw) []byte / Shutdown(ctx) RPC envelopes and logical shutdown; normally used by the C shim, not provider code.
ValidateMetadata(pluginapi.Metadata) error Require nonempty Name/Version/Author/GitHubRepository and an HTTPS GitHub owner/repository URL.
ModelNamespace (Definition field) + ValidateModelNamespace(string) error Fixed vendor namespace for every registered/served model id; required when model capabilities are set, enforced on all three model paths.
CheckModelNamespace(ns, models) error / AssertModelNamespace(t, ns, models) Reusable conformance check for plugin test suites: every emitted id must carry ns exactly once.
APIKeyAuth Stateless SDK AuthProvider; top-level type dispatch, validation, stable IDs, API-key-only login errors, no-op refresh. Optional Validate checks provider fields only when ParseAuth is actually called; the generic file path calls neither method, so validate provider-specific fields in the executor. There is no login flow: StartLogin/PollLogin always fail with api_key_only. API-key providers must omit AuthProvider from their capabilities (no login card is shown) and let auth files load through the host's generic file path; registering APIKeyAuth would advertise a login card that can only fail.
KeyOwnership / ProvisionalParseExpiry(ctx) Shared filename-ownership duplicate guard keyed by a caller-computed credential-identity digest (key alone, or a composite such as account plus key when keys are shared across accounts). Host parses of real auth files commit ownership. The provisional marker (withProvisionalParse) is internal: providers on the generic file path have no pre-save validation flow, so an abandoned validation cannot hold a key. Hand-rolled guards consult ProvisionalParseExpiry for the same commit/reserve split.
PrefixClaims.Claim(file, prefix, credential) / Release(file) Opt-in provider-owned collision tracker, not automatic host enforcement. Claim immediately replaces that file's previous claim; serialize it with committed auth changes, not speculative parses that the host may discard. Release only when the old auth is no longer active.
NormalizePrefix(prefix) (string, error) Canonicalize a direct caller's prefix before both Claim and returning the auth to the host; Claim's internal comparison normalization cannot rewrite the caller's auth. APIKeyAuth.ParseAuth already returns this canonical prefix.
AuthFields(storage, metadata) / APIKey(storage, metadata) Decode object storage with host-compatible metadata precedence; never include credential values in errors.
Passthrough SDK executor using only the injected HostHTTPClient. Supply BuildRequest; optionally PrepareHTTP, ClassifyError, and TokenCount. ExecuteStream re-frames SSE upstreams (text/event-stream, or a missing Content-Type) into one host chunk per data event: bare JSON for OpenAI chat and same-protocol Gemini clients, data: lines for translators, complete named events and standalone comment lines for native Claude/Responses clients. Other Content-Types pass through unchanged; callers must not frame SSE again.
MapError(error) *pluginabi.Error Preserve explicit code/status/retry fields; redact unclassified errors.
RetryAfterSeconds(header, now) int Parse HTTP seconds/date retry guidance without a hidden clock.
NewCachedQuota(provider, display, FetchQuotaFunc) *CachedQuota / NewCachedQuotaWithTTL(provider, display, FetchQuotaFunc, ttl) *CachedQuota Non-resettable SDK QuotaProvider; successful per-auth cache for QuotaCacheTTL = 60 seconds by default, or a per-plugin TTL with NewCachedQuotaWithTTL (non-positive falls back to the default), coalesced concurrent fetches, copy isolation.
CachedQuota.Store(request, response) error / Invalidate(authID) Incorporate provider quota headers by auth identity, or invalidate header observations and all polled credential versions for an auth.
NewModelCatalog(config ModelCatalogConfig) *ModelCatalog / NewModelCatalogWithTTL(config ModelCatalogConfig, ttl time.Duration) *ModelCatalog / OpenAIModelsFetcher(baseURL, path string, headers func(pluginapi.AuthModelRequest) http.Header) FetchModelsFunc Per-auth model discovery for ModelProvider (+ ModelRegistrar): foreground-only fetch through req.HTTPClient on every ModelsForAuth (a TTL never skips a fetch — it is a compatibility field, not a gate), single-flight for concurrent calls only, durable last-good fallback before static (an empty upstream is a failure, a filtered-empty is served), uniform glob/override/namespace pipeline with contributor precedence/dedup (opt out with ModelCatalogConfig.KeepContributorSiblings for upstreams that serve both ids as distinct offerings), credential-scoped last-good (host routing/display edits such as label/priority keep the fallback while key rotation invalidates it; override with ModelCatalogConfig.CredentialKey), executor-mappable ids, and ModelCatalog.Invalidate(authID) for auth lifecycle. Derived 15-second fetch deadline (ModelsFetchTimeout); 1 MiB post-return parse cap (MaxModelsBody) enforced by OpenAIModelsFetcher only — custom Fetch implementations must check response sizes themselves — not a transport bound; redirects and buffering remain host-owned.
NewQuotaHTTPClient(http.RoundTripper) *QuotaHTTPClient Read-only polling, 15-second total timeout, 1 MiB body cap, no redirects, redacted errors; CloseIdleConnections for teardown. The default transport, NewQuotaTransport(), honors WithProxyURL and speaks HTTP/1.1 so pooled connections never cross proxies.
AuthProxyURL(storage, metadata, hostProxyURL) / WithProxyURL(ctx, proxy) / QuotaProxy Resolve the auth's proxy_url, else the host's global proxy-url, attach it to a direct request context. QuotaProxy is the Proxy func behind NewQuotaTransport; build custom transports from NewQuotaTransport() rather than setting QuotaProxy alone, because a transport shared across proxies must not reuse HTTP/2 connections.
NewHostHTTPClient(Caller, callbackID) pluginapi.HostHTTPClient Scoped callback adapter, normally injected by Runtime. Never retain it beyond its request/stream.
Caller / CallerFunc Raw concurrent host callback boundary; useful for isolated contract tests.

Definition.Capabilities supports auth, model registration, model discovery, executor, quota, usage and management. A ManagementAPI must also implement ManagementHandler to dispatch RPC route calls. Registration serializes route metadata, not process-local handlers; the host installs its own RPC handler for each declared management route and resource. Other SDK capabilities are rejected rather than advertised without handlers, including the schema 10 multi-auth forms (Capabilities.AuthProviders, or an AuthProvider implementing MultiAuthProvider): the kit exposes one auth identifier per plugin. Executor definitions must declare input/output formats and may set scope both, static, or oauth (default both). Let the host translate protocols to those formats; the passthrough helper does not translate or rewrite business payloads after host payload rules.

A registered AuthProvider may additionally implement CredentialProviders() []string to advertise credential-provider ownership through the credential_providers field of auth.identifier. The kit forwards those values verbatim; the pinned host interprets and normalizes them. These are ownership claims, not extra login identifiers or a guarantee that the host will install or select the plugin. Providers must claim only credential types they are intended to own. Providers without the extension still return only identifier; definitions without an AuthProvider remain without that capability.

Auth files and persistence

One object per account, using the provider key as top-level type:

{"type":"example-provider","api_key":"example-key","label":"Example","prefix":"example-provider","weight":2,"priority":0,"disabled":false}

Create real auth files privately with mode 0600; never commit them. The default ID is provider: plus the first 12 hex characters of SHA-256 of the trimmed API key, or an explicit string id. Changing filename does not change identity; duplicate files with the same key derive the same identity. Account/file deduplication and rejection policy belong to provider repos, not this stateless parser.

Storage remains a JSON object with provider fields; host-owned id/label/prefix/proxy_url/disabled/weight/priority move to SDK metadata/attributes. The host merges metadata over storage and forces type. APIKey and AuthFields preserve that precedence on execute/refresh/quota calls. The prefix is normalized exactly like the host filestore: surrounding whitespace and / are trimmed, and a prefix still containing / is rejected with invalid_auth — the plugin path pins whatever ParseAuth returns verbatim, so a slashed prefix could otherwise merge two pinning namespaces while looking unpinned. Wrong provider type returns Handled:false. Malformed objects, duplicate top-level fields, missing keys, or wrongly typed routing fields return an error without a half-auth.

Executor, errors and streaming

BuildRequest must choose a trusted endpoint and add the provider's auth headers. Preserve the final ExecutorRequest.Payload; CLIProxy has already applied its payload policy. PrepareHTTP can inject auth and restrict the URL for SDK HttpRequest; without it that method forwards the supplied HTTP request unchanged. Do not copy client credentials blindly into upstream headers. The kit never constructs an independent executor transport.

ExecuteStream opens HTTP through host.http.do_stream, reads raw chunks incrementally, and returns headers before completion. Runtime forwards chunks asynchronously with host.stream.emit and exactly one host.stream.close. It does not buffer the entire stream. Upstream setup failures return a normal RPC error; late stream failures carry structured error_detail. Host operation IDs propagate cancellation to blocked HTTP setup/read calls. Providers must honor contexts, and consumers must drain a stream or cancel its context.

ExecuteStream normalizes SSE (text/event-stream, matched as a media type with optional parameters) into one host chunk per event. It ignores one stream-leading UTF-8 BOM and splits LF/CRLF/CR events across arbitrary HTTP reads. OpenAI output ends at [DONE] and cancels the upstream operation without waiting for HTTP EOF; the sentinel and any later bytes are not forwarded because the host emits its own terminator. OpenAI chat/completions and same-protocol Gemini clients receive bare JSON: their host HTTP writers add the SSE envelope (Gemini default, alt=sse and $alt=sse) or write raw bytes (other Gemini alt modes). Explicit non-SSE bodies such as Gemini alt=json pass through unchanged. Translators receive one data: line per data event, without comments or metadata. Native Claude/Responses clients receive complete SSE events and standalone comment lines, preserving original fields, order, data-line whitespace, comments and metadata-only events (including id: and retry:), with line endings normalized to LF. Native multiline data need not be JSON; downstream clients interpret SSE fields and reconnection metadata. The host records the client route in Metadata["request_path"]; Format and SourceFormat both describe the plugin protocol, not the client. Direct SDK callers without a route hint receive data: lines for OpenAI output and complete events for other output; supply the route hint for cross-protocol non-OpenAI calls.

Upstream comments are preserved only on native SSE routes. A completed standalone comment line is forwarded immediately, without waiting for a blank line, later data or EOF. Comments following pending data/metadata fields stay buffered with those fields to preserve source order and truncated-event semantics. Comments cannot keep a translated, OpenAI bare-JSON or Gemini bare-JSON route alive: the pinned host consumes data chunks there, not raw SSE comments. Such deployments need the host's streaming.keepalive-seconds set above zero for downstream heartbeats (and streaming.send-keepalive-before-ft for pre-first-token heartbeats). Both are disabled by default; the kit does not override server configuration or guarantee idle-connection liveness.

A missing Content-Type retains the SSE assumption. Headerless bare JSON/NDJSON is unsupported and returns invalid_stream rather than silently dropping records. Explicit SSE follows SSE field semantics instead: JSON-looking unknown fields (including colonless true or 42) are ignored by data parsing and preserved on native routes. Other explicit Content-Types are forwarded unchanged; the kit does not split their records or promise that arbitrary raw reads meet every host translator's chunk contract. Pass-through plugins must not frame SSE again. A truncated event cut short by an upstream error is dropped while the error propagates. Native events without a blank-line terminator are also discarded at clean EOF, except completed standalone comment lines already delivered; translator/bare-JSON output retains its historical EOF flush. A colonless data field is an empty data line. The 8 MiB event budget counts data values plus one separator per data line, excluding data: prefixes, identically for complete and split reads. Native events charge all other field/comment bytes plus their separators to that same cumulative budget, including metadata-only events and already-forwarded comments until the actual blank-line delimiter; retaining repeated fields is bounded, not an unbounded metadata buffer. Field prefixes add at most six bytes per data line to native output. A separate line bound also caps comments and other fields. Oversized events, or non-native multiline data that is not one JSON value, return invalid_stream and release the upstream feed. Partial-line buffering scans only new bytes and reuses its allocation.

Classify provider errors explicitly, including exhaustion versus transient throttling, using safe public text:

return &pluginabi.Error{
    Code: "quota_exhausted", Message: "subscription window exhausted",
    HTTPStatus: 429, Retryable: true, RetryAfterSeconds: 60,
}

These fields drive the host's native per-auth cooldown. A non-retryable provider error can still carry a status and cooldown hint. Do not relabel every 403 as bad credentials or every 429 as exhaustion. Never put raw upstream bodies, keys, URLs with credentials, or transport errors in public messages. Unclassified errors and recovered panic values are redacted by the kit. Panics in provider-created goroutines must be recovered by that provider; Go cannot recover another goroutine's panic.

Default CountTokens is an explicitly marked estimate (ceil of Unicode payload codepoints / 4), not a model tokenizer or an upstream call. Supply TokenCount for a real tokenizer or a protocol-specific result. No helper automatically retries inference, rotates accounts, or bypasses provider entitlements. Multiple auth files use CLIProxy's native selection and cooldown; the kit does not claim a global Scheduler capability.

Quota polling and cache

FetchQuotaFunc fetches and parses the provider's official quota format, returning normalized SDK groups/buckets/summary/subscription. The kit does not invent plan limits, reset support, or dollars. Errors are not cached and stale observations are not silently returned. Polled results are keyed by provider, auth identity, and a hash of storage/metadata/attributes, preventing credential changes from reusing old polled quota. Header observations are a separate auth-identity tier, preferred while fresh. When the caller includes the credential context (storage/metadata/attributes) in the Store request, the observation is bound to that credential generation and is never served to a fetch carrying a different credential, which closes the stale-observation window when a key is rotated under a reused identity. Observations stored without credential context (for example from usage records, which carry none) stay unbound and serve every generation of the identity until the TTL expires. Both tiers expire after the cache TTL (60 seconds by default, configurable per plugin with NewCachedQuotaWithTTL) and are pruned on fetch; the cache is in-process and starts empty after reload.

Direct polls follow the host's proxy precedence. CachedQuota passes FetchQuotaFunc a context carrying the auth's proxy_url (else Host.ProxyURL), and QuotaHTTPClient sends the request through it (with its default transport, or a custom one built from NewQuotaTransport()): http, https, socks5 or socks5h, with direct/none bypassing proxies and an empty setting inheriting the environment. An invalid setting fails the request instead of connecting directly. Other direct calls, such as per-auth model discovery, should wrap their context with WithProxyURL(ctx, AuthProxyURL(...)).

A provider UsagePlugin can normalize a quota response header and call CachedQuota.Store using only the identity from pluginapi.UsageRecord:

err := cache.Store(pluginapi.QuotaFetchRequest{
    AuthID: record.AuthID, AuthIndex: record.AuthIndex, Provider: record.Provider,
}, normalizedQuota)

Store and FetchQuota use AuthID, falling back to AuthIndex only when AuthID is absent. Supply AuthID whenever available in the usage record so it matches the host-enriched fetch. Store accepts storage/metadata/attributes and binds the observation to that credential generation when present, but never requires them: usage records carry auth identity without credential context, and store unbound observations served to every generation until the TTL expires. Binding compares the complete storage/metadata/attributes context, not only api_key: metadata can override storage credentials, and provider callbacks may use other fields. A partial context stored before host enrichment intentionally falls back to polling (or an existing matching poll), even if only a label or path changed. Supply the same complete context to both calls to retain the bound-observation fast path; partial context is never downgraded to unbound. An older in-flight poll cannot overwrite the stored header observation. Use one cache instance for both interfaces. Invalidate an identity when reusing it for a different credential without binding context, and recreate/invalidate the cache when configuration changes endpoint or quota semantics.

Transport distinction: v8.0.63's HostHTTPClient follows redirects, without a per-request flag to prohibit them or cap reads. Therefore, the strict polling helper is separate: QuotaHTTPClient.Do allows GET and POST (quota endpoints that mint or refresh a read-only view, such as GraphQL or key-envelope reads, use POST without becoming an inference or mutation path), uses a 15-second total timeout, never follows redirects, caps successful bodies at 1 MiB, and discards error bodies/URL/transport text. Nil uses NewQuotaTransport(): a clone of the standard transport that applies the WithProxyURL setting (else normal environment proxies) and speaks HTTP/1.1 only. Pass a configured http.RoundTripper when deployment proxy policy requires more; it must not follow redirects itself. The client passes it the request and context unchanged, so it honors the WithProxyURL setting only if it routes through QuotaProxy (as NewQuotaTransport() does); otherwise its own proxy behavior applies. An invalid WithProxyURL setting still fails the request with any transport. This helper is not for inference. If using the host client for quota instead, the host's redirect semantics apply and strict no-redirect polling is not guaranteed. Neither helper logs keys; host request logging remains controlled by the host operator.

Lifecycle and ABI ownership

plugin.register / plugin.reconfigure accept a host schema of 10 or newer (an older host is refused with unsupported_schema), answer with schema 10 so the host gates newer features, and return registration metadata/capability flags. Configure gets the plugin config subtree as YAML bytes, not the whole host config. Reconfiguration with changed config first cancels and drains active work; an identical-config reconfigure on an active runtime is a no-op that leaves in-flight work untouched (a non-active runtime that has not shut down, e.g. after quiesce, reruns Configure if set and reactivates on success instead; a shut-down runtime returns not_ready before Configure); a failed Configure leaves the runtime inactive until a successful configuration. plugin.quiesce stops admission (new work is refused not_ready) and drains in-flight executions and streams without canceling them, up to Definition.DrainTimeout (non-positive uses DefaultDrainTimeout, 60 seconds). On deadline expiry quiesce cancels the stragglers and returns a drain_timeout error, so the host aborts the replacement and rolls back by re-registering the old plugin rather than unloading work still in flight. OnQuiesce runs only after a successful drain; its error still aborts the swap. plugin.shutdown and the native shutdown function perform idempotent logical shutdown. Do not call lifecycle methods recursively from a capability/hook.

Capabilities must be concurrency-safe, immutable after construction, and cancelable. Quiesce waits for admitted work to finish on its own; a provider that never finishes blocks quiesce until DrainTimeout expiry, while shutdown and changed-config reconfiguration cancel admitted work. Shutdown cancels streams, but does not unload a resident Go runtime. A native reinitialization uses the factory again.

The shim exports cliproxy_plugin_init, cliproxy_plugin_call, cliproxy_plugin_free_buffer, and cliproxy_plugin_shutdown; the init function populates ABI 1's call/free_buffer/shutdown table. JSON is copied at the C boundary. Plugin-allocated response buffers are freed by the plugin allocator, host callback buffers by the host allocator, exactly once. No Go pointers cross the ABI.

Build and verify

scratch="$(mktemp -d)"
trap 'rm -rf "$scratch"' EXIT
export CGO_ENABLED=1

go vet ./...
go test -race -count=1 ./...
go build -trimpath -buildmode=c-shared -o "$scratch/echo.so" ./examples/echo
# Separate module: root ./... deliberately does not include this harness.
go -C integration/hostcompat vet ./...
go -C integration/hostcompat test -race -count=1 -v ./...

CI runs formatting, module verification, both vet/race suites, a c-shared build, gitleaks, and a generic credential-pattern scan, without provider credentials or repository secrets. All network behavior under test uses httptest loopback upstreams; module/tool downloads use public Go infrastructure only. Tests never call provider inference or quota services.

Why the host compatibility harness is a separate module

The real loader is unexported inside internal/pluginhost. The test-only module at integration/hostcompat intentionally uses import path github.com/router-for-me/CLIProxyAPI/v8/providerkitcompat so Go permits imports of the upstream internal host/config/registry packages. This is an internal-package testing harness, not a fork, an upstream package, or a replacement loader. Its module requires exactly v8.0.63 with normal Go checksum verification; there is no local replace or patched core source.

TestRealHostLoader builds the local echo c-shared library, then calls the pinned host's Host.ApplyConfig: this runs the original dynamicLibraryLoader.Open (dlopen/dlsym), registerRPCPlugin, HTTP callbacks, streaming bridge and executor/auth/quota adapters. Assertions cover metadata/capabilities, host-merged auth persistence, quota, mock execution, incremental streaming, retry-after mapping, reconfiguration, shutdown and resident-image reinitialization. The host test process is disposable; it is not a running proxy service and opens no public listener.

A second test, TestNativeABIContract, compiles testdata/abi_check.c with the system cc against cabi/abi.h and runs it in its own process. It checks that cliproxy_plugin_init rejects missing tables, a wrong ABI version, a missing host free_buffer and a second initialization; that invalid native call arguments return invalid_request; that plugin.register works with schema 10 and plugin.quiesce stops admission; that shutdown is idempotent; and that the resident image can be initialized again.

Source contracts pinned to v8.0.63:

  • sdk/pluginabi/types.go: ABI/schema constants, method names, envelope/error contract.
  • sdk/pluginapi/types.go: auth/model/executor/host HTTP/quota/usage interfaces.
  • internal/pluginhost/loader_unix.go: C layout, buffer allocation/freeing, logical unload.
  • internal/pluginhost/rpc_schema.go, rpc_client.go: registration and scoped requests.
  • internal/pluginhost/host_callbacks.go, rpc_client_stream.go, stream_bridge.go: host HTTP operations and stream emission.
  • internal/pluginhost/auth_provider.go: storage/metadata merge and auth conversion.
  • internal/pluginhost/http_bridge.go: redirect behavior and host request logging.

Provider releases (not kit releases)

This library has no installable .so, registry.json, migration command, or plugin-store entry of its own. Provider repositories own those artifacts and credential migration. For the v8.0.63 GitHub store contract, release <plugin-id>_<version>_linux_amd64.zip with <plugin-id>.so and checksums, and publish a schema-1 registry.json with the provider's GitHub-release installation metadata. Verify the exact contract in internal/pluginstore/github.go before releasing a provider. Keep all credentials out of artifacts and fixtures. Stage each provider before any production cutover; native compatibility executors may take precedence over plugin executors.

The C ABI layout is adapted from CLIProxyAPI's MIT-licensed simple Go plugin example and loader. The original copyright notices are retained in LICENSE, alongside this kit's MIT license.

Documentation

Overview

Card helpers keep every provider quota card close to the official provider card style (one short plan chip, one row per real limit window, no group-specific chrome, dollar balances as text or chips, never bars).

The management panel renders plugin cards generically: every summary metric becomes a chip, each group gets a header, and a bucket's Description becomes its row label (the SDK QuotaBucket has no display name). So card code must:

  • leave Description empty and put the short row label in Window;
  • emit at most MaxCardSummaryChips summary metrics, none repeating a bar;
  • format dollar amounts in text with USD (exactly 2 decimals).

Package providerkit implements the CLIProxy provider plugin RPC contract. Provider implementations use the pinned SDK's interfaces directly.

Index

Constants

View Source
const (
	CardWindowFiveHour = "5h"
	CardWindowWeekly   = "weekly"
	CardWindowMonthly  = "monthly"
	CardWindowDaily    = "daily"
)

Canonical window ids for quota rows, matching the spellings the panel resolves to a countdown timeline and stable sort order ("5h", "weekly"). Window values stay short ids so long labels are never truncated; the panel shows the Window as the row label. Other windows ("daily", "monthly", provider-specific names) render without a countdown but keep their reset text. Keep values lowercase: the panel matches case-insensitively.

View Source
const DefaultDrainTimeout = 60 * time.Second

DefaultDrainTimeout bounds how long plugin.quiesce waits for in-flight work to drain before canceling stragglers and aborting the swap.

View Source
const MaxCardSummaryChips = 3

MaxCardSummaryChips bounds the plan-row chip count (plan chip excluded). The panel renders every summary metric as a chip with no limit, so an unbounded summary becomes a wall of chips next to the official cards.

View Source
const MaxModelsBody = 1 << 20

MaxModelsBody bounds a discovery response parse (default 1 MiB). The check runs against the buffered HTTPResponse.Body length after the host client returns: an oversized buffered body is treated as a failed fetch (last-good/static fallback) and is not parsed. This is a post-return parse cap, not a transport memory bound: the host may already have buffered an arbitrarily large body in memory before the kit sees it. Redirects and hop policy remain host-owned; the kit cannot disable them through HostHTTPClient and does not bypass host proxy/logging policy to try.

View Source
const ModelsFetchTimeout = 15 * time.Second

ModelsFetchTimeout is the kit-owned deadline for one model-discovery fetch. ModelsForAuth derives context.WithTimeout(reqCtx, ModelsFetchTimeout) explicitly; it never assumes a deadline on the incoming callback. When the parent already carries an earlier deadline, the derived context yields the earlier of the two.

View Source
const QuotaCacheTTL = 60 * time.Second

QuotaCacheTTL is the maximum age of a successful per-auth quota observation.

View Source
const QuotaHTTPTimeout = 15 * time.Second

QuotaHTTPTimeout bounds the entire quota HTTP exchange, including body reads.

Variables

This section is empty.

Functions

func APIKey

func APIKey(storage []byte, metadata map[string]any) (string, error)

APIKey reads a nonempty api_key using host metadata precedence. It never includes the key or raw storage in an error.

func AssertModelNamespace added in v0.2.0

func AssertModelNamespace(t TestReporter, ns string, models []pluginapi.ModelInfo)

AssertModelNamespace fails the test if any model id would not be registered under ns.

func AuthFields

func AuthFields(storage []byte, metadata map[string]any) (map[string]any, error)

AuthFields applies the same merge precedence as persisted host auth storage: Metadata overrides StorageJSON. Callers own the returned map.

func AuthProxyURL added in v0.2.0

func AuthProxyURL(storage []byte, metadata map[string]any, hostProxyURL string) string

AuthProxyURL resolves the outbound proxy for direct (non-host) requests made on behalf of one auth, with the host's precedence: the auth's proxy_url (metadata overrides storage), then the host's global proxy-url. An empty result means "inherit": the environment proxy settings apply.

func CheckCard added in v0.2.2

func CheckCard(resp pluginapi.QuotaFetchResponse) []string

CheckCard reports official-card-style violations in a quota response:

  • any bucket Description (leave it empty so Window stays the row label);
  • any bucket with a blank Window (the row would have no label);
  • more than MaxCardSummaryChips summary metrics;
  • a summary metric repeating a bar (same or overlapping label text).

An empty return means the card follows the shared style. Plugin quota tests assert on this alongside their exact-shape golden tests.

func CheckModelNamespace added in v0.2.0

func CheckModelNamespace(ns string, models []pluginapi.ModelInfo) error

CheckModelNamespace is the reusable conformance check for plugin test suites: it reports an error listing every model id that would not be registered under ns. Call it with every id the plugin can emit (static fallback, discovery fixtures and dynamic refresh paths).

func MapError

func MapError(err error) *pluginabi.Error

MapError preserves explicit, public RPC errors. Unclassified errors are intentionally opaque: transport and parsing errors can contain credentials. Providers must put only safe, user-facing text in pluginabi.Error.Message.

An unclassified error may additionally assert account-wide scope with an optional CredentialScoped() bool method (the host's native naming, IsCredentialScoped() bool, is honored as well). When either returns true, the mapped error is credential-scoped, so the host cools the whole credential across all models instead of only the reporting model. Whether a given upstream failure is account-wide is the provider's per-provider classification; MapError only plumbs the claim through.

func NewHostHTTPClient

func NewHostHTTPClient(h Caller, callbackID string) pluginapi.HostHTTPClient

NewHostHTTPClient creates a callback-scoped SDK client. Runtime injects this automatically into each request. Never retain it past that request/stream. Cancellation uses the host's operation IDs, not abandoned blocking goroutines.

func NewQuotaBucket added in v0.2.2

func NewQuotaBucket(window string, remainingFraction float64, resetTime string) pluginapi.QuotaBucket

NewQuotaBucket builds one quota bar row: a canonical window label, the remaining fraction, and the reset time. It never sets Description, so the panel shows the short Window label instead of a sentence. remainingFraction is the fraction remaining in [0, 1] (unclamped; NaN fails host decoding), and resetTime is an RFC 3339 instant or "" when the window has no reset (the panel then shows "-").

func NewQuotaTransport added in v0.2.0

func NewQuotaTransport() *http.Transport

NewQuotaTransport clones the standard transport with QuotaProxy as its proxy (the WithProxyURL setting, else normal environment proxy behavior). It speaks HTTP/1.1 only: Go reuses a cached HTTP/2 connection by destination before it consults Proxy, so one auth's poll could ride another auth's proxy. HTTP/1.1 connections are pooled per proxy.

func NormalizePrefix added in v0.5.0

func NormalizePrefix(prefix string) (string, error)

NormalizePrefix applies the host filestore's prefix contract to a plugin-returned prefix: surrounding whitespace and "/" are trimmed, and an inner "/" is rejected with invalid_auth. The plugin path pins whatever the plugin returns verbatim, so silently mapping a slashed prefix to unpinned would leave two files sharing one host namespace while the tracker believes both are unpinned. The programme rule is never to silently drop: reject.

func ProvisionalParseExpiry added in v0.4.1

func ProvisionalParseExpiry(ctx context.Context) (time.Time, bool)

ProvisionalParseExpiry reports whether ctx belongs to a provisional pre-save validation parse (see withProvisionalParse) and, if so, when the session behind it expires. Parses the host initiates for real auth files are never provisional.

func QuotaProxy added in v0.2.0

func QuotaProxy(r *http.Request) (*url.URL, error)

QuotaProxy is an http.Transport Proxy function that applies the setting from WithProxyURL and otherwise falls back to the environment. Custom transports should start from NewQuotaTransport rather than set it alone: a transport shared across proxies must not reuse HTTP/2 connections.

func QuotaWindowLabel added in v0.2.2

func QuotaWindowLabel(window string) string

Canonical quota card labels.

The plugin QuotaBucket type has no displayName field, so the panel renders each bucket row with the label description ?? window (group displayName is a separate group header). A bucket Description therefore becomes the row label and tooltip while Window is hidden, so long explanations must never be set as Description. Keep bucket labels short and window-style with QuotaWindowLabel and NewQuotaBucket, and put absolute amounts in the label only where they add information (for example, "Weekly key limit · $5.00 of $5.00 left").

Only the listed spellings canonicalize to the CardWindow* constants; anything else (including typos and the empty string) passes through verbatim after trimming, so callers must spell windows deliberately.

func ResetRetryAfterSeconds added in v0.6.0

func ResetRetryAfterSeconds(body []byte, now time.Time) int

ResetRetryAfterSeconds extracts an absolute quota-reset timestamp from an upstream error body and converts it to a retry-after hint in seconds, relative to now. It recognizes a numeric or RFC3339 "resets_at"/"reset_at" field anywhere in the JSON body, or an RFC3339 timestamp following "resets at" in a message string (for example "Your usage window resets at 2026-11-11T02:59:37Z."). A non-JSON body is scanned as raw text. The first future candidate in deterministic order wins; expired candidates are skipped, so a stale reset field cannot mask a later one. Missing or entirely past timestamps yield zero. No wall clock is hidden.

func RetryAfterSeconds

func RetryAfterSeconds(value string, now time.Time) int

RetryAfterSeconds parses HTTP Retry-After seconds or an HTTP date, rounding a future date up. Invalid or past values yield zero. No wall clock is hidden.

func USD added in v0.2.2

func USD(v float64) string

USD formats dollars with exactly 2 decimals, sign first: 25 -> $25.00, -0.02 -> -$0.02. Amounts in bucket text must use this, never a higher-precision debug format; chip currency metrics are formatted panel-side from the numeric value. Non-finite inputs render as $0.00: JSON-decoded amounts are always finite, so this branch is unreachable defense so a card can never show $NaN.

func ValidateMetadata

func ValidateMetadata(m pluginapi.Metadata) error

ValidateMetadata checks the metadata required by the host and public registry.

func ValidateModelNamespace added in v0.2.0

func ValidateModelNamespace(ns string) error

ValidateModelNamespace reports whether ns is usable as a fixed model namespace: a bare segment such as "synthetic", without the trailing slash.

func WithProxyURL added in v0.2.0

func WithProxyURL(ctx context.Context, proxyURL string) context.Context

WithProxyURL attaches a proxy setting to ctx for QuotaHTTPClient requests. It accepts the host's proxy syntax: "" inherits the environment, "direct" or "none" bypasses proxies, otherwise an http, https, socks5 or socks5h URL.

Types

type APIKeyAuth

type APIKeyAuth struct {
	Provider string
	Validate func(map[string]any) error
}

APIKeyAuth implements the SDK auth provider for API-key-only providers. Validate may validate additional provider-owned fields; its errors are redacted by Runtime unless explicitly represented as public pluginabi.Error values. It is stateless: duplicate-file/account policy belongs to the provider.

There is no login flow: StartLogin and PollLogin always fail with api_key_only, so API-key providers show no login card in the management UI and auth files load through the host's generic file path. The kit's script-based API-key login page was removed; OpenRouter-style OAuth/PKCE logins live in the provider, not in the kit.

func (APIKeyAuth) Identifier

func (p APIKeyAuth) Identifier() string

type CachedQuota

type CachedQuota struct {
	// contains filtered or unexported fields
}

CachedQuota implements a non-resettable SDK quota provider with per-account single-flight fetching, copy isolation, and a configurable cache TTL (default 60 seconds). Use Store to incorporate normalized inference response headers by auth identity without another HTTP poll or access to the host's credential context.

func NewCachedQuota

func NewCachedQuota(provider, display string, fetch FetchQuotaFunc) *CachedQuota

NewCachedQuota creates an empty quota cache with the default 60-second TTL. The identity and callback are fixed for its lifetime. No background polls or provider requests are started.

func NewCachedQuotaWithTTL added in v0.5.0

func NewCachedQuotaWithTTL(provider, display string, fetch FetchQuotaFunc, ttl time.Duration) *CachedQuota

NewCachedQuotaWithTTL creates an empty quota cache with a per-plugin TTL for successful per-auth observations. A non-positive TTL falls back to QuotaCacheTTL. Errors are never cached regardless of the TTL. Polls that are expensive or rate-limited (for example a key-envelope read behind a 5-minute connector poll) use a longer TTL so steady state stays within the provider's budget; header observations share the same TTL.

func (*CachedQuota) Identifier

func (q *CachedQuota) Identifier() string

func (*CachedQuota) Invalidate

func (q *CachedQuota) Invalidate(authID string)

Invalidate discards header observations and all polled credential versions for the given auth ID (or AuthIndex when used instead of AuthID).

func (*CachedQuota) Store

Store replaces an auth-identity observation (for example, from a provider quota header). Only AuthID (or AuthIndex when AuthID is absent) and an optional matching Provider are required. When the caller knows the credential context (StorageJSON/Metadata/Attributes), include it: the observation is then bound to that credential generation and is never served to a fetch carrying a different credential, which closes the rotated-key-under-reused-identity staleness window (G-K3). Callers without credential context (UsageRecords) store unbound observations, served to any credential of the identity until the TTL expires. Fresh observations take precedence over polls, including older in-flight polls. Data is cloned on store and read. Invalidate when reusing an identity for a different credential without binding context, or recreate the cache on reconfiguration.

type Caller

type Caller interface {
	Call(context.Context, string, []byte) ([]byte, error)
}

Caller is the raw host callback boundary. Results are pluginabi.Envelope JSON. Implementations must support concurrent calls. Do not log request/response data.

type CallerFunc

type CallerFunc func(context.Context, string, []byte) ([]byte, error)

CallerFunc adapts a callback function to Caller.

func (CallerFunc) Call

func (f CallerFunc) Call(ctx context.Context, m string, b []byte) ([]byte, error)

type Definition

type Definition struct {
	Metadata     pluginapi.Metadata
	Capabilities pluginapi.Capabilities
	// ModelNamespace declares the fixed vendor namespace for every model id
	// this plugin registers or serves.
	ModelNamespace string
	// DrainTimeout bounds how long plugin.quiesce waits for in-flight work
	// to finish before canceling stragglers and aborting the swap. A
	// non-positive value uses DefaultDrainTimeout. It does not affect
	// shutdown or reconfiguration, which always cancel.
	DrainTimeout time.Duration
	Configure    func(context.Context, []byte) error
	OnQuiesce    func(context.Context) error
	OnShutdown   func(context.Context) error
}

Definition wires SDK capabilities and lifecycle hooks. Configure receives only this plugin's configuration subtree, on both registration and reconfiguration. An identical-config reconfigure on an active runtime is a no-op that leaves in-flight work untouched. On a non-active runtime that has not shut down (e.g. after quiesce), reconfiguration runs Configure if set and reactivates the runtime on success. Reconfiguration with changed config and shutdown cancel and drain active calls/streams before their hooks run. Quiesce instead stops admitting new work and drains in-flight work without canceling it, up to DrainTimeout; on expiry it cancels stragglers and returns an error so the host aborts the swap, and OnQuiesce runs only after a successful drain. Capabilities and metadata must not be mutated after New. Capability methods must be concurrency-safe and honor context cancellation.

ModelNamespace is the plugin's fixed vendor namespace (a bare segment such as "synthetic"; the kit appends the slash). Any provider that serves models made by another vendor must namespace its model ids so they never clash when the official provider is added later. A definition that registers or serves models (ModelRegistrar or ModelProvider set) must declare it; an empty, missing or malformed namespace is a startup error. Every static, discovered and dynamic model list is filtered through it: ids outside "<namespace>/" (including doubled "<namespace>/<namespace>/…") are dropped with a logged error and never served bare. The host auth-file `prefix` is an outer host clone and does not replace this namespace.

type FetchModelsFunc added in v0.5.0

type FetchModelsFunc func(ctx context.Context, req pluginapi.AuthModelRequest) ([]pluginapi.ModelInfo, error)

FetchModelsFunc returns candidate models for one auth. Provider-owned protocol parsing stays outside the kit: custom Fetch implementations that use req.HTTPClient directly are responsible for their own post-return parse-size checks (see MaxModelsBody) and must honor context cancellation. Fetch runs foreground-only inside ModelsForAuth with the request-scoped host client; the kit starts no goroutine and retains no client past the callback.

func OpenAIModelsFetcher added in v0.5.0

func OpenAIModelsFetcher(baseURL, path string, headers func(pluginapi.AuthModelRequest) http.Header) FetchModelsFunc

OpenAIModelsFetcher fetches an OpenAI-style {"data":[{"id":…}]} list through the host HTTP client (foreground use inside ModelsForAuth, proxy/logging host-owned). baseURL plus path form the request URL (a single slash joins the parts; an empty path requests baseURL alone). headers, when non-nil, supplies per-request headers such as the auth bearer derived from the request; the kit never derives credentials itself. A non-2xx status, a malformed body, or a body larger than MaxModelsBody is a fetch error (never parsed, never cached); the catalog falls back to last-good/static per the uniform pipeline.

type FetchQuotaFunc

FetchQuotaFunc fetches and normalizes one account. Provider-owned protocol parsing stays outside the kit. Errors are never cached; stale data is not silently returned. Honor context cancellation.

type KeyOwnership added in v0.4.1

type KeyOwnership struct {
	// contains filtered or unexported fields
}

KeyOwnership is the shared filename-ownership duplicate guard for ParseAuth pipelines: one credential digest may be configured in only one auth filename at a time, so a duplicate file is rejected while a watcher's repeated parse of the same file is safe. The caller computes the digest over its full credential identity — just the key when keys are unique per account, or a composite such as account ID plus token when one key may serve several accounts (the same split FilenameFields makes for login file names). Only digests and filenames are retained, never keys.

A committed (host-initiated, non-provisional) parse records ownership until the same filename is parsed with a different digest or the plugin restarts; deleting or renaming an auth file therefore still needs a plugin restart to free its key, as with the hand-rolled guards this replaces. A provisional (login-validation) parse instead reserves the digest only until the login session expires: it rejects a key another filename already owns, and blocks other filenames while the login is in flight, but the reservation is dropped lazily once expired. An abandoned login, an evicted session, or a login the host fails to persist can thus hold a key for at most the login TTL, never until restart. Ownership of a completed login commits when the host parses the auth file it saved.

The zero value is ready to use. KeyOwnership is safe for concurrent use.

func (*KeyOwnership) Claim added in v0.4.1

func (g *KeyOwnership) Claim(ctx context.Context, fileName string, digest [32]byte) error

Claim checks fileName's ownership of digest and records it. digest is caller-chosen: hash the full credential identity (key alone, or account plus key for providers whose keys are shared across accounts). For a provisional parse (ProvisionalParseExpiry reports true for ctx) the claim is only reserved until the login session expires; any other parse commits ownership. Claim returns a duplicate_auth error when another filename owns or has reserved the digest.

type ModelCatalog added in v0.5.0

type ModelCatalog struct {
	// contains filtered or unexported fields
}

ModelCatalog implements pluginapi.ModelProvider and pluginapi.ModelRegistrar: RegisterModels and StaticModels serve Static through the uniform pipeline, and ModelsForAuth runs fetch → glob → overrides → Validate hook → cache → serve with req.HTTPClient.

Every ModelsForAuth invocation attempts a fresh fetch (no TTL skip); only concurrent invocations sharing one key coalesce through single-flight. A waiter whose shared leader failed only because the leader's own context was cancelled retries as a new leader rather than inheriting another caller's cancellation. Results fall back fresh → durable last-good → static, so a transient fetch error never drops a known model: an empty upstream result (before glob filtering) is a failed fetch, while a filtered-empty result is a legitimate policy outcome that is served and cached. A refresh whose candidates are all rejected by the namespace filter is likewise a failed fetch, so an incompatible upstream payload never erases last-good. Errors are never cached and a failed fetch, failed Validate or oversized body never replaces last-good. The last-good store has no TTL eviction and persists until a newer successful fetch replaces it or Invalidate runs; it is still bounded at maxLastGoodGenerations credential generations (FIFO eviction), so plugins sharing one catalog across many auths should call Invalidate on credential rotation. The generation key hashes credential material only (see CredentialKey): host routing/display edits (label, priority, weight, prefix) keep the fallback, while a credential change starts a new generation. Fetch verification is catalog availability only: it does not bypass credential validity, auth disablement, provider entitlements or executor authorization, which the host and executor still enforce per-request.

Concurrency-safe; the constructor starts no goroutine and there is no background fetch. A catalog is fixed once installed; recreate it on reconfiguration, exactly like CachedQuota.

func NewModelCatalog added in v0.5.0

func NewModelCatalog(config ModelCatalogConfig) *ModelCatalog

NewModelCatalog creates an empty model catalog for its lifetime.

func NewModelCatalogWithTTL added in v0.5.0

func NewModelCatalogWithTTL(config ModelCatalogConfig, ttl time.Duration) *ModelCatalog

NewModelCatalogWithTTL records a per-plugin TTL for the freshness markers. Like NewCachedQuotaWithTTL it mirrors the old constructor, but it does not introduce a skip gate: ModelsForAuth fetches on every invocation regardless of the TTL (see ModelCatalogConfig.TTL).

func (*ModelCatalog) Invalidate added in v0.5.0

func (c *ModelCatalog) Invalidate(authID string)

Invalidate discards the last-good entry (and any in-flight fetch) for the given auth ID. A fetch completing after Invalidate returns its result to its own callback but never resurrects the invalidated generation: its last-good write checks the in-flight entry is still current under lock. It fires no host callback (see the package freshness notes: recovery waits for the next host event).

func (*ModelCatalog) ModelsForAuth added in v0.5.0

ModelsForAuth fetches fresh models for one auth and serves the uniform pipeline result. Concurrent calls for the same credential generation share one fetch; sequential calls each fetch.

func (*ModelCatalog) RegisterModels added in v0.5.0

RegisterModels serves the static list through the uniform pipeline, for development-time registration.

func (*ModelCatalog) StaticModels added in v0.5.0

StaticModels serves the static list through the uniform pipeline.

type ModelCatalogConfig added in v0.5.0

type ModelCatalogConfig struct {
	// Namespace is the kit vendor namespace; upstream ids are prefixed
	// with it before serving (the same rule the runtime enforces with
	// filterModels, so it must match Definition.ModelNamespace). An empty
	// namespace serves ids unchanged.
	Namespace string
	// Static is the fallback served through the same glob/override/
	// namespace pipeline as fresh and last-good models, and the list
	// served by RegisterModels and StaticModels.
	Static []pluginapi.ModelInfo
	// Fetch fetches fresh candidates per auth. Nil means static-only: no
	// fetch is attempted and ModelsForAuth serves the static list.
	Fetch FetchModelsFunc
	// TTL is a compatibility field. It does NOT gate ModelsForAuth
	// fetches: every invocation attempts a fresh fetch, so a large TTL
	// cannot lose a one-shot catalog-change callback. It bounds the
	// freshness-marker store for diagnostics only and never erases the
	// durable last-good store.
	TTL time.Duration
	// Overrides keys by upstream (pre-namespace) id.
	Overrides map[string]ModelOverride
	// Allow and Deny are path globs matched against the upstream id;
	// deny wins and an empty allowlist allows everything else. An invalid
	// pattern simply never matches.
	Allow, Deny []string
	// KeepContributorSiblings opts out of contributor dedup: by default,
	// when both an id and its "-contributor" sibling appear, the base
	// entry is dropped and the contributor entry wins. Providers whose
	// upstream serves both ids as distinct offerings (different
	// descriptions or prices) set this to keep both ids through the
	// fresh, static, and last-good paths. The final-id dedup after
	// overrides (an explicit ContributorID rewrite colliding with a real
	// sibling) still keeps the first entry.
	KeepContributorSiblings bool
	// Validate is the post-fetch hook (glob and overrides applied, not
	// yet namespace-prefixed); e.g. probe validation that stays
	// plugin-side. It runs only on the fresh path: last-good and static
	// results were already validated or are trusted as configured. An
	// error is a failed fetch and never replaces last-good. Its
	// returned ids preserve the upstream glob decision: ids the
	// pipeline already approved survive (including ContributorID
	// rewrites whose source was allowed), deny always wins, and ids
	// Validate invented face the full allow/deny check, so Validate
	// cannot bypass the glob policy.
	Validate func(ctx context.Context, req pluginapi.AuthModelRequest, models []pluginapi.ModelInfo) ([]pluginapi.ModelInfo, error)
	// CredentialKey derives the credential-generation bytes that scope
	// the last-good store (besides auth ID and provider). When nil, the
	// key hashes the auth storage plus metadata/attributes with the
	// host-owned display/routing fields (id, label, prefix, proxy_url,
	// disabled, weight, priority, request_retry) stripped — including
	// where the host merges them into storage — so a label or priority
	// edit keeps the fallback while a key rotation still invalidates
	// it. A provider with credential material beyond storage (for
	// example a key that only exists in metadata) supplies this hook
	// to name exactly which request fields constitute one generation:
	// the returned bytes are hashed, never stored or logged. An error
	// (or a panicking hook) fails ModelsForAuth without a fetch; a nil
	// result with no error keys the generation by auth ID and provider
	// only.
	CredentialKey func(req pluginapi.AuthModelRequest) ([]byte, error)
}

ModelCatalogConfig configures a ModelCatalog. Do not mutate the fields after NewModelCatalog; copy isolation covers the static list, overrides and globs taken at construction.

type ModelOverride added in v0.5.0

type ModelOverride struct {
	DisplayName         string
	ContextLength       int64
	MaxCompletionTokens int64
	ContributorID       string
	Thinking            *pluginapi.ThinkingSupport
}

ModelOverride tunes one upstream model. Zero/empty fields keep the discovered value; it never deletes a model. ContributorID, when non-empty, makes the emitted id that value (owner rule: the contributor id is preferred whenever a model has one). Overrides key by upstream (pre-namespace) id.

type Passthrough

Passthrough is a thin executor. BuildRequest selects the provider protocol, endpoint and authentication; it must preserve the host's final business payload (the host already applied payload rules). No protocol translation is done here. ExecuteStream re-frames SSE upstream bytes into one host chunk per data event: data lines for translators, bare JSON for OpenAI chat/Gemini clients, full named events and standalone comment lines for native SSE clients. Other Content-Types pass through unchanged; they are not parsed as SSE. Callers must not frame again. HttpRequest copies the supplied HTTP request unless PrepareHTTP is provided. Every network operation uses the injected HostHTTPClient; there is no fallback. Provider-specific quota/exhaustion classification belongs in ClassifyError. Hooks are immutable after construction and must support concurrent calls.

func (Passthrough) CountTokens

CountTokens uses TokenCount when supplied. Otherwise it returns an explicitly estimated one-token-per-four-Unicode-codepoints upper-rounded payload count, not a model tokenizer or a billable upstream token-count request.

func (Passthrough) Identifier

func (p Passthrough) Identifier() string

type PrefixClaims added in v0.5.0

type PrefixClaims struct {
	// contains filtered or unexported fields
}

PrefixClaims tracks single-segment host prefixes claimed by auth files so two credentials cannot silently share one pinning namespace (G-K1). The host strips a credential's prefix before upstream dispatch, so two accounts sharing a prefix merge into one namespace and defeat per-account pinning. APIKeyAuth stays stateless by design; providers own this optional tracker and the lifecycle that applies or removes their auths. Claim is an immediate update, not a reservation tied to a host parse/install transaction. Providers must serialize claims with committed auth changes; speculative ParseAuth calls alone cannot guarantee uniqueness among installed host auths.

Only non-empty, slash-free prefixes are claimed. The host pins the returned prefix verbatim (TrimSpace only). Use APIKeyAuth's parsed Prefix, or call NormalizePrefix and use its result both for Claim and the returned auth. Claim normalizes for comparison but cannot rewrite a caller's auth value. Empty prefixes are the normal unpinned state shared by any number of files. Credential identity is the SHA-256 of the provider's credential bytes (for example the trimmed API key); only digests are retained, never key material.

func (*PrefixClaims) Claim added in v0.5.0

func (t *PrefixClaims) Claim(file, prefix string, credential []byte) error

Claim records file's use of prefix for the given credential. It returns a duplicate_prefix *pluginabi.Error (via rpcError, HTTP 400) naming both files when a different credential already claims prefix, and duplicate_auth when the same credential is already configured in another file. Reparsing the same file, changing a file's key, or moving a file between prefixes updates the tracker instead of erroring. A prefix containing "/" is rejected with invalid_auth: the host pins it verbatim on the plugin path, so it can never be treated as unpinned.

func (*PrefixClaims) Release added in v0.5.0

func (t *PrefixClaims) Release(file string)

Release forgets every claim held by file. Providers call it after an auth has been removed from active use, not merely when a speculative parse fails, so a removed auth cannot squat on a prefix or mask a genuine duplicate.

type QuotaHTTPClient

type QuotaHTTPClient struct {
	// contains filtered or unexported fields
}

QuotaHTTPClient is a bounded, secret-redacting polling helper for quota/discovery reads. It performs no logging. Errors omit URLs, headers, bodies and transport text. This is deliberately separate from HostHTTPClient: ABI 1 / schema 10 has no no-redirect setting. Executors must always use the host client instead. The transport may implement a provider's deployment proxy policy. Its RoundTrip method must NOT follow redirects itself (the standard transports do not).

func NewQuotaHTTPClient

func NewQuotaHTTPClient(transport http.RoundTripper) *QuotaHTTPClient

NewQuotaHTTPClient sets a 15s timeout, no redirects and a 1MiB response cap. With nil transport it uses NewQuotaTransport.

func (*QuotaHTTPClient) CloseIdleConnections

func (c *QuotaHTTPClient) CloseIdleConnections()

CloseIdleConnections releases pooled polling connections during plugin shutdown.

func (*QuotaHTTPClient) Do

Do reads a quota/discovery resource. Only GET and POST are allowed: quota endpoints that mint or refresh a read-only view (GraphQL, key envelope reads) use POST without becoming an inference or mutation path, and every other method stays rejected. Successful bodies are returned to the provider parser; callers must never log them or credential-bearing headers.

type Runtime

type Runtime struct {
	// contains filtered or unexported fields
}

Runtime owns registration, dispatch and the lifetime of asynchronous work.

func New

func New(d Definition, h Caller) (*Runtime, error)

New validates a definition. The runtime starts inactive until plugin.register. Supported capabilities are auth, models, executor, quota, usage and management; unrelated SDK capabilities are rejected instead of silently advertised without dispatch.

func (*Runtime) Call

func (r *Runtime) Call(ctx context.Context, method string, raw []byte) (out []byte)

Call always returns a valid RPC envelope. Panics on the synchronous dispatch path are contained and redacted. Provider-created goroutines must contain their own panics (Go cannot recover another goroutine's panic).

func (*Runtime) Shutdown

func (r *Runtime) Shutdown(ctx context.Context)

Shutdown cancels and drains work and runs OnShutdown once. Logical shutdown does not unload the c-shared Go runtime from the host address space.

type TestReporter added in v0.2.0

type TestReporter interface {
	Helper()
	Errorf(string, ...any)
}

TestReporter is the subset of testing.T used by AssertModelNamespace, so plugin suites in any harness can use the conformance check.

Directories

Path Synopsis
Package cabi exports the ABI 1 function table for Go c-shared provider plugins.
Package cabi exports the ABI 1 function table for Go c-shared provider plugins.
examples
echo command
Echo is a mock-upstream-only provider example, not a production integration.
Echo is a mock-upstream-only provider example, not a production integration.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL