Documentation
¶
Overview ¶
Package llmgw is the AI-gateway inlet: existing AI clients (Claude Code first) point their base URL at the chassis, their Anthropic Messages request runs once through the tenant's `_llm` stack (which may reject it, rewrite it, or repoint the upstream), and the upstream response streams back byte-transparent. The inlet owns the transport; stacks own policy — the SSE stream is never routed through the processor. After the exchange, completion metadata is recorded asynchronously (usage event + a phase="completed" envelope into the same stack); no client waits on it.
Design doc: thanks-computer-service/docs/todo-ai-gateway-inlet.md.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Gateway ¶
type Gateway struct {
// contains filtered or unexported fields
}
func New ¶
func New(ctx context.Context, pu *processor.Unit, resolver *ingress.DBResolver, guard egress.Guard) (*Gateway, error)
New builds the gateway. The upstream client mirrors the processor's egress-guarded transport (dial-time IP check under the chassis egress policy) but with NO client-level timeout — an LLM stream legitimately runs for many minutes; cancellation is the request context — and with compression disabled so the client's own Accept-Encoding negotiates end-to-end and response bytes pass through untouched.
func (*Gateway) HandleCountTokens ¶
func (g *Gateway) HandleCountTokens(w http.ResponseWriter, r *http.Request)
HandleCountTokens serves POST /v1/messages/count_tokens as a stackless transparent forward. Claude Code calls it constantly for context management; it's a cheap metadata echo of the request, so policy (the _llm stack) doesn't apply — but tenant routing, the _llm self-gate, and auth do, exactly as for /v1/messages. Without this route the path would fall through to the tenant's web stack and answer with something that isn't a token count.
func (*Gateway) HandleMessages ¶
func (g *Gateway) HandleMessages(w http.ResponseWriter, r *http.Request)
HandleMessages serves POST /v1/messages. Registered ahead of the web catch-all: no BasicAuth, no opstack render path — the gateway shapes its own (Anthropic-protocol) responses.
Source Files