Documentation
¶
Overview ¶
Package chutes is a Chutes end-to-end-encrypted, TEE-attested LLM client. It satisfies inference.Client and tunnels OpenAI chat completions through the Chutes /e2e/invoke API sealed with post-quantum ML-KEM-768 + ChaCha20-Poly1305.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type AttestReason ¶
type AttestReason string
AttestReason names the specific attestation check that failed. It lets callers branch on the failure mode via errors.As without parsing strings.
const ( // ReasonQuoteSignatureInvalid: the TDX quote's ECDSA signature or QE // report signature did not verify. ReasonQuoteSignatureInvalid AttestReason = "quote_signature_invalid" // ReasonRootCAUntrusted: the PCK certificate chain did not chain to the // trusted Intel SGX Root CA (or a chain cert was expired). ReasonRootCAUntrusted AttestReason = "root_ca_untrusted" // ReasonBindingMismatch: report_data[:32] did not match // sha256(nonceHex + pubKeyB64). ReasonBindingMismatch AttestReason = "binding_mismatch" // ReasonEvidenceMalformed: the raw quote bytes could not be parsed into a // TDX quote. ReasonEvidenceMalformed AttestReason = "evidence_malformed" // ReasonNvidiaVerdictInvalid: NRAS verification failed — the returned EAT // JWT did not verify (bad/absent signature, wrong alg, unknown kid, // malformed token) or the x-nvidia-overall-att-result claim was not true. ReasonNvidiaVerdictInvalid AttestReason = "nvidia_verdict_invalid" )
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client is a Chutes end-to-end-encrypted, attested chat client. It resolves model names to chute IDs, discovers + attests TEE instances, caches the attested session, and tunnels OpenAI chat requests through POST /e2e/invoke sealed with post-quantum ML-KEM + ChaCha20-Poly1305.
A Client is safe for concurrent use: the model->chute and session caches are guarded by mu.
Connection-binding note (fail-safe asymmetry): unlike the generic transport.Client — which binds one Endpoint and rejects a request whose Model.Provider/BaseURL differs with a pre-I/O *failure.ModelMismatchError — this client binds its gateway endpoints at construction (New's apiBase/llmBase) and enforces model identity per request via NVIDIA TEE attestation. A provider/endpoint mismatch therefore surfaces as an attestation failure (attestation cannot bind to the wrong model/instance), not an *failure.ModelMismatchError. This is fail-safe: the request is never sent when the check fails; only the error type differs.
func New ¶
New builds a Client. apiBase is the e2e/evidence host (https://api.chutes.ai); apiKey is the Chutes bearer token. Defaults (llmBase, NRAS/JWKS URLs, an http.Client, the caches, and the real attestFn) are applied first, then options override.
func (*Client) Invoke ¶
Invoke sends a non-streaming chat completion through the attested e2e channel and returns the decrypted response as a provider-neutral *inference.Response. It validates the model and request features before any network I/O — fail closed.
func (*Client) Stream ¶
func (c *Client) Stream(ctx context.Context, req inference.Request) (*stream.StreamReader[content.Chunk], error)
Stream sends a streaming e2e chat request and returns a *stream.StreamReader[content.Chunk] over the decrypted deltas. It validates the model and request features before any network I/O — fail closed. The returned reader MUST be Closed by the caller.
type Option ¶
type Option func(*Client)
Option configures a Client at construction. Only the knobs that production or tests actually use are exposed; nothing speculative.
func WithHTTPClient ¶
WithHTTPClient sets the HTTP client used for every request (e2e, evidence, model listing, NRAS, JWKS). Useful for timeouts, proxies, and httptest.
func WithLLMBase ¶
WithLLMBase overrides the base URL used to resolve model names via /v1/models. Production uses https://llm.chutes.ai.
Source Files