Documentation
¶
Overview ¶
Package gateway is the daemon half of the LLM gateway's server-side control plane (design 023): it reports governed spend to rysh-server, leases a slice of an org-wide allowance and enforces locally against that slice, and pulls central policy.
THE ONE RULE THIS PACKAGE EXISTS TO OBEY ---------------------------------------- The hot path never leaves the process. A per-request WAN call would put provider-latency-scale delay in front of every LLM request and turn a network blip into what looks like a provider outage (023 §4.2/G4). So:
- Allow() reads local numbers only. No I/O, no locks held across a call, no blocking. Ever.
- Everything that talks to the network — usage flushes, lease renewals, policy polls — runs on this package's own goroutine, off the request path.
INERT BY DEFAULT ---------------- Nothing here does anything unless BOTH `upstream.enabled` and the explicit `upstream.governance` opt-in are set. Enabled() is checked at every entry point, and TestLedgerClient_DisabledMakesNoNetworkCalls holds the line.
Index ¶
- Constants
- type LeaseState
- type LedgerClient
- func (c *LedgerClient) Allow(tenant string) (bool, string)
- func (c *LedgerClient) Enabled() bool
- func (c *LedgerClient) Flush()
- func (c *LedgerClient) Leases() []LeaseState
- func (c *LedgerClient) Policy() ([]byte, time.Time)
- func (c *LedgerClient) PullPolicy(ctx context.Context) []byte
- func (c *LedgerClient) Record(rec msg.MsgUsageRecord)
- func (c *LedgerClient) SetPolicyHook(fn func([]byte))
- func (c *LedgerClient) Spend(tenant string, tokens int64)
- func (c *LedgerClient) Start()
- func (c *LedgerClient) Stop()
Constants ¶
const ( FlushInterval = 10 * time.Second FlushMaxCalls = 256 )
Flush thresholds (design 023 §4.4): every 10 s or 256 metered calls, whichever comes first, and on shutdown.
const ( PartitionLocal = "local" // fall back to the machine-local ceiling (default) PartitionStrict = "strict" // refuse PartitionOpen = "open" // allow )
Partition modes (design 023 §4.3).
const ( VerdictNoGovernance = "no_governance" // feature off VerdictLeased = "leased" // spending inside a valid lease VerdictExhausted = "exhausted" // the SERVER says the scope is out of budget VerdictLeaseSpent = "lease_spent" // local slice used up, renewal not back yet VerdictPartition = "partition" // no usable lease; partition mode decided )
Verdicts explain why Allow answered as it did — for the log line, and for tests that must tell "out of budget" apart from "partitioned". Plain strings so LedgerClient satisfies proxy.LeaseGate without an adapter.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type LeaseState ¶
type LeaseState struct {
Scope string
Granted int64
Spent int64
Expires time.Time
Ceiling int64
Remaining int64
Exhausted bool
}
LeaseState is a snapshot for `##proxy status` / tests. Values only, no pointers into the client's state.
type LedgerClient ¶
type LedgerClient struct {
// contains filtered or unexported fields
}
LedgerClient owns all four flows in design 023 §4.1.
func New ¶
func New(cfg config.UpstreamConfig, daemonID string, pub *msg.NATSPublisher) *LedgerClient
New builds a client. It is safe to construct one even when governance is off — every method is a no-op then, which keeps the call sites free of conditionals.
func (*LedgerClient) Allow ¶
func (c *LedgerClient) Allow(tenant string) (bool, string)
Allow reports whether a tenant may spend right now.
IN-PROCESS AND NON-BLOCKING, ALWAYS. It reads a local number and may kick a renewal off on another goroutine; it never waits for one. This is the property design 023 §7's first risk row is about.
func (*LedgerClient) Enabled ¶
func (c *LedgerClient) Enabled() bool
Enabled reports whether the control plane is active. Both switches must be on AND the connection details must be present: a half-configured upstream must not produce a client that fails a request per flush.
func (*LedgerClient) Flush ¶
func (c *LedgerClient) Flush()
Flush publishes the pending rollups as one batch.
On a publish failure the rollups are put BACK and the sequence number is not consumed, so nothing is lost and the retry is byte-identical — which is exactly what the server's (daemon_id, seq) seen-set is built to absorb.
func (*LedgerClient) Leases ¶
func (c *LedgerClient) Leases() []LeaseState
Leases snapshots every scope the daemon holds a lease for.
func (*LedgerClient) Policy ¶
func (c *LedgerClient) Policy() ([]byte, time.Time)
Policy returns the last-known-good central document and when it was refreshed. Empty when none has ever been fetched.
func (*LedgerClient) PullPolicy ¶
func (c *LedgerClient) PullPolicy(ctx context.Context) []byte
PullPolicy fetches the central policy document, using If-None-Match so an unchanged document costs a 304 (023 §4.7).
It returns the body when it CHANGED, nil otherwise. An unreachable server or an error response leaves the last-known-good document in place: an expired cache must not become an escape hatch around a policy (013's fail-closed rule applies to a central document exactly as it does to a local one).
func (*LedgerClient) Record ¶
func (c *LedgerClient) Record(rec msg.MsgUsageRecord)
Record accumulates one metered call. It is called from the usage path, so it does nothing but arithmetic under a short lock.
The daemon keeps its own rollups regardless (023 §4.4): `##cost` stays a local, offline-capable view and this is a second, additive observer — never a replacement, and never a second source of truth for the local ledger.
func (*LedgerClient) SetPolicyHook ¶
func (c *LedgerClient) SetPolicyHook(fn func([]byte))
SetPolicyHook installs the change callback. Call before Start.
func (*LedgerClient) Spend ¶
func (c *LedgerClient) Spend(tenant string, tokens int64)
Spend records tokens against the current lease. Pure arithmetic; the renewal it may trigger happens on the next Allow.
func (*LedgerClient) Start ¶
func (c *LedgerClient) Start()
Start runs the background loop: usage flushes, lease renewals, policy polls. It returns immediately, and is a no-op when governance is off.
func (*LedgerClient) Stop ¶
func (c *LedgerClient) Stop()
Stop flushes what is pending and shuts the loop down. Safe to call twice.
Source Files