Count returns the number of BPE tokens in s under cl100k_base. Returns 0
for the empty string. Panics if Init has not been called successfully —
thesis-2 loud-fail: a count without init is a wiring bug. The encoder
pointer is the source of truth: nil ⇒ Init never ran or failed.
Init loads the cl100k_base BPE table. Safe to call multiple times;
repeated calls are no-ops. Returns the first init error if loading
failed. Call from server.New so a corrupt embed fails startup loudly,
not on first request.
Documentation
¶
Source Files