README
¶
tokc
tokc counts tokens using pretrainded models from HuggingFace.
Example usage
$ echo "The quick brown fox jumps over the lazy dog" | tokc -model mistralai/Mistral-Small-3.1-24B-Base-2503
11
Install
sh go install github.com/WinPooh32/tokc@latest
Documentation
¶
There is no documentation for this package.
Source Files
Click to show internal directories.
Click to hide internal directories.