Documentation
¶
Overview ¶
Package titlemodel names sessions with a tiny language model that runs on the user's own machine.
A session needs a short title, and asking the chat provider for it costs a request, needs the network and shows the first message to the provider again. A 0.5-billion-parameter instruct model answers in about a second on a CPU, offline and for free. The package downloads two things on request, checks both against sums fixed in this file, and runs them: the llama.cpp server (llama-server) and a small GGUF model. Nothing is downloaded or started unless the user installed the model (`seshat title-model install`), and when it is missing or fails, the engine falls back to the provider as before.
Index ¶
- Constants
- Variables
- func CleanTitle(raw string) string
- func CleanupOrphan(root string) bool
- func CloseShared()
- func LogFile(root string) (*os.File, error)
- func Remove(root string) error
- func SaveState(root string, st State) error
- func StatePath(root string) string
- type Archive
- type Generator
- type Installer
- type Model
- type Progress
- type Server
- type State
Constants ¶
const DefaultModelID = "qwen2.5-0.5b"
DefaultModelID is the model installed when none is chosen.
const HFBaseURL = "https://huggingface.co"
HFBaseURL is where the model files are published.
const LlamaBaseURL = "https://github.com/ggml-org/llama.cpp/releases/download/" + LlamaTag
LlamaBaseURL is where the archives are published.
const LlamaTag = "b11342"
LlamaTag is the llama.cpp release the server comes from. Raising it means checking the new archives and updating every sum below.
Variables ¶
var Models = []Model{
{
ID: "qwen2.5-0.5b", Label: "Qwen2.5 0.5B Instruct", Repo: "Qwen/Qwen2.5-0.5B-Instruct-GGUF",
File: "qwen2.5-0.5b-instruct-q4_k_m.gguf", SHA256: "74a4da8c9fdbcd15bd1f6d01d621410d31c6fc00986f5eb687824e7b93d7a9db",
Size: 491400032, Description: "Default. Small, multilingual, Apache-2.0.",
},
{
ID: "gemma-3-270m", Label: "Gemma 3 270M Instruct", Repo: "unsloth/gemma-3-270m-it-GGUF",
File: "gemma-3-270m-it-Q4_K_M.gguf", SHA256: "b1baabd6b729e4041822220d3e648e00d99cac5df86b10dffb77bcccf0688e39",
Size: 253115424, Description: "Smallest option. Faster, rougher titles. Gemma terms of use apply.",
},
}
Models is the catalog, the same models the desktop offers.
Functions ¶
func CleanTitle ¶
CleanTitle keeps the first line of a model's answer, without quotes, a "Title:" label or closing punctuation, and at most 8 words.
func CleanupOrphan ¶
CleanupOrphan stops a title server that an earlier run of seshat left behind, and reports whether it did.
A process id alone is not proof of anything (the system reuses them), so the process is stopped only if the port of the record still answers as a llama-server that has loaded this very model file.
Types ¶
type Archive ¶
Archive is a llama.cpp CPU build for one platform.
func ArchiveFor ¶
ArchiveFor returns the llama.cpp build for a platform.
type Generator ¶
type Generator struct {
// contains filtered or unexported fields
}
Generator names sessions with the local model. It satisfies the engine's TitleGenerator. The server starts at the first title, is shared by every session of the process, and stops when idle or on Close.
func NewGenerator ¶
NewGenerator returns a generator for an installation.
func Shared ¶
Shared returns the generator of the process for a runtime root, or nil when the model is not installed or is turned off. Several clients are built in one run (changing the model builds a new one) and they must share one server.
type Installer ¶
type Installer struct {
// Root is the runtime root; everything lives in Root/title-model.
Root string
// LlamaBase and HFBase are the download locations (tests point them at a local server).
LlamaBase string
HFBase string
HTTP *http.Client
// Archives and Models override the catalog (tests).
Archives map[string]Archive
Models []Model
// GOOS and GOARCH override the platform (tests).
GOOS, GOARCH string
Progress Progress
}
Installer downloads, checks and records the server and the model.
func NewInstaller ¶
NewInstaller returns an installer for the official locations and the real platform.
type Model ¶
type Model struct {
ID string
Label string
Repo string
File string
SHA256 string
Size int64
Description string
}
Model is a GGUF instruct model small enough to name a session in about a second on a CPU. Only models that answer directly belong here: a reasoning model spends its output budget thinking and never returns a title.
type Server ¶
type Server struct {
// contains filtered or unexported fields
}
Server is a running llama-server bound to the loopback address.
func StartServer ¶
StartServer launches llama-server on a free loopback port and returns once the model is loaded and the server answers. log, when not nil, receives what the server prints.
type State ¶
type State struct {
// Enabled says whether sessions are named with the local model. Installing turns it on; `disable` turns it off
// without deleting anything.
Enabled bool `json:"enabled"`
ModelID string `json:"model_id"`
ModelPath string `json:"model_path"`
ServerPath string `json:"server_path"`
LlamaTag string `json:"llama_tag"`
}
State is what is recorded of an installation, in <runtime root>/title-model/state.json.
Source Files