localmodel

package
v0.8.59 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 11, 2026 License: Apache-2.0 Imports: 6 Imported by: 0

Documentation

Overview

Package localmodel is the host-side brain for Dejima's managed local-model support: a curated, hardware-aware catalog of open-weights models plus the backend-detection glue that lets islands drive them as a shared inference service. It deliberately does NOT reimplement a model registry — the pull / serve / lifecycle is delegated to a backend (Ollama by default); this package is the curation + recommendation + detection layer Dejima adds on top.

See docs/local-models.md for the design of record.

Index

Constants

View Source 
const (
	// LocalProviderName is the providercreds entry auto-registered when a backend
	// is installed, so local models show up in the `v` model editor unprompted.
	LocalProviderName = "local"

	// OllamaEndpoint is the base URL an island uses to reach the host's Ollama.
	OllamaEndpoint = "http://host.docker.internal:11434/v1"
	// OllamaAllowHostPort is the egress-allowlist entry that lets an island reach
	// exactly that endpoint and nothing else.
	OllamaAllowHostPort = "host.docker.internal:11434"
)

In-island wiring for the default (Ollama) backend. Islands reach the host's inference server via host.docker.internal (already in the egress no-proxy path); Ollama exposes an OpenAI-compatible API under /v1 on port 11434.

View Source 
const DefaultBackend = BackendOllama

DefaultBackend is what a bare `dejima local install` provisions.

Variables

View Source 
var Catalog = []Model{
	{Alias: "qwen-coder-3b", Ref: "qwen2.5-coder:3b-instruct-q4_K_M", Params: "3B", MinRAMGiB: 8, Coding: true,
		Note: "Smallest coding model — runs on laptops; light autocomplete/edits."},
	{Alias: "qwen-coder-7b", Ref: "qwen2.5-coder:7b-instruct-q4_K_M", Params: "7B", MinRAMGiB: 16, Coding: true,
		Note: "Solid small coder; good default for 16 GB machines."},
	{Alias: "mistral-small", Ref: "mistral-small:24b-instruct-2501-q4_K_M", Params: "24B", MinRAMGiB: 32, Coding: true,
		Note: "Strong general + coding model at a mid footprint."},
	{Alias: "qwen-coder", Ref: "qwen2.5-coder:32b-instruct-q4_K_M", Params: "32B", MinRAMGiB: 36, Coding: true,
		Note: "Best open coder that fits a workstation; the recommended default."},
	{Alias: "llama-70b", Ref: "llama3.3:70b-instruct-q4_K_M", Params: "70B", MinRAMGiB: 48, Coding: true,
		Note: "Large general model; strong reasoning, heavier to run (fits a 64 GB box)."},
	{Alias: "kimi-k2", Ref: "kimi-k2:q4_K_M", Params: "~1T MoE", MinRAMGiB: 256, Coding: true,
		Note: "Frontier-class open MoE — needs a big rig / server, not a laptop."},
}

Catalog is the curated set — deliberately small, refreshed in code over time rather than mirroring a full model registry. Ordered small→large so callers can scan for the biggest that fits. MinRAMGiB is the *total host RAM* to run the model comfortably at the referenced quantization, leaving room for the OS and a working set — not just the weight footprint.

Functions

func ResolveRef 

func ResolveRef(handle string) (ref string, curated bool, err error)

ResolveRef maps a user-typed handle (a curated alias or a raw backend ref) to the ref to pull. A curated alias resolves to its pinned ref; anything else is passed through verbatim after validation, so power users can pull uncurated models. ok reports whether the handle matched the curated catalog.

func ValidateRef 

func ValidateRef(ref string) error

ValidateRef guards a model ref before it reaches a shell/exec arg. Refs are backend tags like "qwen2.5-coder:32b-instruct-q4_K_M" — alnum plus a small punctuation set; anything else is rejected rather than escaped.

Types

type Backend 

type Backend string

Backend identifies a local inference-server implementation. Ollama is the default; the set is open so vLLM (throughput, Linux/GPU) and LM Studio can slot in behind the same LocalBackend interface. 

const (
	BackendOllama   Backend = "ollama"
	BackendVLLM     Backend = "vllm"
	BackendLMStudio Backend = "lmstudio"
)

type InstalledModel 

type InstalledModel struct {
	Ref   string `json:"ref"`             // backend tag, e.g. "qwen2.5-coder:32b-instruct-q4_K_M"
	Size  string `json:"size,omitempty"`  // human size from the backend, e.g. "20 GB"
	Alias string `json:"alias,omitempty"` // catalog alias, when the ref is one we curate
}

InstalledModel is a model already pulled into the backend on the host.

type LocalBackend 

type LocalBackend interface {
	Name() Backend
	// Detect reports whether the backend binary is installed and whether it's
	// currently responding.
	Detect(ctx context.Context) (installed, running bool)
	// Endpoint is the in-island base URL for the OpenAI-compatible API.
	Endpoint() string
	// AllowHostPort is the egress-allowlist entry islands need to reach Endpoint.
	AllowHostPort() string
	// List returns models already pulled on the host.
	List(ctx context.Context) ([]InstalledModel, error)
	// Pull downloads a model, streaming progress lines. Caller closes the reader.
	Pull(ctx context.Context, ref string) (io.ReadCloser, error)
	// Remove deletes a pulled model.
	Remove(ctx context.Context, ref string) error
	// Install streams a best-effort install of the backend on the host.
	Install(ctx context.Context) (io.ReadCloser, error)
}

LocalBackend abstracts a host inference server so Ollama (default), vLLM, and LM Studio can slot in behind one interface. All methods run on the daemon HOST (where the GPU is), never inside an island.

type Model 

type Model struct {
	Alias     string `json:"alias"`       // dejima handle, e.g. "qwen-coder"
	Ref       string `json:"ref"`         // backend pull ref, e.g. "qwen2.5-coder:32b-instruct-q4_K_M"
	Params    string `json:"params"`      // human parameter size, e.g. "32B"
	MinRAMGiB int    `json:"min_ram_gib"` // host RAM (GiB) to run it comfortably (weights + KV headroom)
	Coding    bool   `json:"coding"`      // suited to agentic / coding work
	Note      string `json:"note"`        // one-line description
}

Model is one curated open-weights model Dejima knows how to recommend and pull. Ref is the backend-native pull reference (an Ollama tag today); Alias is the short, backend-agnostic handle users type (`dejima local pull qwen-coder`).

func Lookup 

func Lookup(handle string) (Model, bool)

Lookup resolves a user-typed handle to a catalog model. It matches the Alias first, then the full Ref (so `dejima local pull qwen2.5-coder:32b-...` also works). ok is false for anything not in the curated set — callers may still pass an arbitrary Ref straight to the backend, but it won't be size-checked.

type Ollama 

type Ollama struct {
	// contains filtered or unexported fields
}

Ollama is the default LocalBackend: the daemon shells out to the host `ollama` CLI. It's the simplest path and Mac-friendly (Metal); vLLM is the better default on a Linux/GPU daemon and can implement this same interface later.

func NewOllama 

func NewOllama() *Ollama

NewOllama returns the default-configured Ollama backend.

func (*Ollama) AllowHostPort 

func (o *Ollama) AllowHostPort() string

func (*Ollama) Detect 

func (o *Ollama) Detect(ctx context.Context) (installed, running bool)

Detect: installed = binary on PATH; running = `ollama list` succeeds (it talks to the local server, so a clean exit means the server is up).

func (*Ollama) Endpoint 

func (o *Ollama) Endpoint() string

func (*Ollama) Install 

func (o *Ollama) Install(ctx context.Context) (io.ReadCloser, error)

Install runs Ollama's official install script (Linux/macOS). It's best-effort and explicitly user-invoked (`dejima local install`); we never auto-install.

func (*Ollama) List 

func (o *Ollama) List(ctx context.Context) ([]InstalledModel, error)

List parses `ollama list` and annotates any ref we curate with its alias.

func (*Ollama) Name 

func (o *Ollama) Name() Backend

func (*Ollama) Pull 

func (o *Ollama) Pull(ctx context.Context, ref string) (io.ReadCloser, error)

Pull streams `ollama pull <ref>` combined output. The model ref is validated by the caller (ValidateRef) before we ever build the command.

func (*Ollama) Remove 

func (o *Ollama) Remove(ctx context.Context, ref string) error

type Recommendation 

type Recommendation struct {
	HostRAMGiB int     `json:"host_ram_gib"`
	Fits       []Model `json:"fits"` // catalog models that fit, largest-first
	Top        *Model  `json:"top"`  // the recommended default (nil if nothing fits)
}

Recommendation is the host-aware answer to "what should I run here?": the models that fit this machine (largest-first) and a single top pick.

func RecommendFor 

func RecommendFor(ramGiB int) Recommendation

RecommendFor curates the catalog for a host with ramGiB total RAM. A model fits when ramGiB meets its MinRAMGiB (which already folds in OS + working-set headroom). Top is the largest coding-capable model that fits — the best local coder this box can run.

type Status 

type Status struct {
	Backend    Backend          `json:"backend"`
	Installed  bool             `json:"installed"` // backend binary present on the host
	Running    bool             `json:"running"`   // backend responding (models listable)
	Endpoint   string           `json:"endpoint"`  // in-island OpenAI-compatible base URL
	Models     []InstalledModel `json:"models"`
	HostRAMGiB int              `json:"host_ram_gib"`
	Recommend  Recommendation   `json:"recommend"`
	Provider   string           `json:"provider"` // the providercreds name islands use ("local")
}

Status is the backend's runtime state on the daemon host, plus the host-aware recommendation — everything the TUI/CLI needs to render "Local models" in one call.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.