Documentation
¶
Index ¶
- Constants
- func CheckHardware(parameterSize string, systemRAMGB float64) (bool, float64)
- func CheckHardwareSafe(parameterSize string, systemRAMGB float64) (bool, float64)
- func ComputeRecommendedScore(swe, speedTPM, ramFit float64) float64
- func Detect(ctx context.Context, baseURL string) bool
- func DetectAppleSiliconBandwidthGBs() float64
- func EstimateCloudSpeedTPM(pricePerToken float64) float64
- func EstimateLocalSpeedTPM(paramsB, bandwidthGBs float64) float64
- func EstimateRAMGB(paramsB float64) float64
- func FetchSWEBenchScores(ctx context.Context, url string) (map[string]float64, error)
- func FindSWEScore(scores map[string]float64, hfRepoID string) (float64, bool)
- func FormatActiveTOMLSnippet(ranked []RankedModel) string
- func FormatTOMLSnippet(ranked []RankedModel) string
- func GetSystemRAMGB() float64
- func InferTaskLabel(info HFModelInfo, ollamaName string) string
- func MatchModelScore(ollamaName string, scores map[string]float64) (float64, bool)
- func ModelFamily(name string) string
- func NormalizeCloudID(id string) string
- func ParseAppleSiliconBandwidth(brandString string) float64
- func ParseParamSize(s string) float64
- type HFModelInfo
- type Model
- type RankedModel
Constants ¶
const DefaultBaseURL = "http://localhost:11434"
DefaultBaseURL is the default ollama API endpoint.
const HuggingFaceAPIURL = "https://huggingface.co/api/models"
HuggingFaceAPIURL is the default HuggingFace Hub API base URL.
const SWEBenchCacheTTL = 6 * time.Hour
FetchSWEBenchScores fetches the SWE-bench Verified leaderboard and returns the best score per model. Keys include lowercased model tags, HuggingFace repo paths extracted from URL tags, and model name parts.
All entries are included (not filtered by os_model) so that HF repo ID matching can find scores for any model. SWEBenchCacheTTL is the on-disk cache lifetime for SWE-bench leaderboard data.
const SWEBenchURL = "https://raw.githubusercontent.com/SWE-bench/swe-bench.github.io/master/data/leaderboards.json"
SWEBenchURL is the default URL for SWE-bench leaderboard data.
Variables ¶
This section is empty.
Functions ¶
func CheckHardware ¶
CheckHardware checks if a model with given parameter size fits in available RAM. Returns (ok, neededGB). If parameter size is unknown, returns (true, 0).
func CheckHardwareSafe ¶ added in v0.5.0
CheckHardwareSafe checks if a model fits within 75% of available RAM. Uses a conservative threshold: a 48 GB model won't run on 48 GB RAM. Returns (ok, neededGB). If parameter size is unknown, returns (true, 0).
func ComputeRecommendedScore ¶ added in v0.5.0
ComputeRecommendedScore computes the composite ranking score. swe: SWE-bench %, speedTPM: tokens/min, ramFit: graduated multiplier (0–1).
SWE quality dominates (90% weight). Speed contributes a small bonus (up to 5 points) so that among models with equal SWE scores, faster wins — but a model with 8% lower SWE cannot overcome a 3-point speed bonus.
func DetectAppleSiliconBandwidthGBs ¶ added in v0.5.0
func DetectAppleSiliconBandwidthGBs() float64
DetectAppleSiliconBandwidthGBs returns 0 on Linux (not Apple Silicon).
func EstimateCloudSpeedTPM ¶ added in v0.5.0
EstimateCloudSpeedTPM estimates cloud model speed (tokens/min) from completion price per token (USD). Cheaper models are typically faster.
func EstimateLocalSpeedTPM ¶ added in v0.5.0
EstimateLocalSpeedTPM estimates local model speed (tokens/min).
When bandwidthGBs > 0 (Apple Silicon detected), uses the physics formula:
speed = bandwidth / model_weight_bytes_per_token
Q4_K_M quantization ≈ 0.5625 bytes/param (4.5 bits/param). Capped at 9000 T/min (150 tok/s) — the compute-bound ceiling for tiny models.
When bandwidthGBs == 0, falls back to generic tier estimates for average consumer GPU hardware.
func EstimateRAMGB ¶
EstimateRAMGB estimates RAM needed (GB) for a Q4_K_M quantized model. Formula: params_in_billions * 0.55 + 2 GB overhead. Returns 0 if paramsB is 0.
func FetchSWEBenchScores ¶
func FindSWEScore ¶
FindSWEScore finds the best SWE-bench score for a HuggingFace repo ID. It checks the scores map using the full repo ID, the model name part, and case-insensitive variants.
func FormatActiveTOMLSnippet ¶ added in v0.3.0
func FormatActiveTOMLSnippet(ranked []RankedModel) string
FormatActiveTOMLSnippet generates an active (uncommented) TOML snippet for devcell.toml from ranked models. The #1 ranked model becomes the default.
func FormatTOMLSnippet ¶
func FormatTOMLSnippet(ranked []RankedModel) string
FormatTOMLSnippet generates a commented-out TOML snippet for devcell.toml from ranked models.
func GetSystemRAMGB ¶
func GetSystemRAMGB() float64
GetSystemRAMGB returns total system RAM in GB (Linux).
func InferTaskLabel ¶
func InferTaskLabel(info HFModelInfo, ollamaName string) string
InferTaskLabel returns a short human-readable label for what a model is good at, based on HuggingFace tags and model name.
func MatchModelScore ¶
MatchModelScore finds the SWE-bench score for an ollama model name. It strips the :size suffix and does case-insensitive matching.
func ModelFamily ¶
ModelFamily strips the :tag suffix from an ollama model name (exported). "deepseek-r1:32b" → "deepseek-r1", "codellama:latest" → "codellama".
func NormalizeCloudID ¶ added in v0.5.0
NormalizeCloudID converts an OpenRouter model ID to a SWE-bench-comparable key. Strips provider prefix, converts dots to dashes, strips :variant suffixes. "anthropic/claude-opus-4.6" → "claude-opus-4-6"
func ParseAppleSiliconBandwidth ¶ added in v0.5.0
ParseAppleSiliconBandwidth parses a CPU brand string like "Apple M4 Pro" and returns the corresponding memory bandwidth in GB/s, or 0 if unrecognised.
func ParseParamSize ¶
ParseParamSize parses a parameter size string like "32B" or "671M" into billions of parameters. Returns 0 if unparseable.
Types ¶
type HFModelInfo ¶
type HFModelInfo struct {
ModelID string `json:"modelId"`
PipelineTag string `json:"pipeline_tag"`
Tags []string `json:"tags"`
}
HFModelInfo holds model metadata from HuggingFace.
func FetchHFModelInfo ¶
FetchHFModelInfo searches the HuggingFace Hub API for a model by name and returns its metadata. The baseURL parameter allows test injection.
type Model ¶
type Model struct {
Name string
Size int64
ParameterSize string
Family string
Provider string // "" or "ollama" = local; otherwise cloud provider (e.g. "anthropic")
CompletionPricePerToken float64 // USD per token; 0 for local models
}
Model represents a locally available ollama model.
type RankedModel ¶
type RankedModel struct {
Model
SWEScore float64
Rank int
ScoreSource string // "SWE", "est", or "" (no score)
SpeedTPM float64 // estimated tokens per minute
RecommendedScore float64 // composite score for default ranking
}
RankedModel is a Model with its SWE-bench score and rank position.
func RankModels ¶
func RankModels(models []Model, limit int, sweScores map[string]float64, hfInfoMap map[string]HFModelInfo, systemRAMGB float64, sortBy string) []RankedModel
RankModels sorts models by composite score (descending) and limits to top N. It tries multiple matching strategies in order:
- Direct family match against sweScores (MatchModelScore)
- HF repo ID match against sweScores (FindSWEScore via hfInfoMap)
- Cloud model NormalizeCloudID match against sweScores
- Hardcoded fallback ratings
ScoreSource is set to "SWE" for live matches, "est" for fallback, "" for no score. sortBy can be "swe", "speed", "size", or "" / "recommended".
Source Files