toolbelt

package module
v2.2.6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 24, 2026 License: GPL-2.0, GPL-3.0 Imports: 34 Imported by: 0

README

toolbelt

Go Reference Go version Test coverage Mutation OpenSSF Best Practices OpenSSF Scorecard

Declarative dev-tool provisioning for container dev boxes: manifest + catalog + reconciler engine

A standalone Go library that provisions developer tools (language servers, CLIs, runtimes, linters) onto a persistent volume, declaratively. A JSON manifest records intent: which tools, at which versions, enabled or disabled. A compiled catalog carries install knowledge for ~700 tools, sourced from the mise and aqua registries plus curated overlays. The engine reconciles installed state against intent through a single-flight job queue: enabled-but-missing tools are installed (checksum-verified when the registry definition declares a source), disabled-but-installed tools are uninstalled with their template kept, and unmanaged files are never touched.

Built for headless and UI consumers alike: everything is driven through the Go API, an optional REST projection (toolbelt/httpapi), or hand edits of the manifest file itself, which the engine picks up on its next operation.

The model

Three data artifacts, all on the consumer's persistent volume:

File Owner Purpose
tools.json user intent (engine-written, hand-editable) which tools exist, versions, pin, disabled
tools-state.json engine what is actually installed, owned bin names, last error
tool-catalog.json image build (baked fallback) + runtime refresh (tool-catalog.cached.json under ConfigDir) install knowledge: sources, artifact templates, checksum locations, dependencies, registry license texts

Tool lifecycle is a three-state machine the reconciler enforces in both directions:

  • absent: not in the manifest. Unmanaged files under the tools dir are never touched.
  • disabled ("disabled": true): a template. Recorded intent, nothing on disk; the engine uninstalls its owned footprint if present.
  • enabled (present, no flag): installed; updated when unpinned.

Sources: aqua:owner/repo (binary artifacts with upstream checksum verification), npm:pkg, pip:pkg (via uv), cargo:crate, go:module, and a manual bash escape hatch. Ecosystem backends are themselves tools: an npm: install pulls node from the catalog automatically, go: pulls the Go toolchain, and so on.

Install

go get github.com/cplieger/toolbelt/v2@latest

Usage

engine, err := toolbelt.New(&toolbelt.Config{
    ConfigDir:   "/config",                        // tools.json + tools-state.json
    ToolsDir:    "/config/tools",                  // bin/, opt/, npm/, python/
    CatalogPath: "/opt/app/tool-catalog.json",     // baked fallback (image build)
    Seed:        toolbelt.DefaultSeed(),           // LSP templates, disabled
    Refresh: &toolbelt.CatalogRefresh{             // optional: runtime catalog refresh
        URL:      "https://example.com/tool-catalog.json",
        Interval: 24 * time.Hour,                  // 0 = on-demand only
        Require:  []string{"gopls", "gh"},         // verified before every swap
    },
})
if err != nil {
    log.Fatal(err)
}
defer engine.Close()

// Boot: converge disk to intent, then gate whatever needs tools on the job.
if job, _ := engine.Reconcile(toolbelt.ReconcileMissing); job != nil {
    _, _ = engine.Wait(ctx, job.ID)
}

// Add a tool by name; the catalog supplies source, version, deps.
job, err := engine.Add(ctx, &toolbelt.AddRequest{Name: "gopls"})

// The enable/disable toggle: disabling uninstalls, the template stays.
on := true
job, err = engine.Patch("gopls", toolbelt.PatchRequest{Disabled: &on})

DefaultSeed() ships five disabled templates: the officially supported language servers plus the GitHub CLI (gopls, typescript-language-server, pyright, rust-analyzer, gh). Nothing downloads until a template is enabled; install knowledge hydrates from the catalog at enable time, so the seed never goes stale. Backend runtimes (node, go) and required packages (typescript) are not seeded: the engine auto-adopts missing dependencies at install time, while a seeded-but-disabled dependency would refuse dependent installs (a disabled entry is user policy).

The REST projection

toolbelt/httpapi serves the engine over HTTP: inventory, catalog search, add, patch (the toggle verb), install, update, remove, jobs, cancel. It is a pure projection: no auth, no middleware; wrap it in your own stack (an origin-checked chain, a loopback-only gate).

h := httpapi.Handler(engine, "/api/tools")
mux.Handle("/api/tools", h)
mux.Handle("/api/tools/", h)

Mutations return 202 {"job": ...}; refusals are 409 with a coded envelope (has_dependents names the blockers, disabled marks install-on-a-template, not_configured marks a catalog refresh without Config.Refresh). Stream job progress via the Config callbacks or poll GET .../jobs.

Runtime catalog refresh

The catalog is data on its own cadence. With Config.Refresh set, the engine fetches the published catalog on the configured interval and on demand via RefreshCatalog or the httpapi route. There is deliberately no fetch at construction: call RefreshCatalog once your boot work is enqueued. Each fetch is verified (a structural entry floor plus your Require list, the same offline checks as toolcatalog verify), re-overlaid with any Config.CatalogOverlays display patches, persisted raw under ConfigDir (tool-catalog.cached.json; at the next boot the newer of cache and baked wins), and swapped in atomically. The last good catalog stands on any failure: a bad fetch degrades to yesterday's knowledge, never to a broken engine. CatalogInfo() reports what is loaded and where it came from (baked, cached, remote, or none), the registry refs, the generation timestamp, and the last refresh error.

Consumer defaults ship with the library. DefaultCatalogURL is the published catalog's latest-download URL. ParseCatalogRefresh(raw, envName) turns a refresh env value into the Interval under the canonical policy: default 24h, clamped to 1h–30d, and off/disabled/0 disables the schedule while keeping on-demand refresh available. ParseRequireList(raw) parses a one-name-per-line requirements list (# comments and blank lines ignored) for Require.

The catalog compiler

cmd/toolcatalog compiles the catalog and verifies it against a required tool set. The compiler versions with the engine in one module, so a catalog is always compiled under the schema and verification semantics of the engine release that consumes it. tool-catalog runs it on every registry bump and publishes the artifact consumers fetch; images run verify against their own required list at build. Importing the library never pulls the compiler's TOML/YAML registry parsers into your build or binary (Go's module graph pruning); they cost consumers a few go.sum metadata lines only:

go run github.com/cplieger/toolbelt/v2/cmd/toolcatalog@latest \
    -mise mise-checkout/registry -aqua aqua-registry-checkout/pkgs \
    -overlay overlays.json -refs mise=<ref>,aqua=<ref> -out tool-catalog.json

go run github.com/cplieger/toolbelt/v2/cmd/toolcatalog@latest \
    verify -catalog tool-catalog.json -require required-tools.txt

verify fails the build when a required name is missing from the catalog or its definition is unusable (no source, unparseable templates, no linux amd64/arm64 support), so registry drift surfaces at publish or image build instead of in a boot job. The command ships a base overlay set covering runtimes, forge CLIs, and the officially supported language servers (gopls, typescript-language-server, pyright, rust-analyzer); it stamps a generated timestamp and embeds both registries' MIT license texts into the artifact, so the notice travels with every copy. Overlay merge semantics are the root module's ApplyOverlay, shared with the runtime refresh.

API

Engine
  • New(cfg *Config) (*Engine, error): construct and start (seed the manifest when absent, launch the job worker; a manifest of any other schema version is an error). Close() stops the worker.
  • Inventory() (*Inventory, error): the full read-side snapshot (every manifest entry joined with install state, the system group, the active job).
  • Search(q string) []CatalogEntry: catalog lookup (empty query = the featured set), hiding names already in the manifest.
  • Add(ctx, *AddRequest) (*Job, error): record a new tool and enqueue its install; present-and-enabled is the default intent. Disabled: true adds a template instead (no job, returns nil).
  • Patch(name, PatchRequest) (*Job, error): merge fields. Disabled is the enable/disable toggle (false→true uninstalls and keeps the template, true→false installs), a version change enqueues a reinstall, and Force permits disabling a tool enabled entries require.
  • Install(name) (*Job, error): retry an existing, enabled entry. Refuses templates with ErrDisabled (install is policy-neutral).
  • Update(names ...string) (*Job, error): refresh unpinned entries (or the named set).
  • Remove(name string, force bool) (*Job, []string, error): uninstall + delete the entry; refuses with the dependents named unless forced (force cascades).
  • Reconcile(mode) (*Job, error): converge disk to intent. ReconcileMissing installs missing enabled entries and uninstalls disabled-but-owned ones (zero network when converged); ReconcileFull also enqueues an update pass. Returns (nil, nil) on an empty manifest.
  • Wait(ctx, jobID) (*Job, error): block until a job settles (boot gates, synchronous flows).
  • EnsureInstalled(ctx, name) error: synchronous "a product action needs this binary now" path (creates from the catalog, enables a disabled template, installs, waits).
  • Jobs() (active *Job, recent []*Job) / CancelJob(id) bool: queue introspection and cancellation.
  • RefreshCatalog() (*Job, error): enqueue an on-demand catalog refresh (ErrRefreshNotConfigured without Config.Refresh).
  • CatalogInfo() CatalogInfo: the live catalog's provenance (refs, generation timestamp, entry count, source, last refresh outcome, schedule state).
Types and errors

Tool (manifest entry), Manifest, ToolStatus/State (machine state), CatalogEntry/Catalog (+ VerifyCatalog), Inventory/ToolInfo/SystemTool/Job (result shapes; also the httpapi wire shapes), DefaultSeed(). Sentinels: ErrNotFound, ErrDisabled, ErrHasDependents (match with errors.Is; *DependentsError carries the blocking names for errors.As).

httpapi routes
Route Engine call Notes
GET {prefix} Inventory
GET {prefix}/search?q= Search results omit embedded install definitions
POST {prefix} Add 202 {job} (null for template adds)
PATCH {prefix}/{name} Patch the toggle verb; 409 has_dependents + names
POST {prefix}/{name}/install Install 409 disabled on templates
POST {prefix}/update Update optional {"names": [...]} body
DELETE {prefix}/{name}?force=1 Remove 202 {job, dependents}; 409 without force
GET {prefix}/jobs Jobs active job carries the output tail
POST {prefix}/jobs/{id}/cancel CancelJob
GET {prefix}/catalog CatalogInfo provenance + freshness of the live catalog
POST {prefix}/catalog/refresh RefreshCatalog 202 {job}; 409 not_configured without Config.Refresh
toolcatalog (catalog compiler command)

compile (default): -mise <dir> -aqua <dir> [-overlay file]... [-no-base-overlays] -refs k=v,... -out tool-catalog.json. verify: -catalog <file> -require <names-file>; exits non-zero when a required name doesn't resolve to usable linux amd64+arm64 install knowledge. Versioned with the module (go run github.com/cplieger/toolbelt/v2/cmd/toolcatalog@vX.Y.Z); the historical cmd/toolcatalog/vX.Y.Z lane tags (≤ v2.2.0) remain resolvable for old builds.

Configuration reference

Config field Purpose
ConfigDir directory holding tools.json + tools-state.json (required)
ToolsDir install tree root: bin/ (the single PATH dir), opt/<name>/<version>/, npm/, python/ (required)
CatalogPath baked catalog path (the first-boot/offline fallback with Refresh set); missing degrades to manual/ecosystem sources with named errors for catalog-dependent entries
Refresh runtime catalog refresh: the published-catalog URL, the schedule interval (0 = on-demand only), and the required names verified before every swap; nil keeps the baked catalog static
CatalogOverlays consumer overlay files re-applied to every loaded catalog (display patches survive refreshes); entries they add must embed any aqua definition inline
Seed manifest written when none exists (fresh volume); nil seeds empty
System image-baked binaries reported read-only in Inventory
OnJobChanged / OnJobOutput job lifecycle + coalesced output callbacks (must not block); nil is silent
Logger slog logger; nil uses the default

Manifest entry fields: source, version, pin (freeze version), disabled (template), requires, and install/uninstall/probe for manual sources. Every field except the name is optional; the catalog completes the rest. A _comment array survives engine rewrites. The engine accepts only manifest schema version 2; any other version fails New (it never rewrites or backs up an unrecognized manifest).

Security model

  • Aqua-sourced artifacts verify against the checksum source their registry definition declares (upstream checksums.txt and equivalents); an install without one is logged as unverified.
  • Fetches ride an SSRF-guarded client (public-IP enforcement at the dial boundary, redirect policy, port allowlist) with transient-failure retry and rate-limit handling via cplieger/httpx.
  • Downloads are size-capped (1 GiB); archive extraction rejects symlink members that escape the install tree; installs land in versioned directories swapped atomically.
  • The manual source runs arbitrary bash by design: it is an operator escape hatch for single-principal volumes, equivalent in trust to editing the manifest itself.

Scope notes

  • Linux only (amd64, arm64). The aqua evaluator resolves definitions for linux and ignores other platforms.
  • No apt/system-package backend: OS packages belong to the image or the consumer's entrypoint, not volume intent.
  • The manifest store's single-writer guarantee is in-process. Run one engine per data directory; other processes go through the consumer's server.

Contributing

Issues and PRs are welcome. See CONTRIBUTING.md for the conventions and how to run the checks locally.

Disclaimer

This project is built with care and follows security best practices, but it is intended for personal / self-hosted use. No guarantees of fitness for production environments. Use at your own risk.

This project was built with AI-assisted tooling using Claude, GPT, and Kiro. The human maintainer defines architecture, supervises implementation, and makes all final decisions.

License

GPL-3.0-or-later. See LICENSE.

Documentation

Overview

Package toolbelt provisions developer tools onto a persistent volume, declaratively. A manifest (tools.json) records intent: which tools, at which versions, enabled or disabled. A compiled catalog carries install knowledge (sources, artifact templates, checksum locations, dependencies). The Engine reconciles installed state against intent through a single-flight job queue: enabled-but-missing tools are installed, disabled-but-installed tools are uninstalled (their template kept), unmanaged files are never touched.

Data files (all under the consumer's persistent config volume):

<ConfigDir>/tools.json       — the manifest: user intent. Toolbelt is
                               the only writer, but out-of-band hand
                               edits are supported by design (the file
                               is re-read on every operation).
<ConfigDir>/tools-state.json — engine-owned machine state (installed
                               version, owned bin names, last error).
<ToolsDir>/opt/<name>/<ver>/ — versioned install trees.
<ToolsDir>/bin               — the single PATH dir (symlinks).

The catalog (CatalogPath, compiled at image build by cmd/toolcatalog) is read-only environment data; a missing catalog degrades to manual and ecosystem sources only, and entries that need catalog knowledge fail their jobs with a named error.

Install sources: aqua-registry artifact definitions (with upstream checksum verification when the definition declares a source), npm, pip (via uv), cargo, go install, and a manual bash escape hatch. No external package-manager binary ships with the library; ecosystem backends are themselves installable tools (npm needs node, pip needs uv, ...), and the engine adopts those backend dependencies automatically.

Index

Constants

View Source
const (
	DefaultCatalogRefresh = 24 * time.Hour
	MinCatalogRefresh     = time.Hour
	MaxCatalogRefresh     = 30 * 24 * time.Hour
)

Canonical catalog refresh cadence policy, shared by every consumer. The published artifact moves at most a few times per day, so the default matches that cadence and the floor keeps an interval typo from hammering the publisher; the ceiling keeps a fat-fingered unit from silently freezing refreshes for months.

View Source
const (
	CatalogSourceNone   = "none"   // no catalog available (degraded)
	CatalogSourceBaked  = "baked"  // Config.CatalogPath (image-baked)
	CatalogSourceCached = "cached" // last fetched copy, reloaded from ConfigDir
	CatalogSourceRemote = "remote" // fetched this process lifetime
)

Catalog sources reported by CatalogInfo.

View Source
const (
	SourceAqua   = "aqua"   // aqua:owner/repo — evaluated aqua-registry definition
	SourceNpm    = "npm"    // npm:package
	SourcePip    = "pip"    // pip:package (installed via uv)
	SourceCargo  = "cargo"  // cargo:crate
	SourceGo     = "go"     // go:module/path
	SourceManual = "manual" // user-provided install command
)

Source prefixes for Tool.Source. A source is "<kind>:<ref>" except SourceManual which stands alone.

View Source
const (
	JobQueued    = "queued"
	JobRunning   = "running"
	JobDone      = "done"
	JobFailed    = "failed"
	JobCancelled = "cancelled"
)

Job states.

View Source
const (
	JobKindInstall        = "install"
	JobKindUninstall      = "uninstall" // Remove: footprint removed, entry deleted
	JobKindDisable        = "disable"   // Patch disabled:true: footprint removed, entry kept
	JobKindUpdate         = "update"
	JobKindReconcile      = "reconcile"       // converge disk to intent (install missing + disable extras)
	JobKindCatalogRefresh = "catalog-refresh" // fetch + verify + swap the published catalog
)

Job kinds.

View Source
const DefaultCatalogURL = "https://github.com/cplieger/tool-catalog/releases/latest/download/tool-catalog.json"

DefaultCatalogURL is the published compiled catalog's stable latest-download URL (the cplieger/tool-catalog release artifact). The consumer contract lives here so every app defaults to the same publisher and a publisher move is a one-line change; apps expose an env override for forks and mirrors.

View Source
const ManifestVersion = 2

ManifestVersion is the manifest schema version this engine reads and writes — the only version it accepts. A manifest with any other version (or one that fails to parse) is an error at engine start.

Variables

View Source
var (
	// ErrNotFound marks an operation on a tool the manifest doesn't have.
	ErrNotFound = errors.New("tool not found")
	// ErrHasDependents marks a refused remove/disable: enabled entries
	// still require the tool (directly or as an implied backend).
	ErrHasDependents = errors.New("tool has dependents")
	// ErrDisabled marks an install attempt on a disabled template.
	// Enabling is an explicit state change (Patch Disabled=false), never
	// a side effect of a retry.
	ErrDisabled = errors.New("tool is disabled")
	// ErrUnknownJob marks a Wait on a job id the queue no longer knows:
	// never enqueued, or its terminal view was evicted by the history
	// cap. Without it a Wait on an evicted id would poll to ctx
	// deadline.
	ErrUnknownJob = errors.New("unknown job")
)

Sentinel errors. Compare with errors.Is; *DependentsError additionally carries the dependent names (errors.As).

View Source
var ErrRefreshNotConfigured = errors.New("catalog refresh not configured")

ErrRefreshNotConfigured marks RefreshCatalog on an engine constructed without Config.Refresh.

Functions

func ApplyOverlay added in v2.1.0

func ApplyOverlay(c *Catalog, data []byte, resolveAqua func(ref string) (*AquaPackage, error)) error

ApplyOverlay merges one overlay JSON document into the catalog. Shared by cmd/toolcatalog (compile time: base + app overlays) and the engine's catalog load/refresh pipeline (runtime: consumer overlay files re-applied over every fetched catalog, so display patches survive refreshes).

resolveAqua, when non-nil, loads an aqua install definition for a source-bearing `aqua:` entry that does not embed one — the compile case, where cmd/toolcatalog passes a registry-checkout loader. At runtime there is no registry checkout: pass nil, and source-bearing aqua entries must carry their definition inline (display-field patches, the common runtime overlay, need no definition at all).

func ParseCatalogRefresh added in v2.2.0

func ParseCatalogRefresh(raw, name string) time.Duration

ParseCatalogRefresh interprets a consumer's refresh env value into the CatalogRefresh.Interval duration under the canonical policy: empty or unparseable falls back to DefaultCatalogRefresh, positive durations clamp to [MinCatalogRefresh, MaxCatalogRefresh], and "off"/"disabled"/"0" return zero (schedule disabled; on-demand refresh stays available). name is the env variable named in fallback and clamp warnings.

func ParseRequireList added in v2.1.0

func ParseRequireList(raw string) []string

ParseRequireList extracts tool names from a requirements list: one name per line, # comments and blank lines ignored — the shape required-tools.txt files use across the fleet (cmd/toolcatalog verify reads the same format). Shared here so consumers embedding their list for CatalogRefresh.Require parse it identically to the build gate.

func VerifyCatalog

func VerifyCatalog(c *Catalog, require []string) []error

VerifyCatalog checks that every required name resolves in the catalog to install knowledge the engine can act on, offline: a non-empty source; for aqua sources an embedded definition that parses its templates and claims linux support on both amd64 and arm64; for manual sources an install command. One error per failing name; nil means the catalog satisfies the requirement set. Consumers run this at image build (cmd/toolcatalog verify) over their required names so a registry drift fails the build instead of a boot job.

Types

type AddRequest

type AddRequest struct {
	Name        string   `json:"name"`
	Source      string   `json:"source,omitempty"`  // optional when the catalog knows the name
	Version     string   `json:"version,omitempty"` // optional: resolve latest
	Description string   `json:"description,omitempty"`
	Origin      string   `json:"origin,omitempty"`
	Install     string   `json:"install,omitempty"`
	Uninstall   string   `json:"uninstall,omitempty"`
	Probe       string   `json:"probe,omitempty"`
	Requires    []string `json:"requires,omitempty"`
	Pin         bool     `json:"pin,omitempty"`
	// Disabled adds the entry as a template: recorded, not installed,
	// no job enqueued (Add then returns a nil Job).
	Disabled bool `json:"disabled,omitempty"`
}

AddRequest is the Add call's body: intent for a new tool. Every field except Name is optional when the catalog knows the name.

type AquaChecksum

type AquaChecksum struct {
	Enabled   *bool  `json:"enabled,omitempty" yaml:"enabled"`
	Type      string `json:"type,omitempty" yaml:"type"` // github_release | http
	Asset     string `json:"asset,omitempty" yaml:"asset"`
	URL       string `json:"url,omitempty" yaml:"url"`
	Algorithm string `json:"algorithm,omitempty" yaml:"algorithm"` // sha256 | sha512 | ...
}

AquaChecksum describes where the artifact's checksum lives.

type AquaFile

type AquaFile struct {
	Name string `json:"name" yaml:"name"`
	Src  string `json:"src,omitempty" yaml:"src"` // template; default = Name
}

AquaFile names one binary inside the extracted artifact.

type AquaOverride

type AquaOverride struct {
	Replacements map[string]string `json:"replacements,omitempty" yaml:"replacements"`
	Checksum     *AquaChecksum     `json:"checksum,omitempty" yaml:"checksum"`
	GOOS         string            `json:"goos,omitempty" yaml:"goos"`
	GOArch       string            `json:"goarch,omitempty" yaml:"goarch"`
	Asset        string            `json:"asset,omitempty" yaml:"asset"`
	URL          string            `json:"url,omitempty" yaml:"url"`
	Format       string            `json:"format,omitempty" yaml:"format"`
	Files        []AquaFile        `json:"files,omitempty" yaml:"files"`
}

AquaOverride adjusts fields for a specific GOOS/GOARCH.

type AquaPackage

type AquaPackage struct {
	Replacements     map[string]string     `json:"replacements,omitempty" yaml:"replacements"`
	Checksum         *AquaChecksum         `json:"checksum,omitempty" yaml:"checksum"`
	VersionConstr    string                `json:"version_constraint,omitempty" yaml:"version_constraint"`
	VersionSource    string                `json:"version_source,omitempty" yaml:"version_source"`
	Asset            string                `json:"asset,omitempty" yaml:"asset"`
	URL              string                `json:"url,omitempty" yaml:"url"`
	Path             string                `json:"path,omitempty" yaml:"path"`
	Format           string                `json:"format,omitempty" yaml:"format"`
	RepoOwner        string                `json:"repo_owner,omitempty" yaml:"repo_owner"`
	RepoName         string                `json:"repo_name,omitempty" yaml:"repo_name"`
	VersionPrefix    string                `json:"version_prefix,omitempty" yaml:"version_prefix"`
	Description      string                `json:"description,omitempty" yaml:"description"`
	Type             string                `json:"type" yaml:"type"`
	VersionFilter    string                `json:"version_filter,omitempty" yaml:"version_filter"`
	VersionOverrides []AquaVersionOverride `json:"version_overrides,omitempty" yaml:"version_overrides"`
	SupportedEnvs    []string              `json:"supported_envs,omitempty" yaml:"supported_envs"`
	Overrides        []AquaOverride        `json:"overrides,omitempty" yaml:"overrides"`
	Files            []AquaFile            `json:"files,omitempty" yaml:"files"`
	NoAsset          bool                  `json:"no_asset,omitempty" yaml:"no_asset"`
}

AquaPackage is the subset of an aqua-registry package definition the engine evaluates. Field names mirror the upstream YAML/JSON schema (https://aquaproj.github.io/docs/reference/registry-config), which the catalog compiler (cmd/toolcatalog) relies on to unmarshal registry files directly. Windows- and darwin-only concerns (rosetta2, windows_arm_emulation, complete_windows_ext) are intentionally absent: the engine resolves for linux only, and the evaluator resolves everything for linux/GOARCH.

func (*AquaPackage) CheckTemplates

func (p *AquaPackage) CheckTemplates() error

CheckTemplates parses (without executing) every Go template the definition carries — asset, url, path, checksum asset/url, file srcs, across version overrides — so catalog verification can catch template syntax drift offline. Returns the first parse error.

func (*AquaPackage) ResolveSpec

func (p *AquaPackage) ResolveSpec(version string) (*InstallSpec, error)

ResolveSpec evaluates the package definition for the given version on linux/GOARCH and returns the download plan. version is the upstream tag (with any version_prefix still attached).

func (*AquaPackage) SupportsLinux

func (p *AquaPackage) SupportsLinux(goarch string) bool

SupportsLinux reports whether the definition claims support for linux/goarch at the root level (supported_envs; empty = everywhere). A static, offline check — no version rendering. Used by catalog verification.

type AquaVersionOverride

type AquaVersionOverride struct {
	Asset         *string            `json:"asset,omitempty" yaml:"asset"`
	URL           *string            `json:"url,omitempty" yaml:"url"`
	Format        *string            `json:"format,omitempty" yaml:"format"`
	Replacements  *map[string]string `json:"replacements,omitempty" yaml:"replacements"`
	Checksum      *AquaChecksum      `json:"checksum,omitempty" yaml:"checksum"`
	NoAsset       *bool              `json:"no_asset,omitempty" yaml:"no_asset"`
	VersionPrefix *string            `json:"version_prefix,omitempty" yaml:"version_prefix"`
	VersionConstr string             `json:"version_constraint,omitempty" yaml:"version_constraint"`
	Files         []AquaFile         `json:"files,omitempty" yaml:"files"`
	Overrides     []AquaOverride     `json:"overrides,omitempty" yaml:"overrides"`
	SupportedEnvs []string           `json:"supported_envs,omitempty" yaml:"supported_envs"`
}

AquaVersionOverride switches definition fields per version range. A nil pointer field means "inherit from the package root".

type Catalog

type Catalog struct {
	// Refs records the upstream registry refs this catalog was
	// compiled from (informational).
	Refs map[string]string `json:"refs,omitempty"`
	// Licenses carries the upstream registries' license texts, keyed by
	// registry name (mise, aqua-registry). The compiled catalog embeds
	// data derived from both (MIT), and MIT requires the copyright +
	// permission notice to travel with copies — embedding the texts
	// makes every copy (baked, cached, fetched) self-contained.
	Licenses map[string]string       `json:"licenses,omitempty"`
	Entries  map[string]CatalogEntry `json:"entries"`

	// Generated is the compile timestamp (RFC 3339 UTC), stamped by
	// cmd/toolcatalog (informational).
	Generated string `json:"generated,omitempty"`
	// contains filtered or unexported fields
}

Catalog is the compiled tool-catalog.json document.

func (*Catalog) Featured

func (c *Catalog) Featured() []CatalogEntry

Featured returns the curated starter set (empty-state content), sorted by name.

func (*Catalog) Lookup

func (c *Catalog) Lookup(name string) (CatalogEntry, bool)

Lookup finds a catalog entry by name or alias.

func (*Catalog) Search

func (c *Catalog) Search(query string) []CatalogEntry

Search ranks catalog entries against a query: exact name, name prefix, alias, name substring, then description substring. Empty query returns the featured set.

type CatalogEntry

type CatalogEntry struct {
	Aqua        *AquaPackage `json:"aqua,omitempty"`
	Name        string       `json:"name"`
	Description string       `json:"description,omitempty"`
	Source      string       `json:"source"`
	// Version is the default pinned version for entries without an
	// upstream version source (manual installs).
	Version   string   `json:"version,omitempty"`
	Install   string   `json:"install,omitempty"`   // manual-source entries
	Uninstall string   `json:"uninstall,omitempty"` // manual-source entries
	Probe     string   `json:"probe,omitempty"`     // manual-source entries
	Aliases   []string `json:"aliases,omitempty"`
	Requires  []string `json:"requires,omitempty"`
	Featured  bool     `json:"featured,omitempty"`
	// Lsp marks language-server entries; drives the consumers'
	// no-LSP-enabled warning and UI badges.
	Lsp bool `json:"lsp,omitempty"`
}

CatalogEntry is one tool in the compiled catalog: the mise-registry name/description joined with the preferred install source and, for aqua sources, the embedded aqua package definition. Overlay entries (curated) may add requires/manual install commands and the lsp marker.

type CatalogInfo added in v2.1.0

type CatalogInfo struct {
	// Refs are the upstream registry refs the catalog was compiled
	// from; Generated its compile timestamp (RFC 3339 UTC). Both are
	// informational pass-throughs of the catalog document.
	Refs      map[string]string `json:"refs,omitempty"`
	Generated string            `json:"generated,omitempty"`
	// Source is where the live catalog came from: baked (the image
	// file), cached (the refresh cache, reloaded at boot), remote
	// (fetched this process lifetime), or none (degraded, no catalog).
	Source string `json:"source"`
	// URL is the configured refresh source (empty when refresh is not
	// configured).
	URL string `json:"url,omitempty"`
	// LastError is the most recent refresh failure ("" after a
	// successful refresh).
	LastError string `json:"last_error,omitempty"`
	Entries   int    `json:"entries"`
	// FetchedAt is the last successful refresh (Unix milliseconds; 0
	// before the first).
	FetchedAt int64 `json:"fetched_at,omitempty"`
	// Scheduled reports whether the engine-owned background refresh
	// loop is running.
	Scheduled bool `json:"scheduled"`
}

CatalogInfo reports the live catalog's provenance and freshness (the Engine.CatalogInfo return and the httpapi GET catalog body).

type CatalogRefresh added in v2.1.0

type CatalogRefresh struct {
	// URL of the published compiled catalog (e.g. a GitHub release's
	// latest-download URL). Fetched with the engine's hardened HTTP
	// client: SSRF-validated on every hop, HTTPS on 443 only.
	URL string
	// Require lists tool names a fetched catalog must resolve (the
	// same offline checks as cmd/toolcatalog verify) before it
	// replaces the current one. The consumer's required set; empty
	// skips the name check (the structural floor still applies).
	Require []string
	// Interval between background refreshes. Positive starts the
	// engine-owned schedule (one refresh per interval, with jitter;
	// deliberately no fire-on-start — consumers trigger the boot
	// fetch via RefreshCatalog once their boot work is enqueued);
	// zero disables the schedule while keeping on-demand refresh
	// (RefreshCatalog / the httpapi route) available.
	Interval time.Duration
}

CatalogRefresh configures the runtime catalog refresh: the engine periodically fetches a published compiled catalog, verifies it, and swaps it in atomically. The catalog is DATA on a daily upstream cadence; refreshing it at runtime decouples catalog freshness from consumer image rebuilds. The last good catalog always stands: a failed fetch, parse, overlay, or verify fails the refresh job and changes nothing.

type Config

type Config struct {
	// OnJobChanged, when non-nil, receives every job state transition
	// (queued, running, done, failed, cancelled). Views carry no output
	// tail; stream output via OnJobOutput or poll Jobs().
	OnJobChanged func(*Job)
	// OnJobOutput, when non-nil, receives coalesced batches of a running
	// job's output lines (~150 ms cadence).
	OnJobOutput func(jobID string, lines []string)
	// Logger receives engine-level log lines. Nil means slog.Default().
	Logger *slog.Logger
	// Seed is the manifest written when none exists (fresh volume).
	// Nil seeds an empty manifest. See DefaultSeed.
	Seed *Manifest
	// ConfigDir holds tools.json + tools-state.json (the persistent
	// config volume root).
	ConfigDir string
	// ToolsDir is the install tree root (bin/, opt/, npm/, python/).
	ToolsDir string
	// CatalogPath is the compiled catalog baked into the consumer's
	// image (optional; missing = degraded search + named install errors
	// for catalog-dependent entries). With Refresh configured this is
	// the first-boot/offline fallback: a valid refresh cache under
	// ConfigDir takes precedence at construction.
	CatalogPath string
	// Refresh, when non-nil, enables runtime catalog refresh: the
	// engine fetches the published catalog at Refresh.URL (on the
	// engine-owned schedule when Interval is positive, and on demand
	// via RefreshCatalog), verifies it against Refresh.Require, caches
	// it under ConfigDir, and swaps it in atomically. The last good
	// catalog stands on any failure. Nil keeps the baked catalog
	// static for the process lifetime (the pre-refresh behavior).
	Refresh *CatalogRefresh
	// CatalogOverlays are consumer overlay JSON files (see
	// ApplyOverlay) applied in order to every catalog the engine
	// loads: the baked file, the refresh cache, and each fetched
	// refresh. This keeps app-specific display patches independent of
	// the published artifact. Entries added by a runtime overlay must
	// embed any aqua definition inline (there is no registry checkout
	// to resolve from at runtime).
	CatalogOverlays []string
	// System names image-baked binaries surfaced read-only in
	// Inventory's System group (informational; not managed).
	System []string
}

Config wires an Engine. ConfigDir and ToolsDir are required.

type DependentsError

type DependentsError struct {
	Dependents []string
}

DependentsError is the ErrHasDependents shape that names the enabled entries blocking a remove/disable.

func (*DependentsError) Error

func (e *DependentsError) Error() string

func (*DependentsError) Is

func (e *DependentsError) Is(target error) bool

Is makes errors.Is(err, ErrHasDependents) match.

type Engine

type Engine struct {
	// contains filtered or unexported fields
}

Engine is the tools subsystem: the single owner of the manifest and install tree, the job queue, and the catalog. Construct with New.

The manifest store's single-writer guarantee is an in-process lock: every other process (a CLI, an agent) must go through the consumer's server rather than linking toolbelt against the same data dirs.

func New

func New(cfg *Config) (*Engine, error)

New constructs and starts an Engine: initializes the manifest files (seeding when absent; a manifest of any other schema version is an error) and launches the job worker.

func (*Engine) Add

func (e *Engine) Add(ctx context.Context, req *AddRequest) (*Job, error)

Add records a new tool in the manifest and, unless the request marks it disabled, enqueues its install. Present-and-enabled is the default intent: adding means "have this installed".

func (*Engine) CancelJob

func (e *Engine) CancelJob(id string) bool

CancelJob aborts a queued or running job.

func (*Engine) CatalogInfo added in v2.1.0

func (e *Engine) CatalogInfo() CatalogInfo

CatalogInfo reports the live catalog's provenance and freshness: what is loaded (refs, generation timestamp, entry count), where it came from (baked file, refresh cache, live fetch), the last refresh error, and whether the background schedule is running.

func (*Engine) Close

func (e *Engine) Close()

Close stops the catalog-refresh schedule, then the job worker (cancelling any running job).

func (*Engine) EnsureInstalled

func (e *Engine) EnsureInstalled(ctx context.Context, name string) error

EnsureInstalled synchronously guarantees a tool: present in the manifest (created from the catalog when missing), enabled, installed, and on PATH. This is the programmatic "a product action needs this binary now" path (a forge login installing gh, an MCP flow installing node); unlike Install it DOES enable a disabled template, because the user explicitly invoked the feature that needs the tool.

func (*Engine) Install

func (e *Engine) Install(name string) (*Job, error)

Install re-enqueues an install for an existing, enabled tool (retry / install-missing). Installing a disabled template is refused with ErrDisabled: install is policy-neutral, enabling rides Patch.

func (*Engine) Inventory

func (e *Engine) Inventory() (*Inventory, error)

Inventory assembles the full read-side snapshot: every manifest entry joined with install state, the system group, and the active job.

func (*Engine) Jobs

func (e *Engine) Jobs() (active *Job, recent []*Job)

Jobs returns the active job (with output tail) and recent history.

func (*Engine) Patch

func (e *Engine) Patch(name string, req PatchRequest) (*Job, error)

Patch merges fields into an existing tool and enqueues the follow-up job the transition needs: enable → install (when missing), disable → footprint uninstall (template kept), version change on an enabled tool → reinstall. Returns nil when no job is needed.

func (*Engine) Reconcile

func (e *Engine) Reconcile(mode ReconcileMode) (*Job, error)

Reconcile enqueues the convergence job: install missing enabled entries, uninstall the engine-owned footprint of disabled ones and of orphaned state rows (a manifest row deleted without its uninstall job completing). ReconcileFull additionally enqueues an update pass over unpinned entries. Returns (nil, nil) when the manifest is empty and no state row exists (nothing to converge). The returned job is the reconcile job; Wait on it to gate consumer readiness (session creation, UI states).

func (*Engine) RefreshCatalog added in v2.1.0

func (e *Engine) RefreshCatalog() (*Job, error)

RefreshCatalog enqueues a catalog-refresh job: fetch the published catalog, verify it, persist it to the refresh cache, and swap it in. Returns ErrRefreshNotConfigured when Config.Refresh is nil. The returned job is observable like any other (jobs API, callbacks); Wait on it to gate on completion.

func (*Engine) Remove

func (e *Engine) Remove(name string, force bool) (*Job, []string, error)

Remove uninstalls a tool and deletes its template. Without force, a tool that enabled entries require is refused and the dependents are returned. The removed manifest entries travel on the uninstall job so source-specific cleanup (npm/pip uninstalls, manual uninstall commands) still knows the sources after the manifest rows are gone.

func (*Engine) Search

func (e *Engine) Search(query string) []CatalogEntry

Search queries the catalog (empty query = featured set), hiding entries already in the manifest.

func (*Engine) Update

func (e *Engine) Update(names ...string) (*Job, error)

Update enqueues an update job over every unpinned, enabled tool (or the given names).

func (*Engine) Wait

func (e *Engine) Wait(ctx context.Context, jobID string) (*Job, error)

Wait blocks until the job reaches a terminal state and returns its final view.

type InstallSpec

type InstallSpec struct {
	URL         string
	Format      string // tar.gz | tar.xz | tar.zst | zip | gz | xz | raw
	ChecksumURL string // empty = no verification
	ChecksumAlg string
	Files       []AquaFile
}

InstallSpec is the fully resolved plan for downloading one tool version on this machine: a URL, an archive format, the binaries to link, and an optional checksum source.

type Inventory

type Inventory struct {
	Job    *Job         `json:"job,omitempty"`
	Tools  []ToolInfo   `json:"tools"`
	System []SystemTool `json:"system"`
}

Inventory is the full read-side snapshot: every manifest entry joined with state, the system group, and the active job.

type Job

type Job struct {
	ID    string   `json:"id"`
	Kind  string   `json:"kind"`
	State string   `json:"state"`
	Error string   `json:"error,omitempty"`
	Names []string `json:"names,omitempty"`
	// OutputTail carries the job's most recent output lines; populated
	// by Jobs() snapshots only (live output streams via the
	// Config.OnJobOutput callback).
	OutputTail []string `json:"output_tail,omitempty"`
	// Timestamps are Unix milliseconds.
	CreatedAt int64 `json:"created_at"`
	StartedAt int64 `json:"started_at,omitempty"`
	EndedAt   int64 `json:"ended_at,omitempty"`
}

Job is one queued/running/finished unit of engine work.

type Manifest

type Manifest struct {
	Tools map[string]Tool `json:"tools"`
	// Comment is a single reserved documentation key preserved across
	// engine rewrites (seed how-to text). Other unknown JSON keys are
	// NOT preserved; the manifest is not a general round-tripping
	// document.
	Comment []string `json:"_comment,omitempty"`
	Version int      `json:"version"`
}

Manifest is the tools.json document (schema ManifestVersion).

func DefaultSeed

func DefaultSeed() *Manifest

DefaultSeed returns the shared starter manifest: the officially supported language servers for Go, TypeScript, and Python plus the GitHub CLI, all disabled. Nothing downloads until an entry is enabled; install knowledge (source, dependencies, version) hydrates from the catalog at enable time. Backend runtimes (node, go) and required packages (typescript) are deliberately NOT seeded: the engine auto-adopts missing dependencies at install time, whereas a seeded-but-disabled dependency REFUSES dependent installs ("dependency X is disabled; enable it first" — a disabled entry is user policy). Returns a fresh copy on every call.

type PatchRequest

type PatchRequest struct {
	Version     *string   `json:"version,omitempty"`
	Pin         *bool     `json:"pin,omitempty"`
	Disabled    *bool     `json:"disabled,omitempty"`
	Description *string   `json:"description,omitempty"`
	Requires    *[]string `json:"requires,omitempty"`
	Install     *string   `json:"install,omitempty"`
	Uninstall   *string   `json:"uninstall,omitempty"`
	// Force permits disabling a tool that enabled entries require,
	// cascading the disable to those dependents (one level, mirroring
	// Remove's force cascade).
	Force bool `json:"force,omitempty"`
}

PatchRequest edits an existing tool. Pointer fields distinguish "absent" from zero values. Disabled is the enable/disable toggle: false→true uninstalls the engine-owned footprint and keeps the template; true→false installs.

type ReconcileMode

type ReconcileMode int

ReconcileMode selects how much a Reconcile job does.

const (
	// ReconcileMissing converges intent without network: installs
	// missing enabled entries, uninstalls the engine-owned footprint of
	// disabled ones. Zero fetches when already converged.
	ReconcileMissing ReconcileMode = iota
	// ReconcileFull is ReconcileMissing plus an update pass over
	// unpinned entries (enqueued as a separate update job).
	ReconcileFull
)

type State

type State struct {
	Tools map[string]ToolStatus `json:"tools"`
}

State is the tools-state.json document.

type SystemTool

type SystemTool struct {
	Name      string `json:"name"`
	Installed bool   `json:"installed"`
}

SystemTool is one image-baked binary surfaced read-only (Config.System).

type Tool

type Tool struct {
	// Source locates the install definition: "aqua:cli/cli",
	// "npm:pyright", "pip:x", "cargo:x", "go:golang.org/x/tools/gopls",
	// or "manual". Empty = hydrate from the catalog.
	Source string `json:"source,omitempty"`
	// Version is the concrete upstream version, exactly as upstream
	// tags it (may or may not carry a leading v). Never a range.
	// Empty = resolve latest when the tool is actively installed.
	Version string `json:"version,omitempty"`
	// Description is display text (catalog-provided or user-written).
	Description string `json:"description,omitempty"`
	// Origin records provenance for linked entries, e.g. "mcp:<name>"
	// for a tool created from an MCP add flow.
	Origin string `json:"origin,omitempty"`
	// Install is the shell command for Source == "manual". It runs via
	// bash with VERSION, BIN, TOOLS, OPT and ARCH_* in the environment.
	Install string `json:"install,omitempty"`
	// Uninstall optionally overrides cleanup for Source == "manual".
	Uninstall string `json:"uninstall,omitempty"`
	// Probe is the bin name whose presence marks the tool installed
	// (manual installs only; other sources derive it). Defaults to the
	// tool name.
	Probe string `json:"probe,omitempty"`
	// Requires lists other manifest/catalog tool names that must be
	// installed before (or alongside) this one, e.g. jdtls -> java.
	// Backend-level needs (npm->node, pip->uv, cargo->rust, go->go)
	// are implied and need not be listed.
	Requires []string `json:"requires,omitempty"`
	// Pin freezes the version: update runs skip this tool.
	Pin bool `json:"pin,omitempty"`
	// Disabled marks the entry a template: recorded intent whose
	// install is explicitly bypassed. The reconciler uninstalls a
	// disabled tool's engine-owned footprint and keeps the entry.
	// Absent (false) means enabled — presence in the manifest is
	// intent to have the tool installed.
	Disabled bool `json:"disabled,omitempty"`
}

Tool is one manifest entry: the user's intent for a single tool. Every field except the map key (the tool name) is optional; empty Source/Version/Requires hydrate from the catalog when the tool is installed or updated.

type ToolInfo

type ToolInfo struct {
	Name             string   `json:"name"`
	Source           string   `json:"source,omitempty"`
	Version          string   `json:"version,omitempty"`
	Description      string   `json:"description,omitempty"`
	Origin           string   `json:"origin,omitempty"`
	InstalledVersion string   `json:"installed_version,omitempty"`
	Latest           string   `json:"latest,omitempty"`
	LastError        string   `json:"last_error,omitempty"`
	Requires         []string `json:"requires,omitempty"`
	Pin              bool     `json:"pin,omitempty"`
	Disabled         bool     `json:"disabled,omitempty"`
	// Lsp marks a language-server entry (catalog knowledge); consumers
	// use it for the no-LSP-enabled warning and UI badges.
	Lsp        bool `json:"lsp,omitempty"`
	Installed  bool `json:"installed"`
	Installing bool `json:"installing"`
}

ToolInfo is one tool row in Inventory: the manifest entry joined with the engine's install state.

type ToolStatus

type ToolStatus struct {
	// UpdatedAt is when this status last changed.
	UpdatedAt time.Time `json:"updated_at"`
	// InstalledVersion is the version last installed successfully.
	InstalledVersion string `json:"installed_version,omitempty"`
	// LastError is the failure message of the most recent install
	// attempt; cleared on success.
	LastError string `json:"last_error,omitempty"`
	// Bins are the names this tool owns in the bin dir (symlinks),
	// removed on uninstall.
	Bins []string `json:"bins,omitempty"`
	// PMBins are package-manager bin names discovered by diffing the
	// pm's bin dir (npm/pip), symlinked into the bin dir.
	PMBins []string `json:"pm_bins,omitempty"`
}

ToolStatus is the engine-owned per-tool machine state.

Directories

Path Synopsis
cmd
toolcatalog command
Command toolcatalog compiles a toolbelt tool catalog from registry data, and verifies a compiled catalog against a consumer's required tool set.
Command toolcatalog compiles a toolbelt tool catalog from registry data, and verifies a compiled catalog against a consumer's required tool set.
Package httpapi is the HTTP projection of a toolbelt Engine: a REST surface over the Engine's Go API, one route per method, JSON in and out.
Package httpapi is the HTTP projection of a toolbelt Engine: a REST surface over the Engine's Go API, one route per method, JSON in and out.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL