modelsync

package
v1.0.0-rc.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 9, 2026 License: Apache-2.0 Imports: 23 Imported by: 0

Documentation

Overview

Package modelsync manages the model routing policy (one canonical document at <home>/.gridctl/models/policy.yaml) and projects it into LiteLLM proxy configuration and client provider config. It is the models-kind tenant of the pkg/project engine.

Three ownership mechanisms, deliberately different per target:

  • The rendered LiteLLM router fragment is a wholly gridctl-owned file (wholesale ownership, contexts-style installed/canonical hashes). The fragment carries only the auto-router model entry: backends stay in the user's own model_list, referenced by name, because LiteLLM's include directive extends model_list across files and a re-emitted backend would silently load-balance against the original.
  • The include: line referencing the fragment from the human-owned LiteLLM config is a single-line text edit (the contexts import- shim pattern adapted to YAML): the parent file is never parsed and re-marshalled, so its comments and formatting survive byte-for-byte outside the one managed line.
  • The provider subtree inside the human-owned opencode.json is a key-level ownership write (the wiring pattern: canonical value hashes with history, foreign and drifted refusals), applied as an RFC 6902 patch through hujson so every byte outside the owned subtree survives, comments included.

gridctl stays off the inference data path: every operation here is a pure file operation, no running gateway or LiteLLM is required, and if gridctl is down inference is unaffected. LiteLLM only reads its config at startup, so sync latches a restart-pending state (Entry.AckedHash) that only an explicit ack clears.

Index

Constants

View Source
const (
	StateInSync        = project.StateInSync
	StateStale         = project.StateStale
	StateDrifted       = project.StateDrifted
	StateTargetMissing = project.StateTargetMissing
	StateNeverSynced   = "never-synced"
)

Projection states: the engine vocabulary plus the context-kind extension for supported-but-never-synced targets.

View Source
const (
	ActionUpdated        = project.ActionUpdated
	ActionUnchanged      = project.ActionUnchanged
	ActionError          = project.ActionError
	ActionSkippedDrift   = project.ActionSkippedDrift
	ActionWouldUpdate    = project.ActionWouldUpdate
	ActionSkippedForeign = "skipped-foreign"
	ActionRemoved        = "removed"
	ActionWouldRemove    = "would-remove"
	ActionKeptDrift      = "kept-drift"
	ActionAlreadyGone    = "already-gone"
	ActionAdopted        = "adopted"
)

Actions: the engine vocabulary plus kind-specific verbs.

View Source
const (
	TierSimple    = "SIMPLE"
	TierMedium    = "MEDIUM"
	TierComplex   = "COMPLEX"
	TierReasoning = "REASONING"
)

Complexity tiers, in LiteLLM's fixed vocabulary and render order.

View Source
const (
	SchemaV1     = "v1"
	SchemaV2     = "v2"
	SchemaDetect = "detect"
)

OpenCode config generations. The v2 config renamed provider -> providers, npm -> package, options -> settings, and requires an env list; both shapes stay renderable because upstream config churn is a release-note event, not a runtime crash.

View Source
const (
	SeverityError   = "error"
	SeverityWarning = "warning"
)

Issue severities.

View Source
const DefaultTemplate = "hybrid"

DefaultTemplate is the scaffold used when init names no template.

View Source
const PolicyKind = "models"

PolicyKind is the required kind: field value in the policy document.

Variables

View Source
var (
	ErrNoPolicy     = errors.New("no models policy; run 'gridctl models init' first")
	ErrPolicyExists = errors.New("models policy already exists (use --force to overwrite)")
	ErrNotSynced    = errors.New("nothing synced for this target")
	// ErrNewerLockVersion is aliased from the engine so callers'
	// errors.Is checks work without importing pkg/project.
	ErrNewerLockVersion = project.ErrNewerLockVersion
)

Sentinel errors callers branch on.

Functions

func HasErrors

func HasErrors(issues []Issue) bool

HasErrors reports whether any issue is error-severity.

func HasFailures

func HasFailures(results []SyncResult) bool

HasFailures reports whether any sync result failed or was skipped.

func NeedsAttention

func NeedsAttention(statuses []Status) bool

NeedsAttention reports whether any status row needs a sync or a decision. Restart-pending alone is not attention.

func RenderLiteLLM

func RenderLiteLLM(p *Policy, policyHash string) ([]byte, error)

RenderLiteLLM renders the router-only LiteLLM fragment: exactly one model_list entry (the auto-router), with complexity_router_default_model as a sibling of complexity_router_config under litellm_params per LiteLLM's documented placement. The renderer is pure and deterministic: fixed key order, maps emitted in sorted order, no timestamps. It never emits backends (the parent owns model_list inventory), router_settings, or any secret material.

func ResolveOpenCodeSchema

func ResolveOpenCodeSchema(declared, configPath string) string

ResolveOpenCodeSchema turns the policy's schema declaration into a concrete generation, sniffing the target file in detect mode: an existing providers key means v2, an existing provider key means v1, and an empty or missing file defaults to v1.

func TemplateNames

func TemplateNames() []string

TemplateNames lists the available --template values, sorted.

Types

type AdoptResult

type AdoptResult struct {
	Target string `json:"target"`
	Client string `json:"client"`
	Path   string `json:"path"`
	Action string `json:"action"`
	Detail string `json:"detail,omitempty"`
}

AdoptResult is one target's adopt outcome.

type Clients

type Clients struct {
	OpenCode *OpenCodeClient `yaml:"opencode"`
}

Clients holds the client projection targets.

type Issue

type Issue struct {
	Severity string `json:"severity"`
	Field    string `json:"field"`
	Message  string `json:"message"`
}

Issue is one validation finding.

type LiteLLMScan

type LiteLLMScan struct {
	// ModelNames are the model_list names in declaration order,
	// deduplicated, excluding auto_router entries.
	ModelNames []string
	// AutoRouterNames are model_list entries whose underlying model is
	// an auto_router/ route (already-routed setups).
	AutoRouterNames []string
}

LiteLLMScan is what gridctl learns from reading a LiteLLM config: the declared model_name values (includes followed, relative to each file) and any auto_router entry already present.

func ParseLiteLLMConfig

func ParseLiteLLMConfig(path string) (*LiteLLMScan, error)

ParseLiteLLMConfig scans a LiteLLM config file, following its include entries (paths resolve relative to each file's directory, matching LiteLLM's own resolution).

type LiteLLMTarget

type LiteLLMTarget struct {
	ConfigPath   string `yaml:"config_path"`
	FragmentPath string `yaml:"fragment_path"`
}

LiteLLMTarget names the human-owned parent config and where the rendered fragment lives (default: next to the parent).

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager owns the models policy store and every projection decision. All paths resolve against home, so tests point it at a temp dir. Mutating operations serialize on mu in-process and on the engine's cross-process lock.

func NewManager

func NewManager() (*Manager, error)

NewManager builds a Manager rooted at the user's home directory. CLI call sites only; anything tests can reach uses NewManagerWithHome.

func NewManagerWithHome

func NewManagerWithHome(home string) *Manager

NewManagerWithHome builds a Manager rooted at an explicit home directory. Tests use this to stay isolated from $HOME.

func (*Manager) AckRestart

func (m *Manager) AckRestart(ctx context.Context) error

AckRestart marks the running LiteLLM as restarted since the last fragment write: the only way the restart-pending latch clears.

func (*Manager) Adopt

func (m *Manager) Adopt(ctx context.Context) ([]AdoptResult, error)

Adopt records the current on-disk state of every recorded target as gridctl-owned, clearing drift without touching any file.

func (*Manager) Dir

func (m *Manager) Dir() string

Dir returns the canonical models store directory.

func (*Manager) FragmentPath

func (m *Manager) FragmentPath(p *Policy) (string, error)

FragmentPath resolves where the rendered LiteLLM fragment lives: the policy's explicit fragment_path, else next to the parent config, else under the models store (render-only, nothing to include it from).

func (*Manager) HasPolicy

func (m *Manager) HasPolicy() bool

HasPolicy reports whether the canonical policy document exists.

func (*Manager) InitFromLiteLLM

func (m *Manager) InitFromLiteLLM(path string, force bool) error

InitFromLiteLLM scaffolds the policy from an existing LiteLLM config: its model_list names become backend references (never copied inventory) with a proposed tier mapping over the first backend, and the config becomes the sync target. The user edits tiers from there.

func (*Manager) InitFromTemplate

func (m *Manager) InitFromTemplate(name string, force bool) error

InitFromTemplate scaffolds the policy document from a named starter.

func (*Manager) LoadPolicy

func (m *Manager) LoadPolicy() (*Policy, error)

LoadPolicy reads and parses the canonical policy document.

func (*Manager) LockPath

func (m *Manager) LockPath() string

LockPath returns the unified projection lockfile path.

func (*Manager) PolicyPath

func (m *Manager) PolicyPath() string

PolicyPath returns the canonical policy document path.

func (*Manager) ResolveOpenCode

func (m *Manager) ResolveOpenCode(p *Policy) (configPath, schema string, err error)

ResolveOpenCode resolves the client config path and the concrete schema generation exactly the way sync does, so render and sync can never disagree about the emitted shape.

func (*Manager) SavePolicy

func (m *Manager) SavePolicy(data []byte) error

SavePolicy writes the canonical policy document with a backup of the previous revision.

func (*Manager) Statuses

func (m *Manager) Statuses(ctx context.Context) ([]Status, error)

Statuses reports every target's projection state. Read-only.

func (*Manager) Sync

func (m *Manager) Sync(ctx context.Context, opts SyncOptions) ([]SyncResult, error)

Sync projects the policy into every declared target. Per-target results never abort the pass; infrastructure errors (no policy, invalid policy, lockfile) do.

func (*Manager) Unsync

func (m *Manager) Unsync(ctx context.Context, opts UnsyncOptions) ([]UnsyncResult, error)

Unsync removes every projected target in reverse dependency order: the provider stanza, the include line, then the fragment file.

func (*Manager) Validate

func (m *Manager) Validate(p *Policy) []Issue

Validate checks the policy and returns findings, most severe first. It never touches the network; the only file it may read is the declared parent LiteLLM config, to warn about unknown backends.

type OpenCodeClient

type OpenCodeClient struct {
	ProviderID string `yaml:"provider_id"`
	BaseURL    string `yaml:"base_url"`
	// APIKeyEnv names the environment variable holding the LiteLLM key.
	// Only the env reference is ever rendered, never a literal.
	APIKeyEnv string `yaml:"api_key_env"`
	// Schema pins the OpenCode config generation: v1 (provider/npm/
	// options), v2 (providers/package/settings/env), or detect (default:
	// choose by which key the target file already has, else v1).
	Schema string `yaml:"schema"`
	// ConfigPath overrides the default opencode.json location.
	ConfigPath string `yaml:"config_path"`
}

OpenCodeClient wires the OpenCode provider stanza.

type OpenCodeRender

type OpenCodeRender struct {
	Schema    string
	Container string
	Value     map[string]any
}

OpenCodeRender is one rendered provider stanza: the container key it lives under and the subtree value gridctl owns. The top-level model key is deliberately not part of the render: users change it through the client's own picker, and owning it would turn every switch into drift.

func RenderOpenCode

func RenderOpenCode(p *Policy, schema string) (OpenCodeRender, error)

RenderOpenCode builds the provider stanza for the resolved schema generation. The API key is always an env reference, never a literal.

type Policy

type Policy struct {
	Name        string `yaml:"name"`
	Kind        string `yaml:"kind"`
	Description string `yaml:"description"`

	Router Router `yaml:"router"`
	// Backends are references to model_name values that already exist in
	// the user's own LiteLLM model_list. The fragment never re-emits
	// them: a duplicate model_name across included files is silently
	// load-balanced by LiteLLM.
	Backends []string           `yaml:"backends"`
	Tiers    Tiers              `yaml:"tiers"`
	Weights  map[string]float64 `yaml:"weights"`
	// Passthrough is opaque YAML merged last into
	// complexity_router_config, for auto-router keys gridctl does not
	// model. Typed keys win: it may not set tiers, and dimension_weights
	// in it is ignored when weights is set.
	Passthrough map[string]any `yaml:"passthrough"`

	Clients Clients `yaml:"clients"`
	Targets Targets `yaml:"targets"`

	// Extra holds unknown top-level keys, preserved for lint and
	// forward compatibility. Never rendered.
	Extra map[string]any `yaml:"-"`
	// contains filtered or unexported fields
}

Policy is the typed model routing policy. Unknown top-level keys land in Extra so a newer document survives an older binary's validation pass (and so dangerous keys can be linted by name).

func ParsePolicy

func ParsePolicy(data []byte) (*Policy, error)

ParsePolicy decodes a policy document. Unknown top-level keys are collected into Extra rather than rejected; validation decides which of them are dangerous.

func (*Policy) Hash

func (p *Policy) Hash() string

Hash fingerprints the policy document with the engine's scheme, CRLF-normalized so a line-ending flip (git autocrlf) never reads as a stale policy.

type Router

type Router struct {
	EntryModel  string `yaml:"entry_model"`
	DefaultTier string `yaml:"default_tier"`
}

Router names what clients call and where unclassified requests land.

type Status

type Status struct {
	Target string `json:"target"`
	Client string `json:"client"`
	State  string `json:"state"`
	// RestartPending annotates the LiteLLM fragment: the running proxy
	// has not been acknowledged as restarted since the last write. An
	// annotation, never a drift state: it does not affect exit codes.
	RestartPending bool       `json:"restart_pending,omitempty"`
	Path           string     `json:"path,omitempty"`
	Detail         string     `json:"detail,omitempty"`
	SyncedAt       *time.Time `json:"synced_at,omitempty"`
}

Status is one target's projection state.

type SyncOptions

type SyncOptions struct {
	// DryRun previews without writing files or lockfile entries.
	DryRun bool
	// Diff attaches unified diffs to would-update results.
	Diff bool
	// Force overwrites drifted and foreign targets. Recorded drift is
	// otherwise skipped with guidance; a foreign file at the fragment
	// path is otherwise never touched.
	Force bool
}

SyncOptions configure a sync pass.

type SyncResult

type SyncResult struct {
	Target     string `json:"target"`
	Client     string `json:"client"`
	Path       string `json:"path"`
	Action     string `json:"action"`
	Detail     string `json:"detail,omitempty"`
	BackupPath string `json:"backup_path,omitempty"`
	Diff       string `json:"diff,omitempty"`
	Error      string `json:"error,omitempty"`
}

SyncResult is one (target) outcome of a sync pass.

type Targets

type Targets struct {
	LiteLLM *LiteLLMTarget `yaml:"litellm"`
}

Targets holds the proxy projection targets.

type Tiers

type Tiers struct {
	Simple    string `yaml:"SIMPLE"`
	Medium    string `yaml:"MEDIUM"`
	Complex   string `yaml:"COMPLEX"`
	Reasoning string `yaml:"REASONING"`
}

Tiers maps each complexity tier to a backend model_name. Scalar values only in v1; pool lists belong in passthrough until typed.

type UnsyncOptions

type UnsyncOptions struct {
	// Force removes drifted targets too. Foreign content is never
	// removed, force or not.
	Force bool
}

UnsyncOptions configure removal.

type UnsyncResult

type UnsyncResult struct {
	Target     string `json:"target"`
	Client     string `json:"client"`
	Path       string `json:"path"`
	Action     string `json:"action"`
	Detail     string `json:"detail,omitempty"`
	BackupPath string `json:"backup_path,omitempty"`
	Error      string `json:"error,omitempty"`
}

UnsyncResult is one (target) removal outcome.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL