promote

package
v0.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 31, 2026 License: Apache-2.0 Imports: 30 Imported by: 0

Documentation

Overview

Package promote turns experiment evidence into a rollout decision.

Evaluate combines internal/stats with the experiment's stopping rules and returns promote | rollback | continue | expired. Acting on a verdict is deliberately asymmetric: rollback may be applied directly to the policy repo working tree (ApplyRollback), while promote only ever opens a PR (OpenPR); nothing in this package merges.

Index

Constants

View Source
const (
	ForgeAuto   = ""       // GitLab if the remote host names it, else the gh CLI
	ForgeGH     = "gh"     // git push + the gh CLI (the default; behaviour unchanged)
	ForgeGitHub = "github" // GitHub REST API (github.com or GitHub Enterprise Server)
	ForgeGitLab = "gitlab" // GitLab REST API v4 (gitlab.com or self-managed)
)

Forge kinds. ForgeAuto (the zero value) picks from the git remote host.

View Source
const (
	SourceGateway = telemetry.SourceGateway // authenticated halo-proxy receiver: may auto-kill
	SourceCLI     = telemetry.SourceCLI     // developer-reachable receiver: client-controlled
	// SourceEval: the authenticated eval receiver (`halo eval online` judge
	// scores). Trusted for verdicts and PRs; never trips the kill switch.
	SourceEval = telemetry.SourceEval
)

Evidence sources (Report.Source): which collector receiver the samples came through.

View Source
const ForgeKinds = "gh|github|gitlab (default: autodetect from the origin remote; gh unless the host names gitlab)"

ForgeKinds lists the values --forge accepts, for flag help.

Variables

View Source
var ErrToggleChange = errors.New("invalid toggle change")

ErrToggleChange marks a toggle change that cannot be applied or would make the policy invalid (as opposed to an infrastructure failure).

View Source
var StatusTransitions = map[string]struct {
	To   string
	From []string
}{
	"start":    {"running", []string{"", "draft", "paused"}},
	"pause":    {"paused", []string{"running"}},
	"conclude": {"concluded", []string{"running", "paused"}},
}

StatusTransitions are the experiment status verbs (halo exp start|pause|conclude and the MCP tools): the status each sets and the statuses it may leave.

Functions

func ApplyRollback

func ApplyRollback(repoDir string, ch *Change) error

ApplyRollback writes a rollback change straight into the working tree (atomically per file). Only rollbacks may be applied directly; promotions go through OpenPR.

func CommitOnBranch

func CommitOnBranch(ctx context.Context, run RunFunc, dir string, bc BranchCommit) (branch, commit string, err error)

CommitOnBranch commits bc.Edit's files onto a new local branch starting at HEAD and returns the branch (with its random suffix) and the commit. It works in a temporary `git worktree add -b` checkout that is removed afterwards, so the user's branch, index and working tree are never touched. It never pushes. run nil means Run.

func DetectForge added in v0.5.0

func DetectForge(hostname string) string

DetectForge maps a remote host to a kind. Only a host naming gitlab selects GitLab; everything else keeps the gh CLI, so GitHub users see no change.

func OpenPR

func OpenPR(ctx context.Context, o PROpener, t Target, ch *Change, exp *policy.Experiment, rep Report, base, roi string) (string, error)

OpenPR builds the PR for a promotion verdict and hands it to the opener. It is the only path by which a promotion reaches the policy repo; a human merges. roi, when non-empty, is a delivery-ROI section (internal/roi, ADR-0027) added to the body for the reviewer: it is informational and plays no part in the verdict.

func ParseForgeKind added in v0.5.0

func ParseForgeKind(s string) (string, error)

ParseForgeKind validates a --forge value.

func PushBranch

func PushBranch(ctx context.Context, run RunFunc, dir string, bc BranchCommit) (string, error)

PushBranch runs CommitOnBranch, pushes the branch to origin and then deletes the local branch whether or not the push succeeded, so a long-lived clone (halo-server's policy writer) never accumulates branches. run nil means Run.

func ResetClone

func ResetClone(ctx context.Context, run RunFunc, dir, base string) error

ResetClone force-checks-out origin/<base> in dir and cleans untracked files (no fetch, no work-tree-root guard; call SyncClone first). The checkout keeps the committed bytes (no core.autocrlf=true CRLF, the Git for Windows default), so artifact digests in the clone match their pins.

func Run

func Run(ctx context.Context, dir, name string, args ...string) ([]byte, error)

Run runs name in dir and returns stdout; errors carry the trimmed stderr. It is the one command runner for git/gh in promote and its callers.

func SyncClone

func SyncClone(ctx context.Context, run RunFunc, dir, base string) error

SyncClone hard-resets dir to a pristine origin/<base>: it refuses unless dir is the root of its own git work tree (the reset is destructive and must never hit a parent repo), then fetches base and force-checks it out, removing untracked files. run is promote.Run or a stand-in.

func TransitionStatus

func TransitionStatus(d *yamledit.Doc, verb string) (from, to string, data []byte, err error)

TransitionStatus checks verb against the experiment document's current status and returns the edited file content (d is not written).

func UnifiedDiff

func UnifiedDiff(path, old, nw string) string

UnifiedDiff renders a single-hunk unified diff (common prefix/suffix trimmed, 3 lines of context). Empty when nothing changed.

func UpsertVerdict

func UpsertVerdict(path, experiment string, rep Report, at time.Time) error

UpsertVerdict replaces (or appends) the experiment's entry in the JSON array at path, atomically. Other entries are kept verbatim. ponytail: read-modify-write without a lock; run one writer (controller or `halo exp analyze`) per file.

func VerdictRow added in v0.3.0

func VerdictRow(experiment string, rep Report, at time.Time) ([]byte, error)

VerdictRow encodes one experiment's entry exactly as UpsertVerdict writes it, for stores that keep one document per experiment (HA backend).

Types

type BranchCommit

type BranchCommit struct {
	Branch  string // prefix; a random suffix is always appended
	Subject string
	Body    string // optional second message paragraph
	// Edit returns the new content (dir-relative, slash-separated path ->
	// data) of every file to change. It is given the committed version of any
	// file via read, so uncommitted edits in the user's checkout never land in
	// the commit. A nil value deletes the file (an empty, non-nil one empties it).
	Edit func(read ReadFunc) (map[string][]byte, error)
}

BranchCommit describes one commit onto a new branch.

type Change

type Change struct {
	Files map[string][]byte // repo-relative path -> new content
	Patch string
	// Edit re-applies just this change to whatever read returns (see
	// BranchCommit.Edit), e.g. to the committed version of each file.
	Edit func(read ReadFunc) (map[string][]byte, error)
}

Change is a proposed edit to the policy repo: new file contents plus a human-readable unified diff.

func PlanPromote

func PlanPromote(t Target, exp *policy.Experiment, ring, release string) (*Change, error)

PlanPromote points ring at release and marks the experiment concluded. release is required: promotion of a traffic-axis experiment (a model route change) is edited by a human in the gateway config, not here.

func PlanRollback

func PlanRollback(t Target, exp *policy.Experiment, ring, release string) (*Change, error)

PlanRollback pauses the experiment and, when release is non-empty, re-points ring at it (the last known-good release).

func PlanStatus

func PlanStatus(t Target, exp *policy.Experiment, status string) (*Change, error)

PlanStatus only sets the experiment's status (e.g. paused, concluded).

func PlanToggle

func PlanToggle(dir string, name string, c ToggleChange) (*Change, error)

PlanToggle builds the policy-repo change that applies c to toggle name. dir is the policy repo. The result is validated with the full guardrails (a copy of the policy with the edit applied must load and have no error-severity issue) before it is returned; nothing in dir is written. Comments and the other documents of the file are kept; the edited file is re-encoded by yaml.v3 (2-space indent), so unusual formatting in it may be normalised.

type ClickHouse

type ClickHouse struct {
	URL      string // e.g. http://clickhouse:8123
	Database string
	User     string
	Password string
	Table    string       // default halo_metrics
	Client   *http.Client // default: 30s timeout
}

ClickHouse reads the normalised halo_metrics table (see deploy/observability/clickhouse/schema.sql) over ClickHouse's HTTP interface. Values are bound with server-side query parameters, never interpolated.

func (*ClickHouse) Insert added in v0.4.0

func (c *ClickHouse) Insert(ctx context.Context, table string, rows []any) error

Insert writes rows, each marshalled as one JSON object, into table ([db.]name) with JSONEachRow. Timestamps may be RFC 3339 strings (date_time_input_format=best_effort).

func (*ClickHouse) Query added in v0.4.0

func (c *ClickHouse) Query(ctx context.Context, sql string, params map[string]string, fn func(json.RawMessage) error) error

Query runs sql (without a FORMAT clause) and streams each JSONEachRow line to fn. Values are bound as {name:Type} server-side parameters from params, never interpolated.

func (*ClickHouse) Samples

func (c *ClickHouse) Samples(ctx context.Context, experiment, metric string) (map[string][]float64, string, error)

Samples returns one value per (variant, unit) and the evidence source: see derivedSamples, else the unit's mean of the metric (client-supplied: SourceCLI).

func (*ClickHouse) Window

func (c *ClickHouse) Window(ctx context.Context, experiment string) (Window, error)

Window: first observation time and total halo.cost.usd for the experiment.

type Forge added in v0.5.0

type Forge interface {
	PROpener
	PRFinder
}

Forge is what callers need from a code host: open a change request for a branch Halos pushes with git, and find one it already opened by marker. A Forge must never merge, enable auto-merge or approve (invariant 5: a human merges). *ForgeOpener implements it for GitHub (gh CLI or REST) and GitLab.

type ForgeConfig added in v0.5.0

type ForgeConfig struct {
	// Kind is gh, github, gitlab or empty (autodetect).
	Kind string
	// BaseURL overrides the API root: https://ghe.example.com/api/v3 for GitHub
	// Enterprise Server, https://gitlab.example.com/api/v4 for self-managed
	// GitLab. Empty derives it from the origin remote host. https only, or
	// http to a loopback host.
	BaseURL string
}

ForgeConfig selects and configures a Forge.

type ForgeOpener added in v0.5.0

type ForgeOpener struct {
	Kind    string
	BaseURL string
	// Exec runs git (and gh); default Run.
	Exec RunFunc
	// HTTP is the API client; default has a 30s timeout.
	HTTP *http.Client
	// Getenv reads token variables; default os.Getenv.
	Getenv func(string) string
}

ForgeOpener is the Forge. Zero value = autodetect. Fields other than Kind and BaseURL are test seams.

func NewForge added in v0.5.0

func NewForge(cfg ForgeConfig) (*ForgeOpener, error)

NewForge validates cfg and returns the Forge. The kind is resolved per call from the repo's origin remote when cfg.Kind is empty, so one value serves a long-lived server whose clone it does not yet know.

func (*ForgeOpener) FindOpenPR added in v0.5.0

func (f *ForgeOpener) FindOpenPR(ctx context.Context, dir, marker string) (string, bool, error)

FindOpenPR implements OpenPRFinder: only a request that is still open.

func (*ForgeOpener) FindPR added in v0.5.0

func (f *ForgeOpener) FindPR(ctx context.Context, dir, marker string) (string, bool, error)

FindPR implements PRFinder: a request in any state (merged, closed, open).

func (*ForgeOpener) OpenPR added in v0.5.0

func (f *ForgeOpener) OpenPR(ctx context.Context, r PRRequest) (string, error)

OpenPR implements PROpener.

type GHOpener

type GHOpener struct {
	// Exec runs a command in dir and returns stdout; default Run.
	Exec func(ctx context.Context, dir, name string, args ...string) ([]byte, error)
}

GHOpener opens PRs with git and the gh CLI. It commits the given files onto a uniquely suffixed branch (so a retry after a partial failure never collides) in a temporary worktree (see CommitOnBranch), so the user's checkout, branch and index are never touched, pushes the branch, and runs `gh pr create`. It never merges.

UNVERIFIED against a real GitHub; tests use real git and a recording gh.

func (GHOpener) FindOpenPR added in v0.5.0

func (g GHOpener) FindOpenPR(ctx context.Context, dir, marker string) (string, bool, error)

FindOpenPR implements OpenPRFinder with `gh pr list --state open`.

func (GHOpener) FindPR added in v0.3.0

func (g GHOpener) FindPR(ctx context.Context, dir, marker string) (string, bool, error)

FindPR implements PRFinder with `gh pr list` (any state), matching the marker exactly in the returned bodies: the search narrows, the check decides.

func (GHOpener) OpenPR

func (g GHOpener) OpenPR(ctx context.Context, r PRRequest) (string, error)

type GuardrailReport

type GuardrailReport struct {
	Metric string                `json:"metric"`
	Source string                `json:"source,omitempty"`
	Result stats.GuardrailResult `json:"result"`
}

GuardrailReport is one guardrail's outcome.

type MemorySource

type MemorySource struct {
	Start    time.Time
	SpendUSD float64
	// Data is metric -> variant -> per-unit observations.
	Data map[string]map[string][]float64
	// Source is reported for every metric ("" = unknown, i.e. untrusted).
	Source string
}

MemorySource is an in-memory MetricSource for tests and local analysis.

func (*MemorySource) Samples

func (m *MemorySource) Samples(_ context.Context, _, metric string) (map[string][]float64, string, error)

func (*MemorySource) Window

func (m *MemorySource) Window(context.Context, string) (Window, error)

type MetricSource

type MetricSource interface {
	// Window reports when data first appeared and total spend so far.
	Window(ctx context.Context, experiment string) (Window, error)
	// Samples returns per-unit (user) observations of a halo.* metric, by
	// variant name, and their evidence source (SourceGateway | SourceCLI |
	// SourceEval; ""
	// = unknown, treated as untrusted).
	Samples(ctx context.Context, experiment, metric string) (map[string][]float64, string, error)
}

MetricSource supplies experiment evidence.

type OpenPRFinder added in v0.5.0

type OpenPRFinder interface {
	FindOpenPR(ctx context.Context, dir, marker string) (url string, found bool, err error)
}

OpenPRFinder finds a pull request with marker that is still open. Callers that dedupe a proposal (not a controller intent) use it, so a merged or closed request never blocks a new one.

type PRFinder added in v0.3.0

type PRFinder interface {
	FindPR(ctx context.Context, dir, marker string) (url string, found bool, err error)
}

PRFinder finds a pull request whose body contains marker (an opener that can, implements it). The controller embeds an intent key in every PR it opens, so a new leader can tell whether a deposed one already opened it.

type PROpener

type PROpener interface {
	OpenPR(ctx context.Context, req PRRequest) (url string, err error)
}

PROpener publishes a change as a pull request. Implementations must never merge.

type PRRequest

type PRRequest struct {
	RepoDir string
	Base    string // base branch; empty = repo default
	Branch  string
	Title   string
	Body    string
	// CommitBody is an optional extra commit-message paragraph (e.g. the reason).
	CommitBody string
	Files      map[string][]byte
	// Labels and Reviewers (usernames) are optional; empty adds none.
	Labels    []string
	Reviewers []string
	// Edit, when set, re-applies the change to the committed (HEAD) version of
	// each file, so uncommitted working-tree edits never ride into the PR.
	// Files is then only the preview.
	Edit func(ReadFunc) (map[string][]byte, error)
}

PRRequest is what a PROpener needs to publish a change for review.

type ReadFunc

type ReadFunc func(rel string) ([]byte, error)

ReadFunc returns the committed (HEAD) content of a dir-relative file.

type Remote added in v0.5.0

type Remote struct {
	Scheme   string // http, https or ssh
	Host     string // host[:port] for http(s); bare host for ssh
	Hostname string
	Path     string // group/sub/repo, no .git
}

Remote is a parsed git remote URL. Userinfo is dropped so it can never reach an error message.

func ParseRemote added in v0.5.0

func ParseRemote(raw string) (Remote, error)

ParseRemote parses an https://, ssh:// or scp-style (git@host:group/repo.git) remote URL.

type Report

type Report struct {
	Verdict    Verdict `json:"verdict"`
	Reason     string  `json:"reason"`
	Control    string  `json:"control"`
	Treatment  string  `json:"treatment"`
	NControl   int     `json:"nControl"`
	NTreatment int     `json:"nTreatment"`
	// PValue and Effect (treatment - control) are for the primary metric.
	PValue     float64           `json:"pValue"`
	Effect     float64           `json:"effect"`
	Guardrails []GuardrailReport `json:"guardrails"`
	// Source is the evidence source of the metric that decided a promote or
	// rollback verdict (the failing guardrail, else the primary); "" otherwise.
	Source string `json:"source,omitempty"`
}

Report explains a verdict. JSON keys are camelCase (the halo-server verdicts format).

func Evaluate

func Evaluate(ctx context.Context, exp *policy.Experiment, data MetricSource) (Report, error)

Evaluate decides using the current time.

func EvaluateAt

func EvaluateAt(ctx context.Context, exp *policy.Experiment, data MetricSource, now time.Time) (Report, error)

EvaluateAt is Evaluate with an explicit clock.

Order: (1) guardrails can trigger rollback once every arm has MinSamples; (2) exhausted MaxDays/MaxSpendUSD -> expired; (3) below MinSamples -> continue; (4) primary metric: significant improvement AND all guardrails passing -> promote, significant regression -> rollback, else continue. Only two-variant experiments (one control, one treatment) are supported.

Guardrails use an always-valid mSPRT confidence sequence (stats.SeqGuardrail) with a Bonferroni-split alpha, so polling on every evaluation does not inflate the false-rollback rate above alpha.

type RunFunc

type RunFunc func(ctx context.Context, dir, name string, args ...string) ([]byte, error)

RunFunc runs a command in dir and returns stdout (see Run).

type Target

type Target struct {
	RepoDir        string
	RingFile       string // YAML containing the Ring document
	ExperimentFile string // YAML containing the Experiment document
}

Target locates the policy-repo files a change edits, relative to RepoDir.

type ToggleChange

type ToggleChange struct {
	Default *bool    `json:"default,omitempty"`
	Expires string   `json:"expires,omitempty"`
	Rule    string   `json:"rule,omitempty"` // rule name or index; required with the fields below
	Percent *float64 `json:"percent,omitempty"`
	// Add*/Remove* edit the rule's rings, groups and users.
	AddRings     []string `json:"addRings,omitempty"`
	RemoveRings  []string `json:"removeRings,omitempty"`
	AddGroups    []string `json:"addGroups,omitempty"`
	RemoveGroups []string `json:"removeGroups,omitempty"`
	AddUsers     []string `json:"addUsers,omitempty"`
	RemoveUsers  []string `json:"removeUsers,omitempty"`
}

ToggleChange is a console-requested edit of one Toggle document: its default state, expiry, and one rule's rollout percent or ring/group/user lists. Zero fields change nothing.

func (ToggleChange) Check

func (c ToggleChange) Check() error

Check rejects a change that cannot be applied, before any file is touched.

func (ToggleChange) Summary

func (c ToggleChange) Summary() string

Summary is a one-line description for a PR title.

type Verdict

type Verdict string

Verdict is the decision for an experiment at this point in time.

const (
	Promote  Verdict = "promote"
	Rollback Verdict = "rollback"
	Continue Verdict = "continue"
	Expired  Verdict = "expired" // budget (days/spend) exhausted without a decision
)

type Window

type Window struct {
	Start    time.Time // zero = no data yet
	SpendUSD float64
}

Window is the experiment's observed span and spend.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL