Documentation
¶
Overview ¶
Package promote turns experiment evidence into a rollout decision.
Evaluate combines internal/stats with the experiment's stopping rules and returns promote | rollback | continue | expired. Acting on a verdict is deliberately asymmetric: rollback may be applied directly to the policy repo working tree (ApplyRollback), while promote only ever opens a PR (OpenPR); nothing in this package merges.
Index ¶
- Constants
- Variables
- func ApplyRollback(repoDir string, ch *Change) error
- func CommitOnBranch(ctx context.Context, run RunFunc, dir string, bc BranchCommit) (branch, commit string, err error)
- func OpenPR(ctx context.Context, o PROpener, t Target, ch *Change, exp *policy.Experiment, ...) (string, error)
- func PushBranch(ctx context.Context, run RunFunc, dir string, bc BranchCommit) (string, error)
- func ResetClone(ctx context.Context, run RunFunc, dir, base string) error
- func Run(ctx context.Context, dir, name string, args ...string) ([]byte, error)
- func SyncClone(ctx context.Context, run RunFunc, dir, base string) error
- func TransitionStatus(d *yamledit.Doc, verb string) (from, to string, data []byte, err error)
- func UnifiedDiff(path, old, nw string) string
- func UpsertVerdict(path, experiment string, rep Report, at time.Time) error
- func VerdictRow(experiment string, rep Report, at time.Time) ([]byte, error)
- type BranchCommit
- type Change
- func PlanPromote(t Target, exp *policy.Experiment, ring, release string) (*Change, error)
- func PlanRollback(t Target, exp *policy.Experiment, ring, release string) (*Change, error)
- func PlanStatus(t Target, exp *policy.Experiment, status string) (*Change, error)
- func PlanToggle(dir string, name string, c ToggleChange) (*Change, error)
- type ClickHouse
- func (c *ClickHouse) Insert(ctx context.Context, table string, rows []any) error
- func (c *ClickHouse) Query(ctx context.Context, sql string, params map[string]string, ...) error
- func (c *ClickHouse) Samples(ctx context.Context, experiment, metric string) (map[string][]float64, string, error)
- func (c *ClickHouse) Window(ctx context.Context, experiment string) (Window, error)
- type GHOpener
- type GuardrailReport
- type MemorySource
- type MetricSource
- type PRFinder
- type PROpener
- type PRRequest
- type ReadFunc
- type Report
- type RunFunc
- type Target
- type ToggleChange
- type Verdict
- type Window
Constants ¶
const ( SourceGateway = telemetry.SourceGateway // authenticated halo-proxy receiver: may auto-kill SourceCLI = telemetry.SourceCLI // developer-reachable receiver: client-controlled // SourceEval: the authenticated eval receiver (`halo eval online` judge // scores). Trusted for verdicts and PRs; never trips the kill switch. SourceEval = telemetry.SourceEval )
Evidence sources (Report.Source): which collector receiver the samples came through.
Variables ¶
var ErrToggleChange = errors.New("invalid toggle change")
ErrToggleChange marks a toggle change that cannot be applied or would make the policy invalid (as opposed to an infrastructure failure).
var StatusTransitions = map[string]struct { To string From []string }{ "start": {"running", []string{"", "draft", "paused"}}, "pause": {"paused", []string{"running"}}, "conclude": {"concluded", []string{"running", "paused"}}, }
StatusTransitions are the experiment status verbs (halo exp start|pause|conclude and the MCP tools): the status each sets and the statuses it may leave.
Functions ¶
func ApplyRollback ¶
ApplyRollback writes a rollback change straight into the working tree (atomically per file). Only rollbacks may be applied directly; promotions go through OpenPR.
func CommitOnBranch ¶
func CommitOnBranch(ctx context.Context, run RunFunc, dir string, bc BranchCommit) (branch, commit string, err error)
CommitOnBranch commits bc.Edit's files onto a new local branch starting at HEAD and returns the branch (with its random suffix) and the commit. It works in a temporary `git worktree add -b` checkout that is removed afterwards, so the user's branch, index and working tree are never touched. It never pushes. run nil means Run.
func OpenPR ¶
func OpenPR(ctx context.Context, o PROpener, t Target, ch *Change, exp *policy.Experiment, rep Report, base, roi string) (string, error)
OpenPR builds the PR for a promotion verdict and hands it to the opener. It is the only path by which a promotion reaches the policy repo; a human merges. roi, when non-empty, is a delivery-ROI section (internal/roi, ADR-0027) added to the body for the reviewer: it is informational and plays no part in the verdict.
func PushBranch ¶
PushBranch runs CommitOnBranch, pushes the branch to origin and then deletes the local branch whether or not the push succeeded, so a long-lived clone (halo-server's policy writer) never accumulates branches. run nil means Run.
func ResetClone ¶
ResetClone force-checks-out origin/<base> in dir and cleans untracked files (no fetch, no work-tree-root guard; call SyncClone first). The checkout keeps the committed bytes (no core.autocrlf=true CRLF, the Git for Windows default), so artifact digests in the clone match their pins.
func Run ¶
Run runs name in dir and returns stdout; errors carry the trimmed stderr. It is the one command runner for git/gh in promote and its callers.
func SyncClone ¶
SyncClone hard-resets dir to a pristine origin/<base>: it refuses unless dir is the root of its own git work tree (the reset is destructive and must never hit a parent repo), then fetches base and force-checks it out, removing untracked files. run is promote.Run or a stand-in.
func TransitionStatus ¶
TransitionStatus checks verb against the experiment document's current status and returns the edited file content (d is not written).
func UnifiedDiff ¶
UnifiedDiff renders a single-hunk unified diff (common prefix/suffix trimmed, 3 lines of context). Empty when nothing changed.
func UpsertVerdict ¶
UpsertVerdict replaces (or appends) the experiment's entry in the JSON array at path, atomically. Other entries are kept verbatim. ponytail: read-modify-write without a lock; run one writer (controller or `halo exp analyze`) per file.
Types ¶
type BranchCommit ¶
type BranchCommit struct {
Branch string // prefix; a random suffix is always appended
Subject string
Body string // optional second message paragraph
// Edit returns the new content (dir-relative, slash-separated path ->
// data) of every file to change. It is given the committed version of any
// file via read, so uncommitted edits in the user's checkout never land in
// the commit. A nil value deletes the file (an empty, non-nil one empties it).
Edit func(read ReadFunc) (map[string][]byte, error)
}
BranchCommit describes one commit onto a new branch.
type Change ¶
type Change struct {
Files map[string][]byte // repo-relative path -> new content
Patch string
// Edit re-applies just this change to whatever read returns (see
// BranchCommit.Edit), e.g. to the committed version of each file.
Edit func(read ReadFunc) (map[string][]byte, error)
}
Change is a proposed edit to the policy repo: new file contents plus a human-readable unified diff.
func PlanPromote ¶
PlanPromote points ring at release and marks the experiment concluded. release is required: promotion of a traffic-axis experiment (a model route change) is edited by a human in the gateway config, not here.
func PlanRollback ¶
PlanRollback pauses the experiment and, when release is non-empty, re-points ring at it (the last known-good release).
func PlanStatus ¶
PlanStatus only sets the experiment's status (e.g. paused, concluded).
func PlanToggle ¶
func PlanToggle(dir string, name string, c ToggleChange) (*Change, error)
PlanToggle builds the policy-repo change that applies c to toggle name. dir is the policy repo. The result is validated with the full guardrails (a copy of the policy with the edit applied must load and have no error-severity issue) before it is returned; nothing in dir is written. Comments and the other documents of the file are kept; the edited file is re-encoded by yaml.v3 (2-space indent), so unusual formatting in it may be normalised.
type ClickHouse ¶
type ClickHouse struct {
URL string // e.g. http://clickhouse:8123
Database string
User string
Password string
Table string // default halo_metrics
Client *http.Client // default: 30s timeout
}
ClickHouse reads the normalised halo_metrics table (see deploy/observability/clickhouse/schema.sql) over ClickHouse's HTTP interface. Values are bound with server-side query parameters, never interpolated.
func (*ClickHouse) Insert ¶ added in v0.4.0
Insert writes rows, each marshalled as one JSON object, into table ([db.]name) with JSONEachRow. Timestamps may be RFC 3339 strings (date_time_input_format=best_effort).
func (*ClickHouse) Query ¶ added in v0.4.0
func (c *ClickHouse) Query(ctx context.Context, sql string, params map[string]string, fn func(json.RawMessage) error) error
Query runs sql (without a FORMAT clause) and streams each JSONEachRow line to fn. Values are bound as {name:Type} server-side parameters from params, never interpolated.
type GHOpener ¶
type GHOpener struct {
// Exec runs a command in dir and returns stdout; default Run.
Exec func(ctx context.Context, dir, name string, args ...string) ([]byte, error)
}
GHOpener opens PRs with git and the gh CLI. It commits the given files onto a uniquely suffixed branch (so a retry after a partial failure never collides) in a temporary worktree (see CommitOnBranch), so the user's checkout, branch and index are never touched, pushes the branch, and runs `gh pr create`. It never merges.
UNVERIFIED against a real GitHub; tests use real git and a recording gh.
type GuardrailReport ¶
type GuardrailReport struct {
Metric string `json:"metric"`
Source string `json:"source,omitempty"`
Result stats.GuardrailResult `json:"result"`
}
GuardrailReport is one guardrail's outcome.
type MemorySource ¶
type MemorySource struct {
Start time.Time
SpendUSD float64
// Data is metric -> variant -> per-unit observations.
Data map[string]map[string][]float64
// Source is reported for every metric ("" = unknown, i.e. untrusted).
Source string
}
MemorySource is an in-memory MetricSource for tests and local analysis.
type MetricSource ¶
type MetricSource interface {
// Window reports when data first appeared and total spend so far.
Window(ctx context.Context, experiment string) (Window, error)
// Samples returns per-unit (user) observations of a halo.* metric, by
// variant name, and their evidence source (SourceGateway | SourceCLI |
// SourceEval; ""
// = unknown, treated as untrusted).
Samples(ctx context.Context, experiment, metric string) (map[string][]float64, string, error)
}
MetricSource supplies experiment evidence.
type PRFinder ¶ added in v0.3.0
type PRFinder interface {
FindPR(ctx context.Context, dir, marker string) (url string, found bool, err error)
}
PRFinder finds a pull request whose body contains marker (an opener that can, implements it). The controller embeds an intent key in every PR it opens, so a new leader can tell whether a deposed one already opened it.
type PRRequest ¶
type PRRequest struct {
RepoDir string
Base string // base branch; empty = repo default
Branch string
Title string
Body string
// CommitBody is an optional extra commit-message paragraph (e.g. the reason).
CommitBody string
Files map[string][]byte
// Edit, when set, re-applies the change to the committed (HEAD) version of
// each file, so uncommitted working-tree edits never ride into the PR.
// Files is then only the preview.
Edit func(ReadFunc) (map[string][]byte, error)
}
PRRequest is what a PROpener needs to publish a change for review.
type Report ¶
type Report struct {
Verdict Verdict `json:"verdict"`
Reason string `json:"reason"`
Control string `json:"control"`
Treatment string `json:"treatment"`
NControl int `json:"nControl"`
NTreatment int `json:"nTreatment"`
// PValue and Effect (treatment - control) are for the primary metric.
PValue float64 `json:"pValue"`
Effect float64 `json:"effect"`
Guardrails []GuardrailReport `json:"guardrails"`
// Source is the evidence source of the metric that decided a promote or
// rollback verdict (the failing guardrail, else the primary); "" otherwise.
Source string `json:"source,omitempty"`
}
Report explains a verdict. JSON keys are camelCase (the halo-server verdicts format).
func Evaluate ¶
func Evaluate(ctx context.Context, exp *policy.Experiment, data MetricSource) (Report, error)
Evaluate decides using the current time.
func EvaluateAt ¶
func EvaluateAt(ctx context.Context, exp *policy.Experiment, data MetricSource, now time.Time) (Report, error)
EvaluateAt is Evaluate with an explicit clock.
Order: (1) guardrails can trigger rollback once every arm has MinSamples; (2) exhausted MaxDays/MaxSpendUSD -> expired; (3) below MinSamples -> continue; (4) primary metric: significant improvement AND all guardrails passing -> promote, significant regression -> rollback, else continue. Only two-variant experiments (one control, one treatment) are supported.
Guardrails use an always-valid mSPRT confidence sequence (stats.SeqGuardrail) with a Bonferroni-split alpha, so polling on every evaluation does not inflate the false-rollback rate above alpha.
type Target ¶
type Target struct {
RepoDir string
RingFile string // YAML containing the Ring document
ExperimentFile string // YAML containing the Experiment document
}
Target locates the policy-repo files a change edits, relative to RepoDir.
type ToggleChange ¶
type ToggleChange struct {
Default *bool `json:"default,omitempty"`
Expires string `json:"expires,omitempty"`
Rule string `json:"rule,omitempty"` // rule name or index; required with the fields below
Percent *float64 `json:"percent,omitempty"`
// Add*/Remove* edit the rule's rings, groups and users.
AddRings []string `json:"addRings,omitempty"`
RemoveRings []string `json:"removeRings,omitempty"`
AddGroups []string `json:"addGroups,omitempty"`
RemoveGroups []string `json:"removeGroups,omitempty"`
AddUsers []string `json:"addUsers,omitempty"`
RemoveUsers []string `json:"removeUsers,omitempty"`
}
ToggleChange is a console-requested edit of one Toggle document: its default state, expiry, and one rule's rollout percent or ring/group/user lists. Zero fields change nothing.
func (ToggleChange) Check ¶
func (c ToggleChange) Check() error
Check rejects a change that cannot be applied, before any file is touched.
func (ToggleChange) Summary ¶
func (c ToggleChange) Summary() string
Summary is a one-line description for a PR title.