engine

package
v1.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 5, 2026 License: MIT Imports: 11 Imported by: 0

Documentation

Overview

Package engine holds the rules and decision logic that adapters consult on every wrapped operation. It depends only on the standard library and the chaotic fault package.

Index

Examples

Constants

View Source 
const (
	ReasonCounter       = "counter"
	ReasonRateLimit     = "rate_limit"
	ReasonMaxConcurrent = "max_concurrent"
	ReasonFailureBudget = "failure_budget"
	// ReasonDisabled and ReasonKillSwitch are reserved for observers that build
	// their own suppression accounting; the engine's disabled and kill-switch
	// paths return Pass without calling RuleSkipped, so they are not emitted.
	ReasonDisabled   = "disabled"
	ReasonKillSwitch = "killswitch"
)

Skip reasons passed to Observer.RuleSkipped. Observers may switch on these instead of matching free-form strings.

Variables

This section is empty.

Functions

This section is empty.

Types

type Action 

type Action interface {
	Before(ctx context.Context) error
	After(ctx context.Context) error
}

Action is what Eval returns; adapters execute it around the wrapped call. Before runs prior to the call. After runs after call. 

var Pass Action = passAction{}

Pass is the canonical no-op action.

type CounterKind added in v1.4.0 

type CounterKind int

CounterKind classifies a rule's counter for introspection. 

const (
	CounterAlways CounterKind = iota
	CounterTimes
	CounterRange
	CounterProbability
	CounterSequence
)

Counter kinds, one per rule counter strategy.

type CounterSpec added in v1.3.0 

type CounterSpec struct {
	Type string  `yaml:"type" json:"type"`
	N    int     `yaml:"n" json:"n"`
	From int     `yaml:"from" json:"from"`
	To   int     `yaml:"to" json:"to"`
	P    float64 `yaml:"p" json:"p"`
	Seed int64   `yaml:"seed" json:"seed"`
}

CounterSpec selects a counter. Type is "always", "times", "range", or "probability" (empty defaults to "always").

type Engine 

type Engine struct {
	// contains filtered or unexported fields
}

Engine holds the rules and decision logic. Engines are not safe to copy - always pass as *Engine. AddRule is safe for concurrent use, Eval is too.

func New 

func New(opts ...Option) *Engine

New constructs an engine with no rules. Options may inject an Observer or a KillSwitch.

func (*Engine) AddRule 

func (e *Engine) AddRule(r Rule) *Engine

AddRule appends a rule. Returns the engine for chaining. Append is implemented as a copy-on-write swap of the rule slice so concurrent Evals never see a torn slice.

func (*Engine) AllHits 

func (e *Engine) AllHits() map[string]int

AllHits returns a snapshot of hit counts for every named rule registered with the engine, including rules that have not yet fired (value 0).

func (*Engine) Disable added in v1.1.0 

func (e *Engine) Disable()

Disable flips an atomic flag so Enabled reports false and adapters take the passthrough path. Faster than Reset for "kill the chaos now". Reversible.

func (*Engine) Enable added in v1.1.0 

func (e *Engine) Enable()

Enable clears the disable flag.

func (*Engine) Enabled 

func (e *Engine) Enabled() bool

Enabled reports whether the engine has any rules. Adapters call this before constructing an Op so the no-op path stays alloc-free. Nil-safe: a nil engine reports false.

func (*Engine) Eval 

func (e *Engine) Eval(ctx context.Context, op Op) Action

Eval evaluates the op against all configured rules and returns the matching Action, or Pass if no rule matches or the engine is disabled.

func (*Engine) Hits 

func (e *Engine) Hits(name string) int

Hits returns the number of times a named rule has fired. Unknown names return 0. Safe for concurrent use.

func (*Engine) ReplaceRules 

func (e *Engine) ReplaceRules(rs RuleSet)

ReplaceRules atomically swaps the active rule set. Used by rule sources on reload. Concurrent Evals see either the old or the new set, never a torn one. Hit counters are rebuilt for the new set's named rules.

func (*Engine) Reset 

func (e *Engine) Reset()

Reset clears all rules and hit counters. Idempotent and cheap.

type FaultEvent added in v1.4.0 

type FaultEvent struct {
	Rule      string
	Op        Op
	FaultKind fault.Kind
	// Latency is the fault's configured sleep: the exact duration for a Latency
	// fault, or the max bound for a Jittered fault (whose per-call draw is not
	// observable). Zero for faults that inject no sleep.
	Latency time.Duration
}

FaultEvent describes a single injected fault delivered to RichObserver. It is emitted only for faults whose Apply returns without error (latency, jittered, and any custom no-op fault); faults that short-circuit the call (error, panic, connection drop) never produce a FaultEvent.

type FaultSpec added in v1.3.0 

type FaultSpec struct {
	Type     string `yaml:"type" json:"type"`
	Duration string `yaml:"duration" json:"duration"` // latency
	Min      string `yaml:"min" json:"min"`           // jittered
	Max      string `yaml:"max" json:"max"`           // jittered
	Message  string `yaml:"message" json:"message"`   // error -> errors.New(Message)
	Value    string `yaml:"value" json:"value"`       // panic -> Panic(value)
}

FaultSpec selects a fault. Type is "latency", "jittered", "error", "panic", or "conn_drop".

type Finding added in v1.4.0 

type Finding struct {
	Severity Severity
	Rule     string
	Message  string
}

Finding is one blast-radius hazard the linter detected. Rule is the offending rule's name (or "<unnamed>" when it has none).

type KillSwitch 

type KillSwitch func(ctx context.Context, op Op) bool

KillSwitch lets a caller short-circuit chaos. If it returns true for the current Op, Eval returns Pass without consulting any rule. The default engine has no kill switch (every call is evaluated).

type Kind 

type Kind int

Kind identifies which adapter produced an Op. Do not renumber existing values. 

const (
	OpHTTPClient Kind = iota + 1
	OpHTTPServer
	OpSQL
	OpGRPCClient
	OpGRPCServer
	OpExplicit // chaos.Point call sites
	OpPGX
	OpRedis // go-redis adapter
)

Op kind constants identify which adapter produced an Op.

type Observer 

type Observer interface {
	RuleFired(ruleName string, op Op, action Action)
	RuleSkipped(ruleName string, op Op, reason string)
}

Observer receives events from the engine each time it evaluates a named rule. v1 ships no concrete implementations. Users supply their own via WithObserver. The always-on per-named-rule hit counter (Engine.Hits) does not require an observer.

Observer methods are called synchronously on the request path. Keep them cheap, do not block.

type Op 

type Op struct {
	Kind   Kind
	Name   string
	Method string
	Attrs  map[string]string
}

Op describes a single intercepted call. Adapters construct an Op only after Engine.Enabled() returns true, so the no-op path allocates nothing.

type Option 

type Option func(*Engine)

Option configures an Engine at construction time.

func WithFailureBudget 

func WithFailureBudget(maxErrorRate float64, window int) Option

WithFailureBudget stops injecting faults once the observed error rate over a sliding window of the last window calls reaches maxErrorRate. Requires the adapters to report outcomes (they do, via OutcomeReporter). Panics if maxErrorRate is outside [0, 1] or window < 1.

func WithKillSwitch 

func WithKillSwitch(ks KillSwitch) Option

WithKillSwitch attaches a kill switch. Pass nil to clear.

func WithMaxConcurrent added in v1.1.0 

func WithMaxConcurrent(n int) Option

WithMaxConcurrent caps the number of simultaneously in-flight faulted calls to n. Matched calls that would exceed the cap return Pass. The slot is held for the duration of the fault (including latency sleeps) and released when the adapter calls After (or when Before short-circuits).

func WithObserver 

func WithObserver(obs Observer) Option

WithObserver attaches an Observer to the engine. Pass nil to clear. If obs also implements RichObserver, the engine additionally delivers per-fault FaultEvents to it; the assertion happens once here, not per call.

func WithProductionGuard added in v1.1.0 

func WithProductionGuard(check func() bool) Option

WithProductionGuard makes New panic if check returns true. Supply a check that detects an environment chaos must not run in (e.g. reads an env var).

func WithRateLimit added in v1.1.0 

func WithRateLimit(rps int) Option

WithRateLimit caps the number of faults that actually fire to rps per second (global across all rules). Matched calls beyond the limit return Pass.

func WithRuleSource 

func WithRuleSource(rs RuleSet) Option

WithRuleSource backs the engine with rs at construction (instead of AddRule).

type OutcomeReporter 

type OutcomeReporter interface {
	Outcome(ctx context.Context, callErr error)
}

OutcomeReporter is an optional interface an Action may implement to receive the result of the wrapped call. Adapters call Outcome (when implemented) after the wrapped boundary returns. callErr is the wrapped call's error (nil or success). It is not invoked when Before short-circuits the call.

type Report added in v1.4.0 

type Report struct {
	Findings []Finding
}

Report is the result of a lint pass.

func Lint added in v1.4.0 

func Lint(rules []Rule) Report

Lint inspects programmatic rules via RuleInfo. It is coarse: closures hide globs and probability values, so it flags only structural hazards visible through introspection — chiefly a rule that matches every operation on every call (no matchers, Always counter), which is far riskier when its faults are terminal (panic or connection drop).

func LintSpecs added in v1.4.0 

func LintSpecs(specs []RuleSpec) Report

LintSpecs inspects declarative specs and can see globs, probabilities, and durations the programmatic Lint cannot. It is the richer analog: it flags a wildcard name glob, a probability that always fires, latency above lintLatencyCeiling, a terminal fault on a globally-scoped spec, and two specs that target the same kind+glob (an overlap whose combined effect is easy to underestimate).

func (Report) OK added in v1.4.0 

func (r Report) OK() bool

OK reports whether the report contains no SeverityHigh findings. Authoring tools can gate on this to fail a build on a high-severity hazard while tolerating warnings.

type RichObserver added in v1.4.0 

type RichObserver interface {
	Observer
	FaultInjected(ctx context.Context, ev FaultEvent)
}

RichObserver is an optional richer sink. An Observer may also implement it to receive per-fault detail the base Observer cannot carry. The engine checks for it once, when WithObserver runs, not per call. FaultInjected fires from the adapter's request path, synchronously, after a fault's sleep completes - keep it cheap and non-blocking, like the base Observer methods.

type Rule 

type Rule struct {
	// contains filtered or unexported fields
}

Rule is a single match-counter-faults triple. Construct with NewRule and pass to Engine.AddRule. Rule values may be copied (e.g., by Named), but they share state via internal pointers - never modify a Rule's selectors or faults after passing it to AddRule.

func BuildRule added in v1.3.0 

func BuildRule(spec RuleSpec) (Rule, error)

BuildRule converts a RuleSpec into a Rule, validating kinds, counter type, fault types, durations, and probability bounds. It is total: it never panics, and it aggregates every invalid field into a single errors.Join result so a caller (and the operator editing config) sees all problems at once.

Example 
package main

import (
	"context"
	"fmt"

	"github.com/ag4r/chaotic/engine"
)

// fire evaluates op against eng and returns the injected fault error (or nil),
// mimicking what an adapter does around a wrapped call.
func fire(eng *engine.Engine, op engine.Op) error {
	ctx := context.Background()
	act := eng.Eval(ctx, op)
	err := act.Before(ctx)
	_ = act.After(ctx)
	return err
}

func main() {
	// BuildRule turns a declarative RuleSpec (e.g. decoded from YAML) into a
	// Rule. It is total: invalid specs return an error instead of panicking.
	rule, err := engine.BuildRule(engine.RuleSpec{
		Name:    "from-config",
		Kinds:   []string{"http_client"},
		Counter: engine.CounterSpec{Type: "times", N: 1},
		Faults:  []engine.FaultSpec{{Type: "error", Message: "boom"}},
	})
	if err != nil {
		fmt.Println("build error:", err)
		return
	}
	eng := engine.New().AddRule(rule)
	fmt.Println("fired:", fire(eng, engine.Op{Kind: engine.OpHTTPClient}) != nil)
}

Output:
fired: true

func NewRule 

func NewRule(opts ...RuleOption) Rule

NewRule constructs a Rule. The default counter is Always, default action is no faults (Pass).

Example 
package main

import (
	"context"
	"errors"
	"fmt"

	"github.com/ag4r/chaotic/engine"
	"github.com/ag4r/chaotic/fault"
)

// fire evaluates op against eng and returns the injected fault error (or nil),
// mimicking what an adapter does around a wrapped call.
func fire(eng *engine.Engine, op engine.Op) error {
	ctx := context.Background()
	act := eng.Eval(ctx, op)
	err := act.Before(ctx)
	_ = act.After(ctx)
	return err
}

func main() {
	eng := engine.New().AddRule(engine.NewRule(
		engine.MatchKind(engine.OpHTTPClient),
		engine.Times(1), // fire on the first match only
		engine.WithFault(fault.Error(errors.New("transient"))),
	).Named("flap"))

	op := engine.Op{
		Kind: engine.OpHTTPClient,
		Name: "/users",
	}
	fmt.Println("call 1:", fire(eng, op))
	fmt.Println("call 2:", fire(eng, op))
	fmt.Println("hits:", eng.Hits("flap"))
}

Output:
call 1: transient
call 2: <nil>
hits: 1

func (Rule) Info added in v1.4.0 

func (r Rule) Info() RuleInfo

Info returns a RuleInfo describing r.

func (Rule) Name 

func (r Rule) Name() string

Name reports the rule's name, or "" if it has none.

func (Rule) Named 

func (r Rule) Named(name string) Rule

Named tags the rule for assertions and observability. Returns a copy of the rule (with shared mutable state - selectors/counter/faults) so the pre-named rule remains usable.

type RuleInfo added in v1.4.0 

type RuleInfo struct {
	Name          string
	Unconstrained bool
	Counter       CounterKind
	Faults        []fault.Kind
}

RuleInfo is a read-only view of a Rule for linting and tooling. It exposes only what closures permit: whether the rule is unconstrained (no matchers, so it matches every Op), its counter kind, and the kinds of its faults.

type RuleOption 

type RuleOption func(*Rule)

RuleOption configures a Rule during construction.

func Always 

func Always() RuleOption

Always is default counter: every match fires the rule.

func MatchAttr 

func MatchAttr(key, value string) RuleOption

MatchAttr matches Ops whose Attrs[key] equals value.

Example 
package main

import (
	"context"
	"errors"
	"fmt"

	"github.com/ag4r/chaotic/engine"
	"github.com/ag4r/chaotic/fault"
)

// fire evaluates op against eng and returns the injected fault error (or nil),
// mimicking what an adapter does around a wrapped call.
func fire(eng *engine.Engine, op engine.Op) error {
	ctx := context.Background()
	act := eng.Eval(ctx, op)
	err := act.Before(ctx)
	_ = act.After(ctx)
	return err
}

func main() {
	eng := engine.New().AddRule(engine.NewRule(
		engine.MatchKind(engine.OpHTTPClient),
		engine.MatchAttr("host", "payments.internal"),
		engine.WithFault(fault.Error(errors.New("degraded"))),
	).Named("payments"))

	payments := engine.Op{
		Kind:  engine.OpHTTPClient,
		Attrs: map[string]string{"host": "payments.internal"},
	}

	search := engine.Op{
		Kind:  engine.OpHTTPClient,
		Attrs: map[string]string{"host": "search.internal"},
	}
	fmt.Println("payments:", fire(eng, payments))
	fmt.Println("search:", fire(eng, search))
}

Output:
payments: degraded
search: <nil>

func MatchKind 

func MatchKind(kinds ...Kind) RuleOption

MatchKind matches Ops whose Kind appears in kinds. With zero arguments, matches nothing.

func MatchName 

func MatchName(pattern string) RuleOption

MatchName matches Ops whose Name satisfies path.Match(pattern, Name). Patterns support *, ?, and [...]. * does not cross /.

func MatchPredicate 

func MatchPredicate(fn func(context.Context, Op) bool) RuleOption

MatchPredicate matches Ops for which fn returns true.

func Probability 

func Probability(p float64, seed int64) RuleOption

Probability makes the rule fire on each match independently with probability p. Seed makes the decision deterministic across runs. Panics if p is outside [0,1].

Example 
package main

import (
	"context"
	"errors"
	"fmt"
	"strings"

	"github.com/ag4r/chaotic/engine"
	"github.com/ag4r/chaotic/fault"
)

// fire evaluates op against eng and returns the injected fault error (or nil),
// mimicking what an adapter does around a wrapped call.
func fire(eng *engine.Engine, op engine.Op) error {
	ctx := context.Background()
	act := eng.Eval(ctx, op)
	err := act.Before(ctx)
	_ = act.After(ctx)
	return err
}

func main() {
	// A seeded probability rule fires  identically every run, so chaos tests are
	// reproducible. Two engines built with the same seed produce same
	// fire/skip sequence.
	build := func() *engine.Engine {
		return engine.New().AddRule(engine.NewRule(
			engine.MatchKind(engine.OpHTTPClient),
			engine.Probability(0.5, 1),
			engine.WithFault(fault.Error(errors.New("x"))),
		).Named("p"))
	}
	seq := func(eng *engine.Engine) string {
		var b strings.Builder
		for range 10 {
			if fire(eng, engine.Op{Kind: engine.OpHTTPClient}) != nil {
				b.WriteByte('F')
			} else {
				b.WriteByte('.')
			}
		}
		return b.String()
	}
	first, second := seq(build()), seq(build())
	fmt.Println("reproducible:", first == second)
}

Output:
reproducible: true

func Range 

func Range(from, to int) RuleOption

Range makes the rule fire on matches[from:to] (1-indexed, inclusive). If from > to or either is < 1, the rule never fires.

func Sequence added in v1.7.0 

func Sequence(fire []bool) RuleOption

Sequence fires on the evaluations whose index is true in fire, in order, and skips the rest. After the slice is exhausted it never fires again. it is the deterministic counter chaostest/golden uses to replay a recorded fire pattern, and is useful generally for "fire on exactly these matches".

func Times 

func Times(n int) RuleOption

Times makes the rule fire on the first n matches. After n, the rule never fires again until Engine.Reset clears the counter.

func WithFault 

func WithFault(f fault.Fault) RuleOption

WithFault attaches a single fault. Equivalent to WithFaults(f).

func WithFaults 

func WithFaults(fs ...fault.Fault) RuleOption

WithFaults attaches faults that execute in order inside Action.Before. The first fault returning a non-nil error short-circuits the chain.

type RuleSet 

type RuleSet interface {
	Len() int
	Snapshot() []Rule
}

RuleSet is the engine's view of its current rules. The engine never holds a snapshot across Eval calls - it loads a fresh one each time.

func NewRuleSet 

func NewRuleSet(rules []Rule) RuleSet

NewRuleSet returns an in-memory RuleSet backed by the given rules. Sources (file/http) build their rules then call ReplaceRules(NewRuleSet(rules)).

type RuleSpec added in v1.3.0 

type RuleSpec struct {
	Name     string            `yaml:"name" json:"name"`
	Kinds    []string          `yaml:"kinds" json:"kinds"`
	NameGlob string            `yaml:"name_glob" json:"name_glob"`
	Attrs    map[string]string `yaml:"attrs" json:"attrs"`
	Counter  CounterSpec       `yaml:"counter" json:"counter"`
	Faults   []FaultSpec       `yaml:"faults" json:"faults"`
}

RuleSpec is the declarative, serializable form of a Rule. Rule sources parse config (YAML/JSON) into RuleSpec and call BuildRule. The struct tags are the on-disk/on-wire field names. MatchPredicate and typed error values cannot be serialized. Config rules support this declarative subset only.

type Severity added in v1.4.0 

type Severity int

Severity ranks a lint Finding. Only SeverityHigh findings fail Report.OK. 

const (
	SeverityInfo Severity = iota
	SeverityWarn
	SeverityHigh
)

Severity levels in ascending order of seriousness.

func (Severity) String added in v1.4.0 

func (s Severity) String() string

String returns the lowercase severity name.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.