Documentation
¶
Overview ¶
Package abstats is the A/B test's decision rule, shared by the sending engine (which decides) and the report (which explains the decision), so the two can't disagree (review AN-1).
A variant wins only when every variant reached the minimum sample and the leader's lead over the runner-up is significant: a two-sided two-proportion z-test at the chosen confidence (95% unless the campaign says otherwise). Otherwise there is no clear winner, and the rest of the audience gets the default variant.
Index ¶
Constants ¶
View Source
const ( DefaultMinSample = 500 DefaultConfidence = 0.95 )
Defaults of the rule (contract C14: campaigns.ab_test.min_sample and confidence).
View Source
const ( // Winner: the leader won. Winner = "winner" // NotSignificant: the lead could be chance. NotSignificant = "not_significant" // TooSmall: a variant didn't reach the minimum sample. TooSmall = "too_small" // Manual: a person picked the variant. Manual = "manual" )
Outcomes of a decision.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Result ¶
type Result struct {
// Leader has the best rate, RunnerUp the best of the rest (ties go to
// the earlier variant).
Leader, RunnerUp string
// Tested: the z-test could run (both had results and the rates
// weren't all 0% or all 100%).
Tested bool
// Lift is the leader's lead in percentage points; P the two-sided
// p-value; Confidence is 1 - P in percent.
Lift, Z, P, Confidence float64
// Enough: every variant reached the minimum sample.
Enough bool
// Outcome is Winner, NotSignificant or TooSmall.
Outcome string
}
Result is the rule applied to the variants.
Click to show internal directories.
Click to hide internal directories.