Documentation
¶
Overview ¶
Package rake is the one record-level raking (rim weighting) and cell weighting solver behind Pulse.Rake: it validates a rake request against a cohort schema (Compile), reads the margin categories of every record (Plan.Load), runs iterative proportional fitting over the RECORDS to the requested margins (Plan.Solve) and reports the convergence, the achieved margins and the Kish efficiency of the resulting weights.
It is a leaf (stdlib + encoding / errors / types + internal/weighting) so the no-execute layer can reuse Compile's static refusals without reaching the engine. It is NOT OVERLAY_RAKE (which rakes a crosstab table and writes nothing) and reads no Populations file (a population is a saved vector fit, not census targets): targets are inline. Contract: .claude/reference/weighting/rake.md.
Index ¶
Constants ¶
const ( // DefaultMaxIter is the sweep ceiling when Request.MaxIter is 0. DefaultMaxIter = 100 // DefaultTol is the convergence tolerance when Request.Tol is 0: the // largest absolute gap between an achieved and a target share, over // every margin category. DefaultTol = 1e-6 // DefaultWeightField names the output weight column when // Request.WeightField is empty. DefaultWeightField = "rake_weight" )
Defaults applied by Compile when the request leaves a knob at zero.
const ( // MissingRefuse (the default) refuses a record whose margin value is // null with PULSE_RAKE_MISSING_VALUE. MissingRefuse = "refuse" // MissingBase keeps the record's current factor for a margin whose // value it lacks: the margin neither adjusts it nor counts it. MissingBase = "base" )
Missing-value policies (Request.Missing).
const ( // or percentages summing to 100. TargetsShares = "shares" // TargetsCounts: the targets are population counts of any positive // total; only their proportions are used. TargetsCounts = "counts" )
Target spellings (Margin.TargetsAs).
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type CategoryReport ¶
type CategoryReport struct {
// Values holds the category, one entry per margin field.
Values []string `json:"values"`
// ObservedN is the number of records in the category.
ObservedN int64 `json:"observed_n"`
// Observed is its share before raking (under the base weight, when
// one is given).
Observed float64 `json:"observed"`
// Target is its requested share (targets normalised by their sum).
Target float64 `json:"target"`
// Achieved is its share under the output weights.
Achieved float64 `json:"achieved"`
}
CategoryReport is one margin category. Root alias: pulse.RakeCategoryReport.
func (CategoryReport) MarshalJSON ¶
func (c CategoryReport) MarshalJSON() ([]byte, error)
MarshalJSON writes an undefined share as null (types.MarshalFinite).
type Cell ¶
type Cell struct {
// Values holds one category per Margin.Fields entry.
Values []string `json:"values"`
// Target is the cell's share or count.
Target float64 `json:"target"`
}
Cell is one joint-margin target. Root alias: pulse.RakeCell.
type Data ¶
type Data struct {
// contains filtered or unexported fields
}
Data is every record's margin categories, group and base weight, read once by Plan.Load.
type Efficiency ¶
type Efficiency struct {
// N is the number of weighted records.
N int64 `json:"n"`
// SumWeights is Σw.
SumWeights float64 `json:"sum_weights"`
// NEff is Kish's effective sample size (Σw)²/Σw².
NEff float64 `json:"n_eff"`
// DeffKish is n / n_eff.
DeffKish float64 `json:"deff_kish"`
// EfficiencyPct is 100 · n_eff / n.
EfficiencyPct float64 `json:"efficiency_pct"`
// WeightMin / WeightMax are the smallest and largest weight.
WeightMin float64 `json:"weight_min"`
WeightMax float64 `json:"weight_max"`
// WeightRangeRatio is max / min (null when min is 0).
WeightRangeRatio float64 `json:"weight_range_ratio"`
// WeightCV is the weights' coefficient of variation, population
// standard deviation over the mean.
WeightCV float64 `json:"weight_cv"`
// NCapped counts records the final sweep held at a cap.
NCapped int64 `json:"n_capped"`
}
Efficiency is the Kish weighting-efficiency report of the output weights. DeffKish is the UNEQUAL-WEIGHTING design effect only (n·Σw²/(Σw)² = 1 + WeightCV²): it carries no clustering, stratification or finite-population component. An undefined figure is NaN in Go and null on the wire. Root alias: pulse.RakeEfficiency.
func (Efficiency) MarshalJSON ¶
func (e Efficiency) MarshalJSON() ([]byte, error)
MarshalJSON writes an undefined figure as null (types.MarshalFinite).
type GroupReport ¶
type GroupReport struct {
// Value is the group's by-field category.
Value string `json:"value"`
Records int64 `json:"records"`
Iterations int `json:"iterations"`
Converged bool `json:"converged"`
Residual float64 `json:"residual"`
Margins []MarginReport `json:"margins"`
Efficiency Efficiency `json:"efficiency"`
}
GroupReport is one `by` group's fit. Root alias: pulse.RakeGroupReport.
func (GroupReport) MarshalJSON ¶
func (g GroupReport) MarshalJSON() ([]byte, error)
MarshalJSON writes an undefined figure as null (types.MarshalFinite).
type Margin ¶
type Margin struct {
// Field is the single margin field.
Field string `json:"field,omitempty"`
// Targets maps each category of Field (its dictionary label, its
// decimal integer, or "true" / "false") to its target.
Targets map[string]float64 `json:"targets,omitempty"`
// Fields are the joint margin's fields, in Cell.Values order.
Fields []string `json:"fields,omitempty"`
// Cells are the joint margin's targets, one per category tuple.
Cells []Cell `json:"cells,omitempty"`
// TargetsAs is "shares" (default) or "counts".
TargetsAs string `json:"targets_as,omitempty"`
}
Margin is one target distribution. Exactly one form is set: Field + Targets (a single-field rim margin) or Fields (two or more) + Cells (a joint margin — cell weighting). Root alias: pulse.RakeMargin.
type MarginReport ¶
type MarginReport struct {
// Fields names the margin's field(s).
Fields []string `json:"fields"`
// NMissing counts records with no value for the margin (missing
// "base"); the shares are over the other records.
NMissing int64 `json:"n_missing"`
Categories []CategoryReport `json:"categories"`
}
MarginReport is one margin's before / target / after audit. Root alias: pulse.RakeMarginReport.
type Plan ¶
type Plan struct {
// contains filtered or unexported fields
}
Plan is a request compiled against a schema: every static refusal has already been raised. Load reads the records' categories, Solve runs the fit.
func Compile ¶
Compile validates req against schema (the source's LOGICAL schema) and returns the plan. Every refusal is PULSE_RAKE_INVALID or PULSE_RAKE_UNKNOWN_CATEGORY and is raised before any record is read, so a predict over the same schema raises the same code.
func (*Plan) Description ¶
Description is the generated description of the output weight column: what it is, what it was raked to, and how it was scaled.
func (*Plan) Load ¶
Load reads n records through at, which returns record i as exact typed values in schema LOGICAL order (CohortReader.RecordAt). It refuses an invalid base weight (PULSE_RAKE_INVALID), a null margin or by value under missing "refuse" (PULSE_RAKE_MISSING_VALUE) and a category without a target (PULSE_RAKE_UNTARGETED_CATEGORY) — after the full read, so the details count every offending record.
func (*Plan) ReplaceIndex ¶
ReplaceIndex is the source field index the weight column replaces, or -1 when it is appended after the source fields.
func (*Plan) Solve ¶
Solve rakes every group to the plan's margins: sweeps of iterative proportional fitting over the records (each margin in request order rescales its categories to target share × the current weight of the records it counts), the caps after every sweep, until the largest share gap is within tol or max_iter sweeps ran; then each group is normalised to its record count (or Total). A category with a positive target and no weight in a group is PULSE_RAKE_EMPTY_CELL; a group that does not converge adds a PULSE_RAKE_NOT_CONVERGED warning and keeps its last sweep.
func (*Plan) WeightField ¶
WeightField is the output weight column's name.
type Report ¶
type Report struct {
Iterations int `json:"iterations"`
Converged bool `json:"converged"`
Residual float64 `json:"residual"`
Margins []MarginReport `json:"margins"`
Groups []GroupReport `json:"groups,omitempty"`
Efficiency Efficiency `json:"efficiency"`
}
Report is a rake run's convergence, achieved margins and efficiency. Without `by` it describes the one fit; with `by`, Iterations is the most any group took, Converged holds only when every group converged, Residual is the worst group's, and Margins / Efficiency pool every record (each group's own figures are in Groups).
type Request ¶
type Request struct {
// Cohort is the source cohort path (a single file, a shard archive
// or an anchored shard).
Cohort string `json:"cohort"`
// Out is the path of the NEW single-file cohort written: every source
// column verbatim plus the nullable f64 weight column.
Out string `json:"out"`
// WeightField names the output weight column; empty selects
// DefaultWeightField. A name the source already carries is refused
// unless Overwrite is set, which replaces that column in place.
WeightField string `json:"weight_field,omitempty"`
// Margins are the target distributions the weights reproduce. One
// margin of several fields is cell weighting (post-stratification).
Margins []Margin `json:"margins"`
// By rakes every group of this field separately to the SAME targets
// (per-wave weighting of a tracker); each group is normalised on its
// own.
By string `json:"by,omitempty"`
// BaseWeight names a design-weight column the raking starts from: the
// output is that weight times the raking adjustment.
BaseWeight string `json:"base_weight,omitempty"`
// Missing is the null-margin-value policy: "refuse" (default) or
// "base".
Missing string `json:"missing,omitempty"`
// MinWeight / MaxWeight cap every weight relative to the mean weight
// of its group (0 = no cap), applied after every sweep.
MinWeight float64 `json:"min_weight,omitempty"`
MaxWeight float64 `json:"max_weight,omitempty"`
// MaxIter is the sweep ceiling (0 = DefaultMaxIter).
MaxIter int `json:"max_iter,omitempty"`
// Tol is the convergence tolerance on shares (0 = DefaultTol).
Tol float64 `json:"tol,omitempty"`
// Total normalises every group's weights to sum to this value; 0
// (the default) normalises to the group's record count, so the mean
// weight is 1.
Total float64 `json:"total,omitempty"`
// Overwrite replaces an existing cohort at Out and an existing source
// column named WeightField.
Overwrite bool `json:"overwrite,omitempty"`
}
Request is a rake run: which cohort to read, which new cohort to write, and the margins its weights must reproduce. Root alias: pulse.RakeRequest.