rake

package
v1.0.0-alpha.17 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 18, 2026 License: MIT Imports: 9 Imported by: 0

Documentation

Overview

Package rake is the one record-level raking (rim weighting) and cell weighting solver behind Pulse.Rake: it validates a rake request against a cohort schema (Compile), reads the margin categories of every record (Plan.Load), runs iterative proportional fitting over the RECORDS to the requested margins (Plan.Solve) and reports the convergence, the achieved margins and the Kish efficiency of the resulting weights.

It is a leaf (stdlib + encoding / errors / types + internal/weighting) so the no-execute layer can reuse Compile's static refusals without reaching the engine. It is NOT OVERLAY_RAKE (which rakes a crosstab table and writes nothing) and reads no Populations file (a population is a saved vector fit, not census targets): targets are inline. Contract: .claude/reference/weighting/rake.md.

Index

Constants

View Source
const (
	// DefaultMaxIter is the sweep ceiling when Request.MaxIter is 0.
	DefaultMaxIter = 100
	// DefaultTol is the convergence tolerance when Request.Tol is 0: the
	// largest absolute gap between an achieved and a target share, over
	// every margin category.
	DefaultTol = 1e-6
	// DefaultWeightField names the output weight column when
	// Request.WeightField is empty.
	DefaultWeightField = "rake_weight"
)

Defaults applied by Compile when the request leaves a knob at zero.

View Source
const (
	// MissingRefuse (the default) refuses a record whose margin value is
	// null with PULSE_RAKE_MISSING_VALUE.
	MissingRefuse = "refuse"
	// MissingBase keeps the record's current factor for a margin whose
	// value it lacks: the margin neither adjusts it nor counts it.
	MissingBase = "base"
)

Missing-value policies (Request.Missing).

View Source
const (
	// TargetsShares (the default): the targets are shares summing to 1,
	// or percentages summing to 100.
	TargetsShares = "shares"
	// TargetsCounts: the targets are population counts of any positive
	// total; only their proportions are used.
	TargetsCounts = "counts"
)

Target spellings (Margin.TargetsAs).

Variables

This section is empty.

Functions

This section is empty.

Types

type CategoryReport

type CategoryReport struct {
	// Values holds the category, one entry per margin field.
	Values []string `json:"values"`
	// ObservedN is the number of records in the category.
	ObservedN int64 `json:"observed_n"`
	// Observed is its share before raking (under the base weight, when
	// one is given).
	Observed float64 `json:"observed"`
	// Target is its requested share (targets normalised by their sum).
	Target float64 `json:"target"`
	// Achieved is its share under the output weights.
	Achieved float64 `json:"achieved"`
}

CategoryReport is one margin category. Root alias: pulse.RakeCategoryReport.

func (CategoryReport) MarshalJSON

func (c CategoryReport) MarshalJSON() ([]byte, error)

MarshalJSON writes an undefined share as null (types.MarshalFinite).

type Cell

type Cell struct {
	// Values holds one category per Margin.Fields entry.
	Values []string `json:"values"`
	// Target is the cell's share or count.
	Target float64 `json:"target"`
}

Cell is one joint-margin target. Root alias: pulse.RakeCell.

type Data

type Data struct {
	// contains filtered or unexported fields
}

Data is every record's margin categories, group and base weight, read once by Plan.Load.

func (*Data) Len

func (d *Data) Len() int

Len is the number of records read.

type Efficiency

type Efficiency struct {
	// N is the number of weighted records.
	N int64 `json:"n"`
	// SumWeights is Σw.
	SumWeights float64 `json:"sum_weights"`
	// NEff is Kish's effective sample size (Σw)²/Σw².
	NEff float64 `json:"n_eff"`
	// DeffKish is n / n_eff.
	DeffKish float64 `json:"deff_kish"`
	// EfficiencyPct is 100 · n_eff / n.
	EfficiencyPct float64 `json:"efficiency_pct"`
	// WeightMin / WeightMax are the smallest and largest weight.
	WeightMin float64 `json:"weight_min"`
	WeightMax float64 `json:"weight_max"`
	// WeightRangeRatio is max / min (null when min is 0).
	WeightRangeRatio float64 `json:"weight_range_ratio"`
	// WeightCV is the weights' coefficient of variation, population
	// standard deviation over the mean.
	WeightCV float64 `json:"weight_cv"`
	// NCapped counts records the final sweep held at a cap.
	NCapped int64 `json:"n_capped"`
}

Efficiency is the Kish weighting-efficiency report of the output weights. DeffKish is the UNEQUAL-WEIGHTING design effect only (n·Σw²/(Σw)² = 1 + WeightCV²): it carries no clustering, stratification or finite-population component. An undefined figure is NaN in Go and null on the wire. Root alias: pulse.RakeEfficiency.

func (Efficiency) MarshalJSON

func (e Efficiency) MarshalJSON() ([]byte, error)

MarshalJSON writes an undefined figure as null (types.MarshalFinite).

type GroupReport

type GroupReport struct {
	// Value is the group's by-field category.
	Value      string         `json:"value"`
	Records    int64          `json:"records"`
	Iterations int            `json:"iterations"`
	Converged  bool           `json:"converged"`
	Residual   float64        `json:"residual"`
	Margins    []MarginReport `json:"margins"`
	Efficiency Efficiency     `json:"efficiency"`
}

GroupReport is one `by` group's fit. Root alias: pulse.RakeGroupReport.

func (GroupReport) MarshalJSON

func (g GroupReport) MarshalJSON() ([]byte, error)

MarshalJSON writes an undefined figure as null (types.MarshalFinite).

type Margin

type Margin struct {
	// Field is the single margin field.
	Field string `json:"field,omitempty"`
	// Targets maps each category of Field (its dictionary label, its
	// decimal integer, or "true" / "false") to its target.
	Targets map[string]float64 `json:"targets,omitempty"`
	// Fields are the joint margin's fields, in Cell.Values order.
	Fields []string `json:"fields,omitempty"`
	// Cells are the joint margin's targets, one per category tuple.
	Cells []Cell `json:"cells,omitempty"`
	// TargetsAs is "shares" (default) or "counts".
	TargetsAs string `json:"targets_as,omitempty"`
}

Margin is one target distribution. Exactly one form is set: Field + Targets (a single-field rim margin) or Fields (two or more) + Cells (a joint margin — cell weighting). Root alias: pulse.RakeMargin.

type MarginReport

type MarginReport struct {
	// Fields names the margin's field(s).
	Fields []string `json:"fields"`
	// NMissing counts records with no value for the margin (missing
	// "base"); the shares are over the other records.
	NMissing   int64            `json:"n_missing"`
	Categories []CategoryReport `json:"categories"`
}

MarginReport is one margin's before / target / after audit. Root alias: pulse.RakeMarginReport.

type Plan

type Plan struct {
	// contains filtered or unexported fields
}

Plan is a request compiled against a schema: every static refusal has already been raised. Load reads the records' categories, Solve runs the fit.

func Compile

func Compile(req *Request, schema *encoding.Schema) (*Plan, error)

Compile validates req against schema (the source's LOGICAL schema) and returns the plan. Every refusal is PULSE_RAKE_INVALID or PULSE_RAKE_UNKNOWN_CATEGORY and is raised before any record is read, so a predict over the same schema raises the same code.

func (*Plan) Description

func (p *Plan) Description() string

Description is the generated description of the output weight column: what it is, what it was raked to, and how it was scaled.

func (*Plan) Load

func (p *Plan) Load(ctx context.Context, n int64, at func(i int64) ([]any, error)) (*Data, error)

Load reads n records through at, which returns record i as exact typed values in schema LOGICAL order (CohortReader.RecordAt). It refuses an invalid base weight (PULSE_RAKE_INVALID), a null margin or by value under missing "refuse" (PULSE_RAKE_MISSING_VALUE) and a category without a target (PULSE_RAKE_UNTARGETED_CATEGORY) — after the full read, so the details count every offending record.

func (*Plan) ReplaceIndex

func (p *Plan) ReplaceIndex() int

ReplaceIndex is the source field index the weight column replaces, or -1 when it is appended after the source fields.

func (*Plan) Solve

func (p *Plan) Solve(ctx context.Context, d *Data) (*Solution, error)

Solve rakes every group to the plan's margins: sweeps of iterative proportional fitting over the records (each margin in request order rescales its categories to target share × the current weight of the records it counts), the caps after every sweep, until the largest share gap is within tol or max_iter sweeps ran; then each group is normalised to its record count (or Total). A category with a positive target and no weight in a group is PULSE_RAKE_EMPTY_CELL; a group that does not converge adds a PULSE_RAKE_NOT_CONVERGED warning and keeps its last sweep.

func (*Plan) WeightField

func (p *Plan) WeightField() string

WeightField is the output weight column's name.

type Report

type Report struct {
	Iterations int            `json:"iterations"`
	Converged  bool           `json:"converged"`
	Residual   float64        `json:"residual"`
	Margins    []MarginReport `json:"margins"`
	Groups     []GroupReport  `json:"groups,omitempty"`
	Efficiency Efficiency     `json:"efficiency"`
}

Report is a rake run's convergence, achieved margins and efficiency. Without `by` it describes the one fit; with `by`, Iterations is the most any group took, Converged holds only when every group converged, Residual is the worst group's, and Margins / Efficiency pool every record (each group's own figures are in Groups).

type Request

type Request struct {
	// Cohort is the source cohort path (a single file, a shard archive
	// or an anchored shard).
	Cohort string `json:"cohort"`
	// Out is the path of the NEW single-file cohort written: every source
	// column verbatim plus the nullable f64 weight column.
	Out string `json:"out"`
	// WeightField names the output weight column; empty selects
	// DefaultWeightField. A name the source already carries is refused
	// unless Overwrite is set, which replaces that column in place.
	WeightField string `json:"weight_field,omitempty"`
	// Margins are the target distributions the weights reproduce. One
	// margin of several fields is cell weighting (post-stratification).
	Margins []Margin `json:"margins"`
	// By rakes every group of this field separately to the SAME targets
	// (per-wave weighting of a tracker); each group is normalised on its
	// own.
	By string `json:"by,omitempty"`
	// BaseWeight names a design-weight column the raking starts from: the
	// output is that weight times the raking adjustment.
	BaseWeight string `json:"base_weight,omitempty"`
	// Missing is the null-margin-value policy: "refuse" (default) or
	// "base".
	Missing string `json:"missing,omitempty"`
	// MinWeight / MaxWeight cap every weight relative to the mean weight
	// of its group (0 = no cap), applied after every sweep.
	MinWeight float64 `json:"min_weight,omitempty"`
	MaxWeight float64 `json:"max_weight,omitempty"`
	// MaxIter is the sweep ceiling (0 = DefaultMaxIter).
	MaxIter int `json:"max_iter,omitempty"`
	// Tol is the convergence tolerance on shares (0 = DefaultTol).
	Tol float64 `json:"tol,omitempty"`
	// Total normalises every group's weights to sum to this value; 0
	// (the default) normalises to the group's record count, so the mean
	// weight is 1.
	Total float64 `json:"total,omitempty"`
	// Overwrite replaces an existing cohort at Out and an existing source
	// column named WeightField.
	Overwrite bool `json:"overwrite,omitempty"`
}

Request is a rake run: which cohort to read, which new cohort to write, and the margins its weights must reproduce. Root alias: pulse.RakeRequest.

type Solution

type Solution struct {
	// Weights holds record i's output weight.
	Weights  []float64
	Report   Report
	Warnings []*errors.CodedError
}

Solution is a solved rake: one weight per record plus the report.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL