Help us shape the future of Go! Take the 2026 Go Developer Survey

regression

package
v1.0.0-alpha.22 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 21, 2026 License: MIT Imports: 24 Imported by: 0

Documentation

Overview

Package regression hosts the REG_* operators that fit a regression model against the filtered record set. Phase 0 lands the API surface only — every registered operator returns PROCESSING_REGRESSION_NOT_IMPLEMENTED via Fit. Phases 1–5 fill in closed-form OLS, regularization, GLM (IRLS), Bayesian linear, and the Resample / Selection modifier wrappers.

The package owns its own registry; the parent processing package consumes the Fit entry point through processing.FitRegressions without an import of regression-engine internals. To avoid an import cycle with processing.Record, this subpackage stays independent of concrete Record types — Phase 1 will introduce a narrow Record interface the same way internal/processing/feature does.

Index

Constants

View Source 
const InterceptKey = "(intercept)"

InterceptKey is the map key used for the synthesized intercept entry in RegressionResult.Coefficients / StdErrors / PValues. Picked to be human-readable and impossible to collide with a real field name (no schema field carries parentheses).

Variables

This section is empty.

Functions

func AttachExtensionIntervals 

func AttachExtensionIntervals(res *types.RegressionResult, spec *types.RegressionSpec)

AttachExtensionIntervals writes an extension (registered) REG_* model's coefficient confidence intervals at the spec's conf_level (0.95 unset) from the result's standard errors — the built-ins' attachIntervals, with the quantile rule of the matching built-in case:

  • resampled (res.Resample set): the standard normal on the replicate SE, as every built-in resample re-attaches;
  • else, when the model reported its residual df (Fit.DF, "df" on ReportedFigures): the Student-t quantile on that df, REG_OLS's rule (and REG_GLM gamma's) — so an OLS-shaped model reporting N* − p − 1 matches REG_OLS's intervals;
  • else the standard normal (Wald), REG_GLM's binomial / poisson rule.

A coefficient whose SE is NaN gets a NaN (null) interval, key kept; a zero SE the point interval. DF then follows the built-in contract (the df of the t the intervals read): the reported df on a t interval (NaN — null — and NaN bounds when it is ≤ 0), absent on a normal one, so a resampled fit drops a reported df from DF and from ReportedFigures exactly as a built-in resample clears it.

func AttachVcovKeys 

func AttachVcovKeys(res *types.RegressionResult, keys []string, v [][]float64)

AttachVcovKeys sets res.Vcov and res.Correlation from v, square over keys in their order — attachVcov's assembly for a caller that names its own coefficients (an extension REG_* model, whose covariance rows follow its reported coefficient order). v must be len(keys) square.

func BuiltinScorer 

func BuiltinScorer(t types.RegressionType) (model.ScorerFactory, bool)

BuiltinScorer is the built-in model.ScorerSource: the RowScorer factory of each scorable built-in regression — REG_OLS (penalties and selection included) and REG_BAYES_LINEAR (its posterior mean) score the linear predictor, REG_GLM its inverse link on the response scale (the linear predictor on the link scale). model.ScorableType is this table's static face (predict cannot import the engine); TestBuiltinScorer_MatchesScorableType keeps them one set.

func CheckConfLevel 

func CheckConfLevel(spec *types.RegressionSpec) error

CheckConfLevel is validateConfLevel for a spec no built-in engine validates — an extension (registered) REG_* model's: PROCESSING_CONFIG on a conf_level outside (0, 1), the message and details predict raises.

func Fit 

func Fit(specs []*types.RegressionSpec, schema *encoding.Schema) ([]*types.RegressionResult, error)

Fit drives Build then invokes Fit on every engine in order. Returns (results, nil) on success; the first per-spec error short-circuits the loop and propagates to the caller.

Phase 0 contract: every spec returns PROCESSING_REGRESSION_NOT_IMPLEMENTED from Fit. Phase 1 retires this path for the buffered orchestrator: callers now use FitBuffered to pass in the filtered record slice. Fit() is retained for callers that have no records (legacy / unit tests); engines that need rows return PROCESSING_REGRESSION_INSUFFICIENT_DATA from Fit() since their accumulator is empty.

func FitBuffered 

func FitBuffered(specs []*types.RegressionSpec, schema *encoding.Schema, records []Record) ([]*types.RegressionResult, error)

FitBuffered is the Phase 1 buffered-orchestrator entry point. It builds engines, then for each engine routes through FitBuffered with the filtered record slice. Engines that implement BufferedEngine consume the records; legacy not-implemented engines ignore the slice and surface PROCESSING_REGRESSION_NOT_IMPLEMENTED via Fit().

The records slice is shared across every spec — every engine sees the same filtered set, so per-spec listwise null filtering happens inside the engine (callers don't pre-filter per spec).

func FitBufferedWith 

func FitBufferedWith(specs []*types.RegressionSpec, schema *encoding.Schema, records []Record, resolve Resolver) ([]*types.RegressionResult, error)

FitBufferedWith is FitBuffered resolving each type through resolve: the first per-spec fit error short-circuits and propagates.

func FitEachBufferedWith 

func FitEachBufferedWith(specs []*types.RegressionSpec, schema *encoding.Schema, records []Record, resolve Resolver) (results []*types.RegressionResult, fitErrs []error, err error)

FitEachBufferedWith builds every engine (a Build error — a configuration fault — is returned as err) and then fits EACH one, never stopping at a failed fit: results and fitErrs are index-aligned with specs, a failed spec holding a nil result and its error. The processor turns a data-dependent fit error into that slot's per-slot failure (internal/slotfail) and returns any other as fatal.

func FitEachBufferedWithContext 

func FitEachBufferedWithContext(ctx context.Context, specs []*types.RegressionSpec, schema *encoding.Schema, records []Record, resolve Resolver) (results []*types.RegressionResult, fitErrs []error, err error)

FitEachBufferedWithContext is FitEachBufferedWith under ctx: an engine implementing ContextBufferedEngine fits through FitBufferedContext, so its refit loop honours ctx. 

func InverseLink(family, link string) (func(float64) float64, error)

InverseLink returns g⁻¹ for a REG_GLM family / link pair as the fit reported them (RegressionResult.Family / .Link) — the very function the engine computes its fitted means with.

func LevelFault 

func LevelFault(spec *types.RegressionSpec, records ...Record) error

LevelFault is the first corrupt categorical code among the fields spec keys rows by (covariance cluster, jackknife_by) across records — ENCODING_INVALID with the CategoricalIndexFault details — or nil. The drivers run it before a fit reads a row (FitEachBufferedWithContext over the buffered rows, the streaming guard per row), so the fault fails the regression slot alone (slotfail) instead of the row being read as a null cluster and dropped, or keyed by its numeric code (U51 E5-S10).

func LeverageForRow 

func LeverageForRow(fit *AttributeFit, x []float64, w float64) float64

LeverageForRow computes hᵢᵢ for a single row of weight w given a centered-Gram inverse and the streaming predictor means. The closed-form identity for (weighted) OLS with an intercept and centered predictors is 

hᵢᵢ = wᵢ · (1/Σw + (xᵢ − μ_x)ᵀ · M2_xx⁻¹ · (xᵢ − μ_x))

— the diagonal of W½X(XᵀWX)⁻¹XᵀW½ (R hatvalues on a weighted lm), with μ_x the weighted means and M2_xx the Σw-weighted centered Gram. It is invariant to rescaling the weights, so identical across kinds; w = 1 (unweighted, Σw = n) is the unweighted 1/n + quad bit for bit.

Returns 0 when fit is nil or fit.GramInverse is nil — callers should have validated upstream. Rows with any null predictor produce a zero leverage; the attribute layer decides whether to emit zero or flag missingness via the schema.

func LinearPredictor 

func LinearPredictor(intercept float64, coefs, x []float64) float64

LinearPredictor is the ONE linear-predictor kernel: 

intercept + Σ coefs[i]·x[i]

summed left to right in predictor order. Every per-row scorer — ATTR_REG_FITTED / RESIDUAL / LEVERAGE, ATTR_REG_PREDICT — and the GLM engine's own η evaluate it, so a scored row equals the fit's own fitted value bit for bit on every platform: one expression, one summation order, one place the compiler may fuse a multiply-add. len(x) must be at least len(coefs).

func LowNEffFloor 

func LowNEffFloor(spec *types.RegressionSpec) int

LowNEffFloor is the raw-row floor p + 1 the fit itself enforces (PROCESSING_REGRESSION_INSUFFICIENT_DATA), p being the predictor count. A probability-weighted OLS fit whose Kish n_eff is at or below it reports PULSE_WEIGHT_LOW_NEFF: its residual df N* − p − 1 is ≤ 0, so σ², the standard errors, the t p-values and the adjusted R² are NaN (U12 review WS-05). A REG_GLM warns only strictly below it.

func NewScorer 

func NewScorer(b *model.Bound, p model.ScoreParams) (model.RowScorer, error)

NewScorer builds b's RowScorer for p through BuiltinScorer (b.Scorer): see newLinearScorer for the terms / bound rule and newGLMScorer for the scale.

func RegisteredTypes 

func RegisteredTypes() []types.RegressionType

RegisteredTypes returns every registered regression type. Order is map-iteration order; callers that need determinism sort the result.

func Remedies 

func Remedies() []func(offered func(string) bool) string

Remedies lists every regression remedy clause, for processing.ExtensionRegistry.ScopeRefusal.

func RunExtensionModifiers 

func RunExtensionModifiers(
	ctx context.Context,
	spec *types.RegressionSpec,
	pValues bool,
	records []Record,
	fit ExtensionFit,
	missingDeviance func() error,
	mismatch func(base, got []string) error,
) (*types.RegressionResult, error)

RunExtensionModifiers fits an extension model under spec's Selection (first) and Resample modifiers, ctx read before every refit.

Selection (the registration declares SupportsSelection) is the built-in greedy search over spec.Predictors, each candidate scored by the information criterion of its fit's reported deviance (−2·log likelihood up to a constant every candidate shares): AIC = deviance + 2k, BIC = deviance + ln(n)·k, with k the fit's coefficient count and n its n_obs. A candidate fit that reports no deviance is PULSE_EXTENSION_REGRESSION_OUTPUT_MISMATCH (reason missing_deviance) via missingDeviance; a candidate's data-dependent refusal (the per-slot codes) drops it, any other error stops the run.

Resample refits the model over the fitted rows (the target, every active predictor and, weighted, a valid positive weight present — plus the jackknife_by value) exactly as REG_OLS does: the jackknife (leave-one-out, or delete-a-group with jackknife_groups / jackknife_by) and the bootstrap with the same seed rule. Each coefficient's standard error becomes the replicate SE, its statistic the Wald z (estimate / SE) and its p-value the Wald-z two-sided p (jackknife) or the percentile p (bootstrap) — NaN on every key when pValues is false (the registration declares no p-values, so predict counts none and no multiplicity family reads them); ci_lower / ci_upper the normal interval at conf_level on the replicate SE (AttachExtensionIntervals); a spec-requested vcov becomes the replicate covariance. A replicate whose coefficient names differ from the base fit's is PULSE_EXTENSION_REGRESSION_OUTPUT_MISMATCH (reason replicate_coefficients) via mismatch.

func ValidateRegression 

func ValidateRegression(spec *types.RegressionSpec) error

ValidateRegression performs semantic spec validation that the engine itself needs but that the lighter descriptor.validateRegressions (header-only, no execution dependencies) cannot reach. Today this is the Penalty / Alpha / L1Ratio coupling introduced in Phase 2.

Returns nil on a well-formed spec; otherwise a PROCESSING_CONFIG CodedError naming the offending field. The engine factory calls this after the schema check and before constructing the accumulator so invalid combos never reach the streaming path.

Validation rules:

  • Penalty=="l1"|"l2" : require Alpha > 0 and L1Ratio == 0 (callers who want elastic-net should set Penalty="elasticnet").
  • Penalty=="elasticnet": require Alpha > 0 and 0 < L1Ratio < 1. L1Ratio == 0 → user should use l2; L1Ratio == 1 → use l1.
  • Penalty=="" (unpenalized): Alpha / L1Ratio must be zero (we reject silent-typo specs like Penalty="" with Alpha=0.5 — the intent is ambiguous).
  • Unknown Penalty values → reject. Descriptor's enum check normally catches this but ValidateRegression is also reachable from hand-built specs in tests.

A spec is then held to validateConfLevel (0 < conf_level < 1, zero the default), when it sets Vcov to validateVcov, and to the shared covariance rule regcov.Check.

Types

type AttributeFit 

type AttributeFit struct {
	// Coefficients are the fitted slopes in the same order as
	// PredictorOrder. Length equals len(PredictorOrder).
	Coefficients []float64

	// Intercept is the fitted intercept term β₀.
	Intercept float64

	// PredictorOrder echoes the spec's Predictors slice. Provided so the
	// attribute's pass-2 loop reads predictors in the same order the
	// solver used. Owned by the caller.
	PredictorOrder []string

	// MeanX is the streaming mean of each predictor over the rows that
	// contributed to the fit (listwise-deleted predictor/target nulls).
	// Length matches PredictorOrder. Populated for every fit.
	MeanX []float64

	// NObs is the number of rows that contributed to the fit (n in the
	// hat-matrix identity 1/n + …).
	NObs int

	// SumW is Σw over the rows that entered the fit — the exact float
	// of NObs unweighted (the hat-matrix identity's n).
	SumW float64

	// GramInverse is M2_xx⁻¹ — the inverse of the centered predictor
	// Gram matrix used to compute leverage. Populated only when the
	// caller requests it (ATTR_REG_LEVERAGE); nil otherwise so the
	// regularized solvers can keep their post-fit cost free of an
	// additional inverse. Stored as a p×p linalg.Sym.
	GramInverse *linalg.Sym
	// contains filtered or unexported fields
}

AttributeFit is the minimal package of post-fit quantities the per-row regression attributes (ATTR_REG_FITTED / ATTR_REG_RESIDUAL / ATTR_REG_LEVERAGE) need to emit their second-pass per-row values.

The full RegressionResult / olsSolveResult bundle is intentionally not returned: attribute callers do not surface regression result blocks; they only need the coefficient vector (for ŷᵢ / residuals) and, in the leverage case, the centered-Gram inverse plus the streaming predictor means (for the standard hat-matrix identity 

hᵢᵢ = 1/n + (xᵢ − μ_x)ᵀ · M2_xx⁻¹ · (xᵢ − μ_x)).

The struct is per-attribute state captured at finalize time; the attribute then iterates records a second time and computes the per-row scalar.

func FitForAttribute 

func FitForAttribute(
	spec *types.RegressionSpec,
	records []Record,
	needLeverage bool,
) (*AttributeFit, error)

FitForAttribute runs the OLS streaming accumulator over the supplied records, then dispatches the same per-Penalty solver the engine uses to produce coefficients + intercept. When needLeverage is true the helper additionally produces the centered-Gram inverse from the (unpenalized) Cholesky factorisation.

The helper enforces the per-attribute spec rules upstream of the solver: ATTR_REG_LEVERAGE accepts only unpenalized OLS (Penalty==""). Callers gate on that constraint before invoking with needLeverage=true; this function returns PROCESSING_CONFIG if the combination ever escapes upstream validation.

Listwise deletion mirrors olsEngine.UpdateRow: rows whose target or any predictor is null contribute nothing. The returned NObs reflects the rows that actually folded into the accumulator.

func (*AttributeFit) RowWeight 

func (f *AttributeFit) RowWeight(rec Record) (float64, bool)

RowWeight is the weight rec carried in the refit: exactly 1 unweighted; ok=false when its weight is invalid or zero (the row was excluded from the fit).

type BufferedEngine 

type BufferedEngine interface {
	Engine
	FitBuffered(records []Record) (*types.RegressionResult, error)
}

BufferedEngine is the Phase 1 entry point for buffered-orchestrator regression fits. Implementations consume an in-memory record slice and emit the populated result. Phase 0 stub engines ignore the slice and return PROCESSING_REGRESSION_NOT_IMPLEMENTED; the unpenalized olsEngine folds every record through its Welford accumulator.

type ContextBufferedEngine 

type ContextBufferedEngine interface {
	BufferedEngine
	FitBufferedContext(ctx context.Context, records []Record) (*types.RegressionResult, error)
}

ContextBufferedEngine is a BufferedEngine whose fit can run long — a Resample / Selection modifier refitting the model many times — and so reads the request ctx between refits: a request deadline (Options.Limits.RequestTimeout) or a cancel stops it with the ctx's own error. FitEachBufferedWithContext prefers it over FitBuffered.

type Engine 

type Engine interface {
	// Fit returns the per-spec result. Phase 0 stubs always return a
	// PROCESSING_REGRESSION_NOT_IMPLEMENTED CodedError; later phases
	// populate a RegressionResult on success. A non-nil error indicates
	// the orchestrator must not partially include this slot in
	// Response.Regressions.
	Fit() (*types.RegressionResult, error)
}

Engine is the per-spec regression fit contract. Phase 1+ engines implement Fit to consume the filtered record set and emit a populated RegressionResult. Phase 0 stubs return PROCESSING_REGRESSION_NOT_IMPLEMENTED.

func Build 

func Build(specs []*types.RegressionSpec, schema *encoding.Schema) ([]Engine, error)

Build returns an Engine for each spec. Unknown types surface PROCESSING_CONFIG; per-spec factory errors propagate untouched. This is the orchestrator's entry point — both streaming and buffered paths invoke Build first, then route the returned engines through the appropriate execution interface. Equivalent to BuildWith with a nil Resolver.

func BuildWith 

func BuildWith(specs []*types.RegressionSpec, schema *encoding.Schema, resolve Resolver) ([]Engine, error)

BuildWith is Build resolving each type through resolve. A type the resolver misses surfaces the same PROCESSING_CONFIG "unknown regression type" error as a type nothing registered.

type ExtensionFit 

type ExtensionFit func(recs []Record, predictors []string) (res *types.RegressionResult, keys []string, err error)

ExtensionFit is one fit of an extension model over recs with predictors as its active set: the mapped result and the fit's coefficient names in the order the model reported them. A fresh model per call — the caller builds it from the slot's spec with predictors substituted and the modifier keys cleared.

type Factory 

type Factory func(spec *types.RegressionSpec, schema *encoding.Schema) (Engine, error)

Factory constructs an Engine for a concrete RegressionSpec against a schema. Per-operator init() functions register their factory in the shared registry; Lookup returns the factory for a given type.

func Lookup 

func Lookup(t types.RegressionType) (Factory, bool)

Lookup returns the factory for a regression type, if registered.

type LabeledRecord 

type LabeledRecord interface {
	Record
	// StringValueChecked returns the field's categorical label and
	// true, or ("", false, nil) when it is null, absent or not
	// categorical; a code its dictionary has no entry for (corrupt
	// record data) is an ENCODING_INVALID error.
	StringValueChecked(name string) (string, bool, error)
}

LabeledRecord is the optional extension of Record that a robust covariance reads: a categorical cluster field's level label. The engine's processing.Record implements it. A Record without it — or a field with no label (a numeric cluster id) — is keyed by its NumericValue instead, so every Record stays usable.

type Record 

type Record interface {
	// NumericValue returns the value and true if present and non-null.
	// Returns (0, false) when the named field is null or absent on the
	// record.
	NumericValue(name string) (float64, bool)
}

Record is the minimal contract the regression engines need from the parent processing.Record. Defining it locally keeps this subpackage free of the processing import and mirrors the pattern used by internal/processing/feature.

processing.Record satisfies this interface; the orchestrator passes records through unchanged.

type Resolver 

type Resolver func(t types.RegressionType) (Factory, bool)

Resolver resolves a regression type to its factory. The engine passes processing.ExtensionRegistry.LookupRegression, which answers a type the instance feature set hides as never registered. A nil Resolver is the built-in registry alone (Lookup).

type StreamingEngine 

type StreamingEngine interface {
	UpdateRow(rec Record) error
	Finalize() (*types.RegressionResult, error)
}

StreamingEngine is the Phase 1 entry point for single-pass orchestrator regression fits. Implementations consume records one at a time via UpdateRow, then close the fit via Finalize. The orchestrator wraps the same record stream that drives streamable aggregators.

Only the unpenalized OLS engine implements this interface today. Phase 2 (regularized OLS), Phase 4 (Bayes), and Phases 5+ (modifiers) will expand the set.

func BuildStreaming 

func BuildStreaming(specs []*types.RegressionSpec, schema *encoding.Schema) ([]StreamingEngine, error)

BuildStreaming constructs StreamingEngine handles for the single-pass orchestrator. Specs whose engines do not yet implement StreamingEngine are rejected — callers are expected to gate on CanStreamRequest / RegressionSpec.Streamable() before reaching this entry point.

Returns nil, nil for an empty spec slice so the orchestrator can call it unconditionally on streaming requests.

func BuildStreamingWith 

func BuildStreamingWith(specs []*types.RegressionSpec, schema *encoding.Schema, resolve Resolver) ([]StreamingEngine, error)

BuildStreamingWith is BuildStreaming resolving each type through resolve.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.