Documentation
¶
Overview ¶
Package limits is the one definition of Pulse's instance resource limits: the Limits struct the root facade aliases as pulse.Limits, the built-in defaults, the per-field precedence resolver, the construction-time validation and the shared constructors every PULSE_LIMIT_* raise site uses.
It is a leaf over the standard library and Pulse's errors package so that predict (internal/descriptor, no-execute) and the runtime (internal/service, internal/processing) can both import it and agree on every rule.
Encoding, identical on every input layer (Options, feature profile, --limit): 0 means "use the default", Unlimited (-1) means no limit, and any other negative value is refused. An EFFECTIVE (resolved) Limits never carries 0: every field is either a positive bound or Unlimited.
Index ¶
- Constants
- Variables
- func Check(l Limits, n Name, observed int64) *errors.CodedError
- func CheckChainStages(l Limits, stages int) *errors.CodedError
- func CheckComposeSlots(l Limits, slots int) *errors.CodedError
- func CheckJoinBuildRows(l Limits, rows int64) *errors.CodedError
- func CheckMatrixDim(l Limits, p int) *errors.CodedError
- func Default(n Name) int64
- func Digest(l Limits) string
- func EstimateMemory(in MemoryInputs) int64
- func Exceeded(n Name, configured, observed int64) *errors.CodedError
- func FirstCertain(fs []Finding) *errors.CodedError
- func Invalid(n Name, value int64) *errors.CodedError
- func IsUnlimited(v int64) bool
- func MatrixStateBytes(blocks, buckets, accumulatorBytes int64) int64
- func MergeBlocks(shardRecords ...int64) int64
- func Option(n Name) string
- func ParseValue(n Name, s string) (int64, error)
- func RecordBytes(schema *encoding.Schema) int64
- func Set(l *Limits, n Name, v int64) bool
- func Spell(n Name, v int64) string
- func Unit(n Name) string
- func Validate(l Limits) error
- func Value(l Limits, n Name) int64
- type Finding
- type Grade
- type Limits
- type MemoryArm
- type MemoryInputs
- type Name
Constants ¶
const ( DefaultRequestTimeout time.Duration = Unlimited DefaultMaxGroups int64 = 10_000_000 DefaultMaxCrosstabCells int64 = 10_000_000 DefaultMaxEstimatedMemory int64 = Unlimited DefaultMaxMatrixDim int64 = 2_048 DefaultMaxComposeSlots int64 = 1_000 DefaultMaxChainStages int64 = 1_000 DefaultMaxJoinBuildRows int64 = 100_000_000 )
Built-in defaults. RequestTimeout and MaxEstimatedMemory are opt-in (Unlimited by default); the others are set high enough that no legitimate request trips them.
const ( UnitNanoseconds = "nanoseconds" UnitBytes = "bytes" UnitCount = "count" )
Units a limit value is expressed in, as the manifest reports them.
const DigestPrefix = "lim1:"
DigestPrefix versions the limits_digest algorithm. A change to the hashed form (not to a limit value) bumps it.
const MergeBlockSize = 4096
MergeBlockSize mirrors linalg.MergeBlockSize (the blocked-merge block length in records; a run holds one CoMoment per populated block per bucket). The leaf may not import linalg, so the value is restated here and pinned equal by TestMergeBlockSizeMatchesLinalg.
const Unlimited = -1
Unlimited disables a limit. Untyped, so it is assignable to both the time.Duration and the int64 fields.
Variables ¶
var ErrUnknownName = stderrors.New("unknown limit")
ErrUnknownName is returned by ParseName for a key that names no limit.
Functions ¶
func Check ¶
func Check(l Limits, n Name, observed int64) *errors.CodedError
Check is the runtime rule for an exact count: PULSE_LIMIT_EXCEEDED when observed exceeds the effective limit n in l, nil otherwise (including when the limit is Unlimited).
func CheckChainStages ¶
func CheckChainStages(l Limits, stages int) *errors.CodedError
CheckChainStages refuses a ProcessChain call carrying more than MaxChainStages stages.
func CheckComposeSlots ¶
func CheckComposeSlots(l Limits, slots int) *errors.CodedError
CheckComposeSlots refuses a Compose call carrying more than MaxComposeSlots requests.
func CheckJoinBuildRows ¶
func CheckJoinBuildRows(l Limits, rows int64) *errors.CodedError
CheckJoinBuildRows refuses a join whose build (right) side holds more than MaxJoinBuildRows records. The runtime calls it twice: on the header-only right-side count before the build decodes a record, and on the running decode count as a backstop for a miscount.
func CheckMatrixDim ¶
func CheckMatrixDim(l Limits, p int) *errors.CodedError
CheckMatrixDim refuses a matrix whose dimension p (its resolved member count) exceeds MaxMatrixDim.
func Default ¶
Default returns the limit's built-in default (nanoseconds for RequestTimeout); 0 for an unknown name.
func Digest ¶
Digest returns the limits_digest of the effective limits l: DigestPrefix + sha256hex over "name=value\n" lines in declaration order. Two instances share a digest iff every effective limit is equal; it is independent of the feature-set digest.
func EstimateMemory ¶
func EstimateMemory(in MemoryInputs) int64
EstimateMemory is the upper-bound resident footprint of one run, in bytes, per arm:
every arm base + 3 x records x stride (the payload + GC headroom)
+ join right rows x (3 x stride + 2 x record bytes)
+ matrix merge-block state
buffered + records x (record bytes + 16 x aggs + 128 x derived)
+ buckets x (512 + 256 x aggs) + cells x 512
streaming + buckets x (512 + 256 x aggs)
fused + cells x 512
plus records x 96 per value-state aggregation on every arm. Saturates at math.MaxInt64 rather than wrapping.
func Exceeded ¶
func Exceeded(n Name, configured, observed int64) *errors.CodedError
Exceeded builds the PULSE_LIMIT_EXCEEDED error every raise site returns, so the details are identical everywhere and the message carries the configured value. Details: {limit, configured, observed, option}; RequestTimeout values are nanoseconds.
func FirstCertain ¶
func FirstCertain(fs []Finding) *errors.CodedError
FirstCertain returns the error of the first Certain finding in fs — the pre-flight refusal — or nil when none is certain.
func Invalid ¶
func Invalid(n Name, value int64) *errors.CodedError
Invalid builds the PULSE_LIMIT_INVALID error for a refused value. Details: {limit, value}.
func IsUnlimited ¶
IsUnlimited reports whether an effective limit value disables the limit (Unlimited, or a non-positive value that never resolves).
func MatrixStateBytes ¶
MatrixStateBytes is one matrix's merge-block state: blocks x buckets x accumulatorBytes — one linalg.CoMoment per populated merge block per bucket, held until finalize. Saturating.
func MergeBlocks ¶
MergeBlocks is the number of merge blocks a run over the given per-shard record counts populates: ceil(n / MergeBlockSize) per shard, since block numbering restarts per shard. A single-file cohort passes one count.
func Option ¶
Option returns the Go spelling of the limit's option, "Options.Limits.<Field>"; empty for an unknown name.
func ParseValue ¶
ParseValue parses one input-layer value of limit n as written in a text form (`pulse mcp --limit`, the feature profile's request_timeout string): "unlimited" or "-1" is Unlimited, "0" is "use the default"; request_timeout takes a Go duration ("30s", "2m"), every other limit a base-10 integer (underscores allowed: "10_000_000"). A malformed value, or any negative other than Unlimited, is refused.
func RecordBytes ¶
RecordBytes is the resident size, in bytes, of one buffered record decoded over schema at full width (a projected decode holds fewer slots, so this bounds it): the record header, one slot per field and per set-mask / parent-group word, the bit planes, and the aux block for decimal128 fields.
func Set ¶
Set stores v in the limit's field of l. It reports false (and leaves l untouched) for an unknown name.
func Spell ¶
Spell renders an effective limit value in the text form ParseValue accepts: "unlimited" for a disabled limit, a Go duration for RequestTimeout, a plain integer otherwise — so a reported value can be pasted back into `--limit name=value`.
func Unit ¶
Unit returns the unit a limit's value is expressed in: nanoseconds for RequestTimeout, bytes for MaxEstimatedMemory, count otherwise; empty for an unknown name.
Types ¶
type Finding ¶
Finding is one limit an estimate exceeds. Estimated is the figure predict (or the pre-flight) computed, in the limit's unit.
func Evaluate ¶
Evaluate grades one estimate against the effective limit n in l: a finding (and true) when the limit is bounded and estimated exceeds it, else false. A limit equal to its bound is not exceeded.
func (Finding) Err ¶
func (f Finding) Err() *errors.CodedError
Err is the PULSE_LIMIT_EXCEEDED error a breach of f raises, with the estimate as the observed figure — what the runtime reports for a certain finding.
type Grade ¶
type Grade string
Grade is how sure a predict-time limit finding is that the run would breach the limit.
const ( // Certain: the estimate is the exact figure the runtime will see // (a matrix dimension, a Compose slot count, a chain stage count). // A certain finding refuses the request — predict marks it invalid // and the process pre-flight raises PULSE_LIMIT_EXCEEDED before any // record is decoded. Certain Grade = "certain" // Possible: the estimate is an upper bound (a Cartesian product of // axis sizes, a dictionary size clamped by the record count). A // possible finding is a warning only: the request may still run. Possible Grade = "possible" )
type Limits ¶
type Limits struct {
// RequestTimeout bounds the wall-clock time of one top-level call
// (Process, ProcessStream / ProcessStreamResult, Facet /
// FacetSchema, the whole Compose / ComposeParallel / ProcessChain
// call). A trip is PULSE_LIMIT_EXCEEDED; the caller's own ctx
// deadline or cancel passes through unchanged. Default: no timeout.
RequestTimeout time.Duration
// MaxGroups bounds the number of distinct group buckets one grouped
// run may create. Default 10,000,000.
MaxGroups int64
// MaxCrosstabCells bounds rows x columns of one crosstab. Default
// 10,000,000.
MaxCrosstabCells int64
// MaxEstimatedMemory bounds the predicted in-memory footprint of a
// run, in bytes. The estimate is an upper bound. Default: no limit.
MaxEstimatedMemory int64
// MaxMatrixDim bounds the dimension p of one requested matrix.
// Default 2,048.
MaxMatrixDim int64
// MaxComposeSlots bounds the number of requests in one Compose call.
// Default 1,000.
MaxComposeSlots int64
// MaxChainStages bounds the number of stages in one ProcessChain
// call. Default 1,000.
MaxChainStages int64
// MaxJoinBuildRows bounds the record count of a join's build (right)
// side, which is held in memory. Default 100,000,000.
MaxJoinBuildRows int64
}
Limits holds the instance resource limits. On an input layer a zero field means "use the default" and Unlimited (-1) means no limit; any other negative value is refused at pulse.New with PULSE_LIMIT_INVALID. Requests can never override an instance limit.
func Resolve ¶
Resolve returns the effective limits, field by field: a non-zero opts value wins, else a non-zero profile value, else the built-in default. profile may be nil (no feature profile, or a profile without a `limits` section). Pure: neither input is validated or modified — Validate the inputs first.
type MemoryArm ¶
type MemoryArm string
MemoryArm is the execution arm a request runs on, which decides the shape of its resident state.
const ( // ArmBuffered materialises every record before processing // (non-streamable Process, the buffered crosstab): O(records). ArmBuffered MemoryArm = "buffered" // ArmStreaming folds records into per-bucket state as they decode: // O(buckets). ArmStreaming MemoryArm = "streaming" // ArmFusedCrosstab folds records into the crosstab grid in the // decode loop: O(cells). ArmFusedCrosstab MemoryArm = "fused_crosstab" )
type MemoryInputs ¶
type MemoryInputs struct {
// Arm is the execution arm the request runs on.
Arm MemoryArm
// Records is the record count the arm reads (a join's left side).
Records int64
// Stride is the schema's physical record stride
// (encoding.Schema.RecordByteSize) — the payload bytes per record.
Stride int64
// RecordBytes is the resident size of one buffered record over the
// schema (RecordBytes).
RecordBytes int64
// Buckets is the group-bucket bound (1 when ungrouped).
Buckets int64
// Aggregations is the number of aggregator slots (the crosstab
// cell counts as one).
Aggregations int64
// ValueStateAggregations is how many of them keep per-value state
// (distinct / mode / frequency families).
ValueStateAggregations int64
// DerivedColumns is the number of attribute, feature and window
// outputs added to every record.
DerivedColumns int64
// Cells is the crosstab grid bound (0 without a crosstab).
Cells int64
// JoinRightRows is a join build side's record count (0 without a
// join); each is a materialised record of RecordBytes plus its
// payload.
JoinRightRows int64
// MatrixBytes is the matrices' merge-block state (MatrixStateBytes,
// summed over every matrix).
MatrixBytes int64
}
MemoryInputs are the figures EstimateMemory reads. Counts are non-negative; a figure the caller cannot know makes the estimate unknowable, and the caller reports no finding instead of guessing.
type Name ¶
type Name string
Name is a limit's snake_case key — the spelling the feature profile `limits` section, `pulse mcp --limit` and every PULSE_LIMIT_* error's `limit` detail use.
const ( RequestTimeout Name = "request_timeout" MaxGroups Name = "max_groups" MaxCrosstabCells Name = "max_crosstab_cells" MaxEstimatedMemory Name = "max_estimated_memory" MaxMatrixDim Name = "max_matrix_dim" MaxComposeSlots Name = "max_compose_slots" MaxChainStages Name = "max_chain_stages" MaxJoinBuildRows Name = "max_join_build_rows" )
The limit names, in declaration order.
Source Files