Documentation
¶
Overview ¶
Package compact is the deterministic shell-output compactor behind the jev tier's fallback path. It is a port of the pieces of ralph's shell-output-compact.py that the fallback needs: command classification, hard passthrough for source-output families, collapse of repeated line runs and byte-budget assembly. Family compactors that summarize (git status, test runners, installers, ...) are not ported; output of those commands is left unchanged.
Index ¶
- Constants
- func Classify(command string) string
- func CollapseRuns(lines []string) ([]string, map[string]int)
- func IsSourceFamily(family string) bool
- func JevCompact(command, stdout, stderr string, exitStatus int, asker Asker, opts JevOptions) (JevResult, Result)
- type Asker
- type Body
- type CorpusCase
- type CorpusMetrics
- type JevOptions
- type JevResult
- type Options
- type OutKind
- type OutLine
- type Policy
- type Result
- type Rule
Constants ¶
const ( FamilyGitDiff = "git_diff" FamilyGitShow = "git_show" FamilyGitLog = "git_log" FamilyGrep = "grep" FamilyFind = "find" FamilyLS = "ls" FamilyTree = "tree" // FamilyGenericLarge labels output shortened by the size fallback. FamilyGenericLarge = "generic_large" )
Family ids. Only source-output families are recognized; they exist so the hard passthrough can name what it protected.
const ( StatusCompacted = "compacted" StatusNotCompacted = "not compacted" )
Status values.
const ( DefaultJevThresholdBytes = 8192 DefaultMaxLinesPerRequest = 255 )
const ( ActionNever = "never-compact" ActionDeterministic = "deterministic-only" ActionEligible = "eligible" )
const DefaultThresholdBytes = 8192
DefaultThresholdBytes is the combined output size above which the size fallback runs.
Variables ¶
This section is empty.
Functions ¶
func Classify ¶
Classify returns the source-output family of a simple command, or "" when the command is unknown, compound, or not a source-output command.
func CollapseRuns ¶
CollapseRuns replaces runs of consecutive noise lines of one class with an omission marker and returns how many lines each class dropped. Preserved lines are never part of a run. Stack traces keep their first two frames.
func IsSourceFamily ¶
IsSourceFamily reports whether a family id is passthrough-only.
func JevCompact ¶
func JevCompact(command, stdout, stderr string, exitStatus int, asker Asker, opts JevOptions) (JevResult, Result)
JevCompact first applies the deterministic eligibility and compaction guards. The model tier runs only with a verified, retrievable raw original; an unavailable model falls back to the deterministic result. Shadow mode always returns the original bytes.
Types ¶
type Body ¶
type Body struct {
// contains filtered or unexported fields
}
Body is assembled only from source indices and local, registered accounting constructors. Classifier answers never enter an append method.
func (*Body) AppendHeader ¶
func (*Body) AppendMarker ¶
func (*Body) AppendVerbatim ¶
type CorpusCase ¶
type CorpusCase struct {
Name string `json:"name"`
Kind string `json:"kind"`
Command string `json:"command"`
Exit int `json:"exit"`
Prefix string `json:"prefix"`
Repeat string `json:"repeat"`
RepeatCount int `json:"repeatCount"`
Suffix string `json:"suffix"`
Gold []string `json:"gold"`
}
CorpusCase is one labeled original. Repeat expands synthetic noise in memory so committed fixtures stay small. Gold contains exact diagnostic source lines.
func ReadCorpus ¶
func ReadCorpus(dir string) ([]CorpusCase, error)
func (CorpusCase) Original ¶
func (c CorpusCase) Original() string
type CorpusMetrics ¶
type CorpusMetrics struct {
Cases int `json:"cases"`
Failures int `json:"failures"`
GoldLines int `json:"goldLines"`
RetainedGold int `json:"retainedGold"`
FailureGoldLines int `json:"failureGoldLines"`
RetainedFailureGold int `json:"retainedFailureGold"`
BytesBefore int `json:"bytesBefore"`
BytesAfter int `json:"bytesAfter"`
}
func EvaluateCorpus ¶
func EvaluateCorpus(cases []CorpusCase, compactFn func(CorpusCase) string) CorpusMetrics
EvaluateCorpus reports recall and savings jointly for a candidate compactor.
func (CorpusMetrics) FailureRecall ¶
func (m CorpusMetrics) FailureRecall() float64
func (CorpusMetrics) Promotable ¶
func (m CorpusMetrics) Promotable() bool
func (CorpusMetrics) Recall ¶
func (m CorpusMetrics) Recall() float64
func (CorpusMetrics) Saved ¶
func (m CorpusMetrics) Saved() int
type JevOptions ¶
type JevOptions struct {
Enabled bool
Shadow bool
ThresholdBytes int
MaxLinesPerRequest int
StateDir string
AuthoritativeExit bool
// RawPointer must contain the complete, byte-identical original output.
RawPointer string
Runtime string
PolicyRuleID string
Policy *Policy
}
JevOptions controls optional model-assisted compaction. The zero value is safe: compaction is disabled by default.
type Options ¶
type Options struct {
// ThresholdBytes is the combined stdout+stderr size at or below which
// output is returned unchanged. Zero or negative selects the default.
ThresholdBytes int
// Policy can further restrict or tune eligibility. It cannot override hard
// source/binary protections, which are checked first.
Policy *Policy
}
Options tunes Compact. The zero value uses the defaults.
type Policy ¶
Policy is a declarative local compaction policy. Rules can only narrow eligibility or select the existing deterministic path; they never execute code or override hard source/binary guards.
func LoadPolicy ¶
LoadPolicy parses a bounded declarative policy. A project policy may only add never-compact rules, because repository content is untrusted.
type Result ¶
type Result struct {
Stdout string
Stderr string
Compacted bool
StdoutCompacted bool
StderrCompacted bool
// Family is the source-output family that forced passthrough, or
// FamilyGenericLarge when the size fallback shortened the output.
Family string
Status string
ExitStatus int
}
Result is the outcome of Compact.
func Compact ¶
Compact shortens oversized command output deterministically.
- Output containing a NUL byte is binary and passes through.
- Source-output families (git diff/show/log, grep/rg, find, ls, tree) pass through verbatim, by command or, when the command is empty, by output shape. Nothing re-enables compaction of them.
- Source-output commands in general (cat, sed, jq, ...), && chains, $() and background jobs are also left alone.
- Otherwise, output over the byte threshold has runs of repeated noise lines collapsed and is windowed to head and tail lines plus the error/summary lines from the elided middle. The result is used only if it is strictly smaller than the input.