compact

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 6, 2026 License: MIT Imports: 14 Imported by: 0

Documentation

Overview

Package compact is the deterministic shell-output compactor behind the jev tier's fallback path. It is a port of the pieces of ralph's shell-output-compact.py that the fallback needs: command classification, hard passthrough for source-output families, collapse of repeated line runs and byte-budget assembly. Family compactors that summarize (git status, test runners, installers, ...) are not ported; output of those commands is left unchanged.

Index

Constants

View Source
const (
	FamilyGitDiff = "git_diff"
	FamilyGitShow = "git_show"
	FamilyGitLog  = "git_log"
	FamilyGrep    = "grep"
	FamilyFind    = "find"
	FamilyLS      = "ls"
	FamilyTree    = "tree"
	// FamilyGenericLarge labels output shortened by the size fallback.
	FamilyGenericLarge = "generic_large"
)

Family ids. Only source-output families are recognized; they exist so the hard passthrough can name what it protected.

View Source
const (
	StatusCompacted    = "compacted"
	StatusNotCompacted = "not compacted"
)

Status values.

View Source
const (
	DefaultJevThresholdBytes  = 8192
	DefaultMaxLinesPerRequest = 255
)
View Source
const (
	ActionNever         = "never-compact"
	ActionDeterministic = "deterministic-only"
	ActionEligible      = "eligible"
)
View Source
const DefaultThresholdBytes = 8192

DefaultThresholdBytes is the combined output size above which the size fallback runs.

Variables

This section is empty.

Functions

func Classify

func Classify(command string) string

Classify returns the source-output family of a simple command, or "" when the command is unknown, compound, or not a source-output command.

func CollapseRuns

func CollapseRuns(lines []string) ([]string, map[string]int)

CollapseRuns replaces runs of consecutive noise lines of one class with an omission marker and returns how many lines each class dropped. Preserved lines are never part of a run. Stack traces keep their first two frames.

func IsSourceFamily

func IsSourceFamily(family string) bool

IsSourceFamily reports whether a family id is passthrough-only.

func JevCompact

func JevCompact(command, stdout, stderr string, exitStatus int, asker Asker, opts JevOptions) (JevResult, Result)

JevCompact first applies the deterministic eligibility and compaction guards. The model tier runs only with a verified, retrievable raw original; an unavailable model falls back to the deterministic result. Shadow mode always returns the original bytes.

Types

type Asker

type Asker interface {
	Ask(ctx context.Context, req jev.Request) (*jev.Response, error)
}

Asker is the slice of the Jev client the compactor uses.

type Body

type Body struct {
	// contains filtered or unexported fields
}

Body is assembled only from source indices and local, registered accounting constructors. Classifier answers never enter an append method.

func (*Body) AppendHeader

func (b *Body) AppendHeader(locus, outcome, kind string, level int)

func (*Body) AppendMarker

func (b *Body) AppendMarker(omitted int)

func (*Body) AppendVerbatim

func (b *Body) AppendVerbatim(srcIdx int, orig []string)

func (*Body) String

func (b *Body) String() string

func (*Body) Verify

func (b *Body) Verify(orig []string) bool

type CorpusCase

type CorpusCase struct {
	Name        string   `json:"name"`
	Kind        string   `json:"kind"`
	Command     string   `json:"command"`
	Exit        int      `json:"exit"`
	Prefix      string   `json:"prefix"`
	Repeat      string   `json:"repeat"`
	RepeatCount int      `json:"repeatCount"`
	Suffix      string   `json:"suffix"`
	Gold        []string `json:"gold"`
}

CorpusCase is one labeled original. Repeat expands synthetic noise in memory so committed fixtures stay small. Gold contains exact diagnostic source lines.

func ReadCorpus

func ReadCorpus(dir string) ([]CorpusCase, error)

func (CorpusCase) Original

func (c CorpusCase) Original() string

type CorpusMetrics

type CorpusMetrics struct {
	Cases               int `json:"cases"`
	Failures            int `json:"failures"`
	GoldLines           int `json:"goldLines"`
	RetainedGold        int `json:"retainedGold"`
	FailureGoldLines    int `json:"failureGoldLines"`
	RetainedFailureGold int `json:"retainedFailureGold"`
	BytesBefore         int `json:"bytesBefore"`
	BytesAfter          int `json:"bytesAfter"`
}

func EvaluateCorpus

func EvaluateCorpus(cases []CorpusCase, compactFn func(CorpusCase) string) CorpusMetrics

EvaluateCorpus reports recall and savings jointly for a candidate compactor.

func (CorpusMetrics) FailureRecall

func (m CorpusMetrics) FailureRecall() float64

func (CorpusMetrics) Promotable

func (m CorpusMetrics) Promotable() bool

func (CorpusMetrics) Recall

func (m CorpusMetrics) Recall() float64

func (CorpusMetrics) Saved

func (m CorpusMetrics) Saved() int

type JevOptions

type JevOptions struct {
	Enabled            bool
	Shadow             bool
	ThresholdBytes     int
	MaxLinesPerRequest int
	StateDir           string
	AuthoritativeExit  bool
	// RawPointer must contain the complete, byte-identical original output.
	RawPointer   string
	Runtime      string
	PolicyRuleID string
	Policy       *Policy
}

JevOptions controls optional model-assisted compaction. The zero value is safe: compaction is disabled by default.

type JevResult

type JevResult struct {
	Body string
	Used bool
	Err  error
}

type Options

type Options struct {
	// ThresholdBytes is the combined stdout+stderr size at or below which
	// output is returned unchanged. Zero or negative selects the default.
	ThresholdBytes int
	// Policy can further restrict or tune eligibility. It cannot override hard
	// source/binary protections, which are checked first.
	Policy *Policy
}

Options tunes Compact. The zero value uses the defaults.

type OutKind

type OutKind uint8
const (
	OutVerbatim OutKind = iota
	OutMarker
	OutHeader
)

type OutLine

type OutLine struct {
	Kind        OutKind
	Text        string
	SrcIdx      int
	Omitted     int
	Locus       string
	Outcome     string
	ContentKind string
	Level       int
}

type Policy

type Policy struct {
	Version int    `yaml:"version"`
	Rules   []Rule `yaml:"rules"`
}

Policy is a declarative local compaction policy. Rules can only narrow eligibility or select the existing deterministic path; they never execute code or override hard source/binary guards.

func LoadPolicy

func LoadPolicy(path string, project bool) (*Policy, error)

LoadPolicy parses a bounded declarative policy. A project policy may only add never-compact rules, because repository content is untrusted.

func (*Policy) Match

func (p *Policy) Match(command, output string) *Rule

Match returns the first matching rule. Hard protections are evaluated by Compact before a policy result can ever be acted on.

type Result

type Result struct {
	Stdout          string
	Stderr          string
	Compacted       bool
	StdoutCompacted bool
	StderrCompacted bool
	// Family is the source-output family that forced passthrough, or
	// FamilyGenericLarge when the size fallback shortened the output.
	Family     string
	Status     string
	ExitStatus int
}

Result is the outcome of Compact.

func Compact

func Compact(command, stdout, stderr string, exitStatus int, opts Options) Result

Compact shortens oversized command output deterministically.

  • Output containing a NUL byte is binary and passes through.
  • Source-output families (git diff/show/log, grep/rg, find, ls, tree) pass through verbatim, by command or, when the command is empty, by output shape. Nothing re-enables compaction of them.
  • Source-output commands in general (cat, sed, jq, ...), && chains, $() and background jobs are also left alone.
  • Otherwise, output over the byte threshold has runs of repeated noise lines collapsed and is windowed to head and tail lines plus the error/summary lines from the elided middle. The result is used only if it is strictly smaller than the input.

type Rule

type Rule struct {
	ID        string `yaml:"id"`
	Command   string `yaml:"command,omitempty"`
	Output    string `yaml:"output,omitempty"`
	Action    string `yaml:"action"`
	Threshold int    `yaml:"threshold_bytes,omitempty"`
	// contains filtered or unexported fields
}

func (*Rule) String

func (r *Rule) String() string

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL