arms

package
v0.51.0-rc.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 15, 2026 License: MIT Imports: 20 Imported by: 0

Documentation

Overview

Package arms implements the encoding arms of the discovery-effectiveness profiler (spec 083). An arm is one deterministic way of rendering tool definitions into agent-context text (full JSON baseline, compact signature, TSCG, TOON, ...). Arms are compared on frozen corpora for token cost and — when they alter what the retrieval index ingests — for retrieval quality.

The behavioral contract every arm must satisfy lives in specs/083-discovery-profiler/contracts/arm-interface.md. In short: byte-deterministic output (FR-010), explicit errors instead of silent truncation (FR-009), self-reported lower-bound labeling when descriptions are dropped, an explicit index-ingestion mapping (FR-008), and ErrArmUnavailable at registry-resolution time when an external runtime is missing (FR-006).

Index

Constants

View Source 
const (
	PayloadClassTabular    = "tabular"
	PayloadClassNonTabular = "non_tabular"
)

PayloadClassTabular / PayloadClassNonTabular are the fixture classification hints (datasets/README.md rule: tabular = a uniform JSON array of flat objects; everything else is non_tabular).

View Source 
const BaselineName = "baseline_json"

BaselineName is the registry key of the full-JSON baseline arm.

View Source 
const CompactName = "compact_sig"

CompactName is the registry key of the compact-signature arm.

View Source 
const TSCGName = "tscg"

TSCGName is the registry key of the TSCG-compiled arm.

View Source 
const ToonListingName = "toon_listing"

ToonListingName is the registry key of the TOON tool-listing arm.

View Source 
const ToonResultsName = "toon_results"

ToonResultsName is the report key of the fixture-driven TOON results arm.

View Source 
const TronName = "tron_dedup"

TronName is the registry key of the TRON named-class schema-dedup arm.

Variables

View Source 
var ErrArmUnavailable = errors.New("arm runtime unavailable")

ErrArmUnavailable is returned (wrapped, with a human-readable reason) when an arm's external runtime is missing — e.g. the TSCG arm without a node binary or bench/tscg/node_modules. It is surfaced at registry-resolution time, before any tool is processed, so the harness reports an arm-level skip-with-reason instead of per-tool failures (contract rule 5).

Functions

func CanonicalJSON 

func CanonicalJSON(raw json.RawMessage) (string, error)

CanonicalJSON re-encodes raw JSON bytes into the canonical form used everywhere in the bench (research D7b): object keys sorted lexicographically at every depth, array order preserved, number literals preserved verbatim (via json.Number — no float round-trip), compact (no insignificant whitespace), and no HTML escaping. Identical JSON content in any key order canonicalizes to identical bytes (FR-010).

The implementation lives in bench.CanonicalJSON so every schema ingestion boundary (corpus loaders, live fetch) canonicalizes with the SAME function the arms render with — contract parity between the baseline arm and Tokenizer.CountToolWithSchema. This alias keeps the arms-package API.

func CanonicalToolText 

func CanonicalToolText(t bench.Tool) (string, error)

CanonicalToolText is THE canonical full-definition renderer (research D7b): name + "\n" + description, plus "\n" + canonical-JSON schema when the tool has one — byte-identical to the text shape the existing Tokenizer.CountToolWithSchema counts for a canonical-schema tool. This single renderer feeds the baseline_json arm, the naive full-menu count, the proxy-menu count, savings-% denominators, and break-even inputs, so every savings percentage shares one denominator.

func MustRegister 

func MustRegister(a Arm)

MustRegister adds an arm to the package default registry, panicking on a registration bug (duplicate/invalid name) — a programmer error, caught by any test importing the package.

func Names 

func Names() []string

Names lists the package default registry's arms in sorted order.

func Register 

func Register(a Arm) error

Register adds an arm to the package default registry.

Types

type Arm 

type Arm interface {
	// Name is the unique registry key (lowercase snake_case, stable across
	// releases): baseline_json, compact_sig, tscg, toon_listing, toon_results,
	// tron_dedup.
	Name() string

	// IndexAltering reports whether the arm changes any text the retrieval
	// index ingests. True obligates retrieval-quality scoring (FR-008); the
	// two-sided contract test (T022) verifies the declaration against
	// EncodeIndexMetadata diffs on corpus_v2.
	IndexAltering() bool

	// LowerBound reports whether the arm drops or truncates descriptions, so
	// its savings are rendered as a lower-bound estimate (contract rule 3).
	LowerBound() bool

	// EncodeTool renders one tool definition. Byte-deterministic; an
	// unencodable tool returns an error (counted as a skip), never a silently
	// truncated encoding.
	EncodeTool(t bench.Tool) (string, error)

	// EncodeListing renders a whole-response tool listing. Formats with a
	// shared preamble or dictionary (TRON classes, TOON header) amortize it
	// here, not per-tool (contract rule 6).
	EncodeListing(ts []bench.Tool) (string, error)

	// EncodeIndexMetadata returns the exact Name/ServerName/Description/
	// ParamsJSON the production index (internal/index.Manager.BatchIndexTools →
	// BleveIndex.IndexTool) ingests for this arm. It is the single mapping the
	// armindex builder and the IndexAltering contract test consume. Rendering-
	// only arms return the tool's fields unchanged.
	EncodeIndexMetadata(t bench.Tool) (config.ToolMetadata, error)
}

Arm is one deterministic tool-definition encoding under measurement.

func Resolve 

func Resolve(name string) (Arm, error)

Resolve resolves an arm from the package default registry.

type AvailabilityChecker 

type AvailabilityChecker interface {
	Available() error
}

AvailabilityChecker is an optional interface for arms with external runtimes. Registry.Resolve calls Available() and propagates its error, which must wrap ErrArmUnavailable when the runtime is missing.

type BaselineArm 

type BaselineArm struct{}

BaselineArm is the mandatory baseline_json arm: the canonical full-definition rendering every other arm's savings are measured against.

func NewBaseline 

func NewBaseline() *BaselineArm

NewBaseline returns the baseline_json arm.

func (*BaselineArm) EncodeIndexMetadata 

func (*BaselineArm) EncodeIndexMetadata(t bench.Tool) (config.ToolMetadata, error)

EncodeIndexMetadata implements Arm: a rendering-only arm returns the tool's production index fields unchanged — the same Name/ServerName/Description/ ParamsJSON mapping internal/upstream/core builds from a live tools/list (ParamsJSON is the tool's input-schema JSON, empty when the tool has none).

func (*BaselineArm) EncodeListing 

func (a *BaselineArm) EncodeListing(ts []bench.Tool) (string, error)

EncodeListing implements Arm: per-tool renderings joined by a fixed separator (the baseline has no shared preamble to amortize).

func (*BaselineArm) EncodeTool 

func (*BaselineArm) EncodeTool(t bench.Tool) (string, error)

EncodeTool implements Arm via the canonical full-definition renderer.

func (*BaselineArm) IndexAltering 

func (*BaselineArm) IndexAltering() bool

IndexAltering implements Arm: the baseline IS the production rendering, so it never alters what the index ingests.

func (*BaselineArm) LowerBound 

func (*BaselineArm) LowerBound() bool

LowerBound implements Arm: descriptions are preserved verbatim.

func (*BaselineArm) Name 

func (*BaselineArm) Name() string

Name implements Arm.

type CompactArm 

type CompactArm struct{}

CompactArm is the compact-signature arm: each tool renders as `name(param:type, opt?:type)|description` — required parameters bare, optional "?"-suffixed, description preserved verbatim. Pre-measured at −92% on live retrieve_tools responses with recall unchanged (research D2).

func NewCompact 

func NewCompact() *CompactArm

NewCompact returns the compact_sig arm.

func (*CompactArm) EncodeIndexMetadata 

func (*CompactArm) EncodeIndexMetadata(t bench.Tool) (config.ToolMetadata, error)

EncodeIndexMetadata implements Arm: Name, ServerName, and Description are ingested unchanged; ParamsJSON is replaced by the compact params text (the parenthesized portion of the signature) — the exact parameter text the index sees under this arm.

func (*CompactArm) EncodeListing 

func (a *CompactArm) EncodeListing(ts []bench.Tool) (string, error)

EncodeListing implements Arm: per-tool signatures joined by the shared listing separator (no preamble to amortize), so listing totals decompose exactly into per-tool costs.

func (*CompactArm) EncodeTool 

func (a *CompactArm) EncodeTool(t bench.Tool) (string, error)

EncodeTool implements Arm.

func (*CompactArm) IndexAltering 

func (*CompactArm) IndexAltering() bool

IndexAltering implements Arm: the arm replaces the ParamsJSON the retrieval index ingests with the compact params text, so retrieval-quality scoring is obligatory (FR-008).

func (*CompactArm) LowerBound 

func (*CompactArm) LowerBound() bool

LowerBound implements Arm: descriptions are preserved verbatim, so the measured savings are exact, not a lower bound.

func (*CompactArm) Name 

func (*CompactArm) Name() string

Name implements Arm.

type Registry 

type Registry struct {
	// contains filtered or unexported fields
}

Registry holds the registered encoding arms.

func NewRegistry 

func NewRegistry() *Registry

NewRegistry returns an empty arm registry.

func (*Registry) Names 

func (r *Registry) Names() []string

Names returns the registered arm names in sorted (deterministic) order.

func (*Registry) Register 

func (r *Registry) Register(a Arm) error

Register adds an arm; duplicate or non-snake_case names are rejected.

func (*Registry) Resolve 

func (r *Registry) Resolve(name string) (Arm, error)

Resolve returns the named arm, checking runtime availability first: an arm implementing AvailabilityChecker whose Available() fails is reported unavailable here — before any tool is processed — so the harness can record an arm-level skip-with-reason (contract rule 5).

type ResultFixture 

type ResultFixture struct {
	ToolID           string          `json:"tool_id"`
	PayloadClassHint string          `json:"payload_class_hint"`
	Payload          json.RawMessage `json:"payload"`
}

ResultFixture is one captured tool-call output payload.

type ResultFixtureSet 

type ResultFixtureSet struct {
	Captured  string          `json:"captured"`
	FixtureID string          `json:"fixture_id"`
	Results   []ResultFixture `json:"results"`
}

ResultFixtureSet mirrors datasets/result_fixtures_v1.json: a frozen, versioned snapshot of deterministic tool-call outputs (T037).

func LoadResultFixtures 

func LoadResultFixtures(path string) (*ResultFixtureSet, error)

LoadResultFixtures reads and validates a result-fixture snapshot. Validation is strict and per-record (the fixture set is frozen and self-describing): missing identity fields, empty/duplicate tool IDs, unknown classification hints, invalid payload JSON, and a tabular hint on a non-array payload are all explicit errors naming the offending record.

type TSCGArm 

type TSCGArm struct {
	// contains filtered or unexported fields
}

TSCGArm measures the reference TSCG implementation (@tscg/core) by spawning the committed Node shim bench/tscg/shim.mjs once per Encode call and exchanging JSONL records keyed by tool_id (research D3). TSCG is a pure deterministic compiler with pinned options, so identical input bytes always produce identical output bytes (FR-010).

func NewTSCG 

func NewTSCG() *TSCGArm

NewTSCG constructs the tscg arm, locating bench/tscg by walking up from the working directory (tests run in bench/arms; the bench CLI runs from the repo root) and resolving the node binary from PATH. Both checks happen here, at construction; Available() reports the cached verdict.

func NewTSCGAt 

func NewTSCGAt(shimDir string) *TSCGArm

NewTSCGAt constructs the tscg arm against an explicit shim directory (tests use this to exercise the unavailable path).

func (*TSCGArm) Available 

func (a *TSCGArm) Available() error

Available implements AvailabilityChecker with the construction-time verdict.

func (*TSCGArm) EncodeIndexMetadata 

func (a *TSCGArm) EncodeIndexMetadata(t bench.Tool) (config.ToolMetadata, error)

EncodeIndexMetadata implements Arm (FR-008). The compiled text has a fixed three-part shape (verified against @tscg/core@1.4.3 output on all of corpus_v2): 

<name>: <rewritten description>
  <param>[*] (<type>): <desc> | <param2> ...   ← absent for parameterless tools
[CLOSURE:<name>(<required params>)]

Mapping decision: Name stays the tool name (TSCG never re-encodes it, and the "<name>: " header prefix would duplicate the Name field the index already ingests); Description gets the TSCG-rewritten prose (the compiled header may span multiple lines — e.g. sequential-thinking — so the split is structural, not line-count-based); ParamsJSON gets the compiled parameter representation — the trailing run of two-space-indented parameter lines plus the CLOSURE signature line — replacing the JSON schema, mirroring how the compiled text itself represents params. The three fields reconstruct the compiled encoding exactly (Name + ": " + Description + "\n" + ParamsJSON), so the index ingests precisely what this arm renders: nothing more, nothing less.

func (*TSCGArm) EncodeListing 

func (a *TSCGArm) EncodeListing(ts []bench.Tool) (string, error)

EncodeListing implements Arm: one node spawn for the whole listing, per-tool compilations joined by the shared separator (TSCG has no listing-level preamble or dictionary to amortize — contract rule 6).

func (*TSCGArm) EncodeTool 

func (a *TSCGArm) EncodeTool(t bench.Tool) (string, error)

EncodeTool implements Arm: a batch of one through the shim.

func (*TSCGArm) IndexAltering 

func (*TSCGArm) IndexAltering() bool

IndexAltering implements Arm: TSCG re-encodes both the description prose and the parameter text, so retrieval-quality scoring is obligatory (FR-008).

func (*TSCGArm) LowerBound 

func (*TSCGArm) LowerBound() bool

LowerBound implements Arm: the balanced profile rewrites descriptions and elides filler phrases (measured on corpus_v2: e.g. filesystem:read_text_file loses "Use this tool when you need to"), so savings are a lower-bound estimate per contract rule 3.

func (*TSCGArm) Name 

func (*TSCGArm) Name() string

Name implements Arm.

type ToonListingArm 

type ToonListingArm struct{}

ToonListingArm renders tool listings as TOON (Token-Oriented Object Notation, https://github.com/toon-format) via the official toon-go encoder (research D1: measured honestly even though the TOON spec itself concedes compact JSON often wins on deeply-nested structures like JSON Schema).

Each tool becomes an ordered TOON object {name, description, inputSchema} where inputSchema is the tool's JSON input schema re-expressed as TOON text. Determinism (FR-010): field order is fixed by explicit toon.Object fields, schema object keys are sorted lexicographically by toon-go's normalizer, and schema numbers are decoded as json.Number (no float round-trip surprises in the literal source).

func NewToonListing 

func NewToonListing() *ToonListingArm

NewToonListing returns the toon_listing arm.

func (*ToonListingArm) EncodeIndexMetadata 

func (*ToonListingArm) EncodeIndexMetadata(t bench.Tool) (config.ToolMetadata, error)

EncodeIndexMetadata implements Arm: Name/ServerName/Description are the production values unchanged; ParamsJSON is replaced by the TOON-encoded schema text — the exact parameter text the retrieval index would ingest under this arm (empty when the tool has no schema, matching baseline).

func (*ToonListingArm) EncodeListing 

func (*ToonListingArm) EncodeListing(ts []bench.Tool) (string, error)

EncodeListing implements Arm: the whole listing is a single TOON array document, so the shared array header ("[N]:") is paid once for the listing rather than per tool.

func (*ToonListingArm) EncodeTool 

func (*ToonListingArm) EncodeTool(t bench.Tool) (string, error)

EncodeTool implements Arm: one tool as a bare TOON object document. The listing array header is amortized in EncodeListing, not here (contract rule 6).

func (*ToonListingArm) IndexAltering 

func (*ToonListingArm) IndexAltering() bool

IndexAltering implements Arm: the TOON schema text replaces ParamsJSON in the index mapping (see EncodeIndexMetadata), so the arm changes text the retrieval index ingests and obligates retrieval-quality scoring (FR-008).

func (*ToonListingArm) LowerBound 

func (*ToonListingArm) LowerBound() bool

LowerBound implements Arm: descriptions are preserved verbatim (TOON quotes and escapes multi-line strings; nothing is dropped or truncated).

func (*ToonListingArm) Name 

func (*ToonListingArm) Name() string

Name implements Arm.

type ToonResultsRun 

type ToonResultsRun struct {
	// Rows: [0] = compact-JSON baseline row (arm baseline_json, savings 0),
	// [1] = the toon_results row (savings vs [0]).
	Rows []bench.ArmResult `json:"rows"`
	// Per-class token totals: the tabular/non-tabular split of each side's
	// TotalTokens (FR-007 classification split).
	TabularToonTokens        int `json:"tabular_toon_tokens"`
	TabularBaselineTokens    int `json:"tabular_baseline_tokens"`
	NonTabularToonTokens     int `json:"non_tabular_toon_tokens"`
	NonTabularBaselineTokens int `json:"non_tabular_baseline_tokens"`
}

ToonResultsRun is one toon_results measurement: the two report rows (compact-JSON baseline of the payloads, then TOON of the same payloads) and the tabular/non-tabular token split behind them. The baseline row makes the savings recomputable from report rows alone (FR-004 spirit).

func RunToonResults 

func RunToonResults(tk *bench.Tokenizer, fx *ResultFixtureSet) (*ToonResultsRun, error)

RunToonResults measures the fixture payloads under both encodings and assembles the two results-class report rows. Determinism (FR-010): payloads are processed in fixture-file order (the file is sorted by tool_id at capture), the baseline is CanonicalJSON (sorted keys, compact), and TOON object keys are sorted by toon-go's normalizer.

func (*ToonResultsRun) NonTabularSavingsPct 

func (r *ToonResultsRun) NonTabularSavingsPct() float64

NonTabularSavingsPct is the TOON savings vs compact JSON on non-tabular payloads only (0 when none was measured).

func (*ToonResultsRun) TabularSavingsPct 

func (r *ToonResultsRun) TabularSavingsPct() float64

TabularSavingsPct is the TOON savings vs compact JSON on tabular payloads only (0 when no tabular payload was measured).

type TronArm 

type TronArm struct{}

TronArm is a minimal, honest in-tree implementation of TRON's named-class schema deduplication (research D2, arXiv:2605.29676 — no Go implementation exists upstream). Mechanism: each tool's input schema is canonicalized (CanonicalJSON — sorted keys, compact); byte-identical canonical schemas share ONE class definition, emitted once in the EncodeListing preamble as 

class C<id> = {…canonical schema…}

and each tool entry references its class by name: 

name|description|C<id>

Deliberate deviations from the paper, documented for honesty:

  • Dedup key is the EXACT canonical schema bytes, not a structural "shape" with per-property descriptions abstracted away. This is lossless (no description is dropped or truncated — LowerBound stays false) but conservative: schemas that differ only in embedded descriptions do not merge, so measured savings are what exact dedup achieves, not the paper's upper bound.
  • Class names are content-addressed (C + first 8 hex of SHA-256 of the canonical schema) rather than sequential C1/C2, so the per-tool index mapping (EncodeIndexMetadata) and the listing agree without shared state, independent of tool order. Content addressing costs a few more tokens per reference than sequential names would — a conservative bias.
  • Every distinct schema gets a class, including schemas used by a single tool. Singletons pay the class overhead ("class … = " plus the reference) instead of inlining; this keeps the format uniform and the honest cost visible rather than optimized away.
  • Field delimiters (| and newlines) are not escaped: the arm measures token cost of the format shape, and descriptions are preserved verbatim because escaping would perturb the very token counts under measurement.

Amortization lives ONLY in EncodeListing (contract rule 6): EncodeTool returns the non-deduped single form name|description|<canonical schema>, so per-tool means remain comparable across arms.

func NewTron 

func NewTron() *TronArm

NewTron returns the tron_dedup arm.

func (*TronArm) EncodeIndexMetadata 

func (*TronArm) EncodeIndexMetadata(t bench.Tool) (config.ToolMetadata, error)

EncodeIndexMetadata implements Arm: under TRON the schema body lives in a shared class definition that is not attached to any single tool, so the per-tool params text the production index ingests is the content-addressed class reference — Name/ServerName/Description are unchanged. Content addressing makes this mapping deterministic per tool, with no listing context, and guarantees it names the same class EncodeListing defines.

func (*TronArm) EncodeListing 

func (*TronArm) EncodeListing(ts []bench.Tool) (string, error)

EncodeListing implements Arm: a class-definition preamble (one line per distinct canonical schema, in first-appearance order) followed by tool entries referencing their class by name. Determinism: class identity is content-addressed, preamble order is input order — no map iteration.

func (*TronArm) EncodeTool 

func (*TronArm) EncodeTool(t bench.Tool) (string, error)

EncodeTool implements Arm: the non-deduped single form name|description|<canonical schema> (name|description when the tool has no schema). Class machinery appears only in EncodeListing.

func (*TronArm) IndexAltering 

func (*TronArm) IndexAltering() bool

IndexAltering implements Arm: the schema body moves into a shared class definition, so the per-tool params text the index ingests becomes a class reference instead of the schema JSON (see EncodeIndexMetadata).

func (*TronArm) LowerBound 

func (*TronArm) LowerBound() bool

LowerBound implements Arm: descriptions (tool-level and inside schemas) are preserved verbatim.

func (*TronArm) Name 

func (*TronArm) Name() string

Name implements Arm.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.