Documentation
¶
Overview ¶
Package accuracy is the corpus scoring harness: it runs the full Phase 11 pipeline against a set of recorded filings (hermetic — the corpus ships the bytes, nothing touches the network) and scores the assembled documents against hand-verified baselines. It measures three things — per-field statement accuracy, section coverage, and confidence calibration (accuracy within each confidence bucket) — so "looks right against one filing" becomes a measured, regression-gated number, and so the confidence thresholds in internal/ixbrl can be checked against reality rather than intuition.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Baseline ¶
type Baseline struct {
Ticker string `json:"ticker"`
Form string `json:"form"`
Format string `json:"format"`
Confidence string `json:"confidence"`
Sections []string `json:"sections"`
Statements []BaselineStatement `json:"statements"`
}
Baseline is the hand-verified truth for one corpus filing: the expected statement values (by statement role, concept, and period) and the expected section item ids in order. It is the independent ground truth the parser's output is scored against — authored by reading the filing, not by running the parser — so a divergence means a parser regression.
type BaselineCell ¶
type BaselineCell struct {
Concept string `json:"concept"`
Period string `json:"period"`
Value *float64 `json:"value"`
}
BaselineCell is one expected value: a concept in a period. A nil Value means the cell is expected to be absent (null) — and must score as null, never 0.
type BaselineStatement ¶
type BaselineStatement struct {
Role string `json:"role"`
Title string `json:"title"`
Cells []BaselineCell `json:"cells"`
}
BaselineStatement is the expected content of one financial statement.
type BucketStats ¶
BucketStats is the calibration evidence for one confidence level: how many filings landed in the bucket and the mean statement accuracy across them. If "high" is calibrated, its accuracy should exceed "medium"'s.
func (BucketStats) Accuracy ¶
func (b BucketStats) Accuracy() float64
Accuracy is the mean statement accuracy of the filings in this bucket.
type FilingScore ¶
type FilingScore struct {
Name string
Ticker string
Format string
ExpectedConfidence string
GotConfidence string
MatchedCells int
TotalCells int
MatchedSections int
TotalSections int
}
FilingScore is one filing's measured accuracy.
func (FilingScore) SectionCoverage ¶
func (f FilingScore) SectionCoverage() float64
SectionCoverage is the fraction of expected sections the parser bounded.
func (FilingScore) StatementAccuracy ¶
func (f FilingScore) StatementAccuracy() float64
StatementAccuracy is the fraction of expected statement cells the parser got right (1.0 when there is nothing to score).
type Report ¶
type Report struct {
Filings []FilingScore
Totals Totals
Buckets map[string]BucketStats
}
Report is the harness's measurement across the corpus: per-filing scores, the corpus totals, and the per-confidence-bucket calibration that says whether a "high" label actually means high accuracy.
func Score ¶
Score runs the pipeline against every filing under corpusDir and scores it against its baseline.json. Each entry is a subdirectory holding the recorded EDGAR bytes (primary document, linkbases, submissions, index) and a baseline.json. It is fully hermetic: a fixture transport serves the recorded bytes, so no request reaches the network. The caller must have SEC_CLI_USER_AGENT set (edgar.NewClient requires it).
type Totals ¶
Totals aggregates the corpus.
func (Totals) SectionCoverage ¶
SectionCoverage is the corpus-wide fraction of sections covered.
func (Totals) StatementAccuracy ¶
StatementAccuracy is the corpus-wide fraction of statement cells matched.
Source Files