attest

package
v0.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 10, 2026 License: MIT Imports: 11 Imported by: 0

Documentation

Overview

Package attest is the machine-read-plus-human-verdict framework that oidio, raglit and kgraph each built separately and none of them built completely.

The job is one job: an ASSET was read by a machine, the machine made claims about locatable pieces of it, and a person has to be able to rule on those claims durably — with an honest account of how much was actually ruled on. What varies between the three is only the LOCATOR. oidio addresses a span of time in a recording; raglit addresses a box on a page at a rotation and a dpi; kgraph addresses a (fact, source) edge. Everything else — the verdict vocabulary, the completeness accounting, the review loop, the discipline of showing a person the artifact the words actually came from — was reimplemented three times, and each implementation has a piece the other two lack.

What each contributed, and why it is here:

  • From oidio: the distinction between CONFIRMED and AFFIRMED. Listening to a recording end to end and accepting the rest is a real act, and without a way to record it the choice is to leave a finished pass looking half-done or to tick every unit and destroy the question "did a person look at THIS one". Also the generated completeness banner, which is what stops a partly-reviewed transcript being cited as a verified one.
  • From raglit: the digest of the exact artifact the model was shown. A verdict is worthless if the person was looking at a different image than the one that produced the text — and for a passage read off a rotated, zoomed crop, handing them the whole page IS a different image.
  • From kgraph: the append-only log, because two machines rewriting one JSON object over a sync is a merge conflict and a lost verdict; and the categorical verdict vocabulary, because "the document does not say this" is not 0.3 of anything.

This file defines what a machine produces. attestation.go defines what a person produces. state.go is how the two resolve into an answer.

Index

Constants

View Source
const (
	KindAudio = "audio"
	KindImage = "image"
	KindPDF   = "pdf"
	KindText  = "text"
)

Asset kinds.

The pairing with Locator is the point: audio takes a Time, image and pdf take an Area, and text takes a Span. KindText is for an asset with no geometry at all — a note, an email, a markdown transcript. A SCANNED page whose text was read out of pixels is a pdf or an image, not text, because what the claim was read from is the crop.

Variables

This section is empty.

Functions

func Append

func Append(assetPath string, entries ...Entry) error

Append adds entries to an asset's log.

O_APPEND with one Write per line, so a concurrent reviewer on the same file interleaves lines rather than losing them. No lock file: the format is designed so that any interleaving of complete lines is a valid log.

func IsLog

func IsLog(path string) bool

IsLog reports whether a path is attest's own verdict log.

func IsReading

func IsReading(path string) bool

IsReading reports whether a path is attest's own machine-read output.

func LogPath

func LogPath(assetPath string) string

LogPath is where an asset's verdicts live: beside it, next to the reading.

func ReadingPath

func ReadingPath(assetPath string) string

ReadingPath is where an asset's machine read lives: beside it.

func SHA256Hex

func SHA256Hex(b []byte) string

SHA256Hex is the digest scheme for producers whose canonical form IS the bytes they serve — raglit's cropped PNG. Producers whose canonical form differs, like a decoded audio window, digest that instead and are responsible for using the same function at read time and at render time.

func Signature added in v0.5.0

func Signature(ctx context.Context, id Identity, author string) (principal, by string, err error)

signature resolves the pair that goes on an entry: the account the work was authorized under, and the person who says they did it.

The author is refused when blank rather than defaulted. A defaulted author — the account name, the hostname, "unattributed" — reads afterwards exactly like a real one, and the first time it matters is the first time someone asks who performed a review and gets an answer that was never true.

func UnitID

func UnitID(u Unit) string

UnitID computes a unit's content address.

The Extra and ID fields are ignored; see the note above.

func VerifyEvidence

func VerifyEvidence(u Unit, a Artifact) error

VerifyEvidence reports whether a rendering is the artifact a unit's text was read from, with an error that says what went wrong rather than just "false".

func WriteReading

func WriteReading(assetPath string, r *Reading) (string, error)

WriteReading replaces the machine read beside an asset.

REPLACES, unlike the attestation log, and the asymmetry is the point: a re-read is a new set of claims and merging it with the old one would produce a reading no machine ever made. Verdicts are untouched by this — they live in a separate file and are keyed by content, so the ones that still apply survive and the ones that do not are reported as orphaned rather than silently re-attached to a claim nobody ruled on.

Types

type Area

type Area struct {
	Page     int  `json:"page"`
	BBox     Rect `json:"bbox"`
	Rotation int  `json:"rotation,omitempty"`
	DPI      int  `json:"dpi,omitempty"`
}

Area addresses a piece of a page, at the rotation and rasterization it was read at.

Rotation and DPI belong here rather than on the document because both vary per region in practice: one cell of a survey sheet holds text at four different angles, and a descent re-rasterizes the page to get more detail on a smaller box. A locator that records the box and not the resolution reproduces the geometry and loses the detail, and detail is the whole subject.

type Artifact

type Artifact struct {
	MIME string
	Body []byte

	// Digest is the producer's digest of what it just rendered, computed the
	// same way it computed Unit.Evidence when it read the unit.
	//
	// Computed by the producer rather than by attest over Body, because the
	// canonical form is not always the served form: an audio window is digested
	// as decoded samples, since re-encoding is not byte-reproducible, but what
	// gets served to a browser is a container with a header. The producer is the
	// only party that knows the difference.
	//
	// This does not defend against a lying producer, and is not meant to. It
	// catches the thing that actually happens: a renderer upgrade that silently
	// produces different pixels at the same nominal dpi, months after the
	// verdicts were recorded.
	Digest string
}

Artifact is one rendering of one unit.

func Render

func Render(ctx context.Context, ev Evidence, asset Asset, u Unit, as Rendering) (Artifact, error)

Render dispatches to whichever rendering the provider supports, and says plainly when it does not support one rather than quietly substituting the crop.

Substitution is the failure mode worth spending an error on: a UI that asks for the humane rendering, silently receives the crop, and labels it "levelled" has told the reviewer something untrue about what they are looking at.

func (Artifact) Matches

func (a Artifact) Matches(u Unit) bool

Matches reports whether this rendering is the artifact the unit's claim was read from.

Byte equality via the digest, not "same box, same rotation". A weaker check would pass an image cropped from a page rasterized by a different tool at the same nominal dpi, which is precisely the substitution a person attesting a quotation is being asked to rule out.

type AsSeenEvidence

type AsSeenEvidence interface {
	AsSeen(ctx context.Context, asset Asset, u Unit) (Artifact, error)
}

AsSeenEvidence is the optional diagnostic rendering — what the model got, where a producer degraded the crop before reading it.

type Asset

type Asset struct {
	// ID is the stable handle a consumer uses to refer to this asset: a path, a
	// URI, a content id. attest does not interpret it.
	ID   string `json:"id"`
	Name string `json:"name,omitempty"`
	// Kind is audio | image | pdf | text. It selects which locator shape is
	// meaningful and which evidence the reviewer must be shown; it is not a MIME
	// type.
	Kind   string `json:"kind"`
	SHA256 string `json:"sha256,omitempty"`
}

Asset is what was read. Digested, because every guarantee below is about pieces of THIS byte sequence — a re-encoded recording or a re-scanned page is a different asset wearing the same filename, and verdicts recorded against the old one do not transfer.

type Entry

type Entry struct {
	Kind Kind `json:"kind"`

	// Unit is what this entry rules on. Empty when Blanket is set, and for
	// Resegment entries that only add units.
	Unit string `json:"unit,omitempty"`

	// Blanket applies this entry to every unit with no ruling EARLIER IN THE
	// LOG. Only meaningful with Affirmed.
	//
	// One line rather than one line per unit, so the sweep keeps its identity:
	// who swept and when is a single fact about the pass, and reconstructing it
	// by counting per-unit flags is how a reader ends up reporting the wrong
	// thing. Units added after this line are not covered — a sweep is a
	// statement about what had been read at the time it was made.
	Blanket bool `json:"blanket,omitempty"`

	// Text and Label carry a correction. Empty means "not disputed", not
	// "cleared to empty" — clearing text is a Resegment or an Unsupported, both
	// of which say something a blank field does not.
	Text  string `json:"text,omitempty"`
	Label string `json:"label,omitempty"`

	// Note is the reason: what the region actually says, why this paragraph is
	// about the septic and not the pole.
	Note string `json:"note,omitempty"`

	// Statement is the reviewer's own words on a blanket affirmation — the terms
	// they are accepting the remainder under.
	//
	// Separate from Note and never generated. An affirmation is a qualified claim
	// with a standard in it: "reasonably certain there are only minor errors or
	// mis-transcriptions that should not be materially relevant to the best of my
	// ability" is a different assertion from "the rest is right", and a tool that
	// paraphrases the first as the second has changed what somebody attested to.
	// Provenance QUOTES this rather than summarising it.
	//
	// Empty is honest and means the terms were not recorded — which is the case
	// for anything imported from a tool that had no field for them.
	Statement string `json:"statement,omitempty"`

	// Units and Supersedes are the payload of a Resegment.
	Units      []Unit   `json:"units,omitempty"`
	Supersedes []string `json:"supersedes,omitempty"`

	// By is the PERSON who made this ruling — or a machine identity like
	// `ocr-text-layer` when a process did.
	//
	// Self-declared, and required. Self-declared because the person at the
	// keyboard is routinely not the account holder: an attorney hands a
	// paralegal the link and the paralegal does the review, which is normal and
	// authorized and has to be recordable. Required because a defaulted author
	// reads afterwards exactly like a real one.
	//
	// The distinction that matters downstream is not who signed but whether a
	// PERSON did: a machine can honestly report that a string is or is not in a
	// text layer; it cannot report that a recording does not SUPPORT what was
	// transcribed.
	By string `json:"by"`

	// Auth is the authenticated account the ruling was made under, supplied by
	// the host's Identity and never by the caller.
	//
	// The other half of the pair. By alone lets any holder of a link type any
	// name; Auth alone files a delegate's work under the account holder's name.
	// Together they say what happened: authorized as X, performed by Y. Empty
	// means the mount is not account-based — a loopback binding, a shared link —
	// which is worth saying plainly rather than papering over with an invented
	// account.
	Auth string `json:"auth,omitempty"`

	At string `json:"at,omitempty"` // RFC3339
}

Entry is one line of the log.

func ReadLog

func ReadLog(assetPath string) ([]Entry, error)

ReadLog loads an asset's verdicts in the order they were made. A missing log is not an error: a corpus nobody has reviewed yet is the normal starting state.

A malformed line is an error rather than a skip. These files are evidence, and quietly dropping the line nobody could parse is how a verdict disappears without anyone being told.

func (Entry) ByMachine

func (e Entry) ByMachine() bool

ByMachine reports whether this entry came from a process rather than a person. A predicate rather than a rank: what a machine ruling is worth is a reader's call, not an invented statistic.

type Evidence

type Evidence interface {
	Render(ctx context.Context, asset Asset, u Unit) (Artifact, error)
}

Evidence renders the crop: the artifact a claim was read from.

type Guest

type Guest struct{}

Guest is the standalone CLI's identity: no account, everything permitted, and the reviewer still has to say who they are.

It is the right stance for a tool bound to the loopback interface, and it is the same one oidio's and raglit's local servers already take. It is the wrong stance for anything reachable, which is why a host that mounts attest supplies its own rather than being offered a flag to loosen this one.

"Authorized guest" is the honest description and it is what lands in the record: Auth is `guest`, so a reader can see that the ruling rests on nothing but access to the machine, and By is whoever typed their name.

func (Guest) Can

func (Guest) Principal

func (Guest) Principal(context.Context) (string, error)

type HumaneEvidence

type HumaneEvidence interface {
	Humane(ctx context.Context, asset Asset, u Unit) (Artifact, error)
}

HumaneEvidence is the optional reviewer-facing rendering, when fidelity and legibility genuinely conflict.

A producer that implements this is asserting that the plain crop is hard for a person to use, not that it is wrong. Everything served through it is labelled, and it can never satisfy a Matches check — by construction, since it is not the artifact.

type Identity

type Identity interface {
	// Principal is the authenticated account, written to Entry.Auth. Empty is
	// legal and means the mount is not account-based — a link, a loopback
	// binding — in which case the permission check is the whole of the
	// authorization and the record says so honestly rather than inventing an
	// account.
	Principal(ctx context.Context) (string, error)

	// Can reports whether this caller may do p to this asset. Asset-scoped
	// rather than global so a host can publish one corpus and restrict another
	// without mounting attest twice.
	Can(ctx context.Context, asset string, p Permission) (bool, error)
}

Identity is the host's answer to "which account is this, and may it do this".

Both methods take the request context, because that is where a host keeps the authenticated caller. attest passes it through untouched and interprets nothing in it. Note what is NOT here: any notion of the person's name. See the delegation note above.

type Kind

type Kind string

Kind is what a person decided. Categorical, never a score: "the document does not say this" is not 0.3 of anything, and a confidence number here would be an invented statistic.

const (
	// Confirmed — a person went to this claim specifically, checked it, and
	// positively affirmed it.
	//
	// The ESCALATION, not the baseline. Going through a recording and accepting
	// what needs no change is the ordinary pass (see Affirmed); this is for the
	// passage something depends on — the sentence a case hinges on, the figure a
	// filing quotes. Somebody was asked to double-check that one, and did.
	//
	// So a low Confirmed count next to a high Affirmed count is not a thin
	// review. It means few claims were load-bearing enough to need re-checking,
	// which is the normal shape of a careful pass.
	Confirmed Kind = "confirmed"

	// Corrected — the machine was wrong. Text and Label carry what it should
	// say; whichever is empty was not disputed.
	//
	// Distinct from Confirmed on purpose, and it is the only thing that makes
	// word error rate meaningful: scoring a recogniser against text nobody
	// retyped measures it against itself and reports a flawless zero.
	Corrected Kind = "corrected"

	// Affirmed — the reviewer went through the asset and accepted this claim
	// under the standard they stated.
	//
	// THE ORDINARY PASS, and the description of it used to be wrong here in a way
	// that mattered. A reviewer listens to every turn, edits the ones that need
	// it, and moves past the ones they agree with; the affirmation at the end is
	// what covers everything they passed over. So an affirmed claim was HEARD.
	// Calling it "swept past, rather than judged" — which this comment did —
	// reports a thorough review as a shallow one, which is the same failure the
	// completeness account exists to prevent, running backwards.
	//
	// It is still not Confirmed, because the two answer different questions.
	// Affirmed says "I went through this and nothing here needed changing, to the
	// standard I stated". Confirmed says "something depends on this one and I
	// checked it again". Collapsing them would lose the ability to escalate.
	//
	// Statement carries the reviewer's own words. Ticking every unit instead
	// would destroy the distinction above, and paraphrasing the statement in the
	// tool would be worse: a qualified attestation with a materiality standard is
	// not "the rest is right".
	Affirmed Kind = "affirmed"

	// Unclear — looked, cannot tell. A verdict on the ARTIFACT, not the claim:
	// an inaudible passage, an illegible scan.
	//
	// It marks the units where neither the machine nor a person should be
	// expected to be right, and it keeps a bad recording from being silently
	// retried forever. It does not subtract: failing to read a scan is a fact
	// about the scan, and treating it as disproof would let a bad photocopy
	// strip a document of its evidence.
	Unclear Kind = "unclear"

	// Unsupported — the asset is SILENT here. Not "says the opposite": absent.
	//
	// The machine invented it. For a transcript that is a hallucinated turn; for
	// a page it is text read off a region that does not contain it. This is the
	// one verdict that subtracts, because a claim resting on nothing must stop
	// counting as read.
	Unsupported Kind = "unsupported"

	// Resegment — the machine's decomposition was wrong, and the person fixed
	// the pieces rather than the labels.
	//
	// This is not a nicety. It is the common case in both media and it is why
	// this framework is an EDITOR and not a survey: a diarization boundary
	// falling mid-sentence leaves one speaker's last word attached to the next
	// speaker's turn, and no amount of relabelling expresses the fix. The
	// raglit analogue is a region whose box was drawn around the wrong thing.
	//
	// Units carries the replacements and Supersedes names what they retire.
	Resegment Kind = "resegment"

	// Retract — take back an earlier ruling; the unit returns to untouched.
	//
	// Needed because the log cannot be edited. An undo is a later line, which is
	// also the honest record: the person did rule, and then withdrew it.
	Retract Kind = "retract"
)

func (Kind) Ruled

func (k Kind) Ruled() bool

Ruled reports whether this kind means a person judged the claim itself, as opposed to sweeping past it or withdrawing.

type Locator

type Locator struct {
	Time *TimeSpan `json:"time,omitempty"`
	Area *Area     `json:"area,omitempty"`
	Span *Span     `json:"span,omitempty"`
}

Locator is where in the asset a claim points. Exactly one field is set.

A tagged union rather than a flat struct with a Kind discriminator, so an audio unit cannot accidentally carry a bounding box and a consumer cannot read one that was never written. New media types add a field here; nothing about the verdict algebra changes when they do, which is the property this whole package is built to have.

type Permission

type Permission string

Permission is what a caller is asking to do.

Three rather than read/write, because the public-serving case needs the middle one to exist on its own: an unattested transcript is worth showing to people who cannot rule on it, and a person authorised to rule on claims is not necessarily authorised to re-cut the machine's decomposition.

const (
	// PermRead — see the reading, the verdicts and the provenance.
	//
	// The public case. A transcript nobody has checked is still worth reading,
	// as long as it says so — which is what State.Provenance is for.
	PermRead Permission = "read"

	// PermAttest — record a verdict on a claim.
	PermAttest Permission = "attest"

	// PermResegment — change what the units ARE: join, split, re-cut.
	//
	// Separate from PermAttest because it is a stronger act. Ruling on a claim
	// leaves the machine's reading intact and disagreeable; resegmenting asserts
	// the decomposition itself was wrong, and every verdict on the retired units
	// stops applying.
	PermResegment Permission = "resegment"
)

type ReadOnly

type ReadOnly struct{ Identity }

ReadOnly wraps an identity so nothing can be written through it — the shape a host wants for a public mount, without having to reimplement Can.

func (ReadOnly) Can

func (r ReadOnly) Can(ctx context.Context, asset string, p Permission) (bool, error)

type Reading

type Reading struct {
	Version int    `json:"version"`
	Asset   Asset  `json:"asset"`
	Units   []Unit `json:"units"`

	// Producer identifies what read the asset — "oidio/parakeet",
	// "raglit/regions". Two readings of the same asset by different producers
	// are different claims and are meant to be comparable, which is what makes
	// scoring one against a human pass possible.
	Producer string `json:"producer,omitempty"`
	Read     string `json:"read,omitempty"` // RFC3339
}

Reading is one machine pass over one asset: the claims, and enough about who made them to tell two passes apart.

Units are a FLAT list with parent references, not a tree. See Unit.Parent.

func ReadReading

func ReadReading(assetPath string) (*Reading, bool, error)

ReadReading loads the sidecar beside an asset. A missing sidecar is not an error: most assets have not been read.

func (*Reading) Children

func (r *Reading) Children(id string) []Unit

Children returns the units nested directly under id, in the order the producer emitted them — which is the producer's reading order and the only order that means anything for a drawing.

func (*Reading) Reproducible

func (r *Reading) Reproducible() (bool, string)

Reproducible reports whether every unit recorded the artifact it was read from, and names the first that did not.

A reading that is not reproducible can still be reviewed. What it cannot do is prove that the reviewer and the machine looked at the same thing, so a consumer that cares — anything publishing an attestation as evidence — should check this rather than assume it.

func (*Reading) Seal

func (r *Reading) Seal() error

Seal assigns content-addressed ids to every unit and returns the reading.

Parents before children: a child's id depends on its parent's, so the list must be walked in dependency order. A producer that emits a child before its parent gets an error rather than a quietly wrong id.

func (*Reading) Unit

func (r *Reading) Unit(id string) (Unit, bool)

Unit resolves a unit by id.

type Rect

type Rect struct {
	X float64 `json:"x"`
	Y float64 `json:"y"`
	W float64 `json:"w"`
	H float64 `json:"h"`
}

Rect is a box in normalized coordinates — 0..1 of the page's width and height. Normalized rather than pixels so it survives a re-render at a different dpi, which is exactly what raglit's descent does when it zooms.

Field names match raglit's Rect and kgraph's Region: the same four numbers spelled the same way in all three, because a translation layer between them is a place for x/y to swap.

type Rendering

type Rendering string

Rendering names which of the three a caller wants.

const (
	// AsCrop is the attestation image and the default. Anything recording a
	// verdict should be showing this.
	AsCrop Rendering = "crop"
	// AsModelSaw is the diagnostic.
	AsModelSaw Rendering = "seen"
	// AsHumane is the legibility-first rendering. Never the basis of a verdict
	// on its own.
	AsHumane Rendering = "humane"
)

func (Rendering) Attestable

func (r Rendering) Attestable() bool

Attestable reports whether a verdict recorded against this rendering means what a verdict is supposed to mean.

Only the crop is. This is a predicate rather than a prohibition: oidio's reviewers work almost entirely from level-corrected audio and the pass is still worth having. What must not happen is a record that cannot tell the difference afterwards.

type Span

type Span struct {
	From int `json:"from"`
	To   int `json:"to"`
}

Span addresses a range of an asset's canonical text, in BYTES from its start.

Byte offsets into the whole text, never an index into a fragment or page list — the same reasoning as TimeSpan. A reviewer may re-fragment or a producer may re-transcribe, and an offset within a fragment stops meaning anything the moment the fragmentation changes. Bytes rather than runes because that is what Go's strings.Builder counts and what raglit's RegionSpan already records; a producer converting between the two is a place for an off-by-one that only shows up on the one document with an accented name.

This is the locator for text that has no geometry: a note, an email, a markdown transcript. Text that DOES have geometry — a block on a scanned page — is an Area, because the crop is what the claim was read from and a person checking it needs the pixels, not the characters.

type State

type State struct {
	Asset    Asset        `json:"asset"`
	Producer string       `json:"producer,omitempty"`
	Units    []UnitStatus `json:"units"`
	Stats    Stats        `json:"stats"`

	// Orphaned names verdicts that rule on units this reading does not contain
	// — the cost of a re-read that changed what the machine claims.
	//
	// Reported rather than dropped, and rather than matched onto whatever is
	// nearest. A verdict silently re-attached to a claim nobody ruled on is a
	// FALSE attestation, which is worse than a lost one. Recovering the overlap
	// is a rebinding pass that proposes matches for a person to accept.
	Orphaned []Entry `json:"orphaned,omitempty"`
}

State is a reading and its verdicts, resolved.

func Load

func Load(assetPath string) (*State, error)

Load resolves an asset's reading and log from disk.

func Resolve

func Resolve(r *Reading, log []Entry) (*State, error)

Resolve applies a verdict log to a machine reading.

Entries are applied in log order and the last one about a unit wins, so an undo is a later line and the record of the original ruling survives.

func (*State) Provenance

func (st *State) Provenance() string

Provenance is the sentence a consumer must print beside anything derived from this asset.

Generated, never written by hand. These banners were hand-written once, which meant they said whatever someone believed at the time — and on the corpus this was built against that produced a header claiming a completeness the data did not carry. A transcript is a source document downstream: facts get extracted from it and cited, and a file that is 18-of-39 confirmed presented as "the verified transcript" launders an unchecked machine guess into the record one step removed.

Two axes, reported separately, because they are the two people conflate: whether the CLAIMS were ruled on, and whether the WORDS were corrected. Checking who spoke says nothing about whether the text is right.

type Stats

type Stats struct {
	Total       int `json:"total"`
	Confirmed   int `json:"confirmed"`
	Corrected   int `json:"corrected"`
	Affirmed    int `json:"affirmed"`
	Unclear     int `json:"unclear"`
	Unsupported int `json:"unsupported"`
	Untouched   int `json:"untouched"`

	// Authored is how many units a person cut themselves. A high count means the
	// machine's decomposition was wrong, which is a different complaint from its
	// text being wrong and is worth surfacing separately.
	Authored int `json:"authored"`

	// SweptBy, SweptAt and SweptStatement record the last blanket affirmation, so
	// a consumer can state provenance without reconstructing it by counting flags.
	//
	// SweptStatement is the reviewer's own words, and an empty one means the terms
	// were not recorded rather than that there were none. Provenance says which.
	SweptBy        string `json:"swept_by,omitempty"`
	SweptAt        string `json:"swept_at,omitempty"`
	SweptStatement string `json:"swept_statement,omitempty"`
}

Stats is the completeness account. The states are reported SEPARATELY and never collapsed into one percentage: a single number hides which failure produced it.

The distinction that has to survive is between a claim NOBODY HAS READ and one a reviewer went through and accepted. Confirmed and Affirmed are both the latter — the difference between them is whether something depended on this particular claim enough to check it twice. Untouched is the one that means nothing is known.

func (Stats) Complete

func (s Stats) Complete() bool

Complete reports whether every unit has been accounted for — read and accepted, or ruled on. Confirmed and Affirmed both count, because both mean a reviewer went through the claim; the separate fields say which.

func (Stats) Ruled

func (s Stats) Ruled() int

Ruled is how many units a person went to individually — checked, corrected, or marked unreadable — as opposed to accepting under an affirmation.

type TimeSpan

type TimeSpan struct {
	Start float64 `json:"start"`
	End   float64 `json:"end"`
}

TimeSpan addresses a window of a recording, in seconds from its start.

Absolute times, never an index into a segment list: the whole point of the review loop is that a person may re-cut the segmentation, and an offset within a segment stops meaning anything the moment the segment changes.

type Unit

type Unit struct {
	// ID is content-addressed — see UnitID. Assigned by Seal, not by the
	// producer.
	ID string `json:"id"`

	// Parent nests this unit under another, for producers that read
	// hierarchically: raglit's descent reads a sheet, then a drawing interior,
	// then a corner of it. Empty for a flat read.
	//
	// A parent REFERENCE rather than nested children, because the id is a digest
	// of the unit and nesting would make a parent's id depend on what was found
	// underneath it — so descending one level deeper would renumber the whole
	// ancestry and orphan every verdict above the new leaf.
	Parent string `json:"parent,omitempty"`

	Locator Locator `json:"locator"`

	// Text is what the machine read here. Empty is legitimate: a unit may carry
	// only a label, or be a container whose children hold the text.
	Text string `json:"text,omitempty"`

	// Label is the machine's CATEGORICAL claim: the diarizer's cluster, the
	// region's kind, the figure's class. Kept separate from Text because they
	// fail differently and a person rules on them separately — the words can be
	// right while the speaker is wrong, and conflating them is what made oidio's
	// first pass ambiguous.
	Label string `json:"label,omitempty"`

	// Evidence digests the EXACT artifact this claim was read from: the cropped
	// PNG, or the decoded PCM of the time window. Not the whole asset — the
	// whole asset is not what produced the claim, and showing a person the whole
	// asset is asking them to attest against a different image.
	//
	// Producers should record this. A unit without it can still be reviewed, but
	// nothing can then say whether what the reviewer saw is what the machine
	// saw, and Reproducible reports false.
	Evidence string `json:"evidence,omitempty"`

	// Flags are conditions that either hold or do not — low-resolution,
	// repetition, exhausted. Deliberately not a confidence score: a number here
	// would be an invented statistic, and the useful signal is categorical.
	Flags []string `json:"flags,omitempty"`

	// Extra is producer-specific data attest does not interpret and carries
	// through untouched — raglit's tokens-per-square-inch and downscale count,
	// oidio's word timings.
	//
	// Deliberately NOT part of the id. A producer adding a diagnostic field must
	// not orphan every verdict in the corpus.
	Extra json.RawMessage `json:"extra,omitempty"`
}

Unit is one machine claim about one piece of the asset.

IMMUTABLE, and that is the load-bearing decision. oidio's workbench started with a single `speaker` field that held both the diarizer's grouping and the human's correction, which made a local fix and a global identity claim indistinguishable in the data. Here the unit is only ever what the MACHINE said; every human change lives in the attestation log. A re-read replaces units and cannot touch a verdict; a verdict cannot rewrite history.

func (Unit) HasFlag

func (u Unit) HasFlag(f string) bool

HasFlag reports whether a condition was recorded on this unit.

type UnitStatus

type UnitStatus struct {
	Unit Unit `json:"unit"`

	// Kind is the ruling in force, or empty for untouched.
	Kind Kind `json:"kind,omitempty"`

	// Text and Label are what this unit says NOW — the correction where one was
	// made, the machine's reading otherwise. A consumer rendering a transcript
	// wants these; a consumer scoring the machine wants Unit.Text and these both,
	// which is why the original is never overwritten.
	Text  string `json:"text,omitempty"`
	Label string `json:"label,omitempty"`

	Note string `json:"note,omitempty"`

	// By is the person who ruled; Auth is the account they were authorized
	// under. Both are carried, because a delegate doing the work is normal and a
	// reader asking "who actually reviewed this" must not be handed the account
	// holder's name instead. See identity.go.
	By   string `json:"by,omitempty"`
	Auth string `json:"auth,omitempty"`
	At   string `json:"at,omitempty"`

	// Authored marks a unit a person created with a Resegment, rather than one
	// the machine proposed. It is not itself a ruling: someone may cut a turn in
	// the right place and still not have decided who is speaking.
	Authored bool `json:"authored,omitempty"`

	// Swept marks a ruling that came from a blanket affirmation rather than a
	// line about this unit. Kind is Affirmed either way; this says whether anyone
	// named it. Not a demotion: the reviewer went through it, and the affirmation
	// they signed says under what terms.
	Swept bool `json:"swept,omitempty"`
}

UnitStatus is one unit's effective state: what the machine claimed, and what a person did about it.

func (UnitStatus) Corrected

func (s UnitStatus) Corrected() bool

Corrected reports whether a person actually retyped this unit's words. The only units that are ground truth for word error rate — scoring a recogniser against text nobody disputed measures it against itself.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL