Documentation
¶
Overview ¶
Package label is the ground-truth plane: what actually turned out to be fraud, who said so, and when they could first have said it.
It closes the loop the model plane cannot close for itself. /v1/risk decides, /v1/ml learns — and learning needs an answer key that arrives LATE, from several places, sometimes in disagreement:
chargeoff our own books writing a balance off dispute a card network's adjudicated chargeback, normalised by commerce case a compliance determination closed under /v1/aml refund a merchant refunding with a fraud reason review one analyst's call on one decision sample a judged draw from the below-the-line reproducible sample
THREE HARD PARTS, EACH ANSWERED BY A NAMED MECHANISM.
LATENCY Every assertion carries three times: `at`, when the judged event
happened; `seen`, when the filer says it became knowable; and
`knowable`, the later of `seen` and the server clock at the
write — derived here, never supplied. A chargeback lands 30 to
120 days after the transaction. Resolve takes an observation
instant and shows only what was knowable then, so a training set
built for an event can never contain a label that did not exist
when the model would have had to act. The guard reads the DERIVED
instant, because a guard whose only input is a value the caller
chose is exactly as strong as the caller's honesty.
resolve.go, Window; fact.go, Fact.Knowable.
CONFLICT Two sources that disagree BOTH stay. A total order over
adjudication weight picks the one in force and RETURNS the losers
beside it, so a contested label is visible as contested rather
than resolved into silence. There is no UPDATE statement in this
package. resolve.go, stronger().
PROVENANCE Every assertion names its source, the evidence record behind it
and the identity that filed it — the last stamped server-side from
the validated principal, never from the body. A label with no
evidence is refused at the door, because a label that cannot be
traced cannot be defended when the adverse action it fed is
challenged. fact.go, admit().
TWO PLANES, ONE DIRECTION. The tenant's own encrypted SQLite file is the record (store.go); hanzo.risk_label is a derived copy for joining at training scale (mirror.go). The record is written first and the mirror's failure is reported rather than fatal. Nothing here rides /v1/event, which is best-effort by design and drops on purpose.
SHIP BEFORE ACK. Every op that writes ships the tenant's file to its durable object before it answers, and an unacked ship fails the request (state.ship). cloud deploys strategy Recreate at one replica: the successor hydrates the durable snapshot OVER the local file, so a write that was acknowledged and not shipped is not merely at risk — it is overwritten by an older copy of the same tenant's history on the ordinary rollout path.
WHY IT IS ITS OWN SUBSYSTEM. Its writers are mostly not the decision plane — commerce adjudicates the dispute, the compliance face closes the case, an analyst files the review — its readers are the dataset materialiser and the evaluator, and its retention clock is its own: a label that fed an adverse action is a compliance record whose life is not the life of the decision that cited it. One owner per file also means the decision plane's single-writer SQLite is never opened by a second process.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Mount ¶
Mount registers /v1/risk/labels.
EVERY INHERITED CAPABILITY IS NAMED HERE, EXPLICITLY:
IAM auth SanitizeIdentity mints the verified org upstream (serve.go).
This package never validates a token and never can.
tenant gate cloud.Bridge(), which the composer installs — the fused host
at its root, a plugin program in its constructor. A typed op
receives only a context; Bridge is what parks the validated
principal in it, and this package only reads it.
durability cloud.WithDurable routes each tenant file through the ha-elected
single writer, and every op that writes calls state.ship before
it answers. Wiring the option alone is NOT durability: it only
makes a ship possible, and a subsystem that never ships holds an
acknowledged record in a local file that the next pod hydrates
an older snapshot over. cloud deploys strategy Recreate at one
replica, so that is the ordinary rollout and not a rare fault.
logs cloud.NewBase gives the scoped luxlog.
NOT metered, deliberately, and this is the one inherited capability declined rather than wired. A label can be the input to an adverse action, so recording one is a compliance obligation and not a purchase; a balance gate in front of it would mean a tenant that fell behind on its bill could no longer record the ground truth for a decision it is about to be challenged on. The DoS bound is carried by the per-op limits in typed.go instead, which cost nothing to be wrong about.
func Order ¶
func Order() string
Order is the precedence rule rendered as a ClickHouse ordering tuple, for the dataset materialiser to resolve labels IN the warehouse — `argMin(disposition, `+label.Order()+`)` picks the same winner the Go comparator would.
It is GENERATED from the same precedence map stronger() reads, so a source added to the vocabulary appears in both orderings and cannot be forgotten in one. That is the only defence available against a rule with two homes: orderCoversEverySource pins it, and mirrorOrderTest pins the rendering.
The terms match stronger() one for one — rank ascending, knowable DESCENDING (so negated), confidence descending (negated), id ascending.
func ResolvedSQL ¶
func ResolvedSQL() string
ResolvedSQL is the warehouse-side read the dataset materialiser joins against: one row per judged event, resolved at that event's OWN as-of instant, over one tenant.
The horizon arithmetic is in SQL rather than in a loop over rows, because the alternative is shipping every assertion to the app to filter it — the read amplification a single-pod warehouse cannot afford. Two predicates carry the whole leakage rule and both are per row, not per batch:
knowable <= at + h an assertion is visible only at the event's own as-of at + h <= now the event has matured and may be admitted at all
The first reads `knowable` — the SERVER-DERIVED instant, the later of the filer's `seen` and the clock at the write — and it is the same column and the same comparison visible() makes in Go. A predicate over `seen` alone would hold the warehouse to a weaker rule than the record plane, and a materialiser would then train on rows the resolve op refuses to return.
`org` is the LEADING bound predicate and the table's first sort term, so a tenant read is a prefix scan. Nothing a caller sends becomes statement text.
ONE argMin OVER A TUPLE, NOT ONE PER COLUMN, for two reasons and the second is the one that matters. The engine refuses the per-column form outright — an alias `AS source` shadows the column the ordering expression reads, and the second argMin then finds an aggregate inside its own argument (ILLEGAL_ AGGREGATION, code 184). And even where it parsed, three independent argMins over one ordering tuple could each answer from a different row on a tie, so "the winner" would be a chimera assembled from two assertions. Picking the whole payload once makes that unrepresentable: every projected field comes from the row that won.
The six placeholders bind, in order:
org the qualified tenant key `<brand>/<org>` from,to the event window, half-open h the maturity horizon IN SECONDS (DateTime + Int adds seconds) h the same horizon, for the maturity predicate now the materialisation instant
The projection carries the winner's whole provenance — `source`, `id`, `knowable` and `confidence` beside the disposition — so a training row and an adverse action can both name what judged them without a second read. `knowable` and not `seen`: the instant published beside a resolved label has to be the one the guard actually applied, or nobody can check the answer. `contested` is true when the visible assertions hold more than one disposition, matching Resolved.Contested exactly: two sources that agree are corroboration.
It is exported and unused inside this package by design: it is the CONTRACT the dataset plane joins against, published beside the table it reads so the two cannot drift.
Types ¶
type Cohort ¶
type Cohort struct {
Kind Kind
Subject string
At time.Time
// AsOf is this event's own observation instant: At plus the horizon.
AsOf time.Time
// Label is the assertion in force at AsOf, valid only when Labelled. An
// unlabelled matured event is the ordinary state of most traffic and it is
// never a negative: manufacturing one is how a fraud model comes to describe
// the incumbent block list.
Label Resolved
Labelled bool
}
Cohort is one MATURED event and what was knowable about it by its own as-of.
Labelled is a field and not the absence of an entry, because "matured" and "judged" are two different counts and an operator divides one by the other. A grouping that returned only the resolvable events would make the denominator exclude exactly the numerator's complement — matured would count what was LABELLED, the ratio would read 1.0 on a plane with one label in it, and the gate on training would say go.
func Group ¶
Group folds a flat set of assertions into one Cohort per MATURED event, each resolved at its OWN as-of instant.
The per-event instant is the point. A single as-of over a batch would give a row from January and a row from June the same knowledge, and the January row would be trained on six extra months of hindsight. Every row observes exactly its own horizon.
type Disposition ¶
type Disposition string
Disposition is what somebody concluded. The three values are the AML engine's own, verbatim (luxfi/aml pkg/replay), and are spelled identically here rather than translated: replay reports a false-positive proportion against them and topology refuses to rank a search without them, so a second vocabulary would leave the exhaustive search permanently unable to name a winner.
vocabularyTest pins the three literals, so a drift from the engine's spelling fails a test here rather than silently halving a training set there.
const ( // Unjudged is an event somebody looked at and could not conclude about. It // is a real assertion, not the absence of one — "we reviewed this and could // not say" is evidence, and it is how an unmatured row is admitted to a // density model while staying invisible to a supervised one. Unjudged Disposition = "" // Productive is an event that led somewhere: escalated, reported, charged back. Productive Disposition = "productive" // Unproductive is an event judged not suspicious. Unproductive Disposition = "unproductive" )
type Fact ¶
type Fact struct {
// Seq is the store's own delivery position, assigned by the record plane at
// the INSERT and monotone for the life of the file. It is the total order the
// derived copy catches up in, and it is not part of the assertion: nothing on
// the wire carries it and the content digest does not fold it.
Seq int64
// ID is the content digest. Idempotence is therefore a property of the
// assertion itself and not of a hand-picked key tuple: the same chargeback
// delivered twice is one row, and anything that differs in any field is a
// DIFFERENT assertion and is recorded as one. Nothing is ever overwritten.
ID string
Kind Kind
Subject string
At time.Time
Seen time.Time
Disposition Disposition
Source Source
// Evidence points at the record this conclusion came from: a dispute id, a
// case id, a decision id. A label with no evidence cannot be defended when
// the adverse action it fed is challenged, so it is required.
Evidence string
// By is the identity that asserted — the credential subject for an API
// write, the analyst for a review. Stamped server-side from the validated
// principal, never taken from the body: an attributable record whose
// attribution the caller chose is not attributable.
By string
// Confidence in [0,1]. A processor chargeback is 1; an analyst's hunch is
// not. It is a tie-breaker in the precedence rule, never a substitute for it.
Confidence float64
// Hold marks a litigation hold: retention never disposes of it, at any age.
//
// READ ONLY on this value. A hold is a fact about the record and not about
// the world, so it is not part of the assertion, it is not folded into the
// digest, and the write path never sets it — store.record does not name the
// column and takes the schema default. The hold op is the one way it moves,
// in either direction.
Hold bool
// Wrote is the server clock at the moment of the write. It is the ONLY time
// on this record the tenant does not supply, and it is what a retention
// sweep measures against — a tenant that could move At or Seen could
// otherwise age its own records out early.
Wrote time.Time
// Knowable is the instant THIS PLANE could first have answered with the
// assertion: the later of Seen and Wrote. It is derived here, from a value
// the caller supplies and a value only the server holds, and it is the ONLY
// time the leakage guard reads.
//
// WHY Seen CANNOT CARRY THE GUARD. Seen is the filer's claim about the
// world's clock, bounded only by At <= Seen <= now+skew. A caller that files
// a dispute today with seen == at — the natural integration mistake, "the
// dispute is about this transaction" — makes a label written today look
// knowable a year ago, and every horizon in the plane becomes decorative:
// a backtest standing two days after the event resolves a row that did not
// exist for another 298. A guard whose input is a caller-declared value is
// exactly as strong as the caller's honesty, which is not a guard.
//
// The later of the two is the honest instant and it is not a compromise: an
// assertion the plane did not hold could not have informed any decision the
// plane served, whatever the world knew. For a live pipeline — the case this
// is built for — Wrote is within minutes of Seen and the derivation changes
// nothing. It bites exactly where it should: on history filed after the fact,
// which is visible from the moment it is filed and never before.
//
// Seen is still recorded, published and returned. It is provenance — what the
// filer claimed — and it no longer decides anything.
Knowable time.Time
}
Fact is one assertion, and it is immutable once written.
THREE TIMES, AND THEY ARE DIFFERENT. At is when the judged event happened; Seen is when the caller says the assertion became knowable; Knowable is when this plane could first have answered with it. A chargeback lands 30 to 120 days after the transaction it judges. Join on At alone and the training set knows the future: offline AUC near 0.98, online worthless. Knowable is the field that stops it — see its own comment for why it is Seen that cannot.
type Kind ¶
type Kind string
Kind is what the subject IS. Closed, because the kind is half the identity of the thing being judged: an open field lets one typo shard a tenant's labels into a partition nothing ever reads, and nothing would say so.
type Resolved ¶
type Resolved struct {
Kind Kind
Subject string
At time.Time
// AsOf is the instant this answer was computed at. It is on the value rather
// than only on the request, because a resolved label handed to a training set
// or an auditor without the instant it was true at is a claim nobody can check.
AsOf time.Time
// Winner is the assertion in force. Its whole provenance travels with it —
// source, evidence, asserter — because that is what an adverse action needs.
Winner Fact
// Conflicts is every OTHER visible assertion, in the same precedence order,
// strongest first. It is horizon-filtered exactly like the winner: an
// assertion that was not knowable yet cannot even be NAMED as a conflict,
// because naming it would leak its existence into a past decision.
Conflicts []Fact
// Contested is true when at least one visible assertion claims a DIFFERENT
// disposition from the winner. Two sources that agree are corroboration, not
// conflict, and reporting them as conflict would make the number useless.
Contested bool
}
Resolved is the answer: what is in force, and what disagreed.
func Resolve ¶
Resolve picks the assertion in force at asOf and reports what it beat.
It reports false when nothing was knowable then — which is a real answer and not an error. "No label yet" is the ordinary state of a fresh transaction, and a plane that answered Unproductive there would be manufacturing negatives.
func (Resolved) Disposition ¶
func (r Resolved) Disposition() Disposition
Disposition is the claim in force — the one question every consumer of a Resolved asks first. It is the winner's, never a vote or an average: an average of two adjudications is a third claim nobody made.
type Source ¶
type Source string
Source names WHO asserted, and it is the primary term of the precedence rule. It is a closed set because an unknown source has no rank, and a precedence rule with an undefined term is not a rule.
const ( // Chargeoff is our own books writing the balance off. Terminal, ours, and // the only source whose claim costs us money to make. Chargeoff Source = "chargeoff" // Dispute is a card network's adjudicated chargeback, normalised by commerce // from every processor into one `dispute.created` fact. External and binding. Dispute Source = "dispute" // Case is a compliance determination closed under the /v1/aml workflow — // a regulated process with a documented file behind it. Case Source = "case" // Refund is a merchant refunding with a fraud reason coded. A unilateral act // with no counterparty adjudicating it, so it ranks below the three above. Refund Source = "refund" // Review is one analyst's call on one decision. Review Source = "review" // Sample is a judged draw from the below-the-line reproducible sample — the // arm that keeps a training set from being a description of the incumbent // block list. Deliberately the weakest claim: it is cheap and plentiful, and // it must never outvote an adjudicated one. Sample Source = "sample" )
type Window ¶
Window is the observation a caller is resolving under, and it is the whole of the leakage guard.
Now is the instant the caller is standing at — the materialisation instant for training, or a past instant for a backtest. Horizon is how long an event must age before it may be admitted at all.
Both are values on the request, not globals, because a backtest that could not move Now would be a backtest that resolves labels with today's knowledge and reports a score no live model could ever have earned.
func (Window) AsOf ¶
AsOf is the instant an event at `at` observes its labels at: the moment its horizon closes. A label seen after it is a label that did not exist when the model would have had to act, and it is invisible here.
func (Window) Matured ¶
Matured reports whether an event at `at` has aged past the horizon. A row that has not may still be scored — the density model does not need a label — but it must not be admitted to a supervised training set as a negative, because the chargeback that would make it a positive has not had time to arrive.
Source Files