Documentation
¶
Overview ¶
Package identity assigns libcat's opaque, two-tier Work/Instance ids (ARCHITECTURE §4). Ids are minted once and never derived from a provider id; on re-ingest a provider record resolves back to its previously minted id via the persisted identity map, so unchanged records keep stable ids -- and therefore stable public URLs.
This file holds the provider-independent primitives: minting opaque ids and computing the clustering key. The persistence and resolve layer (reading the committed map, mint-or-resolve, clobber-safe re-ingest) builds on top.
Index ¶
- Constants
- func Mint(prefix string) string
- func NormalizeKey(s string) string
- func ProviderKey(scheme, value string) string
- func SeedResolver(r *Resolver, grains []GrainIdentity)
- func WorkKey(author, title, lang string) string
- type Assignment
- type GrainIdentity
- type InstanceIdentity
- type Merge
- type Pin
- type Record
- type Resolver
- func (r *Resolver) Conflicts() []string
- func (r *Resolver) Merges() []Merge
- func (r *Resolver) Resolve(rec Record) Assignment
- func (r *Resolver) SeedInstance(instanceID, workID string, providerKeys []string)
- func (r *Resolver) SeedMerge(from, to string)
- func (r *Resolver) SeedPin(instanceID, workID string)
- func (r *Resolver) SeedWorkKey(clusterKey, workID string)
- func (r *Resolver) WorkForProviderKey(key string) (string, bool)
- type WorkIdentity
Constants ¶
const ( WorkPrefix = "w" InstancePrefix = "i" AuthorityPrefix = "a" )
Id prefixes distinguish the two tiers in a bare id string and in the public URL space (/works/w..., an Instance's i...). AuthorityPrefix marks locally minted authority terms -- not a bibliographic tier, but minted through the same opaque-id discipline so local headings get stable IRIs.
const ( SchemeISBN = "isbn" SchemeISSN = "issn" SchemeID = "id" // provider-local identifier (e.g. an OverDrive title/reserve id) )
Provider-key schemes namespace an identifier value so keys from different schemes never collide, and so a key recovered from a grain matches the key ingest builds for the same identifier.
Variables ¶
This section is empty.
Functions ¶
func Mint ¶
Mint returns a fresh opaque id with the given tier prefix (WorkPrefix or InstancePrefix), drawn from crypto/rand so it is provider-independent and unguessable. Callers persist the id on first mint and never regenerate it; use a resolver to avoid re-minting for an already-mapped record.
func NormalizeKey ¶
NormalizeKey folds a string to its clustering-key form: Unicode-lowercased, with every run of non-alphanumeric characters collapsed to a single space and the result trimmed. It deliberately does not fold diacritics -- that needs a Unicode-normalization dependency this framework avoids -- so accent and transliteration variants under-merge and are corrected in the editorial overlay rather than guessed at here.
func ProviderKey ¶
ProviderKey namespaces an identifier value by scheme, e.g. ProviderKey(SchemeISBN, "978...") -> "isbn:978...".
func SeedResolver ¶
func SeedResolver(r *Resolver, grains []GrainIdentity)
SeedResolver seeds r with the committed identity from scanned grains, so a subsequent Resolve reuses existing ids instead of re-minting.
func WorkKey ¶
WorkKey builds the computed clustering key from the primary author, the title, and the original language -- the MARC 1XX+240 access-point key (ARCHITECTURE §4). Two Instances with the same key cluster into one Work unless an external work id or an editorial merge/split decision says otherwise. The title is the main title only (not the subtitle), so editions that vary a subtitle still cluster.
A record with no main title has no usable access point: clustering title-less records by author (or by nothing) would merge unrelated books, so WorkKey returns "" and the caller must mint instead of clustering.
Types ¶
type Assignment ¶
Assignment is the resolved identity for a record: its stable Instance and Work ids, and whether either was freshly minted this ingest (vs resolved to an already-committed id).
type GrainIdentity ¶
type GrainIdentity struct {
Works []WorkIdentity
Instances []InstanceIdentity
}
GrainIdentity is the identity recovered from one grain -- the Work(s) it carries and their Instances. The derive-from-grains model (Decision A, ): the committed grains are themselves the identity map.
func ScanDataset ¶
func ScanDataset(ds *rdf.Dataset) GrainIdentity
ScanDataset is ScanGrain for callers that already hold the parsed dataset (the work index scans identity, summaries, and barcodes off one parse).
func ScanGrain ¶
func ScanGrain(nq []byte) (GrainIdentity, error)
ScanGrain recovers the identities committed in one grain's N-Quads. Node ids come from the #<id>Work / #<id>Instance IRIs; a Work's clustering key is recomputed from its primary author, main title, and language; provider keys come from each bf:identifiedBy value, namespaced by its identifier type. It reads every named graph, since a grain's feed and editorial lines share the file. Only minted fragment IRIs count: the crosswalk types 76X-78X related-resource stubs as bf:Work/bf:Instance too (blank or external nodes), and seeding their keys or identifiers would let an unrelated incoming record resolve onto a stub label instead of minting.
type InstanceIdentity ¶
InstanceIdentity is the committed identity of one Instance recovered from a grain: its minted id, the Work it belongs to, and the provider keys it answers to.
type Merge ¶
Merge is an editorial merge decision recovered from the grains (ARCHITECTURE §4): every reference to From resolves to To. It is the under-merge fix -- two records that should be one Work but clustered apart -- and the computed key cannot undo it. Chains collapse to a single canonical id.
type Pin ¶
Pin is an editorial split decision recovered from the grains (ARCHITECTURE §4): Instance is assigned to Work regardless of the computed clustering key. It is the over-merge fix -- records the key wrongly clustered together are pinned apart -- and makes the split reproducible across re-ingest.
type Record ¶
type Record struct {
// ProviderKeys are resolution keys in priority order, each namespaced so keys
// from different schemes never collide, e.g. "overdrive:3389970" then
// "isbn:9781682308233". The first key that already maps wins; ISBN is the
// cross-provider merge key (ARCHITECTURE §9).
ProviderKeys []string
Author string
Title string
Lang string
}
Record is the identity-relevant projection of one incoming provider record: the keys it can be resolved by and the fields of its computed clustering key.
type Resolver ¶
type Resolver struct {
// contains filtered or unexported fields
}
Resolver assigns stable Work/Instance ids across ingests. Seed it with the identity already committed (from the grains), then Resolve each incoming record to an existing id or a freshly minted one. It is the mint-or-resolve core of ARCHITECTURE §4 / an unchanged record keeps its previously minted ids, so its grain -- and its public URL -- do not churn.
A Resolver is not safe for concurrent use; ingest is single-threaded per corpus.
func NewResolver ¶
func NewResolver() *Resolver
NewResolver returns an empty resolver. Seed it from the committed identity before resolving new records.
func (*Resolver) Conflicts ¶
Conflicts returns provider-key collisions seen during resolution ( §4), for the caller to surface. Nil when there were none.
func (*Resolver) Merges ¶ added in v0.173.1
Merges returns every merge the resolver is applying -- editorial merges seeded from prior grains and feed cluster-merges alike -- as From->To pairs in deterministic order. The retirement pass diffs on this so a Work folded away by a feed's isReplacedBy has its stale grain removed, not just its records re-homed: without it the retired cluster's grain lingers and keeps duplicating the survivor's identifiers.
func (*Resolver) Resolve ¶
func (r *Resolver) Resolve(rec Record) Assignment
Resolve returns the stable identity for a record, minting only what is genuinely new. An Instance resolves by its first already-known provider key. Its Work is an editorial pin if one exists (an over-merge split, applied first), else the existing instance->work link, else the computed cluster key, else a freshly minted Work. Editorial merges are applied last, so a pinned or clustered Work still follows a later merge to its survivor.
func (*Resolver) SeedInstance ¶
SeedInstance records a previously minted Instance: its id, the provider keys it answers to, and the Work it belongs to. Called once per committed Instance before ingest so re-ingest resolves rather than re-mints.
func (*Resolver) SeedMerge ¶
SeedMerge records an editorial merge: every reference to from resolves to to. Merges override the computed key, so a re-ingest cannot undo a human decision. Chains collapse to a single canonical id.
func (*Resolver) SeedPin ¶
SeedPin records an editorial split pin: instanceID is assigned to workID regardless of the computed clustering key, so an over-merge the key would otherwise recreate stays split across re-ingest. The pinned Work id is reserved so it is never minted for anything else.
Two pins for one instance is a data-integrity event (a split written twice before made the endpoint idempotent, or a hand-edited grain). The first pin seen wins deterministically -- pins arrive in canonical quad order, so the choice no longer depends on which random Work id sorts higher -- and the second is reported rather than silently dropped. The discarded id is not reserved: it denotes nothing, and reserving it would burn an id out of the space for good.
func (*Resolver) SeedWorkKey ¶
SeedWorkKey records the computed clustering key of a committed Work, so a new record with the same key clusters onto it. The caller recomputes the key from the Work's data with WorkKey.
func (*Resolver) WorkForProviderKey ¶ added in v0.171.0
WorkForProviderKey returns the Work id a provider key resolves to when the resolver already knows it (seeded from a prior grain), following merges -- and nothing (false) otherwise. It has no side effects, unlike Resolve. Used to translate a feed's cluster-merge statement, which names records by provider id, into the Work-id space SeedMerge operates on.
When the exact key is not indexed, it falls back to the cluster's format buckets: a single-format cluster's grain indexes only the format-suffixed instance key (e.g. "id:coll:51812:ebook"), never the bare cluster key ("id:coll:51812") a feed merge names, so the exact lookup misses and the merge is skipped. Every instance of a cluster resolves to the same Work, so the bare key is resolved through any indexed key that extends it with a ":<suffix>". The trailing colon keeps "id:coll:5181" from matching "id:coll:51812:..."; agreement across all matching buckets is required, so a cluster already split across Works resolves to nothing rather than guessing.
type WorkIdentity ¶
WorkIdentity is the committed identity of one Work recovered from a grain: its minted id and its recomputed clustering key, so a new record with the same key clusters onto it rather than minting a fresh Work.
Source Files