Documentation
¶
Overview ¶
Package export derives the downloadable catalog artifacts from an ingest output root: catalog.nq.gz is the corpus itself -- the catalog.nq found there, which every writer now emits as the merge of the grains, with blank labels namespaced by work id -- and catalog.mrc.gz / catalog.xml.gz are per-grain MARC round-trips via bibframe.DecodeGrainMARC, which honors editorial override shadows and verbatim sidecars (fidelity bounded by docs/marc-fidelity.md). A Manifest records sizes, sha256 digests, and record counts for a downloads page. This is the static-site counterpart of the backend's on-demand export service (backend/export); both read the same grains.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type File ¶
type File struct {
Name string `json:"name"`
Bytes int64 `json:"bytes"`
SHA256 string `json:"sha256"`
Records int `json:"records"`
}
File is one artifact's manifest entry; the sha256 is over the compressed bytes (what a downloader verifies).
type Manifest ¶
type Manifest struct {
Generated string `json:"generated"`
// Works counts the Works the download describes -- the visible ones, so this
// number is comparable with catalog.json's. It used to count grains, which
// cannot disagree with anything: a build that published 3274 records for a
// 31-work catalog reported 3274 and looked healthy.
Works int `json:"works"`
// Hidden counts the grains held back as suppressed or tombstoned. Recorded so
// a takedown is auditable from the build output rather than inferable from it.
Hidden int `json:"hidden"`
Files []File `json:"files"`
}
Manifest is the downloads-page data file: what was generated when, from how many works, with per-artifact integrity and record counts.
type Options ¶
type Options struct {
// In is the ingest output root: the directory holding catalog.nq and
// data/works/ grains.
In string
// Out is the directory the gzipped artifacts are written to.
Out string
// PublicSources, when non-nil, is the allowlist of extra/sources names
// permitted in the nq download; other attributions are stripped, matching
// project.SanitizeSources on the catalog.json side, so no public surface
// discloses more provenance than the site does. Nil keeps every source.
// The on-disk graph of record stays complete either way.
PublicSources map[string]bool
// PublicExtras, when non-nil, is the allowlist of lcat:extra/ *keys*
// permitted in the nq download, matching project.SanitizeExtras on the
// catalog.json side. Other extra quads are dropped whole.
// "sources" is exempt: PublicSources governs it, by value rather than by
// key. Nil keeps every extra.
PublicExtras map[string]bool
// Log receives skip and strip warnings; nil means os.Stderr.
Log io.Writer
// OrgCode is the deployment's MARC organization code; when set, the
// MARC download derives each record's 040 from graph facts at decode
// time.
OrgCode string
// CoversOut, when set, copies uploaded cover images (data/covers/ under
// In) to this directory as flat files, the site-relative covers/ URLs
// the editorial lcat:extra/cover statements point at.
// Empty skips the copy.
CoversOut string
}
Options configures one export run.