span-crossref-fastproc

Overview ¶

span-crossref-fastproc takes one or more raw crossref daily data slices (zstd compressed) and produces solr-importable output by running the equivalent of: span-import -i crossref | span-tag -unfreeze filterconfig.zip | span-export -with-fullrecord.

The filterconfig can be supplied as a frozen zip file (-f) or fetched directly from FOLIO API (via OKAPI_URL and OKAPI_TOKEN env vars), with automatic caching.

With "-o DIR" (the default), each input is written to DIR as a zstd file named after the input. With "-o -", all inputs are streamed uncompressed to stdout, e.g. for piping straight into solrbulk.

Usage:

span-crossref-fastproc -o /output/dir slice-2026-03-02.json.zst
span-crossref-fastproc -f filterconfig.zip slice-a.json.zst slice-b.json.zst
span-crossref-fastproc -o - slice-2026-03-02.json.zst | solrbulk -server ...