span-freeze

Overview ¶

Freeze file containing urls along with the content of all urls into a zip file. Supports two modes:

Legacy mode (default): reads a blob from stdin, extracts URLs, downloads them, and packages everything into a zip file.

FOLIO mode (-f): fetches metadata collections from FOLIO API, builds a span-tag compatible filter configuration, downloads referenced files, and packages everything into a same zip format.

Output zip structure:

/blob          original input or generated filterconfig JSON
/mapping.json  URL to local path mapping
/files/<sha1>  downloaded content

$ curl -s https://queue.acm.org/ | span-freeze -b -o acm.zip
$ OKAPI_TOKEN=xxx span-freeze -f -okapi-url https://... -o folio.zip