Documentation
¶
Overview ¶
Package capturebatch implements the chrest side of the cutting-garden Capture Plugin Protocol (RFC 0002) under the web-archive binding (RFC 0003). The `capture-batch` subcommand reads a batch of capture requests as JSON on stdin, runs them sequentially, assembles each capture's receipt merkle tree of typed hyphence blobs via the shared cutting-garden capture_plugin builder, streams every node through the orchestrator-supplied writer.cmd subprocess, and emits one receipt ref per capture in a JSON result envelope on stdout.
Index ¶
- Constants
- Variables
- func Canonicalize(v any) ([]byte, error)
- func NewBatchHandler(capturerVersion string) capture_serve.BatchFunc
- func Normalize(format string, raw []byte) (normalized []byte, stripped map[string]any, err error)
- func NormalizeStream(format string, src io.Reader) (io.Reader, map[string]any, error)
- type BatchInput
- type BatchOutput
- type CaptureResult
- type CaptureSpec
- type Defaults
- type Extension
- type Options
- type PluginInfo
- type ProtocolError
- type ReceiptRef
- type Resolved
- type WriterResult
- type WriterSpec
Constants ¶
const BatchSchema = "capture-plugin/v1"
BatchSchema is the wire schema token for both the batch input and the batch output (cutting-garden capture-plugin/v1).
const CapturerName = "chrest"
CapturerName is chrest's plugin identifier — the `environment.binary.name` discriminator that identifies chrest as the binary that produced a receipt's bytes (RFC 0002 §Environment).
Variables ¶
var PayloadMediaTypes = map[string]string{
"text": "text/plain; charset=utf-8",
"pdf": "application/pdf",
"screenshot": "image/png",
"mhtml": "multipart/related",
"a11y": "application/json",
"html-monolith": "text/html; charset=utf-8",
"html-outer": "text/html; charset=utf-8",
"markdown-full": "text/markdown; charset=utf-8",
"markdown-reader": "text/markdown; charset=utf-8",
"markdown-selector": "text/markdown; charset=utf-8",
}
PayloadMediaTypes enumerates the supported capture formats (the value is the format's IANA media type; the type is implied by the payload node's RFC 0003 type string). Membership doubles as the format-validity check in runOne.
Functions ¶
func Canonicalize ¶
Canonicalize encodes v as JCS (RFC 8785) bytes.
Our schema uses strings, integers, booleans, objects, arrays, and null — no floating-point numbers. This implementation is correct for that subset:
- map keys are sorted by UTF-16 code units (same as alphabetical for ASCII-only keys, which our schema uses);
- objects and arrays emit with no whitespace;
- strings are escaped per RFC 8785 §3.2.2.2 (only required control chars are escaped; Go's default json.Encoder escapes more);
- booleans and nulls emit as `true` / `false` / `null`;
- integers (int, int64, json.Number) emit in base 10 with no leading zeros or `+`.
If the schema ever grows floating-point fields, this will need the ES6 ToString semantics from RFC 8785 §3.2.2.3; that is intentionally out of MVP scope.
func NewBatchHandler ¶ added in v0.3.0
func NewBatchHandler(capturerVersion string) capture_serve.BatchFunc
NewBatchHandler adapts chrest's receipt-building runner to cutting-garden's RFC 0008 capture_serve.BatchFunc — the plugin-side capture.batch handler Serve calls once per batch received over the JSON-RPC control connection. The receipt-assembly path (runOneWithWriter) is identical to the v1 capture-batch command; only the wire types differ: capture_serve.BatchParams carries no writer.cmd (w already realizes capture_plugin.Writer over the RFC 0008 blob protocol — see server.go's blobProtocolWriter upstream), and per-capture options arrive pre-parsed as map[string]any rather than raw JSON, so they're re-marshaled at the edge to reach the shared Resolved type unchanged.
func Normalize ¶
Normalize produces the payload bytes that the writer should store when split=true. Each format has its own normalization rules specified in RFC 0001 §Payload Artifact. Unsupported formats return a not-implemented error so the runner can surface it as a per-capture error.
MVP scope: "text", "screenshot", "pdf", and "mhtml" are implemented. "a11y" is blocked on chrest#14 (Chrome SIGTRAP on kernel 6.17) and returns the not-implemented error until that lifts.
func NormalizeStream ¶
NormalizeStream is the streaming counterpart to Normalize. It reads the full input into memory, normalizes, and returns a reader plus the stripped map. Normalization is unavoidably buffering for most formats (need to see the whole document), so streaming here is about interface symmetry with StreamCapture rather than memory.
Types ¶
type BatchInput ¶ added in v0.3.0
type BatchInput struct {
Schema string `json:"schema"`
Writer WriterSpec `json:"writer"`
Target string `json:"target"`
Defaults *Defaults `json:"defaults,omitempty"`
Captures []CaptureSpec `json:"captures"`
}
BatchInput is the single JSON document read from stdin.
type BatchOutput ¶ added in v0.3.0
type BatchOutput struct {
Schema string `json:"schema"`
Plugin PluginInfo `json:"plugin"`
Errors []ProtocolError `json:"errors"`
Captures []CaptureResult `json:"captures"`
}
BatchOutput is the single JSON document written to stdout.
func Run ¶
func Run(ctx context.Context, captures []CaptureSpec, opts Options) (BatchOutput, error)
Run executes every capture in order and returns the batch output. The runner never fails fatally on per-capture errors — they become CaptureResult.Error entries. Batch-level failures (e.g. an empty writer.cmd or target) are returned as errors.
type CaptureResult ¶ added in v0.3.0
type CaptureResult struct {
Name string `json:"name"`
Receipt *ReceiptRef `json:"receipt,omitempty"`
Error *ProtocolError `json:"error,omitempty"`
}
CaptureResult is one entry in the batch output `captures` array. Exactly one of Receipt or Error is set.
type CaptureSpec ¶ added in v0.3.0
type CaptureSpec struct {
Name string `json:"name"`
Format string `json:"format"`
Options json.RawMessage `json:"options,omitempty"`
}
CaptureSpec is one entry in the batch input `captures` array. Options stay raw so the format dispatcher and the invocation echo each parse them independently.
type Defaults ¶ added in v0.3.0
type Defaults struct {
Normalize *bool `json:"normalize,omitempty"`
Plugin map[string]any `json:"plugin,omitempty"`
}
Defaults carries batch-level fields applied to every capture. `plugin` is the plugin-namespaced defaults object; chrest reads `plugin.browser`.
type Extension ¶
type Extension struct {
ID string `json:"id"`
Version string `json:"version"`
ManifestDigest string `json:"manifest_digest,omitempty"`
}
Extension is a browser extension echoed into the plugin-environment node. The orchestrator does not yet send extensions on the wire (the CaptureSpec has no extensions field); this type backs the preinstalled-extension mapping until fetched-extension support (#55).
type Options ¶
type Options struct {
CapturerVersion string
Writer WriterSpec
Target string
Defaults *Defaults
}
Options configure the runner; most come from BatchInput.
type PluginInfo ¶ added in v0.3.0
PluginInfo identifies the capture plugin + version.
type ProtocolError ¶ added in v0.3.0
ProtocolError is a batch-level (errors[]) or per-capture (captures[].error) failure.
type ReceiptRef ¶ added in v0.3.0
ReceiptRef points to a capture's root receipt blob by its markl id.
type Resolved ¶
type Resolved struct {
Name string
Format string
Options json.RawMessage
Browser string
Normalize bool
Isolation string
Extensions []Extension
}
Resolved is a capture after batch defaults have been applied.
func Resolve ¶
func Resolve(c CaptureSpec, def *Defaults) Resolved
Resolve applies batch defaults to a single capture spec. Browser comes from `defaults.plugin.browser` (default firefox, the only backend); normalize comes from `defaults.normalize`.
type WriterResult ¶
WriterResult is the shape the writer protocol returns on stdout. RFC 0001 §Writer Protocol allows additional fields; we ignore them.
func WriteThrough ¶
WriteThrough spawns the writer subprocess declared by cmd, streams src into its stdin until EOF, closes stdin, and parses the single JSON object the writer writes to stdout.
Per RFC 0001 §Writer Protocol, the writer MUST exit 0 on success and MUST write exactly one line of JSON to stdout containing `id` and `size`. Non-zero exit or malformed stdout is a hard error; the caller maps it into a per-capture error.
Source Files
¶
- jcs.go
- mapping.go
- mhtml.go
- normalize.go
- pdf.go
- png.go
- receipt.go
- runner.go
- serve.go
- types.go
- writer.go