Documentation
¶
Overview ¶
Package archive is the capture pipeline that turns a target into a tori repository: it plans what to fetch on the cheapest free tier, streams the records through the x-cli engine into the store, localises the media, renders the HTML and Markdown views, and writes the manifest (spec §5, §7). It owns no scraping of its own — every byte from X comes through the x-cli engine (TP1) — and no presentation — every page comes through render/ (TP3). What it adds is the artifact: a self-contained, deterministic, resumable archive.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Logf ¶
Logf is the optional progress sink the capture writes human-readable lines to. The CLI passes a function that prints to stderr; a nil Logf is silent, which is what the tests use so they stay quiet and deterministic.
type Options ¶
type Options struct {
// Output.
Out string // output root; the repo lands at <Out>/x/<root>
Views []string // which rendered shapes to write: "html", "md"
// Media.
Media media.Policy // all | photos | none
Video media.VideoPref // best | worst
VideoKbps int // when >0, pick the rendition nearest this bitrate (overrides Video)
Tool string // optional external downloader for stream-only video (e.g. yt-dlp)
// Record shaping.
Max int // hard record budget; 0 means as many as the tier gives
Thread bool // upgrade a tweet target into a full-conversation capture
WithReplies bool // include replies in a profile/timeline capture
WithRetweets bool // include retweets in a profile/timeline capture
MediaOnly bool // only tweets carrying media
// Timeline bounds.
Since time.Time // only tweets at or after this time
Until time.Time // only tweets before this time
SinceID string // only tweets newer than this id
UntilID string // only tweets older than this id
// Full-history exhaustion. A user timeline caps at roughly 3200 tweets, so to
// archive an account's whole history tori walks month-wide search windows
// (from:<handle> since:.. until:..) from the account's creation to now,
// stitching the results into one timeline (spec §7.1). Requires a GraphQL tier.
ByMonth bool
// Link chasing (bounded), via x.Walk.
Depth int
Fanout int
// Run control.
Date time.Time // capture stamp written into the manifest (TP5); zero means now
Resume bool // continue from state.json when present (default on)
Force bool // ignore held state and re-capture from scratch
DryRun bool // plan only; fetch and write nothing
Verbose bool // log the tier and endpoint per record
Version string // tori build version, recorded in the manifest
}
Options is the resolved configuration for one capture, built from the CLI flags. The zero value is not valid; the CLI fills it. Fetch-shaping fields mirror the x-cli timeline/search readers so a tori flag maps straight onto an engine call (spec §12.1).
type RenderOptions ¶
type RenderOptions struct {
Views []string // which shapes to write: "html", "md"
Date time.Time // footer stamp; zero means now
Version string
}
RenderOptions controls a re-render of an existing repository.
type RenderResult ¶
RenderResult summarises a re-render.
func Render ¶
func Render(root string, o RenderOptions) (*RenderResult, error)
Render re-renders the views of an existing repository from its stored JSON, touching no network (spec §5). It reads the records, the profile, and the manifest's media index (so localised media still resolves), reconstructs the threads, and writes the requested views over the existing ones.
type Result ¶
type Result struct {
Root string // repository directory written
Target Target // what was captured
Profile *User // the captured profile, when the target had one
Added int // tweets newly written this run
Total int // tweets in the repo after this run
Threads int // reconstructed conversations
MediaOK int // media files on disk after this run
MediaFail int // media references that could not be localised
StreamOnly int // video with no progressive rendition
Tiers []string // tiers that served this capture
Oldest string // oldest captured tweet timestamp (RFC3339), empty if none
Newest string // newest captured tweet timestamp
DryRun bool // true when nothing was fetched or written
Windows int // month-windows walked for a full-history profile capture
// contains filtered or unexported fields
}
Result summarises a completed capture for the CLI to print and for the exit code to reflect (a partial capture with failures is still a success on disk).
func Run ¶
Run captures a target into a repository under opts.Out and returns a summary. It is the one entry the CLI's archive/add commands call. The pipeline is: resolve and stream records into the store (writing each as it arrives so a session-limited or interrupted run keeps what it got), localise media, render the requested views, and write the manifest. Records always persist as JSON; media and views are derived and regenerable with `tori render`.
type Selector ¶
Selector carries the flags that pick a non-profile target kind, so a bare argument is read in the light of --search/--bookmarks/--likes/--list/--thread (spec §12.1). Exactly one of the kind-selecting fields may be set.
type Target ¶
Target is a parsed, canonical capture target. Ref is the canonical identity used to both address the source and name the repository root: a profile's handle, a tweet/thread/list id, or a search query. Display is a human label for the page nav and headings.
func ParseTarget ¶
ParseTarget resolves a CLI argument and the kind-selecting flags into a canonical Target. The grammar is x-cli's own ref grammar (TP via ParseTweetRef / ParseUserRef) plus tori's capture keywords: a status URL or numeric id is a tweet (a thread with --thread), a @handle or profile URL is a profile, and the --search/--bookmarks/--likes/--list flags select the timeline kinds. arg may be empty when a flag fully specifies the target (--bookmarks, --search).
Source Files