Documentation
¶
Index ¶
- Variables
- func FastFailReleaseProbe(ctx context.Context, files []FastFailFile, poolManager pool.Manager, ...) (bool, error)
- func ValidateSegmentsForFile(filename string, fileSize int64, segments []*metapb.SegmentData, ...) error
- type FastFailFile
- type FastFailFileResult
- func DeadReleaseResults(files []FastFailFile, missingIDs []string) []FastFailFileResult
- func FastFailCheckFiles(ctx context.Context, files []FastFailFile, poolManager pool.Manager, ...) ([]FastFailFileResult, error)
- func PlaceholderResults(files []FastFailFile, patchIdx PatchIndex) []FastFailFileResult
- type PatchIndex
- type ProbeVerdict
Constants ¶
This section is empty.
Variables ¶
var ( // ErrFastFailInconclusive means bounded retries could not establish whether // one or more sampled articles exist. It is deliberately distinct from a // definitive NNTP 430/423 miss so callers do not discard files as broken. ErrFastFailInconclusive = errors.New("fast-fail validation inconclusive") )
Functions ¶
func FastFailReleaseProbe ¶
func ValidateSegmentsForFile ¶
func ValidateSegmentsForFile( filename string, fileSize int64, segments []*metapb.SegmentData, encryption metapb.Encryption, ) error
ValidateSegmentsForFile performs local structural validation of file segments including size verification. It validates that segments are structurally sound (valid offsets, non-empty IDs) and that their total size matches the expected file size (accounting for encryption overhead). Network reachability is handled solely by the fast-fail pass at import start.
Types ¶
type FastFailFile ¶
type FastFailFile struct {
Filename string
Segments []*metapb.SegmentData
// GroupKey identifies the multi-volume set this file belongs to (e.g. a RAR
// base name). Empty means the file is standalone. When any member of a group
// is found unreachable, FastFailCheckFiles skips the remaining Stats for that
// group and marks every member Broken — a missing volume dooms the whole set
// (no PAR2 repair at import time), so probing the rest is wasted work.
GroupKey string
}
FastFailFile is the minimal file surface needed for early segment reachability checks.
type FastFailFileResult ¶
type FastFailFileResult struct {
Broken bool
MissingSegmentIDs []string // segment IDs whose Stat failed, plus known gap placeholders
// KnownGapCount is how many of MissingSegmentIDs are gap placeholders:
// articles the NZB never listed, known missing without a STAT. Exact,
// not sampled, so callers judge them separately from the sample.
KnownGapCount int
// SampledCount is how many of the file's segments were Stat-checked (the
// sample size), needed to project the release-wide miss rate for the
// tolerant damage policy.
SampledCount int
}
FastFailFileResult records the reachability outcome for a single FastFailFile. Results from FastFailCheckFiles are index-aligned with the input slice.
func DeadReleaseResults ¶
func DeadReleaseResults(files []FastFailFile, missingIDs []string) []FastFailFileResult
DeadReleaseResults is the per-file outcome of a release the probe judged dead: every file with segments is broken, carrying whichever sampled misses were its own. Index-aligned with files, like FastFailCheckFiles' results.
func FastFailCheckFiles ¶
func FastFailCheckFiles( ctx context.Context, files []FastFailFile, poolManager pool.Manager, segmentSamplePercentage int, maxConnections int, timeout time.Duration, progressTracker progress.ProgressTracker, patchIdx PatchIndex, stopFileOnFirstMiss bool, articleDate time.Time, ) ([]FastFailFileResult, error)
FastFailCheckFiles stats a per-file sample of segments from all files. Every file with segments is checked — broken files are excluded from parsing, and if only PAR2 files survive the import fails naturally. Pass nil Segments for files that should be skipped (e.g. PAR2/sidecars) to keep index alignment while avoiding wasted Stat round-trips. Returns one result per input file (index-aligned). Files with no segments are skipped. Infrastructure failures (pool unavailable) are returned as an error; definitive article-not-found results mark the owning file Broken. Operational failures are retried, and exhaustion returns ErrFastFailInconclusive without marking files broken. progressTracker may be nil; when set it reports completed Stats as work progresses. stopFileOnFirstMiss condemns a file (and its group) on its first definitive miss and ends the sweep once no eligible file is left. Callers running with zero missing-segment tolerance set it: the extent of the damage cannot change the verdict, so mapping the rest of a doomed file is wasted work.
func PlaceholderResults ¶
func PlaceholderResults(files []FastFailFile, patchIdx PatchIndex) []FastFailFileResult
PlaceholderResults maps the damage an NZB declares itself: every file whose segments include gap placeholders (articles the NZB never listed) is reported Broken with those ids as known misses, without a single STAT. Index-aligned with files; gap-free files get the zero result. The group is not condemned here: an exactly-known gap is judged against the hole caps by the caller, which can keep a lightly holed archive set importable.
A placeholder with a local patch is not a miss: PAR2 repair rebuilt it under that id and the reader serves the patch. Counting it would park a repaired import for the same repair again.
type PatchIndex ¶
FastFailReleaseProbe is the cheap phase-1 reachability gate for an NZB import. It flattens all candidate segments across the release and Stats a single sample (usenet.SelectSegmentsForValidation: first 3 + last 2 + random middle, min 5 for the whole release), cancelling the remaining Stats on the first definitive miss. Operational errors are retried before the probe is declared inconclusive.
Returns (missing, err):
- err reports infrastructure failures, caller cancellation, or operational STAT failures that remain inconclusive after bounded retries.
- missing reports whether any sampled segment returned a definitive 430/423, so the caller can escalate to the per-file FastFailCheckFiles sweep to map exactly which files are broken. Operational errors and timeouts are retried; exhaustion returns ErrFastFailInconclusive. A clean release returns (false, nil) and proceeds straight to parsing.
PatchIndex reports whether a locally repaired copy of an article exists. Repaired bytes live only in AltMount's patch store, never on usenet, so a segment the providers have dropped still counts as available when patched — without this a repaired release could never be (re-)imported.
type ProbeVerdict ¶
type ProbeVerdict struct {
// Missing reports a definitive 430/423 on at least one sampled article.
Missing bool
// Dead reports that the first wave alone condemned the release (see
// releaseLooksDead): the caller need not map which files are broken.
Dead bool
// MissingIDs are the sampled articles found missing.
MissingIDs []string
}
ProbeVerdict is what the release probe learned from its sample.
func FastFailReleaseProbeVerdict ¶
func FastFailReleaseProbeVerdict( ctx context.Context, files []FastFailFile, poolManager pool.Manager, segmentSamplePercentage int, maxConnections int, timeout time.Duration, patchIdx PatchIndex, articleDate time.Time, ) (verdict ProbeVerdict, err error)
FastFailReleaseProbeVerdict runs the release probe in two waves: a small first wave whose misses can already prove the post dead, then — only when that wave was clean — the rest of the sample, cancelled on the first miss. Errors and the inconclusive rules are those of statIDsWithBoundedRetries.
Source Files