scrape

package

v0.6.1 Latest Latest Go to latest Published: Apr 1, 2026 License: Apache-2.0 Imports: 8 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/datallboy/gonzb

Links

Open Source Insights

Documentation ¶

Index ¶

type GroupStats
type NNTPAdapter
- func NewNNTPAdapter(p nntp.Provider) *NNTPAdapter
type Options
type OverviewHeader
type Service
- func NewService(repo repository, p provider, log logger, opts Options) *Service

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

This section is empty.

Types ¶

type GroupStats ¶

type GroupStats struct {
	Low  int64
	High int64
}

type NNTPAdapter ¶

type NNTPAdapter struct {
	// contains filtered or unexported fields
}

func NewNNTPAdapter ¶

func NewNNTPAdapter(p nntp.Provider) *NNTPAdapter

func (*NNTPAdapter) GroupStats ¶

func (a *NNTPAdapter) GroupStats(ctx context.Context, group string) (GroupStats, error)

func (*NNTPAdapter) ID ¶

func (a *NNTPAdapter) ID() string

func (*NNTPAdapter) XOver ¶

func (a *NNTPAdapter) XOver(ctx context.Context, group string, from, to int64) ([]OverviewHeader, error)

type Options ¶

type Options struct {
	Newsgroups []string
	BatchSize  int64
}

type OverviewHeader ¶

type OverviewHeader struct {
	ArticleNumber int64
	MessageID     string
	Subject       string
	Poster        string
	DateUTC       *time.Time
	Bytes         int64
	Lines         int
	Xref          string
	RawOverview   map[string]any
}

type Service ¶

type Service struct {
	// contains filtered or unexported fields
}

func NewService ¶

func NewService(repo repository, p provider, log logger, opts Options) *Service

func (*Service) RunBackfillOnce ¶

func (s *Service) RunBackfillOnce(ctx context.Context) error

backfill mode walks backward from the most recent known boundary.

func (*Service) RunLatestOnce ¶

func (s *Service) RunLatestOnce(ctx context.Context) error

latest mode prioritizes the head of the group and continues forward.

func (*Service) RunOnce ¶

func (s *Service) RunOnce(ctx context.Context) error

backward-compatible alias. Default scrape mode is latest.

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL