workers

package
v1.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 28, 2026 License: MIT Imports: 8 Imported by: 0

Documentation

Overview

Package workers collects url graph from provided API and stores it into provided storage

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Parser

type Parser interface {
	Parse(ctx context.Context, url string) (model.ParsedArticle, error)
}

type Storage

type Storage interface {
	GetURLToProcess(ctx context.Context) (string, error)
	SaveParsedArticle(ctx context.Context, article model.ParsedArticle) error

	AddPendingURLs(ctx context.Context, urls ...string) error
	ResetInProgressURLs(ctx context.Context) error
	SetFailed(ctx context.Context, url string, err error) error
}

type Workers

type Workers struct {
	// contains filtered or unexported fields
}

func New

func New(workers int, rate time.Duration, parser Parser, storage Storage) *Workers

func (*Workers) Launch

func (ws *Workers) Launch(ctx context.Context, initialUrls ...string) error

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL