ingestionqueue

package

v0.2.5 Latest Latest Go to latest Published: Nov 1, 2026 License: MIT Imports: 6 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/flexdinesh/diffx

Links

Open Source Insights

Documentation ¶

Overview ¶

Package ingestionqueue owns durable admission and worker contracts.

Index ¶

Variables
func Run(ctx context.Context, queue Queue, ingest Ingest, options ...Option) error
type Delivery
type Ingest
type Job
type Option
- func Wake(signal <-chan struct{}) Option
type Queue

Constants ¶

This section is empty.

Variables ¶

View Source

var ErrEmpty = errors.New("no ingestion jobs available")

View Source

var ErrFull = errors.New("ingestion queue full; retry later")

View Source

var ErrLeaseLost = errors.New("ingestion lease lost")

Functions ¶

func Run ¶

func Run(ctx context.Context, queue Queue, ingest Ingest, options ...Option) error

Run uses bounded processing and renews leases without coupling ingestion to the queue implementation. A crash after commit is recovered by ingestion replay. Queue storage failures are logged and retried after a backoff instead of stopping the server; an unacknowledged lease expires and is claimed again.

Types ¶

type Delivery ¶

type Delivery struct {
	Job
	OwnerID string
	Token   string
	Request ingestion.Request
}

type Ingest ¶

type Ingest func(context.Context, string, ingestion.Request) (string, error)

type Job ¶

type Job = ingestion.Job

type Option ¶ added in v0.2.2

type Option func(*worker)

Option adjusts a worker.

func Wake ¶ added in v0.2.2

func Wake(signal <-chan struct{}) Option

Wake lets an idle worker claim as soon as signal delivers, such as when another process admits a job. Polling remains the fallback.

type Queue ¶

type Queue interface {
	Accept(context.Context, string, ingestion.Request) (Job, error)
	Get(context.Context, string, string) (Job, error)
	Claim(context.Context, time.Duration) (Delivery, error)
	Renew(context.Context, Delivery, time.Duration) error
	Complete(context.Context, Delivery, string) error
	Retry(context.Context, Delivery, time.Time, string) error
	Fail(context.Context, Delivery, string) error
}

Queue guarantees durable, idempotent acceptance and fenced job transitions. Completed jobs retain their identity even when the observation is deleted. An external broker adapter must atomically persist scheduling intent (outbox).

Source Files ¶

View all Source files

queue.go

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL