worker

package

v0.1.2 Latest Latest Go to latest Published: Nov 22, 2025 License: MIT Imports: 8 Imported by: 0

Details

This section is empty.

This section is empty.

This section is empty.

type Builder struct {
	// contains filtered or unexported fields
}

Builder creates a configured conduit with processors using builder pattern.

Example:

conduit, err := worker.New().
    Blueprint(bp).
    Job("process").
    Concurrency(4).
    Build()

func New() *Builder

New creates a new conduit builder with default configuration.

func (b *Builder) Build() (*ConduitWithReporter, error)

Build creates the configured conduit with blueprint processor. Returns a wrapped conduit that includes progress reporter in context.

func (b *Builder) Concurrency(n int) *Builder

Concurrency sets the number of parallel processing workers. Default is 1 (sequential processing).

func (b *Builder) ErrorMode(mode conduit.ErrorMode) *Builder

ErrorMode sets how errors are handled during processing. Options: conduit.FailFast (default) or conduit.SkipError.

func (b *Builder) Jsonify(enable bool) *Builder

func (b *Builder) Metrics(fn conduit.MetricsFunc) *Builder

Metrics sets a callback for periodic metrics updates.

func (b *Builder) Progress(fn conduit.ProgressFunc) *Builder

Progress sets a callback for progress updates after each document.

func (b *Builder) Reporter(r *progress.Reporter) *Builder

Reporter sets the progress reporter and wires up callbacks

func (b *Builder) Runtime() *Builder

func (b *Builder) Splitter(conf processor.ChunkConfig) *Builder

func (b *Builder) Workflow(file string, llm chatter.Chatter) *Builder

Workflow sets the blueprint to use for creating processors. This is required.

type ConduitWithReporter struct {
	*conduit.Conduit
	// contains filtered or unexported fields
}

ConduitWithReporter wraps a conduit and injects progress reporter into context

func (c *ConduitWithReporter) Run(ctx context.Context, source iosystem.Source, sink iosystem.Sink) (*conduit.Stats, error)

Run executes the pipeline with progress reporter in context