Documentation
¶
Overview ¶
Package stream moves records from a Source through a Cascade into a Sink. Unclassified records go to an optional review sink. A nil review sink drops them.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Engine ¶
type Engine struct {
// contains filtered or unexported fields
}
Engine drives records through the cascade.
type Options ¶
type Options struct {
// Source yields records and is required.
Source source.Source
// Cascade classifies records and is required.
Cascade *classmesh.Cascade
// Sink receives classified records and is required.
Sink sink.Sink
// Review receives unclassified records with a zero Classification.
// A nil Review drops those records.
Review sink.Sink
// Logger defaults to slog.Default.
Logger *slog.Logger
// Workers controls concurrent classification. Zero or one uses the serial loop.
// Larger values require concurrent-safe stages and preserve serial outcomes.
Workers int
}
Options configures an Engine.
type Stats ¶
type Stats struct {
// Processed counts records read from the source.
Processed int
// Classified counts records some stage decided.
Classified int
// Reviewed counts records no stage decided.
Reviewed int
// ByStage counts classifications per stage name.
ByStage map[string]int
}
Stats summarizes one Run.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package sink defines where classified records go: stdout, files, review queues, downstream pipelines.
|
Package sink defines where classified records go: stdout, files, review queues, downstream pipelines. |
|
jsonl
Package jsonl implements sink.Sink as JSON Lines: one JSON object per classified record, the pipe-friendly lingua franca of log tooling.
|
Package jsonl implements sink.Sink as JSON Lines: one JSON object per classified record, the pipe-friendly lingua franca of log tooling. |
|
Package source defines where Records come from.
|
Package source defines where Records come from. |
|
jsonl
Package jsonl implements source.Source over JSON Lines: one JSON object per line.
|
Package jsonl implements source.Source over JSON Lines: one JSON object per line. |
|
text
Package text implements source.Source over line-oriented text: files and stdin.
|
Package text implements source.Source over line-oriented text: files and stdin. |
Click to show internal directories.
Click to hide internal directories.