pipeline

package
v1.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 2, 2026 License: MIT Imports: 5 Imported by: 0

Documentation

Overview

Package pipeline provides pre-processing and post-processing stages.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type CompressionStats

type CompressionStats struct {
	OriginalSize   int
	CompressedSize int
	TokensSaved    int
	Algorithm      string
}

CompressionStats contains compression statistics.

func (CompressionStats) ReductionPercent

func (s CompressionStats) ReductionPercent() float64

ReductionPercent returns the reduction percentage.

type Pipeline

type Pipeline struct {
	// contains filtered or unexported fields
}

Pipeline coordinates pre and post processing.

func NewPipeline

func NewPipeline() *Pipeline

NewPipeline creates a new processing pipeline.

func (*Pipeline) Process

func (p *Pipeline) Process(content string, compressFunc func(string) (string, int)) (*PipelineResult, error)

Process runs the full pipeline.

type PipelineResult

type PipelineResult struct {
	PreProcess  *PreProcessResult  `json:"pre_process"`
	PostProcess *PostProcessResult `json:"post_process"`
	FinalOutput string             `json:"final_output"`
}

PipelineResult contains the full pipeline results.

type PostProcessOptions

type PostProcessOptions struct {
	AddMarkers   bool
	AddStats     bool
	VerifyOutput bool
}

PostProcessOptions provides post-processing options.

type PostProcessResult

type PostProcessResult struct {
	Content        string  `json:"content"`
	OriginalSize   int     `json:"original_size"`
	CompressedSize int     `json:"compressed_size"`
	TokensSaved    int     `json:"tokens_saved"`
	ReductionPct   float64 `json:"reduction_percent"`
	Marker         string  `json:"marker,omitempty"`
	Verified       bool    `json:"verified"`
}

PostProcessResult contains post-processing results.

type PostProcessor

type PostProcessor struct {
	// contains filtered or unexported fields
}

PostProcessor handles post-processing after compression.

func NewPostProcessor

func NewPostProcessor() *PostProcessor

NewPostProcessor creates a new post-processor.

func (*PostProcessor) Process

func (p *PostProcessor) Process(original, compressed string, stats CompressionStats) *PostProcessResult

Process runs post-processing.

func (*PostProcessor) SetOptions

func (p *PostProcessor) SetOptions(opts PostProcessOptions)

SetOptions sets post-processing options.

type PreProcessOptions

type PreProcessOptions struct {
	StripANSI       bool
	Normalize       bool
	DetectPII       bool
	DetectInjection bool
}

PreProcessOptions provides pre-processing options.

type PreProcessResult

type PreProcessResult struct {
	Content       string             `json:"content"`
	ContentType   string             `json:"content_type"`
	Language      string             `json:"language"`
	SecurityScan  []security.Finding `json:"security_scan,omitempty"`
	ANSIRemoved   bool               `json:"ansi_removed"`
	AppliedGates  []string           `json:"applied_gates,omitempty"`
	OriginalSize  int                `json:"original_size"`
	ProcessedSize int                `json:"processed_size"`
}

PreProcessResult contains pre-processing results.

type PreProcessor

type PreProcessor struct {
	// contains filtered or unexported fields
}

PreProcessor handles pre-processing before compression.

func NewPreProcessor

func NewPreProcessor() *PreProcessor

NewPreProcessor creates a new pre-processor.

func (*PreProcessor) Process

func (p *PreProcessor) Process(content string) *PreProcessResult

Process runs pre-processing.

func (*PreProcessor) SetOptions

func (p *PreProcessor) SetOptions(opts PreProcessOptions)

SetOptions sets pre-processing options.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL