preprocessing

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 4, 2026 License: Apache-2.0 Imports: 6 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func RemoveHeaderFooter 

func RemoveHeaderFooter(ctx context.Context, pages []string, mode CleanupMode, spec *configs.DocSpec, llmModel string, request LLMRequestFunc) ([]string, error)

RemoveHeaderFooter 根据指定模式移除页眉页脚

Types

type CleanupMode 

type CleanupMode string

CleanupMode 页眉页脚清理模式 

const (
	CleanupNone         CleanupMode = "none"         // 不清理
	CleanupProgrammatic CleanupMode = "programmatic" // 程序清理
	CleanupLLM          CleanupMode = "llm"          // LLM清理
	CleanupAuto         CleanupMode = "auto"         // 自动清理(程序+LLM)
)

func ParseCleanupMode 

func ParseCleanupMode(mode string) CleanupMode

ParseCleanupMode 解析清理模式字符串

type CleanupStats 

type CleanupStats struct {
	Mode                 string
	TotalPages           int
	HeaderCleanedPages   int
	FooterCleanedPages   int
	SampleRemovedHeaders []string // 采样的被删除页眉内容
	SampleRemovedFooters []string // 采样的被删除页脚内容
}

CleanupStats 清理统计信息

type HeaderFooterPattern 

type HeaderFooterPattern struct {
	Lines    []string // 具体的页眉/页脚文本行
	LineNums []int    // 对应的行号位置
}

HeaderFooterPattern 页眉页脚模式

type LLMRequestFunc 

type LLMRequestFunc func(ctx context.Context, modelName, prompt string, chunks []string) (string, error)

LLMRequestFunc is the narrow model seam used only by LLM-assisted cleanup. The host owns retries, credentials, provider selection and network policy.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.