preprocessing

package

v0.2.2 Latest Latest Go to latest Published: Aug 11, 2026 License: Apache-2.0 Imports: 6 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/wsnacj/agentx-go

Links

Open Source Insights

Documentation ¶

Index ¶

func RemoveHeaderFooter(ctx context.Context, pages []string, mode CleanupMode, spec *configs.DocSpec, ...) ([]string, error)
type CleanupMode
- func ParseCleanupMode(mode string) CleanupMode
type CleanupStats
type HeaderFooterPattern
type LLMRequestFunc

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

func RemoveHeaderFooter ¶

func RemoveHeaderFooter(ctx context.Context, pages []string, mode CleanupMode, spec *configs.DocSpec, llmModel string, request LLMRequestFunc) ([]string, error)

RemoveHeaderFooter 根据指定模式移除页眉页脚

Types ¶

type CleanupMode ¶

type CleanupMode string

CleanupMode 页眉页脚清理模式

const (
	CleanupNone         CleanupMode = "none"         // 不清理
	CleanupProgrammatic CleanupMode = "programmatic" // 程序清理
	CleanupLLM          CleanupMode = "llm"          // LLM清理
	CleanupAuto         CleanupMode = "auto"         // 自动清理(程序+LLM)
)

func ParseCleanupMode ¶

func ParseCleanupMode(mode string) CleanupMode

ParseCleanupMode 解析清理模式字符串

type CleanupStats ¶

type CleanupStats struct {
	Mode                 string
	TotalPages           int
	HeaderCleanedPages   int
	FooterCleanedPages   int
	SampleRemovedHeaders []string // 采样的被删除页眉内容
	SampleRemovedFooters []string // 采样的被删除页脚内容
}

CleanupStats 清理统计信息

type HeaderFooterPattern ¶

type HeaderFooterPattern struct {
	Lines    []string // 具体的页眉/页脚文本行
	LineNums []int    // 对应的行号位置
}

HeaderFooterPattern 页眉页脚模式

type LLMRequestFunc ¶

type LLMRequestFunc func(ctx context.Context, modelName, prompt string, chunks []string) (string, error)

LLMRequestFunc is the narrow model seam used only by LLM-assisted cleanup. The host owns retries, credentials, provider selection and network policy.

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL