token

package
v0.27.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 30, 2026 License: Apache-2.0 Imports: 36 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewStopTokenFilter

func NewStopTokenFilter(stopwords []string) analysis.TokenFilter

func StopWords

func StopWords(stopwords []string) analysis.TokenMap

Types

type RegexpTokenFilter

type RegexpTokenFilter struct {
	// contains filtered or unexported fields
}

func NewRegexpTokenFilter

func NewRegexpTokenFilter(r *regexp.Regexp, replacement []byte) *RegexpTokenFilter

func (*RegexpTokenFilter) Filter

type TrimTokenFilter

type TrimTokenFilter struct{}

func NewTrimTokenFilter

func NewTrimTokenFilter() *TrimTokenFilter

func (*TrimTokenFilter) Filter

type UpperCaseTokenFilter

type UpperCaseTokenFilter struct{}

func NewUpperCaseTokenFilter

func NewUpperCaseTokenFilter() *UpperCaseTokenFilter

func (*UpperCaseTokenFilter) Filter

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL