README
¶
f2f-match-utils
This repo is public intentionally
This is a shared repo that holds go packages mainly for text matching.
| package | what |
|---|---|
rules |
a pattern language for matching short texts, with validation, explanations and two indexes: rules/textindex for few rules against many texts and rules/ruleindex for many rules against few texts |
tokens |
converts text into word tokens for fast matching |
sanitize |
string normalisation and escaping helpers |
language |
resolves language names and codes to ISO 639-3 |
Directories
¶
| Path | Synopsis |
|---|---|
|
Package language resolves a language name, alias, or ISO 639-1 or ISO 639-2 code to an ISO 639-3 code.
|
Package language resolves a language name, alias, or ISO 639-1 or ISO 639-2 code to an ISO 639-3 code. |
|
internal/gen
command
Command gen regenerates tables_gen.go from languages.tsv.
|
Command gen regenerates tables_gen.go from languages.tsv. |
|
internal/langcode
Package langcode holds the encoding the generated language table is written in.
|
Package langcode holds the encoding the generated language table is written in. |
|
internal/langtable
Package langtable turns languages.tsv into the lookup tables the language package compiles in.
|
Package langtable turns languages.tsv into the lookup tables the language package compiles in. |
|
Package rules parses, validates and explains match rules, a small pattern language for short texts such as job titles, and matches a set of them against one text at a time.
|
Package rules parses, validates and explains match rules, a small pattern language for short texts such as job titles, and matches a set of them against one text at a time. |
|
internal/synth
Package synth generates made-up texts and rule sets for tests and benchmarks.
|
Package synth generates made-up texts and rule sets for tests and benchmarks. |
|
ruleindex
Package ruleindex matches texts against many rule sets at once: the sets are indexed up front, the texts are not.
|
Package ruleindex matches texts against many rule sets at once: the sets are indexed up front, the texts are not. |
|
textindex
Package textindex matches rule sets against many texts at once: the texts are indexed up front, the rule sets are not.
|
Package textindex matches rule sets against many texts at once: the texts are indexed up front, the rule sets are not. |
|
internal/foldtable
Package foldtable builds the sanitize package's fold table: what every non-ASCII code point becomes after lowercasing, removing diacritics and dropping apostrophes.
|
Package foldtable builds the sanitize package's fold table: what every non-ASCII code point becomes after lowercasing, removing diacritics and dropping apostrophes. |
|
internal/gen
command
Command gen regenerates fold_table.go.
|
Command gen regenerates fold_table.go. |
Click to show internal directories.
Click to hide internal directories.