mangling

package module
v0.0.0-...-3579762 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 1, 2026 License: Apache-2.0 Imports: 10 Imported by: 0

README

mangling

Turn arbitrary strings into valid Go identifiers, and apply common recasing operations (camelCase, kebab-case, snake_case, ...).

It supersedes github.com/go-openapi/swag/mangling (v0.x) with an equivalent job and a stricter, more robust, faster implementation. See differences with v1.

import "github.com/go-openapi/swag/mangling/v2"

The three manglers

Mangler — general-purpose recasing.

m := mangling.MakeMangler()
m.Camelize("sample text")  // "sampleText"
m.Snakize("sampleText")    // "sample_text"
m.Titleize("hello world")  // "Hello World"

GoMangler — names a Go code generator can safely emit. Any input yields a valid, non-empty identifier.

g := mangling.MakeGoMangler()
g.IdentExported("sample text") // "SampleText"
g.ConstName("status 200")      // "StatusTwoHundred"
g.File("test.go")              // "test_swagger.go"

numbers.NumberMangler — verbalize numbers in text.

nm := numbers.MakeNumberMangler()
nm.NumberWords("0.25") // "one quarter"

Manglers are immutable values, configured with functional options at construction (MakeXxx returns a value, NewXxx a pointer), and safe for concurrent use.

Documentation

  • Design — the pipeline, the token model, and the roadmap.
  • Go identifiers — the "always a valid Go identifier" guarantee, Ident* / ConstName / File / Package / Module, and the repairs.
  • ASCII-fication — folding and romanization: café → Cafe, Cyrillic / Greek / Arabic, ½ → OneHalf, 😀 → GrinningFace, Japanese kana.
  • Numbers — cardinals, fractions, digit-group reconstruction.
  • Differences with v1 — migration and rationale.

Full API reference: pkg.go.dev.

Not supported

  • CJK Han ideographs (Kanji / Hanzi) — elided (Japanese kana works); native-script identifiers remain available with folding off.
  • Korean Hangul — elided, both syllables and standalone Jamo letters (a future improvement could romanize the Jamo by letter name).
  • Grapheme clusters (flag emoji, ZWJ sequences).

Performance

A typical mangling takes on the order of 1,000 ns. All methods scale linearly with the number of tokens (~160 ns/token) and perform zero internal allocation beyond the returned string — see our performance analysis. This corresponds roughly a x3 improvement on the swag version.

Tests

The Go mangler is fuzzed with the objective of producing a valid Go identifier against all-weather input. Run the suite with go test ./....

Documentation

Overview

Package mangling turns arbitrary strings into identifiers and applies well-known recasing operations (camelCase, snake_case, kebab-case, ...).

Manglers

Three manglers build on one pipeline:

  • Mangler — general-purpose recasing, ruleset-neutral;
  • GoMangler — identifiers a Go code generator can safely emit (ASCII folding on by default);
  • numbers.NumberMangler — verbalize numbers found in text (subpackage).

Every mangler is an immutable value built with functional options (MakeMangler / MakeGoMangler return a value, NewMangler / NewGoMangler a pointer) and is safe for concurrent use.

The sections below describe the transforms shared by Mangler and GoMangler; both type docs refer here rather than repeat them.

Case handling

Casing follows unicode rules: the first letter of a capitalized word is title-cased (via unicode.ToTitle, which differs from uppercase for a handful of digraphs), while ALL-CAPS uses uppercase and the remainder is lower-cased.

Special casing rules (e.g. unicode.SpecialCase) are not supported at this moment. Letters in languages that do not define case remain unchanged.

Symbol verbalization

Common symbols such as "?", "@", "#" are verbalized and replaced by a short word (e.g. "question", "at", "hash"). Multi-rune operators verbalize as a phrase — "!=" → "not equal" → NotEqual, "<=" → LessOrEqual, "->" → To — as do their Unicode glyph twins (≠, ≤, →). Operator verbalization runs regardless of ASCII folding; it is not honored by a target whose symbol policy drops symbols.

The single-rune set is customizable with WithSymbolWords (start from DefaultSymbolWords). It is dual-purpose: its keys decide which runes are symbol tokens rather than separators, and its values are the emitted words — so adding "," makes it a verbalizable token while removing "@" turns it into a separator.

ASCII folding

By default, letters and digits are left unchanged. With ASCII folding enabled (off in Mangler, on in GoMangler), non-ASCII input is reduced to ASCII: Latin letters with diacritics (é, ü) and non-ASCII digits fold to their base letter/value, while other non-Latin runes are "phonetized" using their Unicode rune name.

See ToASCII for the standalone, tokenization-free folder.

Numerals

Numbers embedded in values are verbalized through the numbers subpackage — cardinals, common fractions and digit-group reconstruction. See numbers.NumberMangler for the full description of how numbers are recognized.

Within a mangler:

  • ASCII digits are left as-is;
  • a "." (dot) is verbalized as "dot" (a symbol), a "," (comma) is elided (a separator);
  • unicode numerals such as ½ verbalize as words when folding is on ("½ cup" → "one half cup"); with folding off they are dropped, like other non-foldable runes. (The standalone ToASCII instead renders a numeral as a plain number: "½" → "0.5".)

Limitations

CJK ideographs and Korean Hangul (both the composed syllables and the standalone Jamo letters) are elided — no simple phonetization is available. Unicode grapheme clusters (flag emoji, ZWJ sequences) are not supported at this moment.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func DefaultInitialisms 

func DefaultInitialisms() []string

DefaultInitialisms returns the built-in set of initialisms recognized when Go initialisms are enabled.

These are common acronyms (API, HTTP, ID, JSON, ...) that the Go mangler keeps fully upper-cased in exported identifiers, plus a few mixed-case exceptions (IPv4, IPv6). Use it as a starting point to extend or override the set through the initialism options.

func DefaultSymbolWords 

func DefaultSymbolWords() map[rune]string

DefaultSymbolWords returns a copy of the built-in symbol-verbalization set (rune → word, e.g. '@' → "at").

Use it as a starting point to build a custom set for WithSymbolWords when you want wholesale control rather than a small overlay. The returned map is a fresh copy — mutating it does not affect the mangler.

func RuneShortName 

func RuneShortName[T ~rune | ~byte](r T) string

RuneShortName returns a lowercase phonetic word for a rune with no ASCII folding.

The word is a distinctive Unicode-name fragment (π → "pi", 😀 → "grinning face", ж → "zhe"). ASCII runes are returned as-is.

NOTE: runes the mangler elides (CJK ideographs, combining marks, decorative symbols) return "".

func RuneToASCII 

func RuneToASCII[T ~rune | ~byte](r T) string

RuneToASCII returns the plain-ASCII equivalent of a single rune bearing a diacritic (é → "e", ñ → "n"), a non-ASCII decimal digit folded to its ASCII value (٧ → "7"), the rune itself if already ASCII, or "" if it has no ASCII folding (non-Latin letters, symbols, emoji).

Use RuneShortName for those. Combining marks fold to "".

func ToASCII 

func ToASCII[T ~string | ~[]byte](s T) string

ToASCII transforms a string to plain ASCII.

Latin diacritics are folded (café → cafe), combining marks stripped, and any remaining non-ASCII rune is replaced by its phonetic Unicode-name word (π → pi, 😀 → grinning face), space-separated so it reads as words.

Non-ASCII decimal digits fold to their ASCII value (٧ → 7), and numeral runes render as a plain number (½ → 0.5).

Runes with no known word (CJK ideographs, decorative symbols) are dropped.

This works best for European languages; it falls back to RuneShortName for other scripts and emoji.

Types

type GoMangler 

type GoMangler struct {
	Mangler
	// contains filtered or unexported fields
}

GoMangler turns strings into identifiers that abide by go naming conventions.

It embeds a Mangler — so every base formatter (Mangler.Camelize, Mangler.Snakize, ...) is available — with ASCII folding enabled by default, and adds go-aware producers:

The producers honor a configurable list of initialisms (API, HTTP, ...), resolve collisions with go keywords and builtins, and guarantee a valid, non-empty identifier for any input.

Case handling, symbol verbalization, ASCII folding and numeral handling are shared with Mangler and documented in the package overview. Numbers are verbalized through the numbers subpackage (see numbers.NumberMangler).

func MakeGoMangler 

func MakeGoMangler(opts ...GoOption) GoMangler

MakeGoMangler returns a value GoMangler.

func NewGoMangler 

func NewGoMangler(opts ...GoOption) *GoMangler

NewGoMangler returns a pointer to a GoMangler.

func (GoMangler) ConstName 

func (g GoMangler) ConstName(value string) string

ConstName produces a valid exported Go identifier from an arbitrary value (e.g. an enum member).

Every number in the value is verbalized ("0.25" -> "one quarter", "300" -> "three hundred") and the result is turned into an exported identifier. This is what distinguishes it from GoMangler.IdentExported, which verbalizes only a *leading* number (for identifier validity) and keeps interior digits: "status 200" -> "StatusTwoHundred" here, but "Status200" through IdentExported. Reach for IdentExported when embedded digits should stay digits (status codes, versions, "oauth2"). Type-name prefixing of enum members (Color + Red -> ColorRed) is the code generator's job. 

ConstName("0.25") == "OneQuarter"   ConstName("300") == "ThreeHundred"   ConstName("read only") == "ReadOnly"

func (GoMangler) File 

func (g GoMangler) File(input string) string

File produces a valid go file name, transforming any trailing segment that bears semantics to the go build system.

Mangling applies only to the file **stem**: the directory prefix (either "/" or "\" separated) and any existing extension are reconducted verbatim; the stem is lower-cased and snakized (no ".go" is added).

When the last snake segment is a reserved GOOS/GOARCH/test suffix (which would make the file build-constrained), a repair suffix is appended (default "swagger"):

  • "test.go" -> "test_swagger.go"
  • "config_linux" -> "config_linux_swagger"
  • "some/dir/MyModel" -> "some/dir/my_model"
  • "IPv4Config.json" -> "ipv4_config.json"

func (GoMangler) IdentExported 

func (g GoMangler) IdentExported(str string) string

IdentExported produces a valid exported go variable identifier from a string, possibly containing multiple words.

IdentExported works like Mangler.Pascalize with a few go-specific additions:

  • compiler constraint: identifiers that don't start with a letter get their prefix verbalized (e.g. digits get their numeral wording, symbols get replaced by a word, etc).

Unlike their unexported counterpart, exported identifiers can't conflict with go reserved keywords or builtins.

func (GoMangler) IdentUnexported 

func (g GoMangler) IdentUnexported(str string) string

IdentUnexported produces a valid unexported go variable identifier from a string, possibly containing multiple words.

IdentUnexported works like Mangler.Camelize with a few go-specific additions:

  • compiler constraint: conflicts with go language keywords are resolved (e.g. "type" becomes "typeVar")
  • compiler constraint: identifiers that don't start with a letter get their prefix verbalized (e.g. digits get their numeral wording, symbols get replaced by a word, etc).
  • linter constraint: conflicts with go builtin functions are resolved (e.g. "append" becomes "appendVar")

An unexported identifier is camelized, with the casing of initialisms respected (e.g. "getHTTP" and not "getHttp").

func (GoMangler) Module 

func (g GoMangler) Module(pth string) string

Module produces a legit go module path.

It mangles only the basename (kebab-cased, with the Go ruleset), reconducting the "/"-separated directory prefix verbatim — the same shape as GoMangler.Package's pkg — with the same reserved name and major-version repairs, plus a repair of Windows device names (con, nul, com1…, lpt1…).

Note: the major-version repair means a trailing "v2" becomes "version2", so an actual semantic-import-versioning suffix (".../repo/v2") must be appended by the caller *after* Module, not passed through it.

func (GoMangler) Package 

func (g GoMangler) Package(pth string) (shortName string, pkg string)

Package produces a legit go package import path and its short (declaration) name.

shortName is the one you'd use in a "package {name}" declaration; pkg is the full import path.

Only the **basename** is mangled — kebab-cased (`-` separated) with the Go ruleset (ASCII folding and initialisms), like GoMangler.File but with "-" instead of "_".

A leading directory (only "/" is a separator here — a package path is not a filesystem path; trim any trailing "/" beforehand) is reconducted verbatim. pkg is `{dir}/{x-y-z}` (or `{x-y-z}`); shortName is the last "-" segment (`z`). 

"github.com/toktok/@alpha-beta" -> shortName "beta", pkg "github.com/toktok/at-alpha-beta"

func (GoMangler) PackageWithParts 

func (g GoMangler) PackageWithParts(pth string) (shortName string, pkg string, parts []string)

PackageWithParts is like GoMangler.Package, but also returns the parts of the mangled basename (`[x, y, z]`). shortName is the last of these.

Useful when the caller wants to derive a deconflicted import alias from the other significant parts of the package.

type GoOption 

type GoOption func(goOptions) goOptions

GoOption customizes the behavior of the GoMangler.

func UseGoInitialisms 

func UseGoInitialisms(list ...string) GoOption

UseGoInitialisms replaces the default initialisms with the given list (WithGoInitialisms entries are still appended on top).

Each string is the canonical casing to emit; matching is case-insensitive. Called with no arguments it is a no-op (the defaults stay).

func WithGoFileRepairSuffix 

func WithGoFileRepairSuffix(suffix string) GoOption

WithGoFileRepairSuffix sets the suffix appended to a file stem that would otherwise be build-constrained by a GOOS/GOARCH/`_test` suffix (default "swagger": "config_linux" → "config_linux_swagger").

The go-swagger default is not appropriate for every generator, so it is configurable.

func WithGoIdentFallback 

func WithGoIdentFallback(word string) GoOption

WithGoIdentFallback sets the word substituted when an identifier reduces to nothing — i.e. the input is empty or made up entirely of separators / elided runes (e.g. "___", "@#$" with symbols dropped, or CJK under ASCII folding).

Without it the Go identifier producers would emit an (invalid) empty string.

The word is itself run through the mangler at the producing target, so any input is made valid and cased correctly: GoMangler.IdentExported/GoMangler.ConstName → "Empty", GoMangler.IdentUnexported → "empty", GoMangler.File → "empty".

If the provided word *also* reduces to nothing, the built-in default "empty" is used, so a valid identifier is always produced. Applies to the GoMangler only; the base Mangler may still return "".

func WithGoInitialisms 

func WithGoInitialisms(extra ...string) GoOption

WithGoInitialisms adds entries on top of the initialism list (the defaults, or the list set by UseGoInitialisms).

Each string is the canonical casing to emit — e.g. "OAI", "gRPC"; matching is case-insensitive. Repeated calls accumulate.

func WithGoNumberOptions 

func WithGoNumberOptions(opts ...numbers.NumberOption) GoOption

WithGoNumberOptions configures the numbers.NumberMangler the GoMangler uses to verbalize numbers in GoMangler.ConstName and leading-digit identifiers — e.g. registering special numbers or eliding "and"/"one".

func WithGoReservedSuffix 

func WithGoReservedSuffix(suffix string) GoOption

WithGoReservedSuffix sets the suffix appended to an unexported identifier that collides with a Go keyword or builtin (default "Var": "type" → "typeVar", "append" → "appendVar").

A house-style knob for generators that prefer a different convention.

func WithManglerOptions 

func WithManglerOptions(opts ...Option) GoOption

WithManglerOptions lifts base Mangler options into a GoOption, so a GoMangler can be configured with the same settings as the base mangler it embeds (folding, token, initialism options, ...).

type Mangler 

type Mangler struct {
	tokens.Tokenizer
	// contains filtered or unexported fields
}

Mangler exposes general purpose well-known case formatters (Mangler.Camelize, Mangler.Snakize, Mangler.Kebabize, Mangler.Titleize, ...) with simple recasing rules.

It works best with latin letters and unicode letters that define upper and lowercase classes. ASCII folding is off by default (see GoMangler for the on-by-default variant).

Case handling, symbol verbalization, ASCII folding and numeral handling are shared with GoMangler and documented in the package overview — see the "Case handling", "Symbol verbalization", "ASCII folding" and "Numerals" sections there.

func MakeMangler 

func MakeMangler(opts ...Option) Mangler

MakeMangler builds a Mangler value with optional settings.

func NewMangler 

func NewMangler(opts ...Option) *Mangler

NewMangler builds a Mangler pointer with optional settings.

func (Mangler) AllCaps 

func (m Mangler) AllCaps(str string) string

AllCaps produces all letters capitalized and words snakized.

Like so: 

ALL_CAPS

func (Mangler) Camelize 

func (m Mangler) Camelize(str string) string

Camelize produces camel case.

Like so: 

camelCase

func (Mangler) Humanize 

func (m Mangler) Humanize(str string) string

Humanize produces a space-separated sentence, first letter titleized, the rest lower-cased.

func (Mangler) Kebabize 

func (m Mangler) Kebabize(str string) string

Kebabize produces kebab case (lower-cased)

Like so: 

kebab-case

func (Mangler) Pascalize 

func (m Mangler) Pascalize(str string) string

Pascalize produces pascal case.

Like so: 

PascalCase

func (Mangler) Snakize 

func (m Mangler) Snakize(str string) string

Snakize produces snake case (lower-cased).

Like so: 

snake_case

func (Mangler) Titleize 

func (m Mangler) Titleize(str string) string

Titleize transforms all words in titled case.

The remainder of each word is lower-cased.

Like so: 

This Is A Title.

func (Mangler) Transform 

func (m Mangler) Transform(target TargetTransform, str string) string

Transform renders str through the target recipe: asciify the input (operator verbalization always, rune-name verbalization when folding is on) → segment → ASCII-fold the tokens when folding is on → assemble (casing × separator × symbol policy).

The GoMangler inserts an initialism overlay before assembly; the base Mangler has no initialism stage.

type Option 

type Option func(options) options

Option customizes the behavior of the Mangler.

func WithASCIIFolding 

func WithASCIIFolding(enabled bool) Option

WithASCIIFolding toggles folding of Latin diacritics to ASCII (é→e, ñ→n, ß→ss, combining marks stripped).

It is off by default in the base Mangler and on by default in the GoMangler (gosmopolitan-clean output). Most non-Latin scripts are romanized (with the notable exception of CJK runes, which are elided).

func WithSymbolWords 

func WithSymbolWords(words map[rune]string) Option

WithSymbolWords customizes the symbol-verbalization set on top of the built-in defaults (DefaultSymbolWords).

Each entry overlays the defaults: a new key adds a symbol, an existing key overrides its word, and an **empty word removes** the symbol from the set. The rest of the defaults are kept — call DefaultSymbolWords and build a fresh map if you need wholesale control.

The set is dual-purpose, so an edit affects two stages:

  • segmentation: a rune in the set is a single-rune *symbol token*; a rune not in the set falls to the category rules and is typically a separator. So adding ',' makes "a,b" tokenize as [a , b] (→ verbalizable), while removing '@' makes "a@b" tokenize as [a b] (the '@' becomes a separator, dropped).
  • verbalization: under a target's verbalize policy, the word is what the assembler emits ("@" → "at").

Repeated calls accumulate. Applies to the base Mangler and, via WithManglerOptions, to the GoMangler.

func WithTokenOptions 

func WithTokenOptions(opts ...TokenOption) Option

WithTokenOptions bundles token-level options into a single mangler Option.

Use it to pass tokenizer settings (such as WithTokenSeparator) when configuring a Mangler.

type TargetOption 

type TargetOption func(TargetTransform) TargetTransform

TargetOption customizes a TargetTransform.

func WithSeparator 

func WithSeparator(sep string) TargetOption

WithSeparator sets the output separator emitted between tokens.

type TargetTransform 

type TargetTransform struct {
	// contains filtered or unexported fields
}

TargetTransform is a compiled, immutable recipe describing how to render a segmented token stream: casing × separator × affix × stages × repair.

All fields are unexported; build custom targets with MakeTargetTransform.

The mangler supplies the data (dictionaries) that stages bind to at run time, so a target degrades gracefully across manglers.

Fields are unexported; the assembly recipe is casing × separator × symbol-policy (affix, stages and repair land later).

func MakeTargetTransform 

func MakeTargetTransform(opts ...TargetOption) TargetTransform

MakeTargetTransform builds a custom TargetTransform.

func TargetAllCaps 

func TargetAllCaps() TargetTransform

TargetAllCaps produces underscore-separated upper-case tokens ("ALL_CAPS").

func TargetCamel 

func TargetCamel() TargetTransform

TargetCamel produces joined tokens with a lower-case first token and the rest capitalized ("camelCase").

func TargetKebab 

func TargetKebab() TargetTransform

TargetKebab produces hyphen-separated lower-case tokens ("kebab-case").

func TargetPascal 

func TargetPascal() TargetTransform

TargetPascal produces joined tokens, each capitalized ("PascalCase").

func TargetSentence 

func TargetSentence() TargetTransform

TargetSentence produces space-separated tokens, only the first capitalized ("Sentence case").

func TargetSnake 

func TargetSnake() TargetTransform

TargetSnake produces underscore-separated lower-case tokens ("snake_case").

func TargetTitle 

func TargetTitle() TargetTransform

TargetTitle produces space-separated tokens, each capitalized ("Title Case").

type TokenOption 

type TokenOption func(tokenOptions) tokenOptions

TokenOption customizes the behavior of the tokenizer (see the internal tokens package).

func WithTokenSeparator 

func WithTokenSeparator(separator func(rune) bool) TokenOption

WithTokenSeparator sets the predicate that decides which runes split the input into tokens.

The predicate reports whether a rune acts as a separator (dropped from the output). It replaces the default separator rule wholesale, so it must recognize every character to break on.

Source Files

View all Source files

Directories

Path Synopsis
internal
tokens
Package tokens is the mangler's token engine: the zero-copy token model plus the opinionated segmentation that produces it.
 Package tokens is the mangler's token engine: the zero-copy token model plus the opinionated segmentation that produces it.
Package numbers verbalizes the numbers found in text as English words.
 Package numbers verbalizes the numbers found in text as English words.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.