utf8mod

Overview ¶

Package utf8mod is the native module "utf8": text as characters (code points) instead of bytes (TODO.md Q93).

use utf8;
local s = "héllo wörld";
println(len(s), utf8::len(s));          // 13 11
println(utf8::upper(utf8::slice(s, 0, 5)));   // HÉLLO

Strings stay bytes. Bytes that are not UTF-8 are kept: each counts as one character, whose code is 0xFFFD (the replacement character) and whose string is the byte itself, so join(utf8::chars(s), "") is s for any s. Positions are character positions; utf8::offset and utf8::index convert them to and from the byte positions of find, slice and re.