EqualFold, IndexFold, ContainsFold, HasPrefixFold, and
HasSuffixFold compare ASCII case-insensitively using the SWAR word
loops: only A..Z/a..z fold, every other byte (including bytes
= 0x80) must match exactly. HasPrefixFold and HasSuffixFold cover
the header checks Fiber otherwise spells as an allocation-prone
ToLower + HasPrefix/HasSuffix pair (authorization schemes such as
Bearer, content-type prefixes such as multipart/form-data, suffixes
such as +json), and like the other Fold helpers they take the needle
as a plain string because call sites pass constant tokens. Their
Benchmark_HasPrefixFold/Benchmark_HasSuffixFold numbers are tracked
on the benchmark charts
and join the catalog above on its next regeneration.
HTTP dates
AppendHTTPDate and FormatHTTPDate write a time in the RFC 9110
preferred HTTP date format (Mon, 02 Jan 2006 15:04:05 GMT,
net/http.TimeFormat), byte-identical to time.Format with that layout
but without walking a layout string: the fixed-width template is copied
once and only the fields are overwritten. ParseHTTPDate is the reverse:
canonical preferred-format input takes a strict scalar fast path, and
everything else — the obsolete RFC 850 and asctime forms, unusual casing,
non-GMT zone names, padding — falls back to time.Parse with
net/http.ParseTime semantics, including its errors. Date,
Last-Modified, and If-Modified-Since handling sit on every
request/response, which makes these the highest-leverage helpers in this
group for Fiber.
URL escaping
AppendQueryEscape/AppendPathEscape and
AppendQueryUnescape/AppendPathUnescape produce byte-identical results
(and, for the unescape pair, identical url.EscapeError values) to their
net/url counterparts, as append-style, allocation-free single passes.
The escape tables are pinned to net/url.shouldEscape by exhaustive
per-byte tests. Unescaping jumps between escape sites with the vectorized
scans (IndexAny2 when + needs rewriting, bytes.IndexByte otherwise)
and copies clean spans wholesale, so route parameters and query values
without escapes — the common case — cost one scan and one copy. Decoding
never grows the input, so dst may be s[:0] on a common backing array
to unescape in place; escaping can grow the input, so there dst must
not alias s.
JSON string escaping
AppendJSONString appends a value as a double-quoted JSON string,
byte-identical to encoding/json.Marshal of the same string — including
its default HTML escaping (<, >, &), the \ufffd replacement of
invalid UTF-8 bytes, and the U+2028/U+2029 escapes — without any of
Marshal's reflection or allocation. Clean spans are located with a SWAR
scan (the same first-match-mask technique as IndexNonQuotable) and
copied wholesale. This is the building block for hand-rolled JSON hot
paths such as access-log lines and error bodies.
IP address parsing
ParseIPv4 and ParseIPv6 parse addresses into netip.Addr, accepting
exactly the strings netip.ParseAddr accepts for the respective family
(pinned by fuzzing) while reporting failure with a bool instead of a
constructed error. Both take strings or byte slices, so fasthttp-style
callers skip the string conversion entirely; remote-address and
trusted-proxy checks parse an IP on every request in Fiber.
Header key canonicalization
CanonicalHeaderKey Title-Cases an HTTP header key exactly like
net/http.CanonicalHeaderKey, including its return-unchanged guard for
keys containing non-token bytes. Already-canonical keys — the common case
on receive paths — are validated in a single table pass and returned
as-is with zero allocations for strings and byte slices alike. Keys that
do need rewriting cost one allocation; note that for the ~40 header names
in the stdlib's interning table the stdlib returns a cached string
without allocating, so this helper's edge there is time, not allocations.
These helpers were added on an amd64 machine, so like the simd numbers
their benchmarks are recorded separately from the arm64 catalog above and
join it on its next regeneration:
The swar package exports the SWAR (SIMD within a register)
building blocks the helpers above are composed from: Load8/Store8,
Broadcast, ZeroLanes, MatchByteMask/MatchRangeMask,
ToLowerWord/ToUpperWord, FirstLane/LastLane, and the WordLen,
Ones, HighBits, and LowSeven constants. They are exported so
downstream packages (Fiber itself, middleware) can fuse their own byte
scans — for example, finding a delimiter while classifying the bytes
before it — without re-deriving the bit tricks. The contracts (unchecked
bounds preconditions, ZeroLanes' approximate mask, the little-endian
lane order) and runnable examples live in the package documentation.
The package benchmarks its primitives against their stdlib counterparts
the same way the helpers above do, measuring the canonical loops composed
from them: Load8/Store8 vs encoding/binary's little-endian
Uint64/PutUint64, a ZeroLanes first-match scan vs
strings.IndexByte, a MatchByteMask+LastLane reverse scan vs
bytes.LastIndexByte, a MatchRangeMask digit scan vs
strings.IndexAny, and an in-place ToLowerWord loop vs
bytes.ToLower. The composed loops are pinned to the stdlib results by a
dedicated test, and the numbers are tracked per commit on the
benchmark charts. Note
that strings.IndexByte is hand-written SIMD assembly and overtakes the
portable SWAR scan on large inputs; the SWAR primitives earn their keep
on short HTTP-sized inputs and on fused scans the stdlib has no single
function for. The package's remaining benchmark, Benchmark_Load8_Fusion,
is an advisory codegen guard rather than an API performance promise, so
it is deliberately not part of the catalog above.
SIMD-accelerated search (package simd)
The simd package is the vector-width counterpart to swar: byte
searching and validation whose scan kernels dispatch to AVX2 assembly (four
32-byte vectors per iteration) on amd64 CPUs for inputs of simd.MinLen
(32) bytes or more, and fall back to portable loops built on the swar
primitives everywhere else, so it builds and behaves identically on every
platform.
simd.Accelerated() reports which mode is active. CPU capability detection
uses golang.org/x/sys/cpu,
which also verifies OS support for saving the vector register state.
The AVX2-backed kernels are the multi-needle scans (Memchr2, Memchr3),
the paired-byte scan (MemchrPair), the class scans
(MemchrDigit/MemchrDigitAt, MemchrWord/MemchrNotWord), the ASCII
scans (IsASCII, FirstNonASCII, CountNonASCII), and the substring
search (Memmem), which
prefilters on the needle's rarest bytes (SelectRareBytes, ByteRank)
before verifying candidates — 2-6 byte needles containing two distinct
values are scanned for their two rarest bytes at the exact relative
distance, longer or single-valued needles for the single rarest byte.
Each kernel classifies four 32-byte vectors per iteration. Where the
result is a position or a yes/no, the four masks are combined, so a
128-byte block costs a single VPMOVMSKB and a single branch, and the
scans that report a position re-extract the four masks in address order on
the (rare) hit path. CountNonASCII is the exception — a count cannot be
recovered from a combined mask, so it extracts and population-counts all
four, and it is the one kernel gated on POPCNT in addition to AVX2.
One tier is documented explicitly as unaccelerated:
MemchrInTable/MemchrNotInTable are plain
scalar loops, since an arbitrary 256-entry membership test has no cheap
vector form. There is deliberately no single-needle Memchr:
bytes.IndexByte is already vector-accelerated by the Go runtime. Memmem
delegates to bytes.Index below 128 bytes — a conservative routing point
above the paired scan's measured break-even (Benchmark_Memmem_Prefilter
forces the prefilter helpers at every size to keep it measurable) — and
bails to bytes.Index after a bounded number of failed candidate
verifications, so even adversarial inputs — a rare byte everywhere plus a
long almost-matching needle — stay within a constant of the stdlib's
O(n+m) (see Benchmark_Memmem_Adversarial). The
package operates on []byte only; the generic top-level helpers IsASCII,
IndexAny2, and IndexAny3 dispatch into it automatically for inputs of
simd.MinLen+ bytes on amd64, which is where the AVX2 kernels overtake the
SWAR word loops (-35% to -88% ns/op at 32-512B in benchstat -count=10
runs; shorter inputs keep the existing SWAR paths).
The kernels are adapted from the coregex
project's simd package (MIT License, Copyright (c) 2025 Andrey Kolkov and
contributors; see simd/LICENSE), with the legacy-SSE
register moves in the kernels replaced by VEX-encoded ones to avoid AVX-SSE
transition stalls, the scalar tail loops replaced by overlapping vector
rescans at the buffer end (a 63-byte scan previously cost ~3.7x a 64-byte
one), the 32-byte main loops replaced by 4x unrolled 128-byte blocks whose
bound is tested once at the bottom against a precomputed limit pointer, the
\w classifier rewritten from three VPMINUB/VPMAXUB clamp-and-compare
range tests into two VPSHUFB nibble-table lookups, FirstNonASCII and
CountNonASCII given kernels of their own (they were SWAR-only), and
the fallbacks reworked around the stdlib and the swar package as
described above.
Against the pre-unrolling kernels, benchstat -count=10 on the machine
below reports -26.6%/-49.7% ns/op for Memchr2 at 512B/4096B,
-17.6%/-34.3% for Memchr3, -13.4%/-24.6% for MemchrPair,
-39.2%/-50.5% for MemchrDigit, -53.5%/-63.7% for MemchrNotWord,
-42.2%/-65.7% for IsASCII, and -13.7%/-27.1% for Memmem, which rides
the paired scan. Those percentages come from benchstat over ten runs of
each side; the single-run table below is a catalog, not a base-vs-head
comparison, so subtracting its rows from an older copy of it will not
reproduce them.
Two bands see no benefit and are worth stating plainly. Inputs below
simd.MinLen never reach the kernels at all. Inputs of 32-127 bytes do
reach them but skip the 128-byte block loop, so they pay its setup with
nothing to amortize it against: across that band benchstat reports the
class scans clearly ahead (MemchrNotWord -8.6% at 32B, -18.4% at 64B;
IsASCII -5.0%/-4.3%), most rows statistically unchanged, and two small
regressions — MemchrDigit +5.6% at 32B and Memchr3 +3.8% at 64B. Read
those small-input figures against the run's own noise floor: the default
legs call unchanged stdlib code, so their movement bounds what a delta of
that size means. They held within 1% in the run quoted here, but a later
run on the same machine drifted up to 17% on those same rows, so anything
under ~10% at 32-64B should be taken as "no measured change" unless its
control legs are quiet.
On the portable side, pinning each unrolled group of words with a reslice
before loading at constant offsets inside it removes the per-load bounds
check that swar.Load8's own reslice otherwise repeats. Measured with
benchstat -count=10 against the same fallbacks, at 32B/512B/4KiB:
IsASCII -28%/-60%/-63%, CountNonASCII -24%/-44%/-44%,
FirstNonASCII -23%/-37%/-43%, Memchr2 -19%/-23%/-25%,
MemchrDigit -7%/-12%/-17%, Memchr3 -8%/-13%/-18%,
MemchrWord -11%/-8%/-11%, MemchrNotWord -5%/-7%/-6%. The tier a group
adds costs one compare on inputs too short to use it, which shows up as
+3% to +10% on some of the 8B fallback rows — a few tenths of a
nanosecond, and only below the length where the group pays for itself.
utils.IsASCII carries the same rework, so the two copies of that scan
stay in step.
Making the counting kernel finish its own sub-vector remainder (rather
than splitting the input in Go and calling the SWAR loop for the tail)
also removed a second non-inlinable call from every whole-vector input:
CountNonASCII at 32B went from 11.3ns — behind its own fallback — to
6.8ns, level with it, and it leads from 64B up.
Because the AVX2 kernels only engage on amd64, their benchmark numbers are
recorded separately from the arm64 catalog above:
Fiber is an open-source project that runs on donations to pay the bills, e.g., our domain name, hosting, and serverless infrastructure. If you want to support Fiber, please become a GitHub Sponsor.
Tool Sponsors - supporting Fiber with free IDE licenses and AI credits
AddTrailingSlashBytes appends a trailing '/' to b if it does not already end with one.
If the input already ends with '/', the original slice is returned.
A new slice is returned when a '/' is appended. The original slice is never modified.
AddTrailingSlashString appends a trailing '/' to s if it does not already end with one.
If the input already ends with '/', the original string is returned.
A new string is returned only when a '/' needs to be appended.
AppendHTTPDate appends t in the RFC 9110 preferred HTTP date format
("Mon, 02 Jan 2006 15:04:05 GMT", net/http.TimeFormat) to dst and returns
the extended slice. The output is byte-identical to
t.UTC().AppendFormat(dst, http.TimeFormat) and always 29 bytes for the
years 0..9999 that HTTP dates can represent; times outside that range
delegate to time.AppendFormat.
func AppendJSONString[S byteSeq](dst []byte, s S) []byte
AppendJSONString appends s to dst as a double-quoted JSON string and
returns the extended slice. The output is byte-identical to
encoding/json.Marshal of the same string value, including its default
HTML escaping: '"' and '\\' get backslash escapes; control bytes below
0x20 become \b, \f, \n, \r, \t, or \u00XX; '<', '>', '&' become \u00XX; each
invalid UTF-8 byte becomes the six literal characters \ufffd; and the
line separators U+2028/U+2029 become the \u2028 and \u2029 escapes.
All other bytes, including multi-byte UTF-8 sequences, are copied
verbatim. dst must not alias s: the output is longer than the input (at
minimum by the surrounding quotes), so in-place encoding is impossible
and an aliased dst would overwrite bytes before they are read. Clean
spans are located with a SWAR scan and copied wholesale, so typical log
or header values cost one scan and one copy — with none of
encoding/json.Marshal's reflection or allocation.
func AppendPathEscape[S byteSeq](dst []byte, s S) []byte
AppendPathEscape appends the percent-encoded path-segment form of s to dst
and returns the extended slice. The output is byte-identical to
net/url.PathEscape; the aliasing rule matches AppendQueryEscape.
func AppendPathUnescape[S byteSeq](dst []byte, s S) ([]byte, error)
AppendPathUnescape appends the decoded form of the path component s to dst
and returns the extended slice, decoding %XX and leaving '+' verbatim,
exactly like net/url.PathUnescape. Error and aliasing behavior match
AppendQueryUnescape.
func AppendQueryEscape[S byteSeq](dst []byte, s S) []byte
AppendQueryEscape appends the percent-encoded form of s to dst and returns
the extended slice. The output is byte-identical to net/url.QueryEscape:
unreserved bytes pass through, space becomes '+', everything else becomes
%XX with uppercase hex. dst must not alias s: escaping can grow the input,
so in-place operation is impossible.
func AppendQueryUnescape[S byteSeq](dst []byte, s S) ([]byte, error)
AppendQueryUnescape appends the decoded form of the query component s to
dst and returns the extended slice, converting '+' to space and %XX to the
byte it encodes, exactly like net/url.QueryUnescape including its
url.EscapeError values for malformed escapes. On error the returned slice
is dst with its original length (its backing array may still have been
reallocated by growth). Decoding never grows the input, so dst may be
s[:0] on a common backing array to decode in place; any other overlap is
invalid. Note that an in-place decode that fails has already overwritten
the prefix of s before the malformed escape with decoded bytes — treat s
as consumed once an in-place decode starts, error or not.
ByteSize returns a human-readable byte string of the form 10M, 12.5K, and so forth.
The unit that results in the smallest number greater than or equal to 1 is always chosen.
Maximum supported input is math.MaxUint64 / 10 (≈ 1844674407370955161).
CanonicalHeaderKey returns the canonical form of the HTTP header key:
the first letter and any letter following a hyphen upper-cased, the rest
lower-cased. The output matches net/http.CanonicalHeaderKey byte for
byte, including its guard: a key containing any byte outside the header
token alphabet is returned unchanged. Unlike the stdlib it accepts byte
slices as well as strings and allocates only when the key actually needs
rewriting — already-canonical keys, the overwhelmingly common case on
receive paths, are validated in one table pass and returned as-is
without the stdlib's canonical-key cache lookup.
func ConvertToBytes(humanReadableString string) int
ConvertToBytes returns integer size of bytes from human-readable string, ex. 42kb, 42M
Decimal units are powers of 1000 (42k = 42000); binary units with an 'i' infix
are powers of 1024 (42Ki = 43008). Note that ByteSize formats with powers of 1024,
so use binary suffixes for a lossless round-trip.
Returns 0 if the string is unrecognized or negative.
GenerateSecureToken generates a cryptographically secure random token encoded in base64.
It uses crypto/rand for randomness and base64.RawURLEncoding for URL-safe output.
If length is less than or equal to 0, it defaults to 32 bytes (256 bits of entropy).
Panics if the random source fails.
HasPrefixFold reports whether s begins with prefix, ASCII
case-insensitively: only 'A'..'Z'/'a'..'z' fold, every other byte
(including >= 0x80) must match exactly, mirroring IndexFold. An empty
prefix matches any s. Like the other Fold helpers the needle is a plain
string, since call sites pass constant tokens.
HasSuffixFold reports whether s ends with suffix, ASCII
case-insensitively, under the same folding contract as HasPrefixFold.
An empty suffix matches any s.
IndexAny2 returns the index of the first occurrence in s of either a or b,
or -1 if neither is present. On amd64 CPUs with AVX2, inputs of 32+ bytes
dispatch to package simd instead.
IndexAny3 returns the index of the first occurrence in s of a, b, or c,
or -1 if none is present. On amd64 CPUs with AVX2, inputs of 32+ bytes
dispatch to package simd instead.
IndexFold returns the index of the first ASCII case-insensitive occurrence
of needle in s, or -1 if absent. An empty needle matches at index 0. Only
'A'..'Z'/'a'..'z' fold; every other byte (including >= 0x80) must match
exactly, so e.g. "no\rcache" does NOT match the needle "no-cache".
The needle is a plain string by design: call sites pass constant tokens,
and a []byte needle would cost its callers a conversion either way.
IndexNonQuotable returns the index of the first byte of s that cannot
appear verbatim inside an RFC 9110 quoted-string — that is, a byte
matching c == '\\' || c == '"' || (c < 0x20 && c != '\t') || c == 0x7f —
or -1 if every byte is quotable. qdtext is HTAB / SP / %x21 / %x23-5B /
%x5D-7E / obs-text, so HTAB and bytes >= 0x80 are quotable. Returning the
index (rather than a bool) lets callers copy the clean prefix and start
escaping exactly at the offending byte. Inputs of 8+ bytes finish with
one overlapping word at n-8, shorter ones byte-wise.
IsASCII reports whether s contains only ASCII bytes (no byte >= 0x80).
It ORs four words per iteration where possible (no index is needed, so
detection can be deferred to one test per 32 bytes), then two, then
word-wise; inputs of 8+ bytes finish with one overlapping word at n-8,
shorter ones byte-wise.
On amd64 CPUs with AVX2, inputs of 32+ bytes dispatch to package simd
instead.
The unrolled loops pin their group of words with a reslice and load at
constant offsets inside it. swar.Load8's own reslice is bounds-checked
against the backing array, which the length-based loop condition does
not imply, so a variable index costs a compare and a branch per load;
pinning the window pays that once per group instead. This mirrors
simd.isASCIIGeneric — the same scan, kept in step so a retune of either
is a retune of both.
ParseHTTPDate parses an HTTP date the way net/http.ParseTime does:
the RFC 9110 preferred format ("Mon, 02 Jan 2006 15:04:05 GMT") plus the
obsolete RFC 850 and ANSI C asctime forms, with surrounding ASCII
whitespace tolerated. Canonical preferred-format input takes a fast scalar
path that never calls time.Parse; every other input — legacy formats,
unusual casing, non-GMT zone names, padding — falls back to time.Parse
with byte-for-byte stdlib semantics, including its errors. The returned
time is in time.UTC on the fast path and whatever time.Parse yields on the
fallback; the instants agree in both cases.
ParseIPv4 parses a dotted-decimal IPv4 address into a netip.Addr. It
accepts exactly the IPv4 strings netip.ParseAddr does — four octets 0-255,
no leading zeros, nothing before or after — and reports ok=false for
everything else, including IPv6 forms. It never allocates, so []byte
callers skip both the string conversion and netip's error construction.
ParseIPv6 parses an IPv6 address into a netip.Addr. It accepts exactly
the IPv6 strings netip.ParseAddr does — 16-bit hex fields, one optional
"::", an optional embedded dotted-decimal IPv4 tail, and an optional
non-empty "%zone" suffix — and reports ok=false for everything else,
including plain IPv4 forms (use ParseIPv4 for those). The parse itself
never allocates; only a present zone is materialized as a string because
netip.Addr stores zones as strings.
ParseUint parses a decimal ASCII string or byte slice into a uint64.
It returns the parsed value and nil on success.
If the input contains non-digit characters, it returns 0 and an error.
ParseVendorSpecificContentType check if content type is vendor specific and
if it is parsable to any known types. If its not vendor specific then returns
the original content type.
StartTimeStampUpdater launches a background goroutine that updates the cached timestamp every second.
It is safe to call multiple times and from multiple goroutines; only one updater runs at a time.
StopTimeStampUpdater stops the background updater goroutine.
Call this on app shutdown to avoid leaking goroutines.
It is safe to call multiple times and from multiple goroutines.
Trim removes all leading and trailing occurrences of the byte cutset from s.
Unlike strings/bytes.Trim, cutset is a single byte, not a set of characters.
TrimRight removes all trailing occurrences of the byte cutset from s.
Unlike strings/bytes.TrimRight, cutset is a single byte, not a set of characters.
TrimSpace removes leading and trailing whitespace from a string or byte slice.
This is an optimized version that's faster than strings/bytes.TrimSpace for ASCII strings.
It removes the following ASCII whitespace characters: space, tab, newline, carriage return, vertical tab, and form feed.
Walk walks the filesystem rooted at root, calling walkFn for each file or
directory in the filesystem, including root. All errors that arise visiting files
and directories are filtered by walkFn. The files are walked in lexical
order.
CBORUnmarshal parses the CBOR-encoded data and stores the result
in the value pointed to by v. If v is nil or not a pointer,
Unmarshal returns an error.
JSONUnmarshal parses the JSON-encoded data and stores the result
in the value pointed to by v. If v is nil or not a pointer,
Unmarshal returns an InvalidUnmarshalError.
type MsgPackUnmarshal func(data []byte, v any) error
MsgPackUnmarshal parses the MsgPack-encoded data and stores the result
in the value pointed to by v. If v is nil or not a pointer,
Unmarshal returns an InvalidUnmarshalError.
XMLUnmarshal parses the XML-encoded data and stores the result
in the value pointed to by v. If v is nil or not a pointer,
Unmarshal returns an InvalidUnmarshalError.