textsafe

package module
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 11, 2026 License: Apache-2.0 Imports: 9 Imported by: 2

README

textsafe

Go Reference Go version

textsafe keeps untrusted text from forging your Go log lines, starting terminal escapes or reordering what a reader sees in a JSON document or report.

It replaces the strings.Map over control characters you would otherwise write at each log call, JSON writer and report renderer with one policy they all share. It uses only the standard library, needs Go 1.27 or later and is licensed under Apache-2.0.

Why use it

textsafe is built for Go code that logs or re-emits text it did not write, such as API fields and error messages.

  • It replaces C0 and C1 controls, DEL, the 12 bidi controls, and U+2028 and U+2029 with a space.
  • encoding/json and slog's JSONHandler write C1 and bidi controls unescaped, so JSON alone does not stop a terminal escape or a Trojan Source reordering.
  • Sanitize keeps CR and LF for JSON, and SanitizeSingleLine replaces them for a one-line sink.
  • Caps cut on a rune boundary, and SanitizeCapped, SanitizeSingleLineCapped and Budget bound the sanitizer's work too.
  • An Untrusted value emits its sanitized form whenever slog, an encoder or a fmt verb writes it, as a field, a slice element or a map key. The exceptions are %#v, which prints an ASCII-escaped constructor call, and %T, %p and a %w on a non-error, which fmt prints without calling its methods. It stores no raw unsafe rune for such a reflection-based printer to find.
  • Sanitize, SanitizeSingleLine and Untrusted in slog and in fmt's %v and %s do not allocate for clean text.

Consider strconv.Quote or %q if you want each control shown as an escape such as \u202e. Consider bluemonday for user content in HTML, which it sanitizes against an allowlist.

Install

go get github.com/cplieger/go/textsafe@latest

Usage

Sanitize a value where it is logged or encoded, so comparisons and dedupe keys keep the raw value:

slog.Warn("better release available",
    "title", textsafe.Sanitize(upstream.Title),
    "group", textsafe.Sanitize(upstream.Group))

Each unsafe rune becomes a space, and each invalid UTF-8 byte becomes U+FFFD, so the result is always valid UTF-8. Sanitizing twice gives the same result as sanitizing once.

Where a raw newline would start a new record, such as a plain-text log line or a one-line error message, use SanitizeSingleLine. SanitizeSingleLineBounded also caps the sanitized text on a rune boundary and appends ... when it cuts:

msg := textsafe.SanitizeSingleLine(upstreamErr.Error())

slog.Warn("upstream rejected request",
    "reason", textsafe.SanitizeSingleLineBounded(upstreamErr.Error(), 200))

You can also tag a field once, in the struct that decodes the upstream payload. slog, fmt's %v and %s, and encoding/json then emit it sanitized, and Raw returns the decoded string unchanged:

type Episode struct {
    Title textsafe.Untrusted `json:"title"`
}

slog.Warn("better release available", "title", ep.Title) // sanitized automatically
if ep.Title.Raw() == stored.Title { /* matching stays raw */ }

Tag text that does not come from a decoder with textsafe.NewUntrusted(s). If your program saves a struct and reads it back, store Raw in a plain string field, because json.Marshal writes a tagged field in its sanitized form.

The package has 14 runnable examples on pkg.go.dev, and go test keeps them true. Bounding sanitized text covers hard size limits and bounding the sanitizer's work.

API

  • Sanitize and SanitizeSingleLine sanitize a whole string, keeping or replacing CR and LF.
  • SanitizeSingleLineBounded sanitizes the whole value and then cuts it to the cap, marking a cut with ... outside it.
  • SanitizeCapped and SanitizeSingleLineCapped cut the raw value to the cap before sanitizing, so a huge value costs no more work than a short one. They return whether they cut, and the marker you pass counts toward the cap. A Budget, created with NewBudget or NewSingleLineBudget, does the same for several values that share one cap.
  • RedactSingleLineCapped removes known secrets from the text and from its sanitized form for a single line, then cuts the result to the cap with the marker inside it. It reads the whole value, so it does not bound the work.
  • CapBytes keeps the head and CapBytesTail keeps the tail, both cut on a rune boundary.
  • IsUnsafeMultiLine, IsUnsafeSingleLine, IsUnsafeNonASCII and IsBidiControl classify one rune, for a policy of your own.
  • Untrusted, created with NewUntrusted or by decoding, has Raw and SingleLine, plus the LogValue, String, Format, MarshalText and UnmarshalText methods that slog, fmt and encoders call.

The full reference is on pkg.go.dev.

Sanitize where text leaves your program

Sanitize text where it is logged or encoded, never when it is parsed, so matching and dedupe keys keep working on the raw value. Use one policy for the whole app, through these functions, the Untrusted type or one wrapper around a predicate, so two sinks cannot disagree about what is unsafe.

Some text must be safe in every sink it may ever reach, such as an error body captured into a returned value. Sanitize it once when you build the value, instead of tagging it.

A Markdown table cell, a link URL or an HTML page has injection characters of its own, such as pipes, brackets and angle brackets. Apply that sink's own escaper on top of textsafe.

If you also remove a known secret, such as an API key, from the same text, use RedactSingleLineCapped. It replaces each secret with [redacted], matching it in the text and in its sanitized form, then caps:

reason, cut := textsafe.RedactSingleLineCapped(body, 200, "...", apiKey)

Bounding sanitized text explains why each step is needed.

Unsupported by design

textsafe does no HTML or XSS sanitizing, Unicode normalization or case folding. It leaves zero-width and look-alike runes alone, and its sanitizers always replace with a space and never remove a rune. Unsupported by design gives the reason for each and what to use instead.

Documentation

  • The rune policy lists every rune textsafe replaces, and shows how to build a custom policy from the predicates.
  • Bounding sanitized text covers the byte caps, the marker and truncation rules, the work budget and the redaction order.
  • The Untrusted type explains what the type does at each sink and the two rules for using it.
  • Unsupported by design lists the features left out on purpose, with the reasons.

Contributing

Issues and pull requests are welcome. The shared contributing rules apply.

Disclaimer

This project is built with care and follows security best practices, but it is intended for personal / self-hosted use. No guarantees of fitness for production environments. Use at your own risk.

This project was built with AI-assisted tooling using Claude, GPT, and Kiro. The human maintainer defines architecture, supervises implementation, and makes all final decisions.

License

Apache-2.0. See LICENSE.

Documentation

Overview

Package textsafe classifies runes that are unsafe in untrusted text bound for logs, JSON or rendered output, and provides shared sanitizers that neutralize them.

Untrusted upstream text eventually reaches a slog line read in a terminal, a JSON report or a Markdown table. Four classes of rune keep their control semantics on that trip and let the upstream author forge or garble what the operator sees:

  • C0 controls and DEL: terminal escape sequences, and a raw newline that splits one log record into two.
  • C1 controls: single-rune escape introducers that encoding/json and slog's JSONHandler emit raw.
  • Unicode Bidi_Control characters, which visually reorder rendered text.
  • U+2028 and U+2029, line terminators to JavaScript and many viewers.

The classes are written as code points, never looked up in a Unicode table, so a Unicode upgrade cannot move the policy.

IsUnsafeMultiLine and IsUnsafeSingleLine classify one rune under the two CR/LF policies. Sanitize keeps CR and LF for JSON-encoded sinks, and SanitizeSingleLine replaces them too, each unsafe rune becoming a space. CapBytes, SanitizeSingleLineBounded, the Capped pair and Budget bound the result. The Untrusted type applies Sanitize automatically in slog, fmt and every encoding.TextMarshaler-aware encoder, map keys included (fmt's %#v prints an ASCII-escaped constructor call instead, and %T and %p bypass its methods), and Untrusted.Raw keeps the exact bytes.

Sanitize at the emit boundary, so comparisons and dedupe keys keep the raw value. RedactSingleLineCapped also removes known secrets, matching each in the text and in its sanitized form in one pass, and caps last.

It is not an HTML sanitizer and does not normalize Unicode. docs/policy.md, docs/capping.md, docs/untrusted.md and docs/non-goals.md hold the full contract.

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func CapBytes

func CapBytes(s string, n int) string

CapBytes truncates s to at most n bytes without splitting a multi-byte rune at the cut: the cut backs off to the nearest rune start, so the result never ends in a partial rune. It exists because sanitizing can grow a string (each invalid UTF-8 byte becomes the three-byte U+FFFD), so a pre-sanitize byte cap does not survive sanitization, and a naive re-cap can split a rune — leaving a partial-rune tail whose raw 0x80-0x9F bytes a non-UTF-8 terminal reads as C1 escape introducers, re-minting the very class of byte the sanitizer removed. For valid UTF-8 input (any Sanitize or SanitizeSingleLine output) the backoff discards at most three bytes below n and the result is a valid-UTF-8 prefix of s. A non-positive n returns the empty string.

Example

ExampleCapBytes bounds a sanitized string without splitting a multi-byte rune: the cut backs off to a rune boundary instead of leaving a partial rune's raw tail bytes.

package main

import (
	"fmt"

	"github.com/cplieger/go/textsafe"
)

func main() {
	s := "葬送のフリーレン" // three bytes per rune
	fmt.Printf("%q\n", textsafe.CapBytes(s, 7))
}
Output:
"葬送"

func CapBytesTail

func CapBytesTail(s string, n int) string

CapBytesTail keeps at most the last n bytes of s without splitting a multi-byte rune: the cut advances FORWARD to the next rune start, so the result is a valid-UTF-8 suffix no longer than n bytes. Forward holds that bound where CapBytes' backward backoff would exceed it, and it drops the partial LEADING rune encoding/json rewrites to U+FFFD, so a capped field cannot open on a replacement rune. Non-positive n returns "". The cut is rune-safe, not grapheme-safe: it may fall between a base rune and its combining mark, which is this package's documented Unicode non-goal.

Example

ExampleCapBytesTail keeps the END of a sanitized string, for a value whose identifying part sits at its tail. The marker goes in FRONT and is the caller's own composition, exactly as SanitizeSingleLineBounded appends one behind the head cap.

package main

import (
	"fmt"

	"github.com/cplieger/go/textsafe"
)

func main() {
	s := "葬送のフリーレン" // three bytes per rune
	fmt.Printf("%q\n", textsafe.CapBytesTail(s, 7))
	fmt.Printf("%q\n", "..."+textsafe.CapBytesTail(s, 7))
}
Output:
"レン"
"...レン"

func IsBidiControl

func IsBidiControl(r rune) bool

IsBidiControl reports whether r is one of Unicode's Bidi_Control format characters: the singleton marks U+061C (ALM) and U+200E/U+200F (LRM/RLM), the override/embedding range U+202A-U+202E (LRE/RLE/PDF/LRO/RLO), and the isolate range U+2066-U+2069 (LRI/RLI/FSI/PDI). Any of them in untrusted text can visually reorder rendered output (Trojan-Source-style report and link spoofing), so every output sanitizer treats the full set as unsafe. The set matches unicode.Bidi_Control exactly, without the table lookup; a conformance sweep pins the two rune by rune, so a later Unicode adding a member fails the tests instead of silently diverging.

Example

ExampleIsBidiControl classifies a right-to-left override against a plain letter.

package main

import (
	"fmt"

	"github.com/cplieger/go/textsafe"
)

func main() {
	fmt.Println(textsafe.IsBidiControl('\u202e'), textsafe.IsBidiControl('a'))
}
Output:
true false

func IsUnsafeMultiLine

func IsUnsafeMultiLine(r rune) bool

IsUnsafeMultiLine reports whether r is unsafe in untrusted text bound for a sink whose encoder escapes CR and LF itself (JSON, a quoting logger): a C0 control other than CR/LF, DEL, a C1 control (U+0080-U+009F, single-rune terminal-escape introducers), a Unicode bidi control (IsBidiControl), or the U+2028/U+2029 line separators. CR and LF are SAFE under this policy. It is the per-rune predicate behind Sanitize; the single-line policy is IsUnsafeSingleLine.

Example

ExampleIsUnsafeMultiLine shows the CR/LF policy split: a newline is safe for a sink whose encoder escapes it, and unsafe for a single-line sink.

package main

import (
	"fmt"

	"github.com/cplieger/go/textsafe"
)

func main() {
	fmt.Println(textsafe.IsUnsafeMultiLine('\n'), textsafe.IsUnsafeSingleLine('\n'))
	fmt.Println(textsafe.IsUnsafeMultiLine('\u009b'), textsafe.IsUnsafeSingleLine('\u009b'))
}
Output:
false true
true true

func IsUnsafeNonASCII

func IsUnsafeNonASCII(r rune) bool

IsUnsafeNonASCII reports whether r is an unsafe rune above the ASCII range: a C1 control (U+0080-U+009F), a Unicode bidi control (IsBidiControl), or the U+2028/U+2029 line separators. It is the IsUnsafe policy minus C0, DEL, and the CR/LF axis (all ASCII), for composed escapers whose sink already covers ASCII: a URL percent-encoder escapes C0, DEL, and whitespace itself, but url.Parse accepts these non-ASCII runes raw, and a terminal or Markdown viewer must never receive them. The CR/LF distinction is moot above ASCII, so one predicate serves both policies: IsUnsafeNonASCII(r) equals IsUnsafeMultiLine(r) && r > unicode.MaxASCII, and IsUnsafeSingleLine(r) && r > unicode.MaxASCII.

Example

ExampleIsUnsafeNonASCII classifies the runes a URL percent-escaper must encode even though url.Parse accepts them raw: the C1 escape introducer and the bidi override are flagged, while an ASCII control (the sink's own escaping covers it) and a plain letter are not.

package main

import (
	"fmt"

	"github.com/cplieger/go/textsafe"
)

func main() {
	fmt.Println(textsafe.IsUnsafeNonASCII('\u009b'), textsafe.IsUnsafeNonASCII('\u202e'))
	fmt.Println(textsafe.IsUnsafeNonASCII('\x1b'), textsafe.IsUnsafeNonASCII('é'))
}
Output:
true true
false false

func IsUnsafeSingleLine

func IsUnsafeSingleLine(r rune) bool

IsUnsafeSingleLine reports whether r is unsafe in untrusted text bound for a single-line sink, where a raw CR or LF forges a new record: everything IsUnsafeMultiLine refuses, plus CR and LF. It is the per-rune predicate behind SanitizeSingleLine.

func RedactSingleLineCapped

func RedactSingleLineCapped(s string, n int, marker Marker, secrets ...string) (text string, cut bool)

RedactSingleLineCapped sanitizes s like SanitizeSingleLine, writing "[redacted]" in place of every rune that an occurrence of a non-empty secret touches, in s or in its sanitized form, one mark per run of such runes; then caps the result as SanitizeSingleLineCapped does, marker inside n, after reading all of s, and drops a trailing proper prefix of a secret after a cut. No occurrence in s or its sanitized form survives, though a mark and its neighbours can spell a secret that shares text with "[redacted]". Pass s whole: a secret your own truncation split survives as its prefix whenever s then fits n.

marker is a Marker, not a string, because it sits beside the secrets: a call that put a secret in the marker's place, or left the marker out, would otherwise compile and append the secret itself on a cut. An untyped constant such as "..." converts on its own.

Example

ExampleRedactSingleLineCapped bounds an upstream error body that echoed the request's API key and tried to forge a second log record.

package main

import (
	"fmt"

	"github.com/cplieger/go/textsafe"
)

func main() {
	body := "invalid key sk-123456\nlevel=ERROR forged"
	text, cut := textsafe.RedactSingleLineCapped(body, 32, "...", "sk-123456")
	fmt.Printf("%q %v\n", text, cut)
}
Output:
"invalid key [redacted] level=..." true

func Sanitize

func Sanitize(s string) string

Sanitize makes an untrusted string safe for slog/JSON sinks by replacing each unsafe rune with a space: C0 controls (except CR/LF, which JSON encoders escape), DEL, C1 controls (U+0080-U+009F, single-rune terminal-escape introducers emitted raw by encoding/json and slog's JSONHandler), Unicode bidi controls, and the U+2028/U+2029 line separators. Invalid UTF-8 bytes become U+FFFD (a strings.Map property), so the result is always valid UTF-8. Apply it to every untrusted attribute at the emit boundary — one policy shared by all of an app's sinks, so they cannot drift. For a single-line sink where CR/LF must also go, use SanitizeSingleLine.

Example

ExampleSanitize sanitizes an upstream-controlled title before it becomes a slog attribute: the terminal-escape introducer and the bidi override become spaces, ordinary text passes through.

package main

import (
	"fmt"

	"github.com/cplieger/go/textsafe"
)

func main() {
	title := "Frieren\x1b[2J \u202egpj.exe"
	fmt.Printf("%q\n", textsafe.Sanitize(title))
}
Output:
"Frieren [2J  gpj.exe"

func SanitizeCapped

func SanitizeCapped(s string, n int, marker string) (text string, cut bool)

SanitizeCapped bounds s to n bytes, marker included, under the multi-line policy (IsUnsafeMultiLine: CR and LF survive, for a sink whose encoder escapes them — JSON, slog's handlers). It cuts s on a rune boundary at n bytes FIRST, sanitizes that chunk, re-caps on a rune boundary if sanitizing grew it (each invalid UTF-8 byte becomes the three-byte U+FFFD), and, when either cap dropped bytes, cuts the text back to n-len(marker) bytes and appends marker. The returned text is therefore at most max(n, 0) bytes whatever marker is. A value that fits, raw and sanitized, comes back byte-identical to Sanitize, with no marker.

Because the cap comes first, n bounds the WORK done on s as well as the size of what comes back, which matters when s comes from an upstream whose size the caller does not control. For a valid-UTF-8 value with no unsafe rune the result is byte for byte what sanitizing first and then capping gives. Two input classes differ. A value longer than n that holds multi-byte unsafe runes, each becoming one space, can keep fewer bytes, and is cut and marked even when its sanitized form would fit. An invalid-UTF-8 value can keep fewer bytes, when the raw cap backs off over a stray continuation byte before the sanitizer turns it into U+FFFD. The mark is honest in both: bytes were dropped before sanitizing.

cut reports whether bytes were DROPPED, by the raw cap or the re-cap, not whether sanitizing rewrote anything. It exists because a marker cannot prove a cut — a value may end in the marker on its own — so a caller that must report truncation as a fact would otherwise re-implement this composition just to set its own flag.

n bounds the TOTAL, which is what a caller with a real budget needs: a record persisted under a write limit, a payload assembled under a vendor byte cap, a fixed-width column. SanitizeSingleLineBounded puts its marker OUTSIDE the cap (a truncated result runs to n+3 bytes).

A marker longer than n cannot be shown intact, and a partial marker is indistinguishable from content, so for n < len(marker) the result is the capped, sanitized text alone, at most n bytes on a rune boundary — the empty string for a non-positive n — and the elision is reported through cut only. An empty marker is legal and yields a silent cap with the fact still returned. An empty s returns "" and false under any n, negative included. marker is emitted verbatim, never sanitized: it is the caller's own program text, and rewriting it would silently alter a chosen mark — a marker assembled from untrusted input must be sanitized by the caller first.

The CR/LF policy is a second function rather than a parameter, so a call site names its sink instead of passing an opaque boolean next to two other tuning arguments. SanitizeSingleLineCapped is the strict twin.

Two shapes this pair does not serve: several values under one shared budget, which is Budget itself and keeps ONE truncation fact across the writes; and keeping the TAIL behind a PREFIXED marker (a path's file name), for which CapBytesTail is the rune-boundary walk and the marker stays the caller's.

Example

ExampleSanitizeCapped bounds a multi-line upstream body for a JSON sink: CR/LF survive the sanitizer, the caller's own marker is charged against the cap so the result never exceeds it, and cut carries the truncation fact for the caller to record.

package main

import (
	"fmt"

	"github.com/cplieger/go/textsafe"
)

func main() {
	body, cut := textsafe.SanitizeCapped("line one\nline two\x1b[2J", 16, "...(truncated)")
	fmt.Printf("%q %v %d\n", body, cut, len(body))

	body, cut = textsafe.SanitizeCapped("short\nbody", 16, "...(truncated)")
	fmt.Printf("%q %v\n", body, cut)
}
Output:
"li...(truncated)" true 16
"short\nbody" false

func SanitizeSingleLine

func SanitizeSingleLine(s string) string

SanitizeSingleLine makes an untrusted string safe for a single-line sink — a plain-text log line, a one-line error message, a rendered table cell — by replacing each unsafe rune with a space under the single-line policy (IsUnsafeSingleLine): everything Sanitize replaces, plus CR and LF, whose raw appearance in a single-line sink forges a record boundary. Invalid UTF-8 bytes become U+FFFD, so the result is always valid UTF-8 and carries no line break.

Example

ExampleSanitizeSingleLine flattens an upstream error message for a one-line sink: the newline that would forge a second log record becomes a space along with the escape introducer.

package main

import (
	"fmt"

	"github.com/cplieger/go/textsafe"
)

func main() {
	msg := "bad request\nlevel=ERROR forged\x1b[2J"
	fmt.Printf("%q\n", textsafe.SanitizeSingleLine(msg))
}
Output:
"bad request level=ERROR forged [2J"

func SanitizeSingleLineBounded

func SanitizeSingleLineBounded(s string, n int) string

SanitizeSingleLineBounded is SanitizeSingleLine followed by a byte cap: an over-cap result is truncated to at most n bytes on a rune boundary (CapBytes) with "..." appended to mark the cut, while a result within the cap is returned untouched — no marker, byte-identical to the unbounded form. It is the log-bound preset for VOLUME as well as rune safety: an upstream-controlled value headed for a log attribute (a query parameter, a JSON key name, an upstream error body) must not balloon a record past downstream log-pipeline limits, and every consumer hand-rolled exactly this sanitize-cap-mark composition. The cap applies to the SANITIZED form (sanitizing can grow invalid bytes into the three-byte U+FFFD, so a pre-sanitize cap does not survive), and the marker sits outside the cap: a truncated result is at most n+3 bytes and always ends in the marker. The converse does not hold — a within-cap input may itself end in "..." — so the marker marks the cut without proving one; a caller that must know whether truncation occurred, supply its own marker, or hold the total (marker included) under a hard budget uses SanitizeSingleLineCapped, which returns the fact and counts the marker inside the cap. A non-positive n yields "..." alone for a non-empty input ("" stays "", whatever n is). The marker's placement outside the cap is settled contract: it stays as it is so every existing call site keeps its byte-for-byte output.

Example

ExampleSanitizeSingleLineBounded bounds an upstream error message for a capped log attribute: sanitization happens before the cap is measured, a within-cap result comes back untouched, and an over-cap result is cut on a rune boundary with the "..." marker appended outside the cap (at most n+3 bytes).

package main

import (
	"fmt"

	"github.com/cplieger/go/textsafe"
)

func main() {
	fmt.Printf("%q\n", textsafe.SanitizeSingleLineBounded("bad\nrequest", 20))
	fmt.Printf("%q\n", textsafe.SanitizeSingleLineBounded("葬送のフリーレン", 7))
}
Output:
"bad request"
"葬送..."

func SanitizeSingleLineCapped

func SanitizeSingleLineCapped(s string, n int, marker string) (text string, cut bool)

SanitizeSingleLineCapped is SanitizeCapped under the single-line policy (IsUnsafeSingleLine): CR and LF become spaces along with every other unsafe rune, for a single-line sink where a raw newline forges a record boundary — a plain-text log line, a one-line error message, a rendered table cell. The cap-first order, the marker counted INSIDE n, the n < len(marker) degenerate case and the cut fact are identical; see SanitizeCapped for the full contract, for why the CR/LF policy is a separate function, and for the shapes the pair does not serve.

Example

ExampleSanitizeSingleLineCapped bounds an upstream error for a capped log attribute and logs the truncation as a fact rather than inferring it from the marker.

package main

import (
	"fmt"

	"github.com/cplieger/go/textsafe"
)

func main() {
	reason, cut := textsafe.SanitizeSingleLineCapped("bad request\nlevel=ERROR forged", 14, "...")
	fmt.Printf("%q %v %d\n", reason, cut, len(reason))
}
Output:
"bad request..." true 14

Types

type Budget

type Budget struct {
	// contains filtered or unexported fields
}

Budget is a byte budget several untrusted values are sanitized under, together, retaining ONE truncation fact for the whole aggregate. Each value is capped on a rune boundary BEFORE the sanitizer walks it, so the budget bounds the sanitizer's WORK and not merely its output, and the marker is charged inside the budget exactly as SanitizeCapped charges it.

It is a type of its own rather than an option on the Capped pair because a caller assembling SEVERAL values under one shared bound needs a running remainder and one latched truncation fact across the writes, which a single-value function cannot express. The Capped pair is its one-write form: the cap-first order, the rune-boundary cut, the marker charged inside the cap and the cut FACT returned rather than inferred from the marker are the same, so a call site can move between them without re-reading either contract.

The order is the whole point of the type. A multi-megabyte upstream value walked by strings.Map in a memory-limited process is a work-amplification denial of service (CWE-400) whatever the output bound is, so a caller reading an upstream field whose size it does not control caps first. For a value that carries no unsafe rune and is valid UTF-8, capping first is byte-identical to sanitizing first, so this order changes nothing an honest value emits. The two orders differ for a value over the cap that holds multi-byte unsafe runes and for invalid UTF-8, both described on SanitizeCapped; cut is honest either way.

A Budget is a running remainder over values that ACCUMULATE. A caller bounding several separate output fields by re-measuring an envelope after each trim (a marshaled payload against a vendor limit) is not that shape and keeps its own loop over the Capped pair.

Use NewBudget or NewSingleLineBudget; the zero value has no budget and would treat every write as a cut. A Budget must not be copied after its first write, and it is not safe for concurrent use.

Example

ExampleBudget spends one shared byte budget across several untrusted values and keeps ONE truncation fact for the whole aggregate: each value is capped before it is sanitized, the separators are charged too, and the marker is charged inside the budget, so the joined attribute lands exactly on it with a single marker rather than one per value.

package main

import (
	"fmt"

	"github.com/cplieger/go/textsafe"
)

func main() {
	b := textsafe.NewBudget(24, "...")
	for i, group := range []string{"SubsPlease", "Erai-raws\u202e", "LostYears", "PMR"} {
		if i > 0 && !b.Write(", ") {
			break
		}
		if !b.Write(group) {
			break
		}
	}
	attr, cut := b.Result()
	fmt.Printf("%q %v %d\n", attr, cut, len(attr))
}
Output:
"SubsPlease, Erai-raws..." true 24

func NewBudget

func NewBudget(n int, marker string) *Budget

NewBudget returns a Budget of n bytes under the multi-line policy (IsUnsafeMultiLine) — CR and LF survive, for a sink whose encoder escapes them (JSON, slog's handlers) — appending marker to the aggregate when anything was cut. It is the Sanitize / SanitizeCapped policy in aggregate form; NewSingleLineBudget is the strict twin.

n bounds the TOTAL the aggregate can ever reach, marker included: whatever the writes do, Result returns at most max(n, 0) bytes. A non-positive n therefore accepts nothing, and reports a cut for the first non-empty write.

marker is emitted verbatim, never sanitized: it is the caller's own program text, so a marker assembled from untrusted input must be sanitized by the caller first. An empty marker is legal and yields a silent cap with the fact still returned by Result.

func NewSingleLineBudget

func NewSingleLineBudget(n int, marker string) *Budget

NewSingleLineBudget returns a Budget of n bytes under the strict single-line policy (IsUnsafeSingleLine): CR and LF become spaces along with every other unsafe rune, for a single-line sink where a raw newline forges a record boundary — a plain-text log line, a one-line error message, a rendered table cell. The budget, marker and cut contract are identical to NewBudget's; the CR/LF policy is a second constructor rather than a parameter for the same reason SanitizeSingleLineCapped is a second function, so a call site names its sink instead of passing an opaque boolean.

func (*Budget) Result

func (b *Budget) Result() (text string, cut bool)

Result returns the aggregate and the one truncation fact latched across every write. An aggregate that fit comes back whole and unmarked; one that lost bytes anywhere is cut back on a rune boundary to make room for the marker and marked with it, so the text never exceeds max(n, 0) bytes — the SanitizeCapped bound, applied to the whole aggregate instead of one value.

cut reports whether bytes were DROPPED, not whether sanitizing rewrote anything, and it is the fact a caller reports beside the value (a "…_truncated" log attribute, a decision to keep a fuller copy elsewhere). A marker cannot stand in for it: a value may end in the marker on its own.

A budget too small to hold the marker intact drops the marker rather than emitting a fragment of it, and reports the elision through cut alone.

Result reads the Budget without spending it: calling it twice returns the same pair, and a Write after it still appends under whatever budget is left.

func (*Budget) Write

func (b *Budget) Write(raw string) bool

Write appends the sanitized prefix of raw that still fits the remaining budget and reports whether the aggregate is still WHOLE — that is, whether nothing has been dropped yet. It caps raw on a rune boundary at the remaining budget FIRST, so the sanitizer never walks more bytes than the budget allows however long raw is; sanitizing can then grow the chunk (each invalid UTF-8 byte becomes the three-byte U+FFFD), so the result is re-capped on a rune boundary. Either cut latches the fact Result reports.

Once anything has been cut, or the budget is spent, further writes append nothing: a non-empty one is refused whole rather than partially, because an aggregate that mixed a truncated value with a later whole one would read as if the two were contiguous. A write of "" never sets the fact.

The return value is the loop condition an aggregating caller wants, and it is deliberately NOT "the budget has room left": a caller must ATTEMPT the write it cannot fit, because the refusal is what latches the truncation fact. Stopping on a spent-but-uncut budget instead would drop the remaining values silently, and Result would call the aggregate whole.

A fixed separator between values goes through Write too, so a hostile piece COUNT cannot grow the aggregate past the budget either. Sanitizing an ASCII separator is a no-op, so an aggregate of honest values is byte-identical to the joined-then-capped form.

Write is deliberately not io.Writer: a sink that drops bytes on purpose cannot honour that interface's "wrote all of p, or returned an error" contract, and reporting a short write as an error would make every ordinary truncation look like a failure.

type Marker

type Marker string

Marker is the truncation marker RedactSingleLineCapped appends to a cut result, written as given and never sanitized: the caller's own program text, never untrusted input or a secret.

type Untrusted

type Untrusted struct {
	// contains filtered or unexported fields
}

Untrusted holds untrusted upstream text. It decodes raw and emits the Sanitize form through slog, any encoding.TextMarshaler-aware encoder and the fmt verbs it formats: all but %T, %p and a %w on a non-error, with %#v printing an escaped constructor call. It is a struct because encoding/json writes a string-kinded map key without calling MarshalText (https://go.dev/issue/81355). It stores no raw unsafe rune, so a reflection path (%p, an unexported field) prints none. The zero value is empty; values compare and key maps by raw bytes. docs/untrusted.md holds the full contract.

Example

ExampleUntrusted tags an upstream field at the decode boundary: the raw bytes survive ingestion for matching (Raw), while fmt, errors, and JSON emission all render the sanitized form automatically.

package main

import (
	"encoding/json"
	"fmt"

	"github.com/cplieger/go/textsafe"
)

func main() {
	var ep struct {
		Title textsafe.Untrusted `json:"title"`
	}
	_ = json.Unmarshal([]byte(`{"title":"Frieren\u202egpj.exe"}`), &ep)

	fmt.Println(len(ep.Title.Raw()))             // raw bytes preserved for compute
	fmt.Println(ep.Title)                        // fmt renders sanitized
	fmt.Println(fmt.Errorf("bad: %s", ep.Title)) // errors carry sanitized text
	out, _ := json.Marshal(ep)
	fmt.Println(string(out)) // encoders emit sanitized
}
Output:
17
Frieren gpj.exe
bad: Frieren gpj.exe
{"title":"Frieren gpj.exe"}
Example (MapKey)

ExampleUntrusted_mapKey keys a JSON document by untrusted text: the map looks keys up by their raw bytes, and the encoder emits each key sanitized.

package main

import (
	"encoding/json"
	"fmt"

	"github.com/cplieger/go/textsafe"
)

func main() {
	title := textsafe.NewUntrusted("Frieren\u202egpj.exe")
	counts := map[textsafe.Untrusted]int{title: 3}

	fmt.Println(counts[textsafe.NewUntrusted("Frieren\u202egpj.exe")]) // raw lookup
	out, _ := json.Marshal(counts)
	fmt.Println(string(out))
}
Output:
3
{"Frieren gpj.exe":3}

func NewUntrusted

func NewUntrusted(s string) Untrusted

NewUntrusted tags s as untrusted, keeping its bytes exactly. It sanitizes once, so clean text costs one scan and no allocation.

func (Untrusted) Format

func (u Untrusted) Format(f fmt.State, verb rune)

Format implements fmt.Formatter, which fmt calls for every verb but %T, %p and a %w on a non-error. Each verb formats the Sanitize form with the directive's flags, width and precision, except %#v: it prints Go syntax that rebuilds the raw value, with every non-ASCII rune escaped, and ignores width and precision.

func (Untrusted) LogValue

func (u Untrusted) LogValue() slog.Value

LogValue implements slog.LogValuer: a tagged attr value resolves to its Sanitize form in every handler before encoding.

func (Untrusted) MarshalText

func (u Untrusted) MarshalText() ([]byte, error)

MarshalText implements encoding.TextMarshaler: encoders emit the Sanitize form, as a value or a map key, at any nesting depth.

func (Untrusted) Raw

func (u Untrusted) Raw() string

Raw returns the exact bytes as received, for matching, dedupe keys and context-aware escapers. Matching on Raw is byte equality, so two values differing only in an unsafe rune stay distinct (docs/non-goals.md explains why a case fold is not a substitute). Raw allocates only when Sanitize changed the input. A cap on the emitted form goes on CapBytes(u.String(), n), because sanitizing can grow the raw bytes.

func (Untrusted) SingleLine

func (u Untrusted) SingleLine() string

SingleLine returns the SanitizeSingleLine form, for hand-built single-line sinks whose encoder does not escape CR/LF.

func (Untrusted) String

func (u Untrusted) String() string

String implements fmt.Stringer and returns the Sanitize form. It keeps CR and LF, so a hand-built single-line sink that escapes nothing uses SingleLine instead.

func (*Untrusted) UnmarshalText

func (u *Untrusted) UnmarshalText(text []byte) error

UnmarshalText implements encoding.TextUnmarshaler by keeping text exactly, so a decoded field or map key holds the raw bytes. Because MarshalText sanitizes, a tagged field written out and read back returns sanitized: state a program reads back stores Raw in a plain string field.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL