redact

package module
v1.0.7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 25, 2020 License: Apache-2.0 Imports: 9 Imported by: 154

README

redact

Utilities to redact Go strings for confidentiality

Godoc link: https://pkg.go.dev/github.com/cockroachdb/redact?tab=doc

Documentation

Overview

Package redact provides facilities for separating “safe” and “unsafe” pieces of data when logging and constructing error object.

An item is said to be “safe” if it is proven to not contain PII or otherwise confidential information that should not escape the boundaries of the current system, for example via telemetry or crash reporting. Conversely, data is considered “unsafe” until/unless it is known to be “safe”.

Example use:

redactable := redact.Sprintf("hello %s", "universe")

// At this point, 'redactable' contains "hello ‹universe›".

// This prints "hello universe":
fmt.Println(redactable.StripMarkers())

// This reports "hello ‹×›":
fmt.Println(redactable.Redact())

When defining your own custom types, you can define a SafeFormat method, implementing the redact.SafeFormatter interface in a way you'd otherwise implement fmt.Formatter. This is then recognized by this package's API automatically.

Alternatively:

- you can implement the SafeValue interface, which tells the redact package to always the default formatting of a type as safe and thus not included inside redaction markers.

- you can include a value within redact.Safe() and redact.Unsafe() in redact.Sprintf / redact.Fprintf calls, to force the omission or inclusion of redaction markers.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func EndMarker

func EndMarker() []byte

EndMarker returns the end delimiter for an unsafe string.

func EscapeMarkers

func EscapeMarkers(s []byte) []byte

EscapeMarkers escapes the special delimiters from the provided byte slice.

func Fprint

func Fprint(w io.Writer, args ...interface{}) (n int, err error)

Fprint is like Sprint but outputs the redactable string to the provided Writer.

func Fprintf

func Fprintf(w io.Writer, format string, args ...interface{}) (n int, err error)

Fprintf is like Sprintf but outputs the redactable string to the provided Writer.

func MakeFormat

func MakeFormat(s fmt.State, verb rune) (justV bool, format string)

MakeFormat is a helper for use by implementations of the SafeFormatter interface. It reproduces the format currently active in fmt.State and verb. This is provided because Go's standard fmt.State does not make the original format string available to us.

If the return value justV is true, then the current state was found to be %v exactly; in that case the caller can avoid a full-blown Printf call and use just Print instead to take a shortcut.

func RedactedMarker

func RedactedMarker() []byte

RedactedMarker returns the special string used by Redact.

func RegisterRedactErrorFn

func RegisterRedactErrorFn(fn func(err error, p SafePrinter, verb rune))

RegisterRedactErrorFn registers an error redaction function for use during automatic redaction by this package. Provided e.g. by cockroachdb/errors.

func RegisterSafeType

func RegisterSafeType(t reflect.Type)

RegisterSafeType registers a data type to always be considered safe during the production of redactable strings.

func StartMarker

func StartMarker() []byte

StartMarker returns the start delimiter for an unsafe string.

func StringWithoutMarkers

func StringWithoutMarkers(f SafeFormatter) string

StringWithoutMarkers formats the provided SafeFormatter and strips the redaction markers from the result. This is provided for convenience to facilitate the implementation of String() methods alongside SafeFormat() to avoid code duplication.

Note: if this function is ever found to be a performance bottleneck, one can consider using an alternate implementation of Sprint() which similarly calls the SafeFormat() methods but does not introduce markers and instead writes to a string buffer directly.

func Unsafe

func Unsafe(a interface{}) interface{}

Unsafe turns any value that would otherwise be considered safe, into an unsafe value.

Types

type RedactableBytes

type RedactableBytes []byte

RedactableBytes is like RedactableString but is a byte slice.

Instances of RedactableBytes should not be constructed directly; instead use the facilities from print.go (Sprint, Sprintf) or the methods below.

func EscapeBytes

func EscapeBytes(s []byte) RedactableBytes

EscapeBytes escapes markers inside the given byte slice and encloses the entire byte slice between redaction markers.

func (RedactableBytes) Redact

func (s RedactableBytes) Redact() RedactableBytes

Redact replaces all occurrences of unsafe substrings by the “Redacted” marker, ‹×›.

func (RedactableBytes) SafeFormat

func (s RedactableBytes) SafeFormat(sp SafePrinter, _ rune)

SafeFormat formats the redactable safely.

func (RedactableBytes) StripMarkers

func (s RedactableBytes) StripMarkers() []byte

StripMarkers removes the redaction markers from the RedactableBytes. This returns an unsafe string where all safe and unsafe bits are mixed together.

func (RedactableBytes) ToString

func (s RedactableBytes) ToString() RedactableString

ToString converts the byte slice to a string.

type RedactableString

type RedactableString string

RedactableString is a string that contains a mix of safe and unsafe bits of data, but where it is known that unsafe bits are enclosed by redaction markers ‹ and ›, and occurrences of the markers inside the original data items have been escaped.

Instances of RedactableString should not be constructed directly; instead use the facilities from print.go (Sprint, Sprintf) or the methods below.

func HelperForErrorf added in v1.0.4

func HelperForErrorf(format string, args ...interface{}) (RedactableString, error)

HelperForErrorf is a helper to implement a redaction-aware fmt.Errorf-compatible function in a different package. It formats the string according to the given format and arguments in the same way as Sprintf, but in addition to this if the format contains %w and an error object in the proper argument position it also returns that error object.

Note: This function only works if an error redaction function has been injected with RegisterRedactErrorFn().

func Sprint

func Sprint(args ...interface{}) RedactableString

Sprint prints out the arguments and encloses unsafe bits between redaction markers. If either safe and unsafe bits of data contain the markers in their representation already, they are escaped first. If a RedactableString or RedactableBytes argument is passed, it is reproduced as-is without escaping.

func Sprintf

func Sprintf(format string, args ...interface{}) RedactableString

Sprintf formats the arguments and encloses unsafe bits between redaction markers. If either safe and unsafe bits of data contain the markers in their representation already, they are escaped first. The format is always considered safe and the caller is responsible to ensure that the markers are not present in the format string.

func Sprintfn

func Sprintfn(printer func(w SafePrinter)) RedactableString

Sprintfn produces a RedactableString using the provided SafeFormat-alike function.

func (RedactableString) Redact

Redact replaces all occurrences of unsafe substrings by the “Redacted” marker, ‹×›. The result string is still safe.

func (RedactableString) SafeFormat

func (s RedactableString) SafeFormat(sp SafePrinter, _ rune)

SafeFormat formats the redactable safely.

func (RedactableString) StripMarkers

func (s RedactableString) StripMarkers() string

StripMarkers removes the redaction markers from the RedactableString. This returns an unsafe string where all safe and unsafe bits are mixed together.

func (RedactableString) ToBytes

func (s RedactableString) ToBytes() RedactableBytes

ToBytes converts the string to a byte slice.

type SafeFormatter

type SafeFormatter interface {
	// SafeFormat is like the Format method of fmt.Formatter, except
	// that it operates using a SafePrinter instead of a fmt.State for
	// output.
	//
	// The verb argument is the control character that defines
	// the formatting mode in the surrounding Printf call.
	// For example, if this method is called to format %03d,
	// the verb is 'd'.
	SafeFormat(s SafePrinter, verb rune)
}

SafeFormatter is implemented by object types that want to separate safe and non-safe information when printed out by a Printf-like formatter.

type SafeMessager

type SafeMessager interface {
	SafeMessage() string
}

SafeMessager is an alternative to SafeFormatter used in previous versions of CockroachDB. NB: this interface is obsolete. Use SafeFormatter instead. TODO(knz): Remove this.

type SafePrinter

type SafePrinter interface {
	// SafePrinter inherits fmt.State to access format flags, however
	// calls to fmt.State's underlying Write() as unsafe.
	fmt.State

	// SafePrinter provides the SafeWriter interface.
	SafeWriter
}

SafePrinter is a stateful helper that abstracts an output stream in the context of printf-like formatting, but with the ability to separate safe and unsafe bits of data.

This package provides one implementation of this using marker delimiters for unsafe data, see markers.go. We would like to aim for alternate implementations to generate more structured formats.

type SafeRune

type SafeRune rune

SafeRune aliases rune. See the explanation for SafeString.

func (SafeRune) SafeValue

func (SafeRune) SafeValue()

SafeValue makes SafeRune a SafeValue.

type SafeString

type SafeString string

SafeString aliases string. This is not meant to be used directly; the type definition ensures that SafePrinter's SafeString method can only take constant string literals as arguments. Typically, a Go linter would ensure that ConstantString is never used to cast a value.

func (SafeString) SafeValue

func (SafeString) SafeValue()

SafeValue makes SafeString a SafeValue.

type SafeValue

type SafeValue interface {
	SafeValue()
}

SafeValue is a marker interface to be implemented by types that alias base Go types and whose natural representation via Printf is always safe for reporting.

This is recognized by the SafePrinter interface as an alternative to SafeFormatter.

It is provided to decorate "leaf" Go types, such as aliases to int.

Typically, a linter enforces that a type can only implement this interface if it aliases a base go type. More complex types should implement SafeFormatter instead.

It is advised to build an automatic process during builds to collect all the types that implement this interface, as well as all uses of this type, and produce a report. Changes to this report should receive maximal amount of scrutiny during code reviews.

func Safe

func Safe(a interface{}) SafeValue

Safe turns any value into an object that is considered as safe by the formatter.

This is provided as an “escape hatch” for cases where the other interfaces and conventions fail. Increased usage of this mechanism should be taken as a signal that a new abstraction is missing. The implementation is also slow.

type SafeWriter

type SafeWriter interface {
	// SafeString emits a safe string.
	SafeString(SafeString)

	// SafeRune emits a safe rune.
	SafeRune(SafeRune)

	// Print emits its arguments separated by spaces.
	// For each argument it dynamically checks for the SafeFormatter or
	// SafeValue interface and either use that, or mark the argument
	// payload as unsafe.
	Print(args ...interface{})

	// For printf, a linter checks that the format string is
	// a constant literal, so the implementation can assume it's always
	// safe.
	Printf(format string, arg ...interface{})

	// UnsafeString writes an unsafe string.
	UnsafeString(string)

	// UnsafeByte writes an unsafe byte.
	UnsafeByte(byte)

	// UnsafeBytes writes an unsafe byte slice.
	UnsafeBytes([]byte)

	// UnsafeRune writes an unsafe rune.
	UnsafeRune(rune)
}

SafeWriter provides helper functions for use in implementations of SafeFormatter, to format mixes of safe and unsafe strings.

type StringBuilder added in v1.0.3

type StringBuilder struct {
	// contains filtered or unexported fields
}

StringBuilder accumulates strings with optional redaction markers.

It implements io.Writer but marks direct writes as redactable. To distinguish safe and unsafe bits, it also implements the SafeWriter interface.

func (*StringBuilder) Cap added in v1.0.3

func (b *StringBuilder) Cap() int

Cap returns the capacity of the builder's underlying byte slice. It is the total space allocated for the string being built and includes any bytes already written.

func (*StringBuilder) Len added in v1.0.3

func (b *StringBuilder) Len() int

Len returns the number of accumulated bytes, including redaction markers; b.Len() == len(b.RedactableString()).

func (*StringBuilder) Print added in v1.0.3

func (b *StringBuilder) Print(args ...interface{})

Print is part of the SafeWriter interface.

func (*StringBuilder) Printf added in v1.0.3

func (b *StringBuilder) Printf(format string, args ...interface{})

Printf is part of the SafeWriter interface.

func (*StringBuilder) RedactableBytes added in v1.0.5

func (b *StringBuilder) RedactableBytes() RedactableBytes

RedactableString returns the accumulated bytes, including redaction markers.

func (*StringBuilder) RedactableString added in v1.0.3

func (b *StringBuilder) RedactableString() RedactableString

RedactableString returns the accumulated string, including redaction markers.

func (*StringBuilder) Reset added in v1.0.3

func (b *StringBuilder) Reset()

Reset resets the Builder to be empty.

func (*StringBuilder) SafeFormat added in v1.0.3

func (b *StringBuilder) SafeFormat(p SafePrinter, _ rune)

SafeFormat implements SafeFormatter.

func (*StringBuilder) SafeRune added in v1.0.3

func (b *StringBuilder) SafeRune(s SafeRune)

SafeRune is part of the SafeWriter interface.

func (*StringBuilder) SafeString added in v1.0.3

func (b *StringBuilder) SafeString(s SafeString)

SafeString is part of the SafeWriter interface.

func (*StringBuilder) String added in v1.0.3

func (b *StringBuilder) String() string

String returns the accumulated string, with redaction markers stripped out. To obtain the redactable string, call RedactableString().

func (*StringBuilder) UnsafeByte added in v1.0.3

func (b *StringBuilder) UnsafeByte(s byte)

UnsafeByte is part of the SafeWriter interface.

func (*StringBuilder) UnsafeBytes added in v1.0.3

func (b *StringBuilder) UnsafeBytes(s []byte)

UnsafeBytes is part of the SafeWriter interface.

func (*StringBuilder) UnsafeRune added in v1.0.3

func (b *StringBuilder) UnsafeRune(s rune)

UnsafeRune is part of the SafeWriter interface.

func (*StringBuilder) UnsafeString added in v1.0.3

func (b *StringBuilder) UnsafeString(s string)

UnsafeString is part of the SafeWriter interface.

func (*StringBuilder) Write added in v1.0.3

func (b *StringBuilder) Write(s []byte) (int, error)

Write implements the io.Writer interface.

Directories

Path Synopsis
fmtsort
Package fmtsort provides a general stable ordering mechanism for maps, on behalf of the fmt and text/template packages.
Package fmtsort provides a general stable ordering mechanism for maps, on behalf of the fmt and text/template packages.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL