Documentation
¶
Overview ¶
Package redact removes known secret values from text on its way somewhere that keeps it.
Why this exists ¶
A task that holds a leased credential can echo it. Not maliciously — a `set -x` in a build script, a debug flag, an HTTP library that dumps its request headers, a stack trace carrying an argv. The credential then lands in three places that outlive the lease: the task artifact under .cloop/tasks/, the step log in state.db, and the live-log frame broadcast to every attached browser. The lease expires in minutes; the transcript does not.
pkg/audit already scans for this, which is the tell: the leak was expected and only reported afterwards. This package prevents it instead, at the point output is captured.
Two halves, and which one is the defence ¶
A Set matches exact values: the material a lease just injected. It is the primary defence and the only thing on the streaming path. A Set does not hunt for secret-shaped strings — no entropy heuristic, no "looks like a token" regex — because both would cost on every streamed token and both mangle ordinary text: a base64 blob in a diff is not a credential.
The second half, in patterns.go, is a registry of credential shapes for the places that cannot know the values: an error message from three packages down, a browser's stack trace, a commit made last year. cloop audit, the provider-call audit, the secret broker's audit reasons and the telemetry scrubber all ask it, so a credential shape is recognised everywhere or nowhere. It is a backstop for text that never passed through a Set, not a substitute for one, and nothing on the streaming path calls it.
The short-value guard ¶
A value shorter than MinLen is never matched. A four-character credential is not one, and replacing every occurrence of a short string would corrupt unrelated output for no security benefit. This is the same guard executor.RedactSecrets has always applied; it lives here now so there is one implementation rather than one per call site.
Index ¶
Constants ¶
const EnvKey = "CLOOP_REDACT_ENV"
EnvKey names the environment variable through which a lease declares which *other* variables hold credentials.
It carries names, never values — exactly like executor.SecretBinding.EnvKeys and for the same reason: it is inherited by every child process and would otherwise be one more copy of the secret. The indirection is what lets a workload scrub its own output without a heuristic: GITHUB_TOKEN holds a credential and CLOOP_GITHUB_REPO_ALLOWLIST holds a repository list, and nothing about the two values themselves says which is which.
const LeaseDirKey = "CLOOP_LEASE_DIR"
LeaseDirKey names the variable holding the lease's own credential directory. It is set by pkg/secretbroker; this package only reads it.
const Marker = "[redacted]"
Marker replaces a matched secret. It is the literal pkg/audit's scanners and the existing git-path redaction already use, so a transcript reads the same whichever layer caught the value.
const MinLen = 8
MinLen is the shortest value worth matching; see the package doc.
Variables ¶
This section is empty.
Functions ¶
func IsPlaceholder ¶
IsPlaceholder reports whether v is something a redactor already left behind rather than a credential.
It matters because a Spec is stored with its leased variables replaced by a placeholder — see executorstore's redactLeasedEnv, which writes "<redacted:leased>" — and a spec rehydrated from that store after a hub restart would otherwise hand this package its own output as a secret to match. The result would not be a disclosure, but it would be a Set that exists to replace "[redacted]" with "[redacted]", doing per-chunk work on the streaming path for nothing and mangling a diagnostic string that is supposed to be legible.
A credential that genuinely looked like this would be skipped. That is the right trade: the value is already published in the source of two packages.
func ScrubFunc ¶ added in v0.0.5
ScrubFunc returns text with every credential replaced by what repl returns for it. Overlapping matches are merged into one span first, so a token inside a URL inside a header is replaced once, and the merged Match carries the detector that starts earliest — the most specific on a tie.
It repeats until nothing more changes, at most maxPasses times. One pass is not always enough: a credential glued to another one — a key straight after a PEM block's last dash — fails its boundary check against bytes that the same pass replaces, and only the next pass sees it follow the replacement. Text with nothing in it costs one pass, as before.
Types ¶
type Detector ¶ added in v0.0.5
type Detector struct {
// Name is the kebab-case identifier, e.g. "github-token-long".
Name string
// Label is a short human description for a report line.
Label string
// Recognises says, for the reference page, what the detector matches.
Recognises string
// contains filtered or unexported fields
}
Detector is one named credential shape.
Name is stable: it is what cloop audit reports and what the fixture corpus keys on, so a detector is never renamed — retire it and add another.
type Live ¶ added in v0.0.5
type Live struct {
// contains filtered or unexported fields
}
Live is a Set over a process's lease that follows rewrites of its files. The zero value is not usable; call NewLive. Safe for concurrent use.
func NewLive ¶ added in v0.0.5
NewLive returns a Live seeded from environ exactly as FromEnviron would be.
type Match ¶ added in v0.0.5
type Match struct {
// Detector is the Name of the detector that matched; Label is its Label.
Detector, Label string
// Start and End delimit the credential: the bytes a scanner removes.
// Context a detector needed in order to recognise it — "Bearer ", a URL's
// scheme and host, a "token:" key — lies outside and is kept.
Start, End int
// Kind is the credential's public lead when its format has one — "ghs_",
// "sk-ant-api03-", "cloop_pat_" — and empty otherwise. It sits at Start.
// A replacement may keep it: it says which kind of credential was removed
// and nothing about the value.
Kind string
// contains filtered or unexported fields
}
Match is one credential found in a text.
func Find ¶ added in v0.0.5
Find returns every credential in text, ordered by position.
Detectors are independent, so the same bytes can be reported twice — a GitHub token inside an Authorization header is both. A scanner that only needs to know what is there (cloop audit) wants exactly that; one that rewrites the text wants ScrubFunc, which merges overlaps first.
type Set ¶
type Set struct {
// contains filtered or unexported fields
}
Set is a collection of secret values to remove. A nil *Set is valid and redacts nothing, so callers with no lease need no special case.
Safe for concurrent use: it is immutable after New.
func FromEnviron ¶
FromEnviron builds a Set from a process environment: the values of every variable EnvKey names, plus the contents of every file in the lease directory.
This is how a workload redacts its *own* output. The hub knows what it injected and scrubs at the executor boundary, but the harness runs inside the sandbox and its output is captured there first — by the orchestrator, into an artifact — so the guarantee has to hold on both sides of that boundary.
Unreadable files are skipped rather than reported. A missing lease directory is the ordinary case for a project with no grants, and failing to redact is not a reason to fail a run.
func New ¶
New returns a Set over values, ignoring those shorter than MinLen and any duplicates. It returns nil when nothing survives that filter, which callers may use to skip redaction entirely.
func (*Set) Bytes ¶
Bytes is String over a byte slice. The input is not modified; a copy is returned only when something matched.
func (*Set) Contains ¶
Contains reports whether text holds any value in the Set. It is the cheap question a test or an audit asks; callers scrubbing output want String.
func (*Set) Holdback ¶
Holdback returns how many trailing bytes of text a streaming caller must withhold because they could still turn out to be the start of a secret.
It is exported for callers that do their own buffering rather than writing through a Writer — a log bus fanning one chunk out to many subscribers cannot hand its bytes to an io.Writer and still number them. Zero means the whole of text is safe to publish now, which is the usual answer.
func (*Set) With ¶ added in v0.0.5
With returns a Set over s's values and values, or s itself when values add nothing that survives New's filter. s is not modified — a Set is immutable, which is what lets one be read without a lock — so a holder whose credential changes while it runs (a GitHub App token refreshed mid-run, Task 20375) installs the result in place of the old Set. The old values stay: the credential a workload printed before the refresh must stay scrubbed after it.
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
Writer wraps an io.Writer so that nothing carrying a known secret reaches it.
Chunk boundaries are handled: a value split across two Writes is still caught, because the tail that could be the start of one is withheld until the next Write resolves it. Flush releases any such remainder, and callers that own the stream's end must call it or risk losing a trailing partial line.
Safe for concurrent use, because the streams it wraps — a driver's output pump and its reaper — race by nature.
func NewFollowingWriter ¶ added in v0.0.5
NewFollowingWriter is NewWriter over a Set that may grow while the stream is open: current is asked on every Write and Flush, and its answer scrubs that chunk together with whatever an earlier one withheld. A value printed before it joined the set has already gone; nothing can call it back.
func NewWriter ¶
NewWriter returns a Writer forwarding scrubbed output to dst. A nil Set yields a Writer that forwards verbatim, so the caller needs no branch.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package redacttest is the credential corpus every pattern scanner in cloop is held to: realistic credentials in the context they leak in, and ordinary output that has to come through untouched.
|
Package redacttest is the credential corpus every pattern scanner in cloop is held to: realistic credentials in the context they leak in, and ordinary output that has to come through untouched. |