Documentation
¶
Overview ¶
Package leak is the staging leak check. It answers one question: how many real production values are on this staging database?
Hash samples the columns scan flags on production and keeps keyed hashes of the values, under a key only the customer holds. Compare reads those columns whole on staging, hashes what it finds the same way, and counts the matches. Neither step keeps or prints a value, and a report holds counts only.
Index ¶
- Constants
- Variables
- func CatchChance(n, N, real float64) float64
- func Compare(ctx context.Context, sess *dbconn.Session, f *File, key Key, ...) (*model.LeakResult, error)
- func Gate(res *model.LeakResult, threshold model.Severity) (hits, unchecked int)
- func ShownAt95(n, N float64) float64
- func WriteNewKey(path string) error
- type Allowlist
- type CompareOptions
- type File
- type FileColumn
- type HashOptions
- type Key
Constants ¶
const DefaultSampleRows = 3000
DefaultSampleRows is how many rows a check samples per table, and values per key of a key/value table. It has no flag: a bigger sample is what would move the free check towards the exact one in Pro, and a smaller one says too little. 3,000 values out of 100,000 catch 50 real rows about 78% of the time; 300 would catch them 14% of the time.
Variables ¶
var ( ErrFileUnreadable = errors.New("can't read the file given to --hashes") ErrFileFormat = errors.New("the file given to --hashes isn't a prodmask leak-check hash file, or it's damaged") ErrFileVersion = errors.New("the file given to --hashes was made by another version of prodmask: run prodmask leak-check hash again") ErrWrongKey = errors.New("this key isn't the one the hash file was made with") )
The errors reading a hash file can give. None quotes what it read.
var ( ErrNothingToHash = errors.New("no flagged column holds a value this check counts, so there is nothing to compare") ErrNothingSampled = errors.New("some flagged columns couldn't be sampled, and no other holds a value this check counts, so there is nothing to compare") )
The errors of a production with nothing to compare.
var ( ErrKeyFormat = errors.New("a key is at least 64 hex characters: make one with prodmask leak-check key, or openssl rand -hex 32") ErrKeyUTF16 = errors.New("the key file isn't plain text; it looks like UTF-16, which PowerShell's > writes: make one with prodmask leak-check key") )
The errors a key can give. None quotes what it read, which may be a key.
var ErrAllowValue = errors.New("a value of --allow-email is neither an address nor a domain with a dot")
ErrAllowValue is the error of an --allow-email value that is neither an address nor a domain with a dot. It doesn't quote the value.
Functions ¶
func CatchChance ¶
CatchChance is the chance that a sample of n values out of N catches at least one of real real values: close to 1 - (1 - n/N)^real.
func Compare ¶
func Compare(ctx context.Context, sess *dbconn.Session, f *File, key Key, opts CompareOptions) (*model.LeakResult, error)
Compare reads on staging the columns a hash file names, whole, and counts the sampled values it finds. It reads through sess, a read-only session, and never runs detection to choose columns. Only names and counts leave it.
func Gate ¶
func Gate(res *model.LeakResult, threshold model.Severity) (hits, unchecked int)
Gate says how a check stands against --fail-on: how many columns at or above the threshold, not excepted, hold a sampled value, and how many couldn't be checked in full. A hit counts in a stopped column too. A gate fails closed whenever it couldn't look, which includes a column renamed on staging, since hashes are bound to their location. A column left out on production for a reason a rerun might cure counts at any threshold, as it has no kind.
func WriteNewKey ¶
WriteNewKey makes a key and writes it to a new file, owner-only where the system has that. It never overwrites a file: that would orphan the hashes made under the old key.
Types ¶
type Allowlist ¶
type Allowlist struct {
Domains []string
// contains filtered or unexported fields
}
Allowlist is what --allow-email keeps out of a check: domains with their subdomains, and exact addresses, in lower case. Addresses are real people's: they're never written to a file or a report, only counted.
func NewAllowlist ¶
NewAllowlist reads --allow-email's values.
type CompareOptions ¶
type CompareOptions struct {
Allow *Allowlist
FailExcept []string
// MaxTime is the limit on the whole read; 0 means none.
MaxTime time.Duration
Version string
// Starting says what's about to be read, once it's known.
Starting func(tables int, rowsEstimate int64)
// Progress reports the table being read and the rows read so far.
Progress func(done, total int, table string, rows int64)
}
CompareOptions control Compare.
type File ¶
type File struct {
Format string `json:"format"`
Version int `json:"version"`
Notice string `json:"notice"`
ToolVersion string `json:"tool_version"`
CreatedAt time.Time `json:"created_at"`
Database string `json:"database"`
Salt []byte `json:"salt"`
KeyCheck []byte `json:"key_check"`
SampleRows int `json:"sample_rows"`
AllowDomains []string `json:"allow_domains"`
AllowedAddresses int `json:"allowed_addresses"`
ValuesSkipped int64 `json:"values_skipped"`
Columns []FileColumn `json:"columns"`
LeftOut []model.LeakLeftOut `json:"left_out"`
// Warnings are the sample's own, for compare's report, as the two steps
// can run apart. They hold names and error numbers only.
Warnings []string `json:"warnings,omitempty"`
}
File is a hash file: what hash leaves for compare. It holds names, counts and keyed hashes, never a value, and the key's only trace is KeyCheck. It's unsigned on purpose: it isn't evidence.
func Hash ¶
func Hash(ctx context.Context, sess *dbconn.Session, key Key, opts HashOptions) (*File, []string, error)
Hash runs a scan of production, samples the columns it flags, and returns the hash file, with warnings to show. It reads through sess, a read-only session, and what it reads stays in memory.
func ReadFile ¶
ReadFile reads a hash file. Its errors don't quote the path, which may be a key typed in the wrong place.
type FileColumn ¶
type FileColumn struct {
Table string `json:"table"`
Column string `json:"column"`
KeyColumn string `json:"key_column,omitempty"`
// MetaKey is a key of a key/value table as scan shows it: a name, or a
// pattern whose * stands for the part that varies.
MetaKey string `json:"meta_key,omitempty"`
Kind model.LeakKind `json:"kind"`
// Whole marks the kinds read with a column's own rule: the loose phone
// finder, or a whole value.
Whole bool `json:"whole,omitempty"`
MaxChars int `json:"max_chars"`
Sample string `json:"sample"`
Exact bool `json:"exact,omitempty"`
RowsEstimate int64 `json:"rows_estimate"`
RowsRead int64 `json:"rows_read"`
EstimatedValues int64 `json:"estimated_values"`
Values int `json:"values"`
// Hashes holds the 16-byte hashes one after another, sorted, so their
// order says nothing about rows.
Hashes []byte `json:"hashes"`
}
FileColumn is the sample of one kind of value in one column or key.
type HashOptions ¶
type HashOptions struct {
Allow *Allowlist
Preset string
Version string
SampleRows int // 0 means DefaultSampleRows; only tests set it
// ScanProgress and Progress report the scan and then the sampling.
ScanProgress func(done, total int, table string)
Progress func(done, total int, table string)
// Pick picks a number below n, where a sample needs a random spot; nil
// picks at random. Only tests set it.
Pick func(n uint64) uint64
}
HashOptions control Hash.
type Key ¶
type Key struct {
// contains filtered or unexported fields
}
Key is the secret the hashes are made under. It prints as [key]. The bytes sit behind a function, so even a verb or a struct that skips Format can't print them: fmt follows pointers, and doesn't call functions.
func ParseKey ¶
ParseKey reads a key as a key file or $PRODMASK_LEAK_KEY holds it: hex, with whitespace and a UTF-8 BOM around it allowed. Anything else is refused, passphrases above all, the one way a keyed hash becomes guessable.
func ReadKeyFile ¶
ReadKeyFile reads a key file. Its errors don't quote the path, which may be a key typed in the wrong place. worldReadable says others can read the file, which only means something outside Windows.
Source Files