Documentation
¶
Overview ¶
Package sqlsanitize turns one application SQL statement into the two forms the application-SQL ingest path keeps: a literal-free text that is safe to store, and a canonical form whose hash is the statement's deduplication key.
The rules it applies are:
N1 every literal becomes `?`, and a PostgreSQL `$n` is rewritten to the same `?`
so that one statement has one stored form however the application wrote it.
An application may already have replaced its literals, which is why the
rewrite has to be idempotent
N2 comments are removed; all other spacing is preserved byte for byte
N3 identifiers and keywords are copied verbatim, quotes and case included
N4 an IN list of placeholders is one placeholder
N5 a VALUES list of repeated placeholder groups is one group
N1-N3 shape both forms; N4 and N5 shape the fingerprint alone, so the stored text keeps what the application wrote. Nothing else folds: an operator and a placeholder count are part of a statement's identity. See fold.go. Any later change to these rules must bump Version, so that a deployment can tell which rows were fingerprinted under which rules.
Classification reads the token's source text rather than omni's token-type constants, which are unexported in every dialect package. See token.go.
Index ¶
Constants ¶
const ( // Version identifies the rules applied by this package, and is stored beside every // fingerprint. It is 1 because this is the first rule set to run anywhere: the // foldings in fold.go were added while the feature was still unreleased, so they are // part of version 1 rather than a second version with rows to migrate. // // Its whole purpose is the next change. Rewriting a rule splits one statement into // rows fingerprinted under the old one, and a deployment that already holds rows // needs to know that happened; Version is how it finds out. Version = 1 // Placeholder is the wire and stored form of every literal. Applications send // it, and it is the only form that is ever persisted. Placeholder = "?" )
Variables ¶
var ErrUnlexable = errors.New("statement could not be lexed")
ErrUnlexable reports a statement that could not be tokenized: empty input, text that is not UTF-8 at all, or a dialect whose lexer stopped on a malformed token. Such a statement is never stored.
Functions ¶
func Analyzable ¶
Analyzable returns the text an analyzer can parse. Every placeholder becomes a string literal, because the MySQL family's grammar has no production that accepts a placeholder at all. The rewrite walks tokens, so a `?` inside a quoted identifier is never touched.
Types ¶
type Statement ¶
type Statement struct {
// Text is what gets stored: every literal replaced by Placeholder, comments
// removed, and all other spacing left as the application wrote it. Spacing is
// preserved because the analyzer names an unaliased expression column after its
// source text, and that name is stored in column_lineage.
Text string
// Canonical is Text with its tokens re-joined by single spaces. It exists only to
// be hashed: whitespace must not split one statement into two.
Canonical string
// Fingerprint is the statement's deduplication key, the sha256 of Canonical.
Fingerprint string
}
Statement is one statement in the two forms the ingest path keeps.
Source Files