audit

package
v0.510.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 30, 2026 License: Apache-2.0 Imports: 22 Imported by: 0

README

Application Audit Module

Application orchestration for parsing and, later, evaluating SQL audit requests.

Files

File Responsibility
parse.go Normalizes one leading UTF-8 BOM, parses bounded top-level statement slices independently by dialect, and translates successful locations from each slice so failed text cannot capture a valid sibling's source match
statement_boundary.go Splits top-level SQL only at semicolons outside supported dialect strings, quoted identifiers, comments, and PostgreSQL dollar-quoted bodies (including Unicode identifier tags) while distinguishing dollar signs inside unquoted identifiers; it does not infer statement semantics
parse_pg.go Implements PostgreSQL parsing when built with the postgresql tag
parse_pg_stub.go Returns the PG-capable build guidance error when PostgreSQL support is not compiled in
parse_test.go Verifies that application parsing hides parser-specific AST details
extract.go Converts parsed statements into first-pass domain Statement values by invoking parser-neutral extractors and attaching shape-only DML impact facts
extract_test.go Verifies representative DDL and DML extraction behavior, including create-like/create-as/partition flags plus enriched create-table facts, preserved backticked-keyword and unnamed-index names, extracted column charset/collation facts, normalized row-format and auto-increment-init options, explicit DDL lifecycle operations, MySQL/TiDB ALTER index/constraint action normalization with ALGORITHM/LOCK preservation, richer alter-table detail including explicit nullability and other statement-local change facts, multi-column add expansion, non-index constraint handling, mutation-target-only DML tables, and predicate-shape facts
corpus_helpers_test.go Provides shared corpus metadata conversion and semantic assertions, including optional statement-level impact contracts
corpus_postgresql_tag_test.go Runs PostgreSQL corpus fixtures, including the representative modern-shape pack and tagged partial parser-error checks, through the full audit pipeline and semantic extraction checks
corpus_test.go Runs MySQL/TiDB corpus fixtures through the full audit pipeline and semantic extraction checks
corpus_testdata_test.go Validates corpus fixture shape and supported expected-field enums, including impact expectations
corpus_inventory_test.go Prints the supported-rule and dialect corpus inventory
corpus_coverage_test.go Verifies every default rule has checked-in SQL corpus coverage for its supported dialects
cross_dialect_ddl_coverage_census_test.go Classifies representative MySQL/TiDB DDL forms for coverage and generated catalog evidence
postgresql_ddl_coverage_census_postgresql_tag_test.go Classifies representative PostgreSQL DDL forms and records which forms have corpus fixtures for catalog generation
postgresql_ddl_consolidated_census_postgresql_tag_test.go Verifies the per-source and consolidated PostgreSQL DDL census totals
ddl_coverage_catalog_test.go Generates and validates the checked-in DDL coverage catalog from the cross-dialect and PostgreSQL census sources
ddl_coverage_catalog_query_test.go Verifies embedded and checked-in catalog parity plus catalog query behavior
impact_postgresql_tag_test.go Verifies PostgreSQL offline primary-key equality and planner impact-source precedence
impact.go Maps extracted DML predicate shapes to conservative offline impact estimates, populates statement impact objects with estimated_rows, estimated_ratio, risk_level, confidence, source, reason_codes, and optional notes, upgrades shape-derived sources to metadata after enrichment, and refines the narrow primary-key-on-id case when metadata snapshots confirm PRIMARY(id) plus optional table_rows facts
impact_test.go Verifies shape-only impact estimation plus post-enrichment metadata source upgrades, unique-equality refinement, and offline preservation behavior for representative UPDATE and DELETE shapes and their additive impact payloads
evaluate.go Applies registered rules, enriches findings with explanation metadata, and aggregates statement/global findings into report output while preserving statement-level DML impact estimates
evaluate_test.go Verifies application-owned report-flow integration and explanation enrichment over the rule registry
explain.go Joins evaluated findings with shipped catalog metadata and statement metadata availability notes
service.go Normalizes one leading UTF-8 BOM before empty-input validation, then orchestrates policy loading, per-statement parser recovery, extraction, metadata enrichment, impact refinement, evaluation, preserved partial results, a review floor for otherwise-passing partial parser failures, and fail-closed diagnostics for parser-error and unsupported outcomes
service_test.go Verifies parser recovery, diagnostics, the partial-parser review floor, defaults/config overrides, metadata enrichment including MySQL/TiDB MODIFY nullability state, DML impact, schema plumbing, and existing unsupported contracts
corpus_coverage_test.go Verifies every non-deferred shipped rule has corpus coverage for its supported dialect targets, including MySQL/TiDB-only metadata rules
metadata.go Defines the optional metadata-provider, index-owner resolver, plan estimator, and object-resolver seams, then attaches resolved target schema, instance, target-table, and non-table object snapshots to statements before evaluation
dml_table_existence_test.go Verifies MySQL/TiDB missing and existing DML target behavior, qualified-schema enrichment, joined mutation-target extraction, and metadata lookup error propagation
diagnostics.go Defines diagnostic evidence constants (classification, reason, action_hint, guidance codes, evidence refs) and helpers for constructing parser-error and unsupported statement diagnostics with optional guidance classification
ddl_coverage_catalog_query.go Defines CatalogEntry, CatalogQuery, CatalogResult, LoadEmbeddedCatalog, LoadCatalogFile, LoadCatalog, QueryCatalog, and Validate for reading the generated (embedded) DDL coverage catalog and filtering it without invoking the audit engine
catalogdata/ddl-coverage-catalog.json Generated catalog copy compiled into release binaries; kept byte-identical to docs/reference/ddl-coverage-catalog.json

Exports

  • Parse(sql string, dialect spec.Dialect)
  • Extract(parsed ParsedSQL)
  • EvaluateStatements(registry, statements)
  • AuditSQL(ctx, request)
  • Request
  • MetadataRequest
  • MetadataProvider
  • IndexOwnerResolver
  • PlanEstimator
  • ObjectResolver
  • Service
  • NewService()
  • Service.Audit(ctx, request)
  • ParsedStatement
  • ParsedSQL
  • PostgreSQLCapabilityBoundaryError
  • CatalogEntry
  • CatalogQuery
  • CatalogResult
  • LoadEmbeddedCatalog() (string, []CatalogEntry, error)
  • LoadCatalogFile(path string) (string, []CatalogEntry, error)
  • LoadCatalog(path string) ([]CatalogEntry, error)
  • QueryCatalog(entries []CatalogEntry, q CatalogQuery) CatalogResult
  • CatalogQuery.Validate() error

Notes

  • report.Result carries additive diagnostics. A parser failure affects only its bounded top-level statement: valid siblings continue through extraction, metadata, evaluation, findings, impact, and source-location attachment in original order. Each failed statement produces one audited=false parser diagnostic with optional 1-based line/column; an otherwise-pass partial result is floored to review, while existing review/reject results remain unchanged. The overall call still returns an error so process surfaces exit 2. Diagnostics never contain raw SQL text, parser near ... fragments, or other forbidden payload.
  • Statement-boundary recovery is lexical and deliberately narrow. It recognizes supported quote/comment forms needed to avoid false semicolon boundaries, then delegates every slice to the existing dialect parser. It does not add grammar fallbacks or guess semantics for failed text.

Dependencies

  • Upstream: future CLI and public audit entrypoints
  • Downstream: context, embed, fmt, internal/application, internal/application/policy, internal/domain/report, internal/domain/rule, internal/domain/rule/ddl, internal/domain/rule/dml, internal/domain/spec, internal/infrastructure/parser/postgresql, internal/infrastructure/parser/tidb

Update Rule

  • If members/interfaces/dependencies change, update this file in same change.

Documentation

Overview

Package audit orchestrates audit use cases at the application layer. input: generated DDL coverage catalog JSON (embedded at compile time) and optional filesystem catalog paths output: catalog entries, catalog version, and filtered query results pos: application catalog query core for the CLI ddl-coverage command note: if this file changes, update this header and module README.md.

Package audit orchestrates audit use cases at the application layer. input: extracted domain statements and the registered rule engine output: aggregated report results with statement/global findings and preserved statement-level impact estimates pos: application evaluation step between extraction/metadata refinement and reporting note: if this file changes, update this header and module README.md.

Package audit enriches evaluated findings with shared explanation metadata. input: rule findings, shipped catalog entries, and optional statement metadata context output: additive per-finding explanation data without changing verdict semantics pos: application explanation enrichment between evaluation and report aggregation note: if this file changes, update this header and module README.md.

Package audit orchestrates audit use cases at the application layer. input: application-owned parsed SQL statements and parser-neutral extractors output: first-pass StatementSpec values plus attached shape-only impact facts for later rule evaluation pos: application extraction step between parsing and rule execution note: if this file changes, update this header and module README.md.

Package audit orchestrates audit use cases at the application layer. input: extracted statement-local DML shape facts plus optional metadata snapshots for refinement output: conservative DML impact estimates attached during extraction and upgraded after metadata enrichment pos: application impact estimation step between extraction, metadata enrichment, and rule evaluation note: if this file changes, update this header and module README.md.

Package audit orchestrates audit use cases at the application layer. input: optional metadata providers plus parsed statement targets for enrichment output: metadata-enriched statements with resolved target schemas for rules that can use live instance or schema facts pos: application-layer bridge between provider-backed metadata and domain statements note: if this file changes, update this header and module README.md.

Package audit provides application-layer parser mismatch hints. input: parser-failed MySQL or TiDB SQL text masked of literals and comments output: bounded PostgreSQL-syntax notice tokens without raw SQL payloads pos: application diagnostic hinting beside parser-error aggregation note: if this file changes, update this header and module README.md.

Package audit orchestrates audit use cases at the application layer. input: SQL text, selected dialect, shared input normalization, statement boundaries, and infrastructure-backed parser adapters output: application-owned parsed statements plus bounded per-statement parse failures for later audit aggregation pos: application parsing entrypoint between interfaces and parser infrastructure note: if this file changes, update this header and module README.md.

Package audit orchestrates audit use cases at the application layer. input: audit requests carrying SQL text, dialect, optional policy override paths, optional metadata providers, and shared input normalization output: end-to-end audit results assembled from policy loading, parsing, extraction, metadata enrichment, rule evaluation, and a review floor for partial parser failures pos: application service entrypoint for the unified offline/metadata-aware SQL audit use case with preserved statement impact estimates note: if this file changes, update this header and module README.md.

Package audit orchestrates audit use cases at the application layer. input: normalized SQL text and its selected MySQL, TiDB, or PostgreSQL dialect output: ordered top-level statement slices with 1-based source start locations pos: bounded lexical statement-boundary scanner before dialect parser adapters note: if this file changes, update this header and module README.md.

Index

Constants

View Source 
const (
	// DiagnosticParserError classifies parser-error outcomes.
	DiagnosticParserError = "parser_error"
	// DiagnosticUnsupportedStatement classifies structured unsupported outcomes.
	DiagnosticUnsupportedStatement = "unsupported_statement"

	// ParserErrorActionHint is the generic safe next step for parser-error diagnostics.
	ParserErrorActionHint = "" /* 138-byte string literal not displayed */
	// UnsupportedActionHint is the generic safe next step for unsupported-statement diagnostics.
	UnsupportedActionHint = "" /* 131-byte string literal not displayed */

	// DiagnosticGuidanceParserUpgradeCandidate identifies parser-error cases that would become
	// parseable after an upstream parser/library upgrade.
	DiagnosticGuidanceParserUpgradeCandidate = "parser_upgrade_candidate"

	// ParserUpgradeCandidateEvidenceRef is the stable GitHub documentation URL for parser-upgrade
	// candidate evidence.
	ParserUpgradeCandidateEvidenceRef = "https://github.com/Fanduzi/DeltaScope/blob/main/docs/reference/cli.md#parser-upgrade-candidate-evidence-v02500"
)

Variables

View Source 
var (
	// ErrEmptySQL indicates the request did not include auditable SQL text.
	ErrEmptySQL = errors.New("audit SQL must not be empty")
	// ErrUnknownDialect indicates the request did not specify a supported dialect.
	ErrUnknownDialect = errors.New("audit dialect must be mysql, tidb, or postgresql")
	// ErrUnsupportedStatement indicates at least one parsed statement is recognized but unsupported.
	ErrUnsupportedStatement = errors.New("audit includes unsupported statements")
)

Functions

func AuditSQL 

func AuditSQL(ctx context.Context, request Request) (report.Result, error)

AuditSQL is the convenience application entrypoint used by outer adapters.

func EvaluateStatements 

func EvaluateStatements(ctx context.Context, registry *rule.Registry, statements []spec.Statement) (report.Result, error)

EvaluateStatements applies registered rules and aggregates their findings into a report result.

func Extract 

func Extract(ctx context.Context, parsed ParsedSQL) ([]spec.Statement, error)

Extract converts parsed statements into first-pass domain StatementSpec values.

Types

type CatalogEntry added in v0.280.0 

type CatalogEntry struct {
	Dialect        string   `json:"dialect"`
	Family         string   `json:"family"`
	Form           string   `json:"form"`
	Classification string   `json:"classification"`
	FindingRuleIDs []string `json:"finding_rule_ids"`
	GuidanceCode   string   `json:"guidance_code,omitempty"`
	EvidenceRef    string   `json:"evidence_ref,omitempty"`
	Notes          string   `json:"notes"`
}

CatalogEntry represents a single DDL coverage catalog entry returned by QueryCatalog. Fields mirror the v0.270.0 catalog JSON schema.

func LoadCatalog added in v0.280.0 

func LoadCatalog(path string) ([]CatalogEntry, error)

LoadCatalog reads the DDL coverage catalog from the given file path and returns entries in their canonical (deterministic) order.

func LoadCatalogFile added in v0.490.0 

func LoadCatalogFile(path string) (string, []CatalogEntry, error)

LoadCatalogFile reads the generated catalog from path and returns its version plus entries in their canonical (deterministic) order.

func LoadEmbeddedCatalog added in v0.490.0 

func LoadEmbeddedCatalog() (string, []CatalogEntry, error)

LoadEmbeddedCatalog returns the generated catalog compiled into the binary.

type CatalogQuery added in v0.280.0 

type CatalogQuery struct {
	Dialect        string
	Classification string
	GuidanceCode   string
	Family         string
	Form           string
	Search         string
	Limit          int
}

CatalogQuery holds filter parameters for querying the DDL coverage catalog. All string fields are optional; zero values mean "no filter".

func (CatalogQuery) Validate added in v0.280.0 

func (q CatalogQuery) Validate() error

Validate checks that enum filter values are recognized. Returns an error describing the first invalid field, or nil.

type CatalogResult added in v0.280.0 

type CatalogResult struct {
	Entries []CatalogEntry `json:"entries"`
	Total   int            `json:"total"`
}

CatalogResult holds the query output: a filtered slice of entries plus summary metadata.

func QueryCatalog added in v0.280.0 

func QueryCatalog(entries []CatalogEntry, q CatalogQuery) CatalogResult

QueryCatalog filters entries according to the query parameters. It returns a CatalogResult with the matching entries preserving their original deterministic order, and a total count. Empty results are a success, not an error.

type IndexOwnerResolver added in v0.18.0 

type IndexOwnerResolver interface {
	ResolveTableForIndex(ctx context.Context, dialect spec.Dialect, schema string, index string) (string, error)
}

IndexOwnerResolver optionally resolves standalone index statements back to owning tables.

type MetadataProvider 

type MetadataProvider interface {
	LoadInstanceFacts(ctx context.Context, dialect spec.Dialect, schema string) (*spec.InstanceFacts, error)
	LoadTableSnapshot(ctx context.Context, dialect spec.Dialect, schema string, table string) (*spec.TableSnapshot, error)
}

MetadataProvider supplies optional instance and schema facts for one audit run.

type MetadataRequest 

type MetadataRequest struct {
	Schema   string
	Provider MetadataProvider
}

MetadataRequest describes one optional metadata-aware audit invocation.

type ObjectResolver added in v0.90.0 

type ObjectResolver interface {
	ResolveObject(ctx context.Context, dialect spec.Dialect, request spec.ObjectLookupRequest) (*spec.ObjectSnapshot, error)
}

ObjectResolver optionally resolves non-table database objects from live metadata.

type ParsedSQL 

type ParsedSQL struct {
	Dialect    spec.Dialect      `json:"dialect"`
	Statements []ParsedStatement `json:"statements"`
	Warnings   []string          `json:"warnings,omitempty"`
	// contains filtered or unexported fields
}

ParsedSQL is the application-owned parsing result used by later extraction steps.

func Parse 

func Parse(ctx context.Context, sql string, dialect spec.Dialect) (ParsedSQL, error)

Parse delegates SQL parsing to the dialect-specific parser adapter.

type ParsedStatement 

type ParsedStatement struct {
	Kind      spec.Kind               `json:"kind"`
	RawSQL    string                  `json:"raw_sql"`
	Line      int                     `json:"line,omitempty"`
	Column    int                     `json:"col,omitempty"`
	Extractor spec.StatementExtractor `json:"-"`
}

ParsedStatement keeps application-facing statement metadata while hiding parser nodes.

type PlanEstimator added in v0.18.0 

type PlanEstimator interface {
	LoadPlanEstimate(ctx context.Context, statement spec.Statement) (*spec.ImpactEstimate, error)
}

PlanEstimator optionally loads planner-backed DML impact estimates.

type PostgreSQLCapabilityBoundaryError added in v0.20.0 

type PostgreSQLCapabilityBoundaryError struct {
	Message string
}

PostgreSQLCapabilityBoundaryError reports that PostgreSQL parsing needs a PostgreSQL-capable build.

func (*PostgreSQLCapabilityBoundaryError) Error added in v0.20.0 

func (e *PostgreSQLCapabilityBoundaryError) Error() string

type Request 

type Request struct {
	SQL              string
	Dialect          spec.Dialect
	ConfigPath       string
	Schema           string
	MetadataProvider MetadataProvider
	Metadata         *MetadataRequest
}

Request describes one application-level audit invocation.

type Service 

type Service struct{}

Service coordinates the full audit use case.

func NewService 

func NewService() Service

NewService returns a ready-to-use audit service.

func (Service) Audit 

func (s Service) Audit(ctx context.Context, request Request) (report.Result, error)

Audit executes the full SQL audit flow.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.