rules

package
v1.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 23, 2026 License: Apache-2.0 Imports: 14 Imported by: 0

Documentation

Overview

Package rules implements the individual MariaDB-to-TiDB transformation rules applied by the engine, plus the registry that orders them.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CheckRuleNames added in v1.3.0

func CheckRuleNames(cfg *config.Config) error

CheckRuleNames reports rule names in enabled_rules / disabled_rules that do not exist: a typo in enabled_rules would otherwise silently disable every rule.

Types

type AutoRandomRule added in v1.3.0

type AutoRandomRule struct {
	// contains filtered or unexported fields
}

AutoRandomRule converts a single-column BIGINT AUTO_INCREMENT primary key to AUTO_RANDOM(shard_bits, range_bits) for the tables listed in auto_random.tables, spreading inserts across regions instead of one hot region. It refuses UNSIGNED columns: without a sign bit the shard bits land in bit 63 and IDs exceed 2^63-1 (PHP_INT_MAX, Java long). https://docs.pingcap.com/tidb/stable/auto-random

func NewAutoRandomRule added in v1.3.0

func NewAutoRandomRule(cfg *config.Config, r *compat.Reporter) *AutoRandomRule

NewAutoRandomRule creates the rule from cfg.AutoRandom.

func (*AutoRandomRule) Apply added in v1.3.0

func (r *AutoRandomRule) Apply(node ast.Node) (ast.Node, error)

Apply implements Rule.

func (*AutoRandomRule) Description added in v1.3.0

func (r *AutoRandomRule) Description() string

Description implements Rule.

func (*AutoRandomRule) Name added in v1.3.0

func (r *AutoRandomRule) Name() string

Name implements Rule.

func (*AutoRandomRule) Priority added in v1.3.0

func (r *AutoRandomRule) Priority() int

Priority implements Rule.

func (*AutoRandomRule) ShouldApply added in v1.3.0

func (r *AutoRandomRule) ShouldApply(node ast.Node) bool

ShouldApply implements Rule.

type CollationRule

type CollationRule struct {
	// contains filtered or unexported fields
}

CollationRule transforms collation and charset specifications based on configuration

func NewCollationRule

func NewCollationRule(cfg *config.Config) *CollationRule

NewCollationRule creates a CollationRule with the provided configuration

func (*CollationRule) Apply

func (r *CollationRule) Apply(node ast.Node) (ast.Node, error)

Apply applies the transformation to the AST node and returns the modified node

func (*CollationRule) Description

func (r *CollationRule) Description() string

Description returns a human-readable description of what the rule does

func (*CollationRule) Name

func (r *CollationRule) Name() string

Name returns the unique name of the rule

func (*CollationRule) Priority

func (r *CollationRule) Priority() int

Priority returns the priority of the rule (lower number = higher priority)

func (*CollationRule) ShouldApply

func (r *CollationRule) ShouldApply(node ast.Node) bool

ShouldApply checks if the rule should be applied to the given AST node

type EngineRule added in v1.3.0

type EngineRule struct {
	// contains filtered or unexported fields
}

EngineRule replaces storage engines TiDB rejects (MariaDB's Aria, ColumnStore, Spider, CONNECT, S3, ROCKSDB, MRG_MyISAM, ...) with InnoDB.

func NewEngineRule added in v1.3.0

func NewEngineRule(r *compat.Reporter) *EngineRule

NewEngineRule creates the rule; r receives a warning per replacement.

func (*EngineRule) Apply added in v1.3.0

func (r *EngineRule) Apply(node ast.Node) (ast.Node, error)

Apply implements Rule.

func (*EngineRule) Description added in v1.3.0

func (r *EngineRule) Description() string

Description implements Rule.

func (*EngineRule) Name added in v1.3.0

func (r *EngineRule) Name() string

Name implements Rule.

func (*EngineRule) Priority added in v1.3.0

func (r *EngineRule) Priority() int

Priority implements Rule.

func (*EngineRule) ShouldApply added in v1.3.0

func (r *EngineRule) ShouldApply(node ast.Node) bool

ShouldApply implements Rule.

type FunctionDefaultRule

type FunctionDefaultRule struct {
	// contains filtered or unexported fields
}

FunctionDefaultRule removes unsupported function-based default values. TiDB allows only a small set of functions in DEFAULT clauses without the expression syntax (for example, CURRENT_TIMESTAMP); newer TiDB versions accept more expressions, so the allowlist is configurable via allowed_default_functions. https://docs.pingcap.com/tidb/stable/data-type-default-values

func NewFunctionDefaultRule added in v1.1.0

func NewFunctionDefaultRule(cfg *config.Config) *FunctionDefaultRule

NewFunctionDefaultRule creates the rule using the allowlist from cfg, falling back to the built-in defaults.

func (*FunctionDefaultRule) Apply

func (r *FunctionDefaultRule) Apply(node ast.Node) (ast.Node, error)

Apply removes the default clause if it uses a disallowed function

func (*FunctionDefaultRule) Description

func (r *FunctionDefaultRule) Description() string

Description returns rule description

func (*FunctionDefaultRule) Name

func (r *FunctionDefaultRule) Name() string

Name returns rule name

func (*FunctionDefaultRule) Priority

func (r *FunctionDefaultRule) Priority() int

Priority defines rule execution order

func (*FunctionDefaultRule) ShouldApply

func (r *FunctionDefaultRule) ShouldApply(node ast.Node) bool

ShouldApply checks if the column has a function-based default that TiDB disallows

type GeneratedValuesRule added in v1.3.0

type GeneratedValuesRule struct {
	// contains filtered or unexported fields
}

GeneratedValuesRule replaces values given for generated columns in INSERT/REPLACE with DEFAULT. mariadb-dump writes them (MariaDB ignores them on load); TiDB rejects the statement with error 3105. Generated columns are learned from CREATE TABLE statements earlier in the same input, so this covers full dumps (schema and data in one file). Tables are keyed by database, following USE, so equal names in different databases do not mix.

func NewGeneratedValuesRule added in v1.3.0

func NewGeneratedValuesRule(r *compat.Reporter) *GeneratedValuesRule

NewGeneratedValuesRule creates the rule.

func (*GeneratedValuesRule) Apply added in v1.3.0

func (r *GeneratedValuesRule) Apply(node ast.Node) (ast.Node, error)

Apply implements Rule.

func (*GeneratedValuesRule) Description added in v1.3.0

func (r *GeneratedValuesRule) Description() string

Description implements Rule.

func (*GeneratedValuesRule) Name added in v1.3.0

func (r *GeneratedValuesRule) Name() string

Name implements Rule.

func (*GeneratedValuesRule) Priority added in v1.3.0

func (r *GeneratedValuesRule) Priority() int

Priority implements Rule.

func (*GeneratedValuesRule) ShouldApply added in v1.3.0

func (r *GeneratedValuesRule) ShouldApply(node ast.Node) bool

ShouldApply implements Rule.

type IndexPrefixRule

type IndexPrefixRule struct{}

IndexPrefixRule adds a prefix length to indexed TEXT/BLOB columns that lack one. Oversized explicit prefixes and wide CHAR/VARCHAR keys are handled separately by KeyLengthRule.

func (*IndexPrefixRule) Apply

func (r *IndexPrefixRule) Apply(node ast.Node) (ast.Node, error)

Apply sets prefix lengths for indexed TEXT/BLOB columns

func (*IndexPrefixRule) Description

func (r *IndexPrefixRule) Description() string

Description returns rule description

func (*IndexPrefixRule) Name

func (r *IndexPrefixRule) Name() string

Name returns rule name

func (*IndexPrefixRule) Priority

func (r *IndexPrefixRule) Priority() int

Priority determines rule order

func (*IndexPrefixRule) ShouldApply

func (r *IndexPrefixRule) ShouldApply(node ast.Node) bool

ShouldApply checks if the table has indexed TEXT/BLOB columns without a prefix length

type JSONCheckRule

type JSONCheckRule struct{}

JSONCheckRule removes CHECK constraints that use JSON_VALID(). Handles both column-level and table-level check constraints. Reference: Step 5 in legacy universal_tidb_transform.sh

func (*JSONCheckRule) Apply

func (r *JSONCheckRule) Apply(node ast.Node) (ast.Node, error)

Apply removes JSON_VALID check constraints

func (*JSONCheckRule) Description

func (r *JSONCheckRule) Description() string

Description returns description

func (*JSONCheckRule) Name

func (r *JSONCheckRule) Name() string

Name returns rule name

func (*JSONCheckRule) Priority

func (r *JSONCheckRule) Priority() int

Priority defines rule execution order

func (*JSONCheckRule) ShouldApply

func (r *JSONCheckRule) ShouldApply(node ast.Node) bool

ShouldApply determines if node contains JSON_VALID check

type JSONGeneratedRule

type JSONGeneratedRule struct{}

JSONGeneratedRule converts generated columns using JSON functions into regular columns. Modern TiDB supports JSON functions in generated columns (https://docs.pingcap.com/tidb/stable/generated-columns), so this rule is disabled by default and only needed for legacy TiDB targets. Reference: Step 12 in legacy universal_tidb_transform.sh. The rule strips the generated expression (and related options) while keeping the column definition intact so that column counts stay consistent between source and target databases.

func (*JSONGeneratedRule) Apply

func (r *JSONGeneratedRule) Apply(node ast.Node) (ast.Node, error)

Apply converts columns with JSON generated expressions into standard columns by removing the generated options while retaining the column and any associated constraints.

func (*JSONGeneratedRule) Description

func (r *JSONGeneratedRule) Description() string

Description returns description

func (*JSONGeneratedRule) Name

func (r *JSONGeneratedRule) Name() string

Name returns rule name

func (*JSONGeneratedRule) Priority

func (r *JSONGeneratedRule) Priority() int

Priority defines rule execution order

func (*JSONGeneratedRule) ShouldApply

func (r *JSONGeneratedRule) ShouldApply(node ast.Node) bool

ShouldApply checks if any column in the table uses a JSON function in generated expression

type JSONValueRule added in v1.3.0

type JSONValueRule struct {
	// contains filtered or unexported fields
}

JSONValueRule rewrites MariaDB's JSON_VALUE(doc, path), which TiDB does not have (error 1305; in generated columns 3102), everywhere: generated columns, views, CHECK constraints, defaults. The replacement

CASE WHEN JSON_VALID(doc) AND JSON_TYPE(JSON_EXTRACT(doc, path))
     NOT IN ('OBJECT', 'ARRAY', 'NULL')
THEN JSON_UNQUOTE(JSON_EXTRACT(doc, path)) END

matches MariaDB's NULL for invalid documents (a bare JSON_EXTRACT fails the whole INSERT with error 3140), JSON null, objects and arrays. Scalars can still be formatted differently (1.50 vs 1.5, true vs 1), which the warning says.

func NewJSONValueRule added in v1.3.0

func NewJSONValueRule(r *compat.Reporter) *JSONValueRule

NewJSONValueRule creates the rule; r receives a one-time note on the remaining semantic difference.

func (*JSONValueRule) Apply added in v1.3.0

func (r *JSONValueRule) Apply(node ast.Node) (ast.Node, error)

Apply implements Rule. Nested JSON_VALUE calls in the arguments are rewritten too, bottom-up, because the engine does not descend into a node a rule replaced.

func (*JSONValueRule) Description added in v1.3.0

func (r *JSONValueRule) Description() string

Description implements Rule.

func (*JSONValueRule) Name added in v1.3.0

func (r *JSONValueRule) Name() string

Name implements Rule.

func (*JSONValueRule) Priority added in v1.3.0

func (r *JSONValueRule) Priority() int

Priority implements Rule.

func (*JSONValueRule) ShouldApply added in v1.3.0

func (r *JSONValueRule) ShouldApply(node ast.Node) bool

ShouldApply implements Rule.

type KeyLengthRule

type KeyLengthRule struct{}

KeyLengthRule caps index key prefix lengths at 768 characters so utf8mb4 keys stay within TiDB's 3072-byte max-index-length limit. Column definitions are left untouched: TiDB supports VARCHAR up to 16383 characters, only the indexed prefix is limited.

func (*KeyLengthRule) Apply

func (r *KeyLengthRule) Apply(node ast.Node) (ast.Node, error)

Apply caps oversized index key prefixes at maxKeyLenChars.

func (*KeyLengthRule) Description

func (r *KeyLengthRule) Description() string

Description returns description

func (*KeyLengthRule) Name

func (r *KeyLengthRule) Name() string

Name returns rule name

func (*KeyLengthRule) Priority

func (r *KeyLengthRule) Priority() int

Priority determines rule order

func (*KeyLengthRule) ShouldApply

func (r *KeyLengthRule) ShouldApply(node ast.Node) bool

ShouldApply checks if any index key exceeds the TiDB key length limit, either through an explicit prefix or via the full width of a CHAR/VARCHAR column.

type Registry

type Registry struct {
	// contains filtered or unexported fields
}

Registry manages all transformation rules

func NewRegistry

func NewRegistry(cfg *config.Config) *Registry

NewRegistry creates a new rules registry

func NewRegistryWithReporter added in v1.3.0

func NewRegistryWithReporter(cfg *config.Config, r *compat.Reporter) *Registry

NewRegistryWithReporter creates a registry whose rules record migration warnings in r (nil disables them).

func (*Registry) GetRuleByName

func (r *Registry) GetRuleByName(name string) Rule

GetRuleByName returns a rule by its name

func (*Registry) GetRules

func (r *Registry) GetRules() []Rule

GetRules returns all registered rules in priority order

func (*Registry) ListRules

func (r *Registry) ListRules() []RuleInfo

ListRules returns information about all registered rules

func (*Registry) Register

func (r *Registry) Register(rule Rule)

Register adds a rule to the registry

type Rule

type Rule interface {
	// Name returns the unique name of the rule
	Name() string

	// Description returns a human-readable description of what the rule does
	Description() string

	// Priority returns the priority of the rule (lower number = higher priority)
	Priority() int

	// ShouldApply checks if the rule should be applied to the given AST node
	ShouldApply(node ast.Node) bool

	// Apply applies the transformation to the AST node and returns the modified node
	Apply(node ast.Node) (ast.Node, error)
}

Rule represents a single transformation rule

type RuleCategory

type RuleCategory string

RuleCategory represents different categories of transformation rules

const (
	CategoryDataType      RuleCategory = "datatype"
	CategoryConstraint    RuleCategory = "constraint"
	CategoryFunction      RuleCategory = "function"
	CategoryIndex         RuleCategory = "index"
	CategoryCollation     RuleCategory = "collation"
	CategoryCompatibility RuleCategory = "compatibility"
)

Rule categories group transformations by the schema aspect they touch.

type RuleInfo

type RuleInfo struct {
	Name        string
	Description string
	Priority    int
}

RuleInfo contains metadata about a rule

type ShardRowIDRule added in v1.3.0

type ShardRowIDRule struct {
	// contains filtered or unexported fields
}

ShardRowIDRule adds SHARD_ROW_ID_BITS (and optionally PRE_SPLIT_REGIONS) to tables whose rows are keyed by the hidden _tidb_rowid: tables without a PRIMARY KEY, or with a NONCLUSTERED one. Every other primary key is clustered by default (tidb_enable_clustered_index=ON) and TiDB rejects the option there (error 8200). Opt-in via shard_row_id_bits. https://docs.pingcap.com/tidb/stable/shard-row-id-bits

func NewShardRowIDRule added in v1.3.0

func NewShardRowIDRule(cfg *config.Config) *ShardRowIDRule

NewShardRowIDRule creates the rule from cfg.

func (*ShardRowIDRule) Apply added in v1.3.0

func (r *ShardRowIDRule) Apply(node ast.Node) (ast.Node, error)

Apply implements Rule.

func (*ShardRowIDRule) Description added in v1.3.0

func (r *ShardRowIDRule) Description() string

Description implements Rule.

func (*ShardRowIDRule) Name added in v1.3.0

func (r *ShardRowIDRule) Name() string

Name implements Rule.

func (*ShardRowIDRule) Priority added in v1.3.0

func (r *ShardRowIDRule) Priority() int

Priority implements Rule.

func (*ShardRowIDRule) ShouldApply added in v1.3.0

func (r *ShardRowIDRule) ShouldApply(node ast.Node) bool

ShouldApply implements Rule.

type TextBlobDefaultRule

type TextBlobDefaultRule struct{}

TextBlobDefaultRule removes default values from TEXT/BLOB/JSON columns TiDB does not allow default values on these types Reference: Step 4 in legacy universal_tidb_transform.sh This rule also removes DEFAULT value expressions for JSON columns but does not handle CHECK constraints (handled by JsonCheckRule)

func (*TextBlobDefaultRule) Apply

func (r *TextBlobDefaultRule) Apply(node ast.Node) (ast.Node, error)

Apply removes default options from column definition

func (*TextBlobDefaultRule) Description

func (r *TextBlobDefaultRule) Description() string

Description returns description

func (*TextBlobDefaultRule) Name

func (r *TextBlobDefaultRule) Name() string

Name returns rule name

func (*TextBlobDefaultRule) Priority

func (r *TextBlobDefaultRule) Priority() int

Priority determines rule order

func (*TextBlobDefaultRule) ShouldApply

func (r *TextBlobDefaultRule) ShouldApply(node ast.Node) bool

ShouldApply checks if node is a ColumnDef with TEXT/BLOB/JSON type and has default value

type ZeroDateRule added in v1.3.0

type ZeroDateRule struct {
	// contains filtered or unexported fields
}

ZeroDateRule handles DEFAULT values such as '0000-00-00 00:00:00' on DATE/DATETIME/TIMESTAMP columns. MariaDB accepts them by default; TiDB rejects them (error 1067) under its default sql_mode, which includes NO_ZERO_DATE and NO_ZERO_IN_DATE. They still load where the session sql_mode is permissive (mariadb-dump output sets one, TiDB Lightning uses one), so the default is to keep them and warn; zero_date_defaults: drop removes the DEFAULT clause instead.

func NewZeroDateRule added in v1.3.0

func NewZeroDateRule(cfg *config.Config, r *compat.Reporter) *ZeroDateRule

NewZeroDateRule creates the rule from cfg.ZeroDateDefaults.

func (*ZeroDateRule) Apply added in v1.3.0

func (r *ZeroDateRule) Apply(node ast.Node) (ast.Node, error)

Apply implements Rule.

func (*ZeroDateRule) Description added in v1.3.0

func (r *ZeroDateRule) Description() string

Description implements Rule.

func (*ZeroDateRule) Name added in v1.3.0

func (r *ZeroDateRule) Name() string

Name implements Rule.

func (*ZeroDateRule) Priority added in v1.3.0

func (r *ZeroDateRule) Priority() int

Priority implements Rule.

func (*ZeroDateRule) ShouldApply added in v1.3.0

func (r *ZeroDateRule) ShouldApply(node ast.Node) bool

ShouldApply implements Rule.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL