processor

package
v0.0.10 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 23, 2024 License: Apache-2.0 Imports: 34 Imported by: 0

Documentation

Overview

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed p. in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright © 2020 Marvin

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Cmp added in v0.0.7 

func Cmp(src map[string]int64, dest map[string]int64) (map[string]int64, map[string]int64)

Cmp used for the src and dest store key-value pair , and key data row and value data row counts

func DivideDatabaseTableColumnStatisticsBucket added in v0.0.7 

func DivideDatabaseTableColumnStatisticsBucket(database database.IDatabase, schemaName, tableName string, cons *structure.Selectivity, chunkRange *structure.Range, divideCountCnt int) ([]*structure.Range, error)

DivideDatabaseTableColumnStatisticsBucket splits a chunk to multiple chunks by random Notice: If the `count <= 1`, it will skip splitting and return `chunk` as a slice directly.

func ExtractDatabaseTableStatisticsValuesFromBuckets added in v0.0.7 

func ExtractDatabaseTableStatisticsValuesFromBuckets(divideDbType, valueString string, columnNames []string) ([]string, error)

ExtractDatabaseTableStatisticsValuesFromBuckets analyze upperBound or lowerBound to string for each column. upperBound and lowerBound are looks like '(123, abc)' for multiple fields, or '123' for one field.

func GenMYSQLCompatibleDatabaseDeleteStmtSQL added in v0.0.7 

func GenMYSQLCompatibleDatabaseDeleteStmtSQL(schemaName, tableName, sqlHint string, columnDetailSlice []string, columnDataString []string, columnDataCounts int) string

func GenMYSQLCompatibleDatabaseInsertStmtSQL added in v0.0.7 

func GenMYSQLCompatibleDatabaseInsertStmtSQL(schemaName, tableName, sqlHint string, columnDetailSlice []string, columnDataString string, columnDataCounts int, safeMode bool) string

func GenMYSQLCompatibleDatabasePrepareStmt added in v0.0.7 

func GenMYSQLCompatibleDatabasePrepareStmt(
	schemaName, tableName string, sqlHint, columnDetailSlice string, insertBatchSize int, safeMode bool) string

func GenOracleCompatibleDatabaseDeleteStmtSQL added in v0.0.7 

func GenOracleCompatibleDatabaseDeleteStmtSQL(schemaName, tableName, sqlHint string, columnDetailSlice []string, columnDataString []string, columnDataCounts int) string

func GenOracleCompatibleDatabaseInsertStmtSQL added in v0.0.7 

func GenOracleCompatibleDatabaseInsertStmtSQL(schemaName, tableName, sqlHint string, columnDetailSlice []string, columnDataString string, columnDataCounts int, safeMode bool) string

func GetDownstreamTableColumnDatatype added in v0.0.9 

func GetDownstreamTableColumnDatatype(schemaNameT, tableNameT string, databaseT database.IDatabase, originColumnNameSli []string, columnNameRouteRule map[string]string) ([]string, error)

func InspectOracleMigrateTask added in v0.0.7 

func InspectOracleMigrateTask(taskName, taskFlow, taskMode string, databaseS database.IDatabase, connectDBCharsetS, connectDBCharsetT string) (string, error)

func OptimizerMYSQLCompatibleDataMigrateColumnS added in v0.0.7 

func OptimizerMYSQLCompatibleDataMigrateColumnS(columnName, datatype, datetimePrecision string) (string, error)

func OptimizerMYSQLCompatibleMigrateOracleDataCompareColumnST added in v0.0.7 

func OptimizerMYSQLCompatibleMigrateOracleDataCompareColumnST(columnNameS, datatypeS string, datetimePrecisionS, dataLengthS int64, dataPrecisionS, dataScaleS, dbCharsetSDest, columnNameT, dbCharsetTFrom, dbCharsetTDest string) (string, string, error)

func OptimizerOracleDataMigrateColumnS added in v0.0.7 

func OptimizerOracleDataMigrateColumnS(columnName, datatype, dataScale string) (string, error)

func OptimizerOracleMigrateMYSQLCompatibleDataCompareColumnST added in v0.0.7 

func OptimizerOracleMigrateMYSQLCompatibleDataCompareColumnST(columnNameS, datatypeS string, dataLengthS int64, dataPrecisionS, dataScaleS, dbCharsetSFrom, dbCharsetSDest, columnNameT, dbCharsetTDest string) (string, string, error)

func ProcessDownstreamStatisticsBucket added in v0.0.9 

func ProcessDownstreamStatisticsBucket(dbTypeT, dbCharsetT string, bs []*structure.Range, r *structure.Rule) ([]*structure.Range, error)

Types

type CsvMigrateRow added in v0.0.7 

type CsvMigrateRow struct {
	Ctx        context.Context
	TaskMode   string
	TaskFlow   string
	BufioSize  int
	Dmt        *task.DataMigrateTask
	DatabaseS  database.IDatabase
	DBCharsetS string
	DBCharsetT string
	TaskParams *pb.CsvMigrateParam
	ReadChan   chan []string
	WriteChan  chan string
}

func (*CsvMigrateRow) MigrateApply added in v0.0.7 

func (r *CsvMigrateRow) MigrateApply() error

func (*CsvMigrateRow) MigrateProcess added in v0.0.7 

func (r *CsvMigrateRow) MigrateProcess() error

func (*CsvMigrateRow) MigrateRead added in v0.0.7 

func (r *CsvMigrateRow) MigrateRead() error

type DataCompareFile added in v0.0.7 

type DataCompareFile struct {
	Ctx        context.Context `json:"-"`
	Mutex      *sync.Mutex     `json:"-"`
	CompFile   *os.File        `json:"-"`
	CompWriter *bufio.Writer   `json:"-"`
	TaskName   string          `json:"taskName"`
	TaskFlow   string          `json:"taskFlow"`
	OutputDir  string          `json:"outputDir"`
}

func NewDataCompareFile added in v0.0.7 

func NewDataCompareFile(ctx context.Context,
	taskName, taskFlow, outputDir string) *DataCompareFile

func (*DataCompareFile) Close added in v0.0.7 

func (s *DataCompareFile) Close() error

func (*DataCompareFile) InitFile added in v0.0.7 

func (s *DataCompareFile) InitFile() error

func (*DataCompareFile) SyncFile added in v0.0.7 

func (s *DataCompareFile) SyncFile() error

type DataCompareRow added in v0.0.7 

type DataCompareRow struct {
	Ctx            context.Context
	StartTime      time.Time
	TaskMode       string
	TaskFlow       string
	Dmt            *task.DataCompareTask
	DatabaseS      database.IDatabase
	DatabaseT      database.IDatabase
	BatchSize      int
	WriteThread    int
	CallTimeout    int
	DBCharsetS     string
	DBCharsetT     string
	RepairStmtFlow string
}

func (*DataCompareRow) CompareCRC32 added in v0.0.7 

func (r *DataCompareRow) CompareCRC32() error

func (*DataCompareRow) CompareMd5ORCrc32 added in v0.0.8 

func (r *DataCompareRow) CompareMd5ORCrc32() error

func (*DataCompareRow) CompareMethod added in v0.0.7 

func (r *DataCompareRow) CompareMethod() string

func (*DataCompareRow) CompareRows added in v0.0.7 

func (r *DataCompareRow) CompareRows() error

type DataCompareRule added in v0.0.7 

type DataCompareRule struct {
	Ctx                         context.Context    `json:"-"`
	TaskName                    string             `json:"taskName"`
	TaskMode                    string             `json:"taskMode"`
	TaskFlow                    string             `json:"taskFlow"`
	SchemaNameS                 string             `json:"schemaNameS"`
	TableNameS                  string             `json:"tableNameS"`
	SchemaNameT                 string             `json:"schemNameT"`
	TableNameT                  string             `json:"tableNameT"`
	GlobalSqlHintS              string             `json:"globalSqlHintS"`
	GlobalSqlHintT              string             `json:"globalSqlHintT"`
	TableTypeS                  map[string]string  `json:"tableTypeS"`
	ColumnNameSliS              []string           `json:"columnNameSliS"`
	IgnoreSelectFields          []string           `json:"ignoreSelectFields"`
	GlobalIgnoreConditionFields []string           `json:"globalIgnoreConditionFields"`
	OnlyDatabaseCompareRow      bool               `json:"onlyDatabaseCompareRow"`
	OnlyProgramCompareCRC32     bool               `json:"onlyProgramCompareCRC32"`
	DisableDatabaseCompareMd5   bool               `json:"disableDatabaseCompareMd5"`
	DatabaseS                   database.IDatabase `json:"-"`
	DatabaseT                   database.IDatabase `json:"-"`
	DBCharsetS                  string             `json:"DBCharsetS"`
	DBCharsetT                  string             `json:"DBCharsetT"`
	CaseFieldRuleS              string             `json:"caseFieldRuleS"`
	CaseFieldRuleT              string             `json:"caseFieldRuleT"`
}

func (*DataCompareRule) GenSchemaNameRule added in v0.0.7 

func (r *DataCompareRule) GenSchemaNameRule() (string, string, error)

func (*DataCompareRule) GenSchemaTableColumnSelectRule added in v0.0.7 

func (r *DataCompareRule) GenSchemaTableColumnSelectRule() (string, string, string, string, error)

func (*DataCompareRule) GenSchemaTableCompareMethodRule added in v0.0.7 

func (r *DataCompareRule) GenSchemaTableCompareMethodRule() string

func (*DataCompareRule) GenSchemaTableCustomRule added in v0.0.7 

func (r *DataCompareRule) GenSchemaTableCustomRule() (string, string, string, []string, string, string, error)

func (*DataCompareRule) GenSchemaTableNameRule added in v0.0.7 

func (r *DataCompareRule) GenSchemaTableNameRule() (string, string, error)

func (*DataCompareRule) GenSchemaTableTypeRule added in v0.0.7 

func (r *DataCompareRule) GenSchemaTableTypeRule() string

func (*DataCompareRule) GetSchemaTableColumnNameRule added in v0.0.7 

func (r *DataCompareRule) GetSchemaTableColumnNameRule() (map[string]string, error)

type DataCompareTask added in v0.0.9 

type DataCompareTask struct {
	Ctx         context.Context
	Task        *task.Task
	DatabaseS   database.IDatabase
	DatabaseT   database.IDatabase
	SchemaNameS string

	DBCharsetS string
	DBCharsetT string
	TaskParams *pb.DataCompareParam

	WaiterC chan *WaitingRecs
	ResumeC chan *WaitingRecs
}

func (*DataCompareTask) Init added in v0.0.9 

func (dmt *DataCompareTask) Init() error

func (*DataCompareTask) PrepareStatisticsRange added in v0.0.9 

func (dmt *DataCompareTask) PrepareStatisticsRange(globalScnS, globalScnT string, attsRule *database.DataCompareAttributesRule, upRanges, downRanges []*structure.Range) ([]*task.DataCompareTask, error)

func (*DataCompareTask) Process added in v0.0.9 

func (dmt *DataCompareTask) Process(s *WaitingRecs) error

func (*DataCompareTask) ProcessStatisticsScan added in v0.0.9 

func (dmt *DataCompareTask) ProcessStatisticsScan(ctx context.Context, dbTypeS, dbTypeT, globalScnS, globalScnT string, tableRows uint64, tableSize float64, attsRule *database.DataCompareAttributesRule) error

func (*DataCompareTask) ProcessTableScan added in v0.0.9 

func (dmt *DataCompareTask) ProcessTableScan(ctx context.Context, schemaNameS, tableNameS, globalScnS, globalScnT string, tableRows uint64, tableSize float64, attsRule *database.DataCompareAttributesRule) error

func (*DataCompareTask) Resume added in v0.0.9 

func (dmt *DataCompareTask) Resume() error

func (*DataCompareTask) Run added in v0.0.9 

func (dmt *DataCompareTask) Run() error

type DataMigrateRule added in v0.0.7 

type DataMigrateRule struct {
	Ctx            context.Context    `json:"-"`
	TaskName       string             `json:"taskName"`
	TaskMode       string             `json:"taskMode"`
	TaskFlow       string             `json:"taskFlow"`
	SchemaNameS    string             `json:"schemaNameS"`
	TableNameS     string             `json:"tableNameS"`
	GlobalSqlHintS string             `json:"globalSqlHintS"`
	TableTypeS     map[string]string  `json:"tableTypeS"`
	DatabaseS      database.IDatabase `json:"databaseS"`
	DBCharsetS     string             `json:"DBCharsetS"`
	CaseFieldRuleS string             `json:"caseFieldRuleS"`
	CaseFieldRuleT string             `json:"caseFieldRuleT"`
}

func (*DataMigrateRule) GenSchemaNameRule added in v0.0.7 

func (r *DataMigrateRule) GenSchemaNameRule() (string, string, error)

func (*DataMigrateRule) GenSchemaTableColumnSelectRule added in v0.0.7 

func (r *DataMigrateRule) GenSchemaTableColumnSelectRule() (string, string, string, string, error)

func (*DataMigrateRule) GenSchemaTableCustomRule added in v0.0.7 

func (r *DataMigrateRule) GenSchemaTableCustomRule() (bool, string, string, error)

func (*DataMigrateRule) GenSchemaTableNameRule added in v0.0.7 

func (r *DataMigrateRule) GenSchemaTableNameRule() (string, string, error)

func (*DataMigrateRule) GenSchemaTableTypeRule added in v0.0.7 

func (r *DataMigrateRule) GenSchemaTableTypeRule() string

func (*DataMigrateRule) GetSchemaTableColumnNameRule added in v0.0.7 

func (r *DataMigrateRule) GetSchemaTableColumnNameRule() (map[string]string, error)

type DataMigrateTask added in v0.0.9 

type DataMigrateTask struct {
	Ctx                  context.Context
	Task                 *task.Task
	DBRoleS              string
	DBVersionS           string
	DBCharsetS           string
	DBCharsetT           string
	DatabaseS            database.IDatabase
	DatabaseT            database.IDatabase
	SchemaNameS          string
	TableThread          uint64
	GlobalSqlHintS       string
	GlobalSqlHintT       string
	EnableCheckpoint     bool
	EnableConsistentRead bool
	ChunkSize            uint64
	BatchSize            uint64
	WriteThread          uint64
	CallTimeout          uint64
	SqlThreadS           uint64

	StmtParams *pb.StatementMigrateParam
	CsvParams  *pb.CsvMigrateParam

	WaiterC chan *WaitingRecs
	ResumeC chan *WaitingRecs
}

func (*DataMigrateTask) Init added in v0.0.9 

func (cmt *DataMigrateTask) Init() error

func (*DataMigrateTask) PrepareStatisticsRange added in v0.0.9 

func (cmt *DataMigrateTask) PrepareStatisticsRange(globalScn string, attsRule *database.DataMigrateAttributesRule, r *structure.Range, chunkID int) (*task.DataMigrateTask, error)

func (*DataMigrateTask) Process added in v0.0.9 

func (cmt *DataMigrateTask) Process(s *WaitingRecs) error

func (*DataMigrateTask) ProcessChunkScan added in v0.0.9 

func (cmt *DataMigrateTask) ProcessChunkScan(ctx context.Context, globalScn string, tableRows uint64, tableSize float64, attsRule *database.DataMigrateAttributesRule) error

func (*DataMigrateTask) ProcessStatisticsScan added in v0.0.9 

func (cmt *DataMigrateTask) ProcessStatisticsScan(ctx context.Context, dbTypeS, globalScn string, tableRows uint64, tableSize float64, attsRule *database.DataMigrateAttributesRule) error

func (*DataMigrateTask) ProcessTableScan added in v0.0.9 

func (cmt *DataMigrateTask) ProcessTableScan(ctx context.Context, schemaNameS, tableNameS, globalScn string, tableRows uint64, tableSize float64, attsRule *database.DataMigrateAttributesRule) error

func (*DataMigrateTask) Resume added in v0.0.9 

func (cmt *DataMigrateTask) Resume() error

func (*DataMigrateTask) Run added in v0.0.9 

func (cmt *DataMigrateTask) Run() error

type DataScanFile added in v0.0.7 

type DataScanFile struct {
	Ctx        context.Context `json:"-"`
	Mutex      *sync.Mutex     `json:"-"`
	CompFile   *os.File        `json:"-"`
	CompWriter *bufio.Writer   `json:"-"`
	TaskName   string          `json:"taskName"`
	TaskMode   string          `json:"taskMode"`
	TaskFlow   string          `json:"taskFlow"`
	OutputDir  string          `json:"outputDir"`
}

func NewDataScanFile added in v0.0.7 

func NewDataScanFile(ctx context.Context,
	taskName, taskMode, taskFlow, outputDir string) *DataScanFile

func (*DataScanFile) Close added in v0.0.7 

func (s *DataScanFile) Close() error

func (*DataScanFile) InitFile added in v0.0.7 

func (s *DataScanFile) InitFile() error

func (*DataScanFile) SyncFile added in v0.0.7 

func (s *DataScanFile) SyncFile() error

type DataScanRow added in v0.0.7 

type DataScanRow struct {
	Ctx        context.Context
	StartTime  time.Time
	TaskName   string
	TaskMode   string
	TaskFlow   string
	Dst        *task.DataScanTask
	DatabaseS  database.IDatabase
	DBCharsetS string
}

func (*DataScanRow) ScanRows added in v0.0.7 

func (r *DataScanRow) ScanRows() error

type DataScanRule added in v0.0.7 

type DataScanRule struct {
	Ctx               context.Context    `json:"-"`
	TaskName          string             `json:"taskName"`
	TaskMode          string             `json:"taskMode"`
	TaskFlow          string             `json:"taskFlow"`
	SchemaNameS       string             `json:"schemaNameS"`
	TableNameS        string             `json:"tableNameS"`
	GlobalSamplerateS string             `json:"globalSamplerateS"`
	GlobalSqlHintS    string             `json:"globalSqlHintS"`
	TableTypeS        map[string]string  `json:"tableTypeS"`
	DatabaseS         database.IDatabase `json:"-"`
	DBCharsetS        string             `json:"DBCharsetS"`
}

func (*DataScanRule) GenSchemaNameRule added in v0.0.7 

func (r *DataScanRule) GenSchemaNameRule() (string, error)

func (*DataScanRule) GenSchemaTableCustomRule added in v0.0.7 

func (r *DataScanRule) GenSchemaTableCustomRule() (string, string, error)

func (*DataScanRule) GenSchemaTableTypeRule added in v0.0.7 

func (r *DataScanRule) GenSchemaTableTypeRule() string

func (*DataScanRule) GenTableColumnNameRule added in v0.0.7 

func (r *DataScanRule) GenTableColumnNameRule() (string, string, error)

func (*DataScanRule) GenTableNameRule added in v0.0.7 

func (r *DataScanRule) GenTableNameRule() (string, error)

type DataScanTask added in v0.0.9 

type DataScanTask struct {
	Ctx         context.Context
	Task        *task.Task
	DatabaseS   database.IDatabase
	SchemaNameS string
	DBRoleS     string
	DBCharsetS  string
	DBVersionS  string

	TaskParams *pb.DataScanParam
	WaiterC    chan *WaitingRecs
	ResumeC    chan *WaitingRecs
}

func (*DataScanTask) Init added in v0.0.9 

func (dst *DataScanTask) Init() error

func (*DataScanTask) PrepareStatisticsRange added in v0.0.9 

func (dst *DataScanTask) PrepareStatisticsRange(globalScn string, attsRule *database.DataScanAttributesRule, ranges []*structure.Range) ([]*task.DataScanTask, error)

func (*DataScanTask) Process added in v0.0.9 

func (dst *DataScanTask) Process(s *WaitingRecs) error

func (*DataScanTask) ProcessChunkScan added in v0.0.9 

func (dst *DataScanTask) ProcessChunkScan(ctx context.Context, schemaNameS, tableNameS, globalScn string, tableRows uint64, tableSize float64, attsRule *database.DataScanAttributesRule) error

func (*DataScanTask) ProcessStatisticsScan added in v0.0.9 

func (dst *DataScanTask) ProcessStatisticsScan(ctx context.Context, dbTypeS, schemaNameS, tableNameS, globalScn string, tableRows uint64, tableSize float64, attsRule *database.DataScanAttributesRule) error

func (*DataScanTask) ProcessTableScan added in v0.0.9 

func (dst *DataScanTask) ProcessTableScan(ctx context.Context, schemaNameS, tableNameS, globalScn string, tableRows uint64, tableSize float64, attsRule *database.DataScanAttributesRule) error

func (*DataScanTask) Resume added in v0.0.9 

func (dst *DataScanTask) Resume() error

func (*DataScanTask) Run added in v0.0.9 

func (dst *DataScanTask) Run() error

type Datasource added in v0.0.7 

type Datasource struct {
	DBTypeS     string             `json:"dbTypeS"`
	DBVersionS  string             `json:"dbVersionS"`
	DatabaseS   database.IDatabase `json:"-"`
	SchemaNameS string             `json:"schemaNameS"`
	TableNameS  string             `json:"tableNameS"`
	TableTypeS  string             `json:"tableTypeS"`
}

func (*Datasource) GetTableCharsetCollation added in v0.0.7 

func (d *Datasource) GetTableCharsetCollation() (string, string, error)

func (*Datasource) GetTableCheckKey added in v0.0.7 

func (d *Datasource) GetTableCheckKey() ([]map[string]string, error)

func (*Datasource) GetTableColumnComment added in v0.0.7 

func (d *Datasource) GetTableColumnComment() ([]map[string]string, error)

func (*Datasource) GetTableColumns added in v0.0.7 

func (d *Datasource) GetTableColumns() ([]map[string]string, error)

func (*Datasource) GetTableComment added in v0.0.7 

func (d *Datasource) GetTableComment() ([]map[string]string, error)

func (*Datasource) GetTableForeignKey added in v0.0.7 

func (d *Datasource) GetTableForeignKey() ([]map[string]string, error)

func (*Datasource) GetTableNormalIndex added in v0.0.7 

func (d *Datasource) GetTableNormalIndex() ([]map[string]string, error)

func (*Datasource) GetTableOriginStruct added in v0.0.7 

func (d *Datasource) GetTableOriginStruct() (string, error)

func (*Datasource) GetTablePrimaryKey added in v0.0.7 

func (d *Datasource) GetTablePrimaryKey() ([]map[string]string, error)

func (*Datasource) GetTableUniqueIndex added in v0.0.7 

func (d *Datasource) GetTableUniqueIndex() ([]map[string]string, error)

func (*Datasource) GetTableUniqueKey added in v0.0.7 

func (d *Datasource) GetTableUniqueKey() ([]map[string]string, error)

func (*Datasource) String added in v0.0.7 

func (d *Datasource) String() string

type Divide added in v0.0.9 

type Divide struct {
	DBTypeS     string
	DBCharsetS  string
	SchemaNameS string
	TableNameS  string
	ChunkSize   int64
	DatabaseS   database.IDatabase
	Cons        *structure.Selectivity
	RangeC      chan []*structure.Range
}

func (*Divide) ProcessUpstreamStatisticsBucket added in v0.0.9 

func (d *Divide) ProcessUpstreamStatisticsBucket() error

type MySQLProcessor 

type MySQLProcessor struct {
	Ctx        context.Context `json:"-"`
	TaskName   string          `json:"taskName"`
	TaskFlow   string          `json:"taskFlow"`
	SchemaName string          `json:"schemaName"`
	TableName  string          `json:"tableName"`

	DBCharset                string                           `json:"dbCharset"`
	BuildinDatatypeRules     []*buildin.BuildinDatatypeRule   `json:"-"`
	BuildinDefaultValueRules []*buildin.BuildinDefaultvalRule `json:"-"`
	ColumnRouteRules         map[string]string                `json:"columnRouteRules"`
	IsBaseline               bool                             `json:"isBaseline"`
	Database                 database.IDatabase               `json:"-"`
}

func (*MySQLProcessor) GenDatabaseSchemaTable 

func (p *MySQLProcessor) GenDatabaseSchemaTable() (string, string)

func (*MySQLProcessor) GenDatabaseTableCharset 

func (p *MySQLProcessor) GenDatabaseTableCharset() (string, error)

func (*MySQLProcessor) GenDatabaseTableCheckConstraintDetail 

func (p *MySQLProcessor) GenDatabaseTableCheckConstraintDetail() (map[string]structure.ConstraintCheck, error)

func (*MySQLProcessor) GenDatabaseTableCollation 

func (p *MySQLProcessor) GenDatabaseTableCollation() (string, error)

func (*MySQLProcessor) GenDatabaseTableColumnDetail 

func (p *MySQLProcessor) GenDatabaseTableColumnDetail() (map[string]structure.NewColumn, map[string]map[string]structure.OldColumn, error)

func (*MySQLProcessor) GenDatabaseTableComment 

func (p *MySQLProcessor) GenDatabaseTableComment() (string, error)

func (*MySQLProcessor) GenDatabaseTableForeignConstraintDetail 

func (p *MySQLProcessor) GenDatabaseTableForeignConstraintDetail() (map[string]structure.ConstraintForeign, error)

func (*MySQLProcessor) GenDatabaseTableIndexDetail 

func (p *MySQLProcessor) GenDatabaseTableIndexDetail() (map[string]structure.Index, error)

func (*MySQLProcessor) GenDatabaseTablePartitionDetail 

func (p *MySQLProcessor) GenDatabaseTablePartitionDetail() ([]structure.Partition, error)

func (*MySQLProcessor) GenDatabaseTablePrimaryConstraintDetail 

func (p *MySQLProcessor) GenDatabaseTablePrimaryConstraintDetail() (map[string]structure.ConstraintPrimary, error)

func (*MySQLProcessor) GenDatabaseTableUniqueConstraintDetail 

func (p *MySQLProcessor) GenDatabaseTableUniqueConstraintDetail() (map[string]structure.ConstraintUnique, error)

type OracleProcessor 

type OracleProcessor struct {
	Ctx                      context.Context                  `json:"-"`
	TaskName                 string                           `json:"taskName"`
	TaskFlow                 string                           `json:"taskFlow"`
	SchemaName               string                           `json:"schemaName"`
	TableName                string                           `json:"tableName"`
	DBCharset                string                           `json:"dbCharset"`
	Database                 database.IDatabase               `json:"-"`
	BuildinDatatypeRules     []*buildin.BuildinDatatypeRule   `json:"-"`
	BuildinDefaultValueRules []*buildin.BuildinDefaultvalRule `json:"-"`
	ColumnRouteRules         map[string]string                `json:"columnRouteRules"`
	IsBaseline               bool                             `json:"isBaseline"`
	TableCollation           string                           `json:"tableCollation"`
	SchemaCollation          string                           `json:"schemaCollation"`
	NLSComp                  string                           `json:"NLSComp"`
	DBCollation              bool                             `json:"DBCollation"`
}

func (*OracleProcessor) GenDatabaseSchemaTable 

func (p *OracleProcessor) GenDatabaseSchemaTable() (string, string)

func (*OracleProcessor) GenDatabaseTableCharset 

func (p *OracleProcessor) GenDatabaseTableCharset() (string, error)

func (*OracleProcessor) GenDatabaseTableCheckConstraintDetail 

func (p *OracleProcessor) GenDatabaseTableCheckConstraintDetail() (map[string]structure.ConstraintCheck, error)

func (*OracleProcessor) GenDatabaseTableCollation 

func (p *OracleProcessor) GenDatabaseTableCollation() (string, error)

func (*OracleProcessor) GenDatabaseTableColumnDetail 

func (p *OracleProcessor) GenDatabaseTableColumnDetail() (map[string]structure.NewColumn, map[string]map[string]structure.OldColumn, error)

func (*OracleProcessor) GenDatabaseTableComment 

func (p *OracleProcessor) GenDatabaseTableComment() (string, error)

func (*OracleProcessor) GenDatabaseTableForeignConstraintDetail 

func (p *OracleProcessor) GenDatabaseTableForeignConstraintDetail() (map[string]structure.ConstraintForeign, error)

func (*OracleProcessor) GenDatabaseTableIndexDetail 

func (p *OracleProcessor) GenDatabaseTableIndexDetail() (map[string]structure.Index, error)

func (*OracleProcessor) GenDatabaseTablePartitionDetail 

func (p *OracleProcessor) GenDatabaseTablePartitionDetail() ([]structure.Partition, error)

func (*OracleProcessor) GenDatabaseTablePrimaryConstraintDetail 

func (p *OracleProcessor) GenDatabaseTablePrimaryConstraintDetail() (map[string]structure.ConstraintPrimary, error)

func (*OracleProcessor) GenDatabaseTableUniqueConstraintDetail 

func (p *OracleProcessor) GenDatabaseTableUniqueConstraintDetail() (map[string]structure.ConstraintUnique, error)

type ScanResultMYSQLCompatible added in v0.0.7 

type ScanResultMYSQLCompatible struct {
	ColumnName     string
	Bigint         string
	BigintUnsigned string
	DecimalInt     string
	DecimalPoint   string
	Unknown        string
}

type SequenceMigrateDatabase added in v0.0.7 

type SequenceMigrateDatabase struct {
	Ctx            context.Context             `json:"-"`
	TaskName       string                      `json:"taskName"`
	TaskMode       string                      `json:"taskMode"`
	TaskFlow       string                      `json:"taskFlow"`
	TaskStartTime  time.Time                   `json:"-"`
	DatasourceT    database.IDatabase          `json:"-"`
	SeqMigrateTask []*task.SequenceMigrateTask `json:"seqMigrateTask"`
}

func NewSequenceMigrateDatabase added in v0.0.7 

func NewSequenceMigrateDatabase(ctx context.Context,
	taskName, taskMode, taskFlow string, datasourceT database.IDatabase,
	taskStartTime time.Time, seqs []*task.SequenceMigrateTask) *SequenceMigrateDatabase

func (*SequenceMigrateDatabase) SyncSequenceDatabase added in v0.0.7 

func (s *SequenceMigrateDatabase) SyncSequenceDatabase() error

func (*SequenceMigrateDatabase) WriteSequenceDatabase added in v0.0.7 

func (s *SequenceMigrateDatabase) WriteSequenceDatabase() error

type SqlMigrateRow added in v0.0.7 

type SqlMigrateRow struct {
	Ctx           context.Context
	TaskMode      string
	TaskFlow      string
	Smt           *task.SqlMigrateTask
	DatabaseS     database.IDatabase
	DatabaseT     database.IDatabase
	DatabaseTStmt *sql.Stmt
	DBCharsetS    string
	DBCharsetT    string
	SqlThreadT    int
	BatchSize     int
	CallTimeout   int
	SafeMode      bool
	ReadChan      chan []interface{}
	WriteChan     chan []interface{}
}

func (*SqlMigrateRow) MigrateApply added in v0.0.7 

func (r *SqlMigrateRow) MigrateApply() error

func (*SqlMigrateRow) MigrateProcess added in v0.0.7 

func (r *SqlMigrateRow) MigrateProcess() error

func (*SqlMigrateRow) MigrateRead added in v0.0.7 

func (r *SqlMigrateRow) MigrateRead() error

type SqlMigrateRule added in v0.0.7 

type SqlMigrateRule struct {
	Ctx             context.Context    `json:"-"`
	TaskName        string             `json:"taskName"`
	TaskMode        string             `json:"taskMode"`
	TaskFlow        string             `json:"taskFlow"`
	SchemaNameT     string             `json:"schemaNameT"`
	TableNameT      string             `json:"tableNameT"`
	SqlHintT        string             `json:"sqlHintT"`
	GlobalSqlHintT  string             `json:"globalSqlHintT"`
	DatabaseS       database.IDatabase `json:"databaseS"`
	DBCharsetS      string             `json:"DBCharsetS"`
	SqlQueryS       string             `json:"sqlQueryS"`
	ColumnRouteRule map[string]string  `json:"columnRouteRule"`
	CaseFieldRuleS  string             `json:"caseFieldRuleS"`
	CaseFieldRuleT  string             `json:"caseFieldRuleT"`
}

func (*SqlMigrateRule) GenSqlMigrateSchemaNameRule added in v0.0.7 

func (r *SqlMigrateRule) GenSqlMigrateSchemaNameRule() (string, error)

func (*SqlMigrateRule) GenSqlMigrateTableColumnRule added in v0.0.7 

func (r *SqlMigrateRule) GenSqlMigrateTableColumnRule() (string, string, string, error)

func (*SqlMigrateRule) GenSqlMigrateTableCustomRule added in v0.0.7 

func (r *SqlMigrateRule) GenSqlMigrateTableCustomRule() (string, string)

func (*SqlMigrateRule) GenSqlMigrateTableNameRule added in v0.0.7 

func (r *SqlMigrateRule) GenSqlMigrateTableNameRule() (string, error)

type SqlMigrateTask added in v0.0.9 

type SqlMigrateTask struct {
	Ctx        context.Context
	Task       *task.Task
	DatabaseS  database.IDatabase
	DatabaseT  database.IDatabase
	DBRoleS    string
	DBCharsetS string
	DBCharsetT string
	DBVersionS string

	TaskParams *pb.SqlMigrateParam
}

func (*SqlMigrateTask) Init added in v0.0.9 

func (smt *SqlMigrateTask) Init() error

func (*SqlMigrateTask) Process added in v0.0.9 

func (smt *SqlMigrateTask) Process() error

func (*SqlMigrateTask) Resume added in v0.0.9 

func (smt *SqlMigrateTask) Resume() error

func (*SqlMigrateTask) Run added in v0.0.9 

func (smt *SqlMigrateTask) Run() error

type StmtMigrateRow added in v0.0.7 

type StmtMigrateRow struct {
	Ctx           context.Context
	TaskMode      string
	TaskFlow      string
	Dmt           *task.DataMigrateTask
	DatabaseS     database.IDatabase
	DatabaseT     database.IDatabase
	DatabaseTStmt *sql.Stmt
	DBCharsetS    string
	DBCharsetT    string
	SqlThreadT    int
	BatchSize     int
	CallTimeout   int
	SafeMode      bool
	ReadChan      chan []interface{}
	WriteChan     chan []interface{}
}

func (*StmtMigrateRow) MigrateApply added in v0.0.7 

func (r *StmtMigrateRow) MigrateApply() error

func (*StmtMigrateRow) MigrateProcess added in v0.0.7 

func (r *StmtMigrateRow) MigrateProcess() error

func (*StmtMigrateRow) MigrateRead added in v0.0.7 

func (r *StmtMigrateRow) MigrateRead() error

type StructCompareFile added in v0.0.7 

type StructCompareFile struct {
	Ctx        context.Context `json:"-"`
	Mutex      *sync.Mutex     `json:"-"`
	CompFile   *os.File        `json:"-"`
	CompWriter *bufio.Writer   `json:"-"`
	TaskName   string          `json:"taskName"`
	TaskFlow   string          `json:"taskFlow"`
	OutputDir  string          `json:"outputDir"`
}

func NewStructCompareFile added in v0.0.7 

func NewStructCompareFile(ctx context.Context,
	taskName, taskFlow, outputDir string) *StructCompareFile

func (*StructCompareFile) Close added in v0.0.7 

func (s *StructCompareFile) Close() error

func (*StructCompareFile) InitFile added in v0.0.7 

func (s *StructCompareFile) InitFile() error

func (*StructCompareFile) SyncFile added in v0.0.7 

func (s *StructCompareFile) SyncFile() error

type StructCompareTask added in v0.0.9 

type StructCompareTask struct {
	Ctx                       context.Context
	Task                      *task.Task
	DBTypeS                   string
	DBTypeT                   string
	DatabaseS                 database.IDatabase
	DatabaseT                 database.IDatabase
	DBCharsetS                string
	DBCharsetT                string
	SchemaNameS               string
	SchemaNameT               string
	StartTime                 time.Time
	TaskParams                *pb.StructCompareParam
	BuildInDatatypeRulesS     []*buildin.BuildinDatatypeRule
	BuildInDefaultValueRulesS []*buildin.BuildinDefaultvalRule
	BuildInDatatypeRulesT     []*buildin.BuildinDatatypeRule
	BuildInDefaultValueRulesT []*buildin.BuildinDefaultvalRule

	ReadyInit chan bool
}

func (*StructCompareTask) Init added in v0.0.9 

func (dmt *StructCompareTask) Init() error

func (*StructCompareTask) Process added in v0.0.9 

func (dmt *StructCompareTask) Process() error

func (*StructCompareTask) Resume added in v0.0.9 

func (dmt *StructCompareTask) Resume() error

func (*StructCompareTask) Run added in v0.0.9 

func (dmt *StructCompareTask) Run() error

type StructMigrateDatabase added in v0.0.7 

type StructMigrateDatabase struct {
	Ctx           context.Context       `json:"-"`
	TaskName      string                `json:"taskName"`
	TaskFlow      string                `json:"taskFlow"`
	TaskStartTime time.Time             `json:"-"`
	DatasourceT   database.IDatabase    `json:"-"`
	TableStruct   *database.TableStruct `json:"tableStruct"`
}

func NewStructMigrateDatabase added in v0.0.7 

func NewStructMigrateDatabase(ctx context.Context,
	taskName, taskFlow string, datasourceT database.IDatabase,
	taskStartTime time.Time, tableStruct *database.TableStruct) *StructMigrateDatabase

func (*StructMigrateDatabase) GenTableStructDDL added in v0.0.7 

func (s *StructMigrateDatabase) GenTableStructDDL() ([]string, []string, error)

func (*StructMigrateDatabase) GenTableStructDigest added in v0.0.7 

func (s *StructMigrateDatabase) GenTableStructDigest() (string, string, string, error)

func (*StructMigrateDatabase) SyncStructDatabase added in v0.0.7 

func (s *StructMigrateDatabase) SyncStructDatabase() error

func (*StructMigrateDatabase) WriteStructDatabase added in v0.0.7 

func (s *StructMigrateDatabase) WriteStructDatabase() error

WriteStructDatabase used for sync file, current only write database, but not sync target database

type StructMigrateFile added in v0.0.7 

type StructMigrateFile struct {
	Ctx              context.Context `json:"-"`
	Mutex            *sync.Mutex     `json:"-"`
	CompFile         *os.File        `json:"-"`
	InCompFile       *os.File        `json:"-"`
	CompWriter       *bufio.Writer   `json:"-"`
	InCompWriter     *bufio.Writer   `json:"-"`
	TaskName         string          `json:"taskName"`
	TaskFlow         string          `json:"taskFlow"`
	DBCharsetS       string          `json:"dbCharsetS"`
	DBCollationS     bool            `json:"dbCollationS"`
	SchemaCollationS string          `json:"schemaCollationS"`
	DBNlsComp        string          `json:"DBNlsComp"`
	SchemaNameS      string          `json:"schemaNameS"`
	SchemaNameT      string          `json:"schemaNameT"`
	OutputDir        string          `json:"outputDir"`
}

func NewStructMigrateFile added in v0.0.7 

func NewStructMigrateFile(ctx context.Context,
	taskName, taskFlow, schemaNameS string, outputDir string) *StructMigrateFile

func (*StructMigrateFile) Close added in v0.0.7 

func (s *StructMigrateFile) Close() error

func (*StructMigrateFile) InitOutputFile added in v0.0.7 

func (s *StructMigrateFile) InitOutputFile() error

func (*StructMigrateFile) SyncSequenceFile added in v0.0.7 

func (s *StructMigrateFile) SyncSequenceFile() error

func (*StructMigrateFile) SyncStructFile added in v0.0.7 

func (s *StructMigrateFile) SyncStructFile() error

type StructMigrateRule added in v0.0.7 

type StructMigrateRule struct {
	Ctx                      context.Context                  `json:"-"`
	TaskName                 string                           `json:"taskName"`
	TaskMode                 string                           `json:"taskMode"`
	TaskFlow                 string                           `json:"taskFlow"`
	SchemaNameS              string                           `json:"schemaNameS"`
	TableNameS               string                           `json:"tableNameS"`
	TablePrimaryAttrs        []map[string]string              `json:"tablePrimaryAttrs"`
	TableColumnsAttrs        []map[string]string              `json:"tableColumnsAttrs"`
	TableCommentAttrs        []map[string]string              `json:"tableCommentAttrs"`
	CaseFieldRuleT           string                           `json:"caseFieldRuleT"`
	CreateIfNotExist         bool                             `json:"createIfNotExist"`
	DBVersionS               string                           `json:"dbVersionS"`
	DBCharsetS               string                           `json:"dbCharsetS"`
	DBCharsetT               string                           `json:"dbCharsetT"`
	BuildinDatatypeRules     []*buildin.BuildinDatatypeRule   `json:"-"`
	BuildinDefaultValueRules []*buildin.BuildinDefaultvalRule `json:"-"`
}

func (*StructMigrateRule) GetCaseFieldRule added in v0.0.7 

func (r *StructMigrateRule) GetCaseFieldRule() string

func (*StructMigrateRule) GetCreatePrefixRule added in v0.0.7 

func (r *StructMigrateRule) GetCreatePrefixRule() string

func (*StructMigrateRule) GetSchemaNameRule added in v0.0.7 

func (r *StructMigrateRule) GetSchemaNameRule() (map[string]string, error)

func (*StructMigrateRule) GetTableAttributesRule added in v0.0.7 

func (r *StructMigrateRule) GetTableAttributesRule() (string, error)

func (*StructMigrateRule) GetTableColumnCollationRule added in v0.0.7 

func (r *StructMigrateRule) GetTableColumnCollationRule() (map[string]string, error)

func (*StructMigrateRule) GetTableColumnCommentRule added in v0.0.7 

func (r *StructMigrateRule) GetTableColumnCommentRule() (map[string]string, error)

func (*StructMigrateRule) GetTableColumnRule added in v0.0.7 

func (r *StructMigrateRule) GetTableColumnRule() (map[string]string, map[string]string, map[string]string, error)

GetTableColumnRule used for get custom table column rule column datatype rule priority: - column level - table level - task level - default level

func (*StructMigrateRule) GetTableCommentRule added in v0.0.7 

func (r *StructMigrateRule) GetTableCommentRule() (string, error)

func (*StructMigrateRule) GetTableNameRule added in v0.0.7 

func (r *StructMigrateRule) GetTableNameRule() (map[string]string, error)

func (*StructMigrateRule) String added in v0.0.7 

func (r *StructMigrateRule) String() string

type StructMigrateTable added in v0.0.9 

type StructMigrateTable struct {
	TaskName            string                                `json:"taskName"`
	TaskFlow            string                                `json:"taskFlow"`
	DatasourceS         *Datasource                           `json:"datasourceS"`
	TableAttributes     *database.StructMigrateAttributes     `json:"tableAttributes"`
	TableAttributesRule *database.StructMigrateAttributesRule `json:"tableAttributesRule"`
	DBCharsetT          string                                `json:"DBCharsetT"`
}

func (*StructMigrateTable) GenSchemaNameS added in v0.0.9 

func (t *StructMigrateTable) GenSchemaNameS() string

func (*StructMigrateTable) GenSchemaNameT added in v0.0.9 

func (t *StructMigrateTable) GenSchemaNameT() (string, error)

func (*StructMigrateTable) GenTableCheckKey added in v0.0.9 

func (t *StructMigrateTable) GenTableCheckKey() ([]string, error)

func (*StructMigrateTable) GenTableColumnComment added in v0.0.9 

func (t *StructMigrateTable) GenTableColumnComment() ([]string, error)

func (*StructMigrateTable) GenTableColumns added in v0.0.9 

func (t *StructMigrateTable) GenTableColumns() ([]string, error)

func (*StructMigrateTable) GenTableComment added in v0.0.9 

func (t *StructMigrateTable) GenTableComment() (string, error)

func (*StructMigrateTable) GenTableCreatePrefixT added in v0.0.9 

func (t *StructMigrateTable) GenTableCreatePrefixT() string

func (*StructMigrateTable) GenTableForeignKey added in v0.0.9 

func (t *StructMigrateTable) GenTableForeignKey() ([]string, error)

func (*StructMigrateTable) GenTableNameS added in v0.0.9 

func (t *StructMigrateTable) GenTableNameS() string

func (*StructMigrateTable) GenTableNameT added in v0.0.9 

func (t *StructMigrateTable) GenTableNameT() (string, error)

func (*StructMigrateTable) GenTableNormalIndex added in v0.0.9 

func (t *StructMigrateTable) GenTableNormalIndex() ([]string, []string, error)

func (*StructMigrateTable) GenTableOriginDDlS added in v0.0.9 

func (t *StructMigrateTable) GenTableOriginDDlS() string

func (*StructMigrateTable) GenTablePrimaryKey added in v0.0.9 

func (t *StructMigrateTable) GenTablePrimaryKey() (string, error)

func (*StructMigrateTable) GenTableSuffix added in v0.0.9 

func (t *StructMigrateTable) GenTableSuffix() (string, error)

func (*StructMigrateTable) GenTableTypeS added in v0.0.9 

func (t *StructMigrateTable) GenTableTypeS() string

func (*StructMigrateTable) GenTableUniqueIndex added in v0.0.9 

func (t *StructMigrateTable) GenTableUniqueIndex() ([]string, []string, error)

func (*StructMigrateTable) GenTableUniqueKey added in v0.0.9 

func (t *StructMigrateTable) GenTableUniqueKey() ([]string, error)

func (*StructMigrateTable) String added in v0.0.9 

func (t *StructMigrateTable) String() string

type StructMigrateTask added in v0.0.9 

type StructMigrateTask struct {
	Ctx                      context.Context
	Task                     *task.Task
	SchemaNameS              string
	SchemaNameT              string
	DatabaseS                database.IDatabase
	DatabaseT                database.IDatabase
	DBTypeS                  string
	DBVersionS               string
	DBCharsetS               string
	DBCharsetT               string
	StartTime                time.Time
	BuildInDatatypeRules     []*buildin.BuildinDatatypeRule
	BuildInDefaultValueRules []*buildin.BuildinDefaultvalRule
	TaskParams               *pb.StructMigrateParam

	ReadyInit chan bool
}

func (*StructMigrateTask) Init added in v0.0.9 

func (st *StructMigrateTask) Init() error

func (*StructMigrateTask) Process added in v0.0.9 

func (st *StructMigrateTask) Process() error

func (*StructMigrateTask) Resume added in v0.0.9 

func (st *StructMigrateTask) Resume() error

func (*StructMigrateTask) Run added in v0.0.9 

func (st *StructMigrateTask) Run() error

type Table 

type Table struct {
	TaskName string
	TaskFlow string
	Source   *structure.Table
	Target   *structure.Table
}

Table structure comparison Based on the upstream Oracle table structure information, compare the downstream MySQL table structure 1. If the upstream exists and the downstream does not exist, the record will be output. If the upstream does not exist and the downstream exists, the record will not be output by default. 2. Ignore the comparison of different index names and constraint names between the upstream and downstream, and only compare whether the same fields exist under the same constraints downstream. 3. Partitions only compare partition types, partition keys, partition expressions, etc., and do not compare the specific conditions of each partition.

func (*Table) ComparePartitionTableType 

func (t *Table) ComparePartitionTableType() string

func (*Table) CompareTableCharsetCollation 

func (t *Table) CompareTableCharsetCollation() string

func (*Table) CompareTableCheckConstraint 

func (t *Table) CompareTableCheckConstraint() (string, error)

func (*Table) CompareTableColumnCharsetCollation 

func (t *Table) CompareTableColumnCharsetCollation() string

func (*Table) CompareTableColumnCounts 

func (t *Table) CompareTableColumnCounts() string

func (*Table) CompareTableColumnDetail 

func (t *Table) CompareTableColumnDetail() (string, error)

func (*Table) CompareTableComment 

func (t *Table) CompareTableComment() string

func (*Table) CompareTableForeignConstraint 

func (t *Table) CompareTableForeignConstraint() (string, error)

func (*Table) CompareTableIndexDetail 

func (t *Table) CompareTableIndexDetail() (string, error)

func (*Table) CompareTablePartitionDetail 

func (t *Table) CompareTablePartitionDetail() (string, error)

func (*Table) CompareTablePrimaryConstraint 

func (t *Table) CompareTablePrimaryConstraint() (string, error)

func (*Table) CompareTableUniqueConstraint 

func (t *Table) CompareTableUniqueConstraint() (string, error)

type WaitingRecs added in v0.0.9 

type WaitingRecs struct {
	TaskName    string
	SchemaNameS string
	TableNameS  string
	SchemaNameT string
	TableNameT  string
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.