Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ExportSourceByPKRange ¶
ExportSourceByPKRange exports source rows into shard text files. Each output line format is: id\tbiz_key\tmetric
func ImportReduceOutputs ¶
ImportReduceOutputs imports mr-out-* files into target table with batch upsert.
Types ¶
type SinkAdapter ¶
type SinkAdapter struct {
// contains filtered or unexported fields
}
func NewSinkAdapter ¶
func NewSinkAdapter(cfg SinkConfig) SinkAdapter
func (SinkAdapter) InputGlob ¶
func (a SinkAdapter) InputGlob() string
type SinkConfig ¶
type SinkConfig struct {
TargetTable string `json:"targettable"`
KeyColumn string `json:"keycolumn"`
ValColumn string `json:"valcolumn"`
InputGlob string `json:"inputglob"`
Replace bool `json:"replace"`
BatchSize int `json:"batchsize"`
}
SinkConfig configures reduce output import into MySQL.
func (*SinkConfig) WithDefaults ¶
func (c *SinkConfig) WithDefaults()
type SourceAdapter ¶
type SourceAdapter struct {
// contains filtered or unexported fields
}
func NewSourceAdapter ¶
func NewSourceAdapter(cfg SourceConfig) SourceAdapter
type SourceConfig ¶
type SourceConfig struct {
Table string `json:"table"`
PKColumn string `json:"pkcolumn"`
KeyColumn string `json:"keycolumn"`
ValColumn string `json:"valcolumn"`
Where string `json:"where"`
Shards int `json:"shards"`
Parallel int `json:"parallel"`
OutputDir string `json:"outputdir"`
FilePrefix string `json:"fileprefix"`
}
SourceConfig configures source export from MySQL table to text shards.
func (*SourceConfig) WithDefaults ¶
func (c *SourceConfig) WithDefaults()
Source Files
Click to show internal directories.
Click to hide internal directories.