plugins

package
v0.13.15 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 30, 2022 License: MPL-2.0 Imports: 22 Imported by: 7

Documentation

Overview

Package plugins defines APIs for source and destination plugins

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func SetDestinationManagedCqColumns added in v0.13.0 

func SetDestinationManagedCqColumns(tables []*schema.Table)

Overwrites or adds the CQ columns that are managed by the destination plugins (_cq_sync_time, _cq_source_name).

func TestSourcePluginSync added in v0.0.4 

func TestSourcePluginSync(t *testing.T, plugin *SourcePlugin, spec specs.Source, opts ...TestSourcePluginOption)

Types

type DestinationClient added in v0.2.3 

type DestinationClient interface {
	Migrate(ctx context.Context, tables schema.Tables) error
	Write(ctx context.Context, table string, data map[string]interface{}) error
	Metrics() DestinationMetrics
	DeleteStale(ctx context.Context, tables string, sourceName string, syncTime time.Time) error
	Close(ctx context.Context) error
}

type DestinationMetrics added in v0.13.15 

type DestinationMetrics struct {
	// Errors number of errors / failed writes
	Errors uint64
	// Writes number of successful writes
	Writes uint64
}

type DestinationPlugin 

type DestinationPlugin struct {
	// contains filtered or unexported fields
}

func NewDestinationPlugin added in v0.2.3 

func NewDestinationPlugin(name string, version string, newDestinationClient NewDestinationClientFunc) *DestinationPlugin

func (*DestinationPlugin) Close added in v0.11.4 

func (p *DestinationPlugin) Close(ctx context.Context) error

func (*DestinationPlugin) DeleteStale added in v0.12.0 

func (p *DestinationPlugin) DeleteStale(ctx context.Context, tables []string, sourceName string, syncTime time.Time) uint64

func (*DestinationPlugin) Init added in v0.12.0 

func (p *DestinationPlugin) Init(ctx context.Context, logger zerolog.Logger, spec specs.Destination) error

we need lazy loading because we want to be able to initialize after

func (*DestinationPlugin) Metrics added in v0.13.15 

func (p *DestinationPlugin) Metrics() DestinationMetrics

func (*DestinationPlugin) Migrate added in v0.0.4 

func (p *DestinationPlugin) Migrate(ctx context.Context, tables schema.Tables) error

we implement all DestinationClient functions so we can hook into pre-post behavior

func (*DestinationPlugin) Name added in v0.0.4 

func (p *DestinationPlugin) Name() string

func (*DestinationPlugin) Version added in v0.0.4 

func (p *DestinationPlugin) Version() string

func (*DestinationPlugin) Write added in v0.0.4 

func (p *DestinationPlugin) Write(ctx context.Context, sourceName string, syncTime time.Time, res <-chan *schema.Resource) *WriteSummary

type NewDestinationClientFunc added in v0.12.0 

type NewDestinationClientFunc func(context.Context, zerolog.Logger, specs.Destination) (DestinationClient, error)

type ResourceTestCase 

type ResourceTestCase struct {
	Plugin *SourcePlugin
	Spec   specs.Source
	// ParallelFetchingLimit limits parallel resources fetch at a time
	ParallelFetchingLimit uint64
	// SkipIgnoreInTest flag which detects if schema.Table or schema.Column should be ignored
	SkipIgnoreInTest bool
}

type SourceMetrics added in v0.13.15 

type SourceMetrics struct {
	TableClient map[string]map[string]*TableClientMetrics
}

func (*SourceMetrics) Equal added in v0.13.15 

func (s *SourceMetrics) Equal(other *SourceMetrics) bool

Equal compares to stats. Mostly useful in testing

func (*SourceMetrics) TotalErrors added in v0.13.15 

func (s *SourceMetrics) TotalErrors() uint64

func (*SourceMetrics) TotalPanics added in v0.13.15 

func (s *SourceMetrics) TotalPanics() uint64

func (*SourceMetrics) TotalResources added in v0.13.15 

func (s *SourceMetrics) TotalResources() uint64

type SourceNewExecutionClientFunc added in v0.0.4 

type SourceNewExecutionClientFunc func(context.Context, zerolog.Logger, specs.Source) (schema.ClientMeta, error)

type SourcePlugin 

type SourcePlugin struct {
	// contains filtered or unexported fields
}

SourcePlugin is the base structure required to pass to sdk.serve We take a declarative approach to API here similar to Cobra

func NewSourcePlugin added in v0.0.4 

func NewSourcePlugin(name string, version string, tables []*schema.Table, newExecutionClient SourceNewExecutionClientFunc) *SourcePlugin

NewSourcePlugin returns a new plugin with a given name, version, tables, newExecutionClient and additional options.

func (*SourcePlugin) GenerateSourcePluginDocs added in v0.6.4 

func (p *SourcePlugin) GenerateSourcePluginDocs(dir string) error

GenerateSourcePluginDocs creates table documentation for the source plugin based on its list of tables

func (*SourcePlugin) Metrics added in v0.13.15 

func (p *SourcePlugin) Metrics() SourceMetrics

func (*SourcePlugin) Name 

func (p *SourcePlugin) Name() string

Name return the name of this plugin

func (*SourcePlugin) SetLogger added in v0.0.4 

func (p *SourcePlugin) SetLogger(logger zerolog.Logger)

func (*SourcePlugin) Sync added in v0.0.4 

func (p *SourcePlugin) Sync(ctx context.Context, spec specs.Source, res chan<- *schema.Resource) error

Sync is syncing data from the requested tables in spec to the given channel

func (*SourcePlugin) Tables 

func (p *SourcePlugin) Tables() schema.Tables

Tables returns all supported tables by this source plugin

func (*SourcePlugin) Version 

func (p *SourcePlugin) Version() string

Version returns the version of this plugin

type TableClientMetrics added in v0.13.15 

type TableClientMetrics struct {
	Resources uint64
	Errors    uint64
	Panics    uint64
	StartTime time.Time
	EndTime   time.Time
}

func (*TableClientMetrics) Equal added in v0.13.15 

func (s *TableClientMetrics) Equal(other *TableClientMetrics) bool

type TestSourcePluginOption added in v0.13.8 

type TestSourcePluginOption func(*testSourcePluginOptions)

func WithTestSourcePluginNoParallel added in v0.13.8 

func WithTestSourcePluginNoParallel() TestSourcePluginOption

type WriteSummary added in v0.12.0 

type WriteSummary struct {
	SuccessWrites uint64
	FailedWrites  uint64
	FailedDeletes uint64
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.