Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
var Connectors = make(map[string]Connector)
Connectors tracks all registered connector drivers.
Functions ¶
func ConsumeAsFiles ¶ added in v0.19.0
Types ¶
type Connector ¶
type Connector interface {
Spec() Spec
ConsumeAsFiles(ctx context.Context, env *Env, source *Source) ([]string, error)
}
Connector is a driver for ingesting data from an external system.
type Env ¶ added in v0.15.0
Env contains contextual information for a source, such as the repo it came from and (in the future) secrets configured by the user.
type PropertySchema ¶
type PropertySchema struct {
Key string
Type PropertySchemaType
Required bool
DisplayName string
Description string
Placeholder string
Hint string
Href string
}
PropertySchema provides the schema for a property supported by a connector.
func (PropertySchema) ValidateType ¶
func (ps PropertySchema) ValidateType(val any) bool
ValidateType checks that val has the correct type.
type PropertySchemaType ¶
type PropertySchemaType int
PropertySchemaType is an enum of types supported for connector properties.
const ( UnspecifiedPropertyType PropertySchemaType = iota StringPropertyType NumberPropertyType BooleanPropertyType InformationalPropertyType )
type SamplePolicy ¶
SamplePolicy tells the connector to only ingest a sample of data from the source. Support for it is currently not implemented.
type Source ¶
type Source struct {
Name string
Connector string
SamplePolicy *SamplePolicy
Properties map[string]any
}
Source represents a dataset to ingest using a specific connector (like a connector instance).
func (*Source) PropertiesEquals ¶ added in v0.13.0
type Spec ¶
type Spec struct {
DisplayName string
Description string
Properties []PropertySchema
}
Spec provides metadata about a connector and the properties it supports.
Source Files
Directories