plugin

package
v0.10.0-preview4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 14, 2026 License: MIT Imports: 17 Imported by: 0

Documentation

Index

Constants

View Source 
const CacheDirEnvVar = "MARMOT_PLUGIN_CACHE_DIR"

CacheDirEnvVar overrides the directory downloaded plugins are cached in.

View Source 
const PluginsDirEnvVar = "MARMOT_PLUGINS_DIR"

PluginsDirEnvVar overrides the directory scanned for locally installed plugins.

View Source 
const SensitiveMask = "********"

Variables

This section is empty.

Functions

func CacheDir added in v0.10.0 

func CacheDir() string

CacheDir resolves the plugin cache directory. Downloaded plugins are stored under <registry>/<name>/<version>/<os>_<arch>/ inside it.

func DecryptConfig added in v0.5.0 

func DecryptConfig(config map[string]interface{}, configSpec []pluginsdk.ConfigField, encryptor *crypto.Encryptor) error

DecryptConfig decrypts sensitive fields in a plugin config using the ConfigSpec

func DecryptConfigForPlugin added in v0.5.0 

func DecryptConfigForPlugin(pluginID string, config map[string]interface{}, encryptor *crypto.Encryptor) error

DecryptConfigForPlugin decrypts sensitive fields for a specific plugin by ID

func DefaultPluginsDir added in v0.10.0 

func DefaultPluginsDir() string

DefaultPluginsDir returns ~/.marmot/plugins, the default directory for locally installed plugins.

func EncryptConfig added in v0.5.0 

func EncryptConfig(config map[string]interface{}, configSpec []pluginsdk.ConfigField, encryptor *crypto.Encryptor) error

EncryptConfig encrypts sensitive fields in a plugin config using the ConfigSpec

func EncryptConfigForPlugin added in v0.5.0 

func EncryptConfigForPlugin(pluginID string, config map[string]interface{}, encryptor *crypto.Encryptor) error

EncryptConfigForPlugin encrypts sensitive fields for a specific plugin by ID

func FilterDiscoveryResult added in v0.7.0 

func FilterDiscoveryResult(result *DiscoveryResult, rawConfig RawPluginConfig)

FilterDiscoveryResult filters a DiscoveryResult based on the Filter in the config. It filters assets by name, then removes lineage, documentation, statistics, and run history entries that reference excluded assets.

func GetSensitiveFields added in v0.5.0 

func GetSensitiveFields(configSpec []pluginsdk.ConfigField) map[string]bool

GetSensitiveFields returns a map of sensitive field names from ConfigSpec

func LoadBinary added in v0.10.0 

func LoadBinary(path string)

LoadBinary registers the external plugin binary at path in the global registry. The first registration of an ID wins; a binary whose ID is already registered is skipped. Load failures are logged, not returned: one broken plugin must not take down the rest.

func PluginsDir added in v0.10.0 

func PluginsDir() string

PluginsDir resolves the local plugins directory from the environment, falling back to the default.

func UnmarshalPluginConfig 

func UnmarshalPluginConfig[T any](raw RawPluginConfig) (*T, error)

UnmarshalPluginConfig unmarshals raw config into a specific plugin config type

Types

type AssetRunHistory added in v0.5.0 

type AssetRunHistory struct {
	AssetMRN string            `json:"asset_mrn"`
	Runs     []RunHistoryEvent `json:"runs"`
}

AssetRunHistory contains run history events for an asset

type BaseConfig 

type BaseConfig = pluginsdk.BaseConfig

BaseConfig mirrors pluginsdk.BaseConfig; kept as a type alias so host code can embed it in configs without importing the SDK package name.

type Config 

type Config struct {
	Name string      `json:"name" yaml:"name"`
	Runs []SourceRun `json:"runs" yaml:"runs"`
}

type DataFetcher added in v0.9.0 

type DataFetcher interface {
	FetchSampleData(ctx context.Context, config RawPluginConfig, a *asset.Asset) (columnNames []string, rows [][]interface{}, err error)
}

DataFetcher is an optional interface that plugins can implement to support data preview functionality. Plugins that can query/fetch sample data from their data sources should implement this interface.

type DiscoveryResult 

type DiscoveryResult struct {
	Assets        []asset.Asset             `json:"assets"`
	Lineage       []lineage.LineageEdge     `json:"lineage"`
	Documentation []assetdocs.Documentation `json:"documentation"`
	Statistics    []Statistic               `json:"statistics"`
	RunHistory    []AssetRunHistory         `json:"run_history,omitempty"`
}

DiscoveryResult contains all discovered assets, lineage, and documentation

type ExternalDataFetcherSource added in v0.10.0 

type ExternalDataFetcherSource struct {
	ExternalSource
}

ExternalDataFetcherSource is an ExternalSource whose plugin advertises data-preview support; it additionally implements DataFetcher.

func (*ExternalDataFetcherSource) FetchSampleData added in v0.10.0 

func (s *ExternalDataFetcherSource) FetchSampleData(ctx context.Context, config RawPluginConfig, a *asset.Asset) ([]string, [][]interface{}, error)
type ExternalLink = pluginsdk.ExternalLink

type ExternalSource added in v0.10.0 

type ExternalSource struct {
	// contains filtered or unexported fields
}

ExternalSource adapts an external plugin binary to the internal Source interface. Each call spawns the plugin process, performs the call over gRPC, and kills the process again.

func (*ExternalSource) Discover added in v0.10.0 

func (s *ExternalSource) Discover(ctx context.Context, config RawPluginConfig) (*DiscoveryResult, error)

func (*ExternalSource) Validate added in v0.10.0 

func (s *ExternalSource) Validate(config RawPluginConfig) (RawPluginConfig, error)

type Filter 

type Filter = pluginsdk.Filter

type LoadState added in v0.10.0 

type LoadState struct {
	// contains filtered or unexported fields
}

func GetLoadState added in v0.10.0 

func GetLoadState() *LoadState

func (*LoadState) Done added in v0.10.0 

func (l *LoadState) Done() <-chan struct{}

Done returns a channel closed when loading finishes.

func (*LoadState) MarkReady added in v0.10.0 

func (l *LoadState) MarkReady()

MarkReady signals that loading is complete. Idempotent.

func (*LoadState) Ready added in v0.10.0 

func (l *LoadState) Ready() bool

Ready reports whether loading has finished, without blocking.

type PluginConfig 

type PluginConfig struct {
	BaseConfig `json:",inline"`
	Source     string `json:"source,omitempty"`
}

PluginConfig combines base config with plugin-specific fields

type RawPluginConfig 

type RawPluginConfig map[string]interface{} // @name RawPluginConfig

RawPluginConfig holds the raw JSON configuration for a plugin It uses a `map[string]interface{}` to unmarshal arbitrary JSON data for each plugin's specific config.

func MaskSensitiveFieldsFromSpec added in v0.5.0 

func MaskSensitiveFieldsFromSpec(config RawPluginConfig, configSpec []pluginsdk.ConfigField) RawPluginConfig

MaskSensitiveFieldsFromSpec masks sensitive fields in a config map using the ConfigSpec

func (RawPluginConfig) MaskSensitiveFields added in v0.3.0 

func (r RawPluginConfig) MaskSensitiveFields(configStruct interface{}) RawPluginConfig

type Registry added in v0.5.0 

type Registry struct {
	// contains filtered or unexported fields
}

func GetRegistry added in v0.5.0 

func GetRegistry() *Registry

func (*Registry) Get added in v0.5.0 

func (r *Registry) Get(id string) (*RegistryEntry, error)

func (*Registry) GetSource added in v0.5.0 

func (r *Registry) GetSource(id string) (Source, error)

func (*Registry) List added in v0.5.0 

func (r *Registry) List() []pluginsdk.Meta

func (*Registry) Register added in v0.5.0 

func (r *Registry) Register(meta pluginsdk.Meta, source Source) error

type RegistryEntry added in v0.5.0 

type RegistryEntry struct {
	Meta   pluginsdk.Meta
	Source Source
}

RegistryEntry pairs a plugin's metadata with the runtime source that dispatches Validate/Discover calls to the plugin binary.

type Run added in v0.3.0 

type Run struct {
	ID           string          `json:"id"`
	PipelineName string          `json:"pipeline_name"`
	SourceName   string          `json:"source_name"`
	RunID        string          `json:"run_id"`
	Status       RunStatus       `json:"status"`
	StartedAt    time.Time       `json:"started_at"`
	CompletedAt  *time.Time      `json:"completed_at,omitempty"`
	ErrorMessage string          `json:"error_message,omitempty"`
	Config       RawPluginConfig `json:"config,omitempty"`
	Summary      *RunSummary     `json:"summary,omitempty"`
	CreatedBy    string          `json:"created_by"`

} // @name PluginRun

Run represents a single run

type RunCheckpoint added in v0.3.0 

type RunCheckpoint struct {
	ID           string    `json:"id"`
	RunID        string    `json:"run_id"`
	EntityType   string    `json:"entity_type"` // 'asset', 'lineage', 'documentation'
	EntityMRN    string    `json:"entity_mrn"`
	Operation    string    `json:"operation"`     // 'created', 'updated', 'deleted', 'skipped'
	SourceFields []string  `json:"source_fields"` // Which fields this source contributed
	CreatedAt    time.Time `json:"created_at"`
}

RunCheckpoint tracks what entities were processed in a run

type RunHistoryEvent added in v0.5.0 

type RunHistoryEvent struct {
	RunID        string                 `json:"run_id"`
	JobNamespace string                 `json:"job_namespace"`
	JobName      string                 `json:"job_name"`
	EventType    string                 `json:"event_type"` // START, RUNNING, COMPLETE, FAIL, ABORT
	EventTime    time.Time              `json:"event_time"`
	RunFacets    map[string]interface{} `json:"run_facets,omitempty"`
	JobFacets    map[string]interface{} `json:"job_facets,omitempty"`
}

RunHistoryEvent represents a single run event (START, COMPLETE, FAIL, etc.)

type RunStatus added in v0.3.0 

type RunStatus string // @name RunStatus

const (
	StatusRunning   RunStatus = "running"
	StatusCompleted RunStatus = "completed"
	StatusFailed    RunStatus = "failed"
	StatusCancelled RunStatus = "cancelled"
)

type RunSummary added in v0.3.0 

type RunSummary struct {
	AssetsCreated      int `json:"assets_created"`
	AssetsUpdated      int `json:"assets_updated"`
	AssetsDeleted      int `json:"assets_deleted"`
	LineageCreated     int `json:"lineage_created"`
	LineageUpdated     int `json:"lineage_updated"`
	DocumentationAdded int `json:"documentation_added"`
	ErrorsCount        int `json:"errors_count"`
	TotalEntities      int `json:"total_entities"`
	DurationSeconds    int `json:"duration_seconds"`

} // @name RunSummary

RunSummary contains summary statistics for a run

type Source 

type Source interface {
	Validate(config RawPluginConfig) (RawPluginConfig, error)
	Discover(ctx context.Context, config RawPluginConfig) (*DiscoveryResult, error)
}

type SourceRun 

type SourceRun map[string]RawPluginConfig

SourceRun maps source names to their raw configurations

type Statistic added in v0.3.1 

type Statistic struct {
	AssetMRN   string  `json:"asset_mrn"`
	MetricName string  `json:"metric_name"`
	Value      float64 `json:"value"`
}

type TagsConfig 

type TagsConfig = pluginsdk.TagsConfig

TagsConfig, Filter, and ExternalLink are wire types shared with plugins. Marmot host code uses them as aliases of the SDK types so plugin authors and marmot always agree on the shape.

Source Files

View all Source files

Directories

Path Synopsis
Package install downloads external Marmot plugins from an OCI registry and caches them on disk, where the plugin loader picks them up alongside locally installed plugins.
 Package install downloads external Marmot plugins from an OCI registry and caches them on disk, where the plugin loader picks them up alongside locally installed plugins.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.