README
¶
ML Datafeed Resource
This resource creates and manages Machine Learning datafeeds in Elasticsearch. Datafeeds retrieve data from Elasticsearch for analysis by an anomaly detection job. Each anomaly detection job can have only one associated datafeed.
Key Features
-
Complete API Coverage: Supports all ML datafeed API options including:
- Index patterns and queries for data retrieval
- Aggregations for time-based data summaries
- Chunking configuration for long time periods
- Delayed data check configuration
- Runtime mappings and script fields
- Frequency and query delay settings
- Scroll size optimization
- Custom headers support
-
Orchestration: Manages datafeed lifecycle including:
- Automatic stopping before updates (as required by Elasticsearch)
- Restarting after successful updates
- Proper cleanup on deletion
-
Validation: Comprehensive field validation including:
- Datafeed ID format validation (lowercase alphanumeric, hyphens, underscores)
- Duration format validation for frequency and query_delay
- JSON validation for complex objects
Implementation Details
This resource follows the Plugin Framework architecture and includes:
- Complete schema definition with all API fields
- Proper model conversion between API and Terraform types
- Comprehensive CRUD operations
- Extensive acceptance tests
- Import functionality for existing datafeeds
API References
Special Considerations
- Update Orchestration: Datafeeds must be stopped before updating and can be restarted afterward
- Job Association: Each datafeed is associated with exactly one anomaly detection job
- Index Permissions: Requires read permissions on the source indices
- ML Privileges: Requires
manage_mlcluster privilege
Documentation
¶
Index ¶
- func GetChunkingConfigAttrTypes() map[string]attr.Type
- func GetDelayedDataCheckConfigAttrTypes() map[string]attr.Type
- func GetIndicesOptionsAttrTypes() map[string]attr.Type
- func GetSchema() schema.Schema
- func NewDatafeedResource() resource.Resource
- func WaitForDatafeedState(ctx context.Context, client *clients.ApiClient, datafeedId string, ...) (bool, diag.Diagnostics)
- type ChunkingConfig
- type Datafeed
- type DelayedDataCheckConfig
- type IndicesOptions
- type State
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func GetChunkingConfigAttrTypes ¶
GetChunkingConfigAttrTypes returns the attribute types for chunking_config
func GetDelayedDataCheckConfigAttrTypes ¶
GetDelayedDataCheckConfigAttrTypes returns the attribute types for delayed_data_check_config
func GetIndicesOptionsAttrTypes ¶
GetIndicesOptionsAttrTypes returns the attribute types for indices_options
func NewDatafeedResource ¶
Types ¶
type ChunkingConfig ¶
type ChunkingConfig struct {
Mode types.String `tfsdk:"mode"`
TimeSpan types.String `tfsdk:"time_span"`
}
ChunkingConfig represents the chunking configuration
type Datafeed ¶
type Datafeed struct {
ID types.String `tfsdk:"id"`
ElasticsearchConnection types.List `tfsdk:"elasticsearch_connection"`
DatafeedID types.String `tfsdk:"datafeed_id"`
JobID types.String `tfsdk:"job_id"`
Indices types.List `tfsdk:"indices"`
Query jsontypes.Normalized `tfsdk:"query"`
Aggregations jsontypes.Normalized `tfsdk:"aggregations"`
ScriptFields customtypes.JSONWithDefaultsValue[map[string]any] `tfsdk:"script_fields"`
RuntimeMappings jsontypes.Normalized `tfsdk:"runtime_mappings"`
ScrollSize types.Int64 `tfsdk:"scroll_size"`
ChunkingConfig types.Object `tfsdk:"chunking_config"`
Frequency types.String `tfsdk:"frequency"`
QueryDelay types.String `tfsdk:"query_delay"`
DelayedDataCheckConfig types.Object `tfsdk:"delayed_data_check_config"`
MaxEmptySearches types.Int64 `tfsdk:"max_empty_searches"`
IndicesOptions types.Object `tfsdk:"indices_options"`
}
Datafeed represents the Terraform resource model for ML datafeeds
func (*Datafeed) FromAPIModel ¶
FromAPIModel populates the Terraform model from an API model
func (*Datafeed) ToAPIModel ¶
ToAPIModel converts the Terraform model to an API model for creating/updating
type DelayedDataCheckConfig ¶
type DelayedDataCheckConfig struct {
Enabled types.Bool `tfsdk:"enabled"`
CheckWindow types.String `tfsdk:"check_window"`
}
DelayedDataCheckConfig represents the delayed data check configuration
type IndicesOptions ¶
type IndicesOptions struct {
ExpandWildcards types.List `tfsdk:"expand_wildcards"`
AllowNoIndices types.Bool `tfsdk:"allow_no_indices"`
IgnoreThrottled types.Bool `tfsdk:"ignore_throttled"`
}
IndicesOptions represents the indices options for search
Source Files