README
¶
Elasticsearch adaptor
The elasticsearch adaptor sends data to defined endpoints. List of supported versions is below.
| Version | Note |
|---|---|
| 1.X | This version does not support bulk operations and will thus be much slower. |
| 2.X | Will only receive bug fixes, please consider upgrading. |
| 5.X | Most recent and supported version. |
IMPORTANT
If you want to keep the source _id as the elasticsearch document _id, transporter will
automatically do this. If you wish to use the auto-generated _id field for elasticsearch but would
like to retain the originating _id from the source, you'll need to include a transform function
similar to the following (assumes MongoDB source):
module.exports = function(msg) {
msg.data["mongo_id"] = msg.data._id['$oid']
msg.data = _.omit(msg.data, ["_id"]);
return msg;
}
NOTE
By using the elasticsearch auto-generated _id, it is not currently possible for transporter to
process update/delete operations. Future work is planned in #39
to address this problem.
Configuration:
- es:
type: elasticsearch
uri: https://username:password@hostname:port/thisgetsignored
timeout: 10s # optional, defaults to 30s
aws_access_key: XXX # optional, used for signing requests to AWS Elasticsearch service
aws_access_secret: XXX # optional, used for signing requests to AWS Elasticsearch service
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type AWSTransport ¶
type AWSTransport struct {
Credentials awsauth.Credentials
// contains filtered or unexported fields
}
AWSTransport handles wrapping requests to AWS Elasticsearch service
type Config ¶
type Config struct {
URI string `json:"uri" doc:"the uri to connect to, in the form mongodb://user:password@host.com:27017/auth_database"`
Namespace string `json:"namespace" doc:"mongo namespace to read/write"`
Timeout string `json:"timeout" doc:"timeout for establishing connection, format must be parsable by time.ParseDuration and defaults to 10s"`
AWSAccessKeyID string `json:"aws_access_key" doc:"credentials for use with AWS Elasticsearch service"`
AWSAccessSecret string `json:"aws_access_secret" doc:"credentials for use with AWS Elasticsearch service"`
}
Config provides configuration options for an elasticsearch adaptor the notable difference between this and dbConfig is the presence of the Timeout option
type Elasticsearch ¶
type Elasticsearch struct {
// contains filtered or unexported fields
}
Elasticsearch is an adaptor to connect a pipeline to an elasticsearch cluster.
func (*Elasticsearch) Description ¶
func (e *Elasticsearch) Description() string
Description for the Elasticsearcb adaptor
func (*Elasticsearch) SampleConfig ¶
func (e *Elasticsearch) SampleConfig() string
SampleConfig for elasticsearch adaptor
func (*Elasticsearch) Start ¶
func (e *Elasticsearch) Start() error
Start the adaptor as a source (not implemented)
Source Files
Directories