esio

Elasticsearch Snapshot Index Orchestrator

RESTful API for orchestrating the restoration of time-based Elasticsearch snapshots.

This API gives you a simple interface for restoring snapshots based on a given time range.

For example, if snapshots are stored in the following daily (repo/snap/index) pattern: logs-%Y/logs-%Y-%m-%d/logs-v1-%Y-%m-%d then requests made to the POST /{start}/{end} route would trigger a restore of indices in the range that maps to the pattern.

High level of what the API does:

• Accepts requests to restore indices from snapshot repository.
• Incoming requests are queued for restore and queue is processed every 2 seconds.
• Requests for additional index restorations are queued while other recoveries are in progress.
• Available index snapshots are listed via the ES Snapshot/Restore API.
• Ongoing recoveries and online indices are obtained via the ES Index API.

OpenAPI Spec

The OpenAPI 2.0 spec is here: ./swagger.yml

Configuration

ESIO Flags:
      --es-host=         Elasticsearch Host [$ES_HOST]
      --max-restore=     Maximum number of indices allowed to restore at once, default is 1 [$MAX_RESTORE]
      --resolution=      Resolution of indices being restored (day, month, year) [$INDEX_RESOLUTION]
      --repo-pattern=    Snapshot repo pattern (repo/snap/index), ex: logs-%y/logs-%y-%m-%d/logs-v1-%y-%m-%d,
                         [$REPO_PATTERN]

Development

Use the make targets to build and test the esio-server.

• make: validate the swagger spec, fetch dependencies, compile the source and run the server in the foreground.
• make test: Start Elasticsearch via Docker in background, start server in background and run all tests
• make gen: Regenerate swagger framework from swagger.yml
• make init-test-data: Creates indices with faux timeseries data and snapshot repo in Elasticsearch for testing.
• make start-elastic: start Elasticsearch and Kibana
• make stop-elastic: stop Elasticsearch and Kibana

Resources

• go-swagger v0.7.4
• OpenAPI v2.0 Spec
• Snapshot/Restore API
• Golang Elastic V2 Library