indy-ingest

Indy Ingest

Personal learning project for Go with Serverless AWS or Kubernetes

Provides an API for validation and storage of metadata/data with data storage S3 and state storage in DynamoDB.

Can be deployed to Kubernetes or serverless with AWS API Gateway and AWS Lambda.

Original Objective

• Create an efficient, generic metadata / data submission system that performs well on minimal infrastructure
• The system must run performantly on a 3-node Kubernetes cluster of Rasperry Pi 3s
• The system must be capable of running as a AWS serverless application at minimal cost by keeping to the free tier whenever possible

Setup and Build

Requires npm for serverless framework and Go 1.11 with module support.

Run ./scripts/setup.sh to resolve dependencies and build.

##Docker Build Run the make file

make

Build Docker image

docker build -f Dockerfile-arm -t codetaming/indy-ingest-arm .
docker build -f Dockerfile -t codetaming/indy-ingest .

Run (Local)

docker run --publish 9000:9000 -t codetaming/indy-ingest

Run (Pi)

docker run --publish 9000:9000 -t codetaming/indy-ingest-arm

Push to Docker Hub

docker login
docker push codetaming/indy-ingest-arm
docker push codetaming/indy-ingest

Uses

• Negroni : HTTP Middleware
• Gorilla Mux : URL router and dispatcher
• Envconfig : Configuration management
• JWT Middleware : Token based authentication