sparta

Sparta

Overview

Sparta takes a set of golang functions and automatically provisions them in AWS Lambda as a logical unit.

Functions must implement

type LambdaFunction func(*json.RawMessage,
                          *LambdaContext,
                          http.ResponseWriter,
                          *logrus.Logger)

where

• json.RawMessage : The arbitrary json.RawMessage event data provided to the function.
• LambdaContext : golang compatible representation of the AWS Lambda Context
• http.ResponseWriter : Writer for response. The HTTP status code & response body is translated to a pass/fail result provided to the context.done() handler.
• logrus.Logger : logrus logger with JSON output. See an example for including JSON fields.

Given a set of registered golang functions, Sparta will:

• Either verify or provision the defined IAM roles
• Build a deployable application via Provision()
• Zip the contents and associated JS proxying logic
• Dynamically create a CloudFormation template to either create or update the service state.
• Optionally:
  • Register with S3 and SNS for push source configuration
  • Provision an API Gateway service to make your functions publicly available

Note that Lambda updates may be performed with no interruption in service.

Sample Lambda Application

1. Create application.go :

```go
package main

import (
  "encoding/json"
  "fmt"
  "github.com/Sirupsen/logrus"
  sparta "github.com/mweagle/Sparta"
  "net/http"
)

func echoEvent(event *sparta.LambdaEvent,
               context *sparta.LambdaContext,
               w http.ResponseWriter,
               logger *logrus.Logger) {

  logger.WithFields(logrus.Fields{
    "RequestID": context.AWSRequestID,
  }).Info("Request received")

  eventData, err := json.Marshal(*event)
  if err != nil {
    logger.Error("Failed to marshal event data: ", err.Error())
    http.Error(*w, err.Error(), http.StatusInternalServerError)
  }
  logger.Info("Event data: ", string(eventData))
}

func main() {
  var lambdaFunctions []*sparta.LambdaAWSInfo

  lambdaEcho := sparta.NewLambda(sparta.IAMRoleDefinition{},
                                  echoEvent,
                                  nil)
  lambdaFunctions = append(lambdaFunctions, lambdaEcho)
  sparta.Main("SpartaEcho",
               "This is a sample Sparta application",
               lambdaFunctions)
}
```

1. go get ./...
2. go run application.go provision --s3Bucket MY_S3_BUCKET_NAME
   • You'll need to change MY_S3_BUCKET_NAME to an accessible S3 bucketname
3. Visit the AWS Lambda console and confirm your Lambda function is accessible

See also the Sparta Application for an example.

Examples - Advanced

The []sparta.LambdaAWSInfo.Permissions slice allows Lambda functions to automatically manage remote event source subscriptions. Push-based event sources are updated via CustomResources that are injected into the CloudFormation template if appropriate.

Examples:

• S3 Subscriber
• SNS Subscriber

The per-service API logic is inline NodeJS ZipFile code. See the provision directory for more.

See also the Sparta Application for a standalone example.

Examples - API Gateway (Preliminary)

It's possible to expose to your golang functions over HTTPS by associating an API Gateway. To enable API Gateway support, you must:

• Define a Stage
• Define an API Gateway name and provide the previously defined stage.
• Create one or more (resource, httpMethod) pairs that are bound to your lambda function.

Example:

package main

import (
	"encoding/json"
	"fmt"
	"net/http"

	"github.com/Sirupsen/logrus"
	sparta "github.com/mweagle/Sparta"
)

////////////////////////////////////////////////////////////////////////////////
// Echo handler
//
func echoEvent(event *json.RawMessage,
                context *sparta.LambdaContext,
                w http.ResponseWriter,
                logger *logrus.Logger) {
	logger.WithFields(logrus.Fields{
		"RequestID": context.AWSRequestID,
		"Event":     string(*event),
	}).Info("Request received")

	fmt.Fprintf(*w, "Hello World!")
}

func main() {
	stage := sparta.NewStage("test")
	apiGateway := sparta.NewAPIGateway("MySpartaAPI", stage)

	var lambdaFunctions []*sparta.LambdaAWSInfo
	lambdaFn := sparta.NewLambda(sparta.IAMRoleDefinition{}, echoEvent, nil)
	apiGatewayResource, _ := apiGateway.NewResource("/hello/echo", lambdaFn)
	apiGatewayResource.NewMethod("GET")

	lambdaFunctions = append(lambdaFunctions, lambdaFn)
	sparta.Main("SampleApplication",
		"Sample application with API Gateway support",
		lambdaFunctions,
		apiGateway)
}

This API can be deployed via:

go run api.go --level debug provision --s3Bucket $MY_S3_BUCKET_NAME

The provisioning log will output the AWS-assigned API Gateway URL as in:

...
Outputs: [{
    Description: "API Gateway URL",
    OutputKey: "URL",
    OutputValue: "https://vpv0e9nv83.execute-api.us-west-2.amazonaws.com/test"
  }],
...

You can then access a specific resource by appending the path component to the OutputValue of the URL value as in:

$ curl -vs https://jhn4bubx7h.execute-api.us-west-2.amazonaws.com/test/hello/echo

*   Trying 54.230.147.237...
* Connected to jhn4bubx7h.execute-api.us-west-2.amazonaws.com (54.230.147.237) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: *.execute-api.us-west-2.amazonaws.com
* Server certificate: Symantec Class 3 Secure Server CA - G4
* Server certificate: VeriSign Class 3 Public Primary Certification Authority - G5
> GET /test/hello/echo HTTP/1.1
> Host: jhn4bubx7h.execute-api.us-west-2.amazonaws.com
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Content-Length: 14
< Connection: keep-alive
< Date: Mon, 23 Nov 2015 17:02:01 GMT
< x-amzn-RequestId: e968e0f1-9203-11e5-9134-61048221e24a
< X-Cache: Miss from cloudfront
< Via: 1.1 5687015cb50d88319b87aae0ee898267.cloudfront.net (CloudFront)
< X-Amz-Cf-Id: lxP1CTKaV5ArYkfZCNJeWnUaF-63AwCM3-puXo315d_HwSvdhz7ibQ==
<
* Connection #0 to host jhn4bubx7h.execute-api.us-west-2.amazonaws.com left intact
"Hello World!"

NOTE: Providing nil as the Stage argument to sparta.NewAPIGateway() will provision an API instance, but will not deploy it.

Prerequisites

1. Verify your golang SDK credentials are properly configured
2. If referring to pre-existing IAM Roles, verify that the Lambda IAM Permissions are properly configured and that the correct IAM RoleName is provided to sparta.NewLambda()
   • More information on the Lambda permission model is available here

Lambda Flow Graph

It's also possible to generate a visual representation of your Lambda connections via the describe command line argument.

go run application.go describe --out ./graph.html && open ./graph.html

Additional documentation

View the latest versions at GoDoc or run make docs in the source directory & visit http://localhost:8090.

Caveats

1. golang isn't officially supported by AWS (yet) - But, you can vote to make golang officially supported. - Because of this, there is a per-container initialization cost of:
   • Copying the embedded binary to /tmp
   • Changing the binary permissions
   • Launching it from the new location
   • See the AWS Forum for more background - Depending on container reuse, this initialization penalty (~700ms) may prove burdensome. - See the JAWS project for a pure NodeJS alternative. - See the PAWS project for a pure Python alternative.
2. There are Lambda Limits that may affect your development

Outstanding

• Eliminate NodeJS CustomResources
• Support API Gateway updates
  • Currently API reprovisioning is done by delete => create
• Optimize CONSTANTS.go for deployed binary
• Implement APIGateway graph
• Support APIGateway inline Model definition
• Support custom domains