Skip to content

Obviate.io

To anticipate and prevent

  • Home
  • About Us
  • History
  • Privacy Policy
  • Toggle search form

Tutorial: AWS API Gateway to Lambda to DynamoDB

Posted on 2015-08-05 By Jon No Comments on Tutorial: AWS API Gateway to Lambda to DynamoDB

AmazonWebservices_Logo.svg
AmazonWebservices_Logo.svg

After last week’s Internet Of Things hack session, I became fascinated with all the fun IoT projects and technologies there are to play with. Everything you do, requires data to go into the cloud, so I figured I’d start there. In the hack series I learned how to push data into AWS Kinesis which is amazing, however it has its disadvantages. On the flip side there is the new AWS API Gateway which I also haven’t spent much time on, so that’s a perfect thing to learn. After some tinkering, I found that while AWS API Gateway -> Lambda -> DynamoDB might sound complicated, it’s actually quite easy.

Before we get started, why API over Kinesis? First, Kinesis currently costs $0.015 per hour, per shard. You need a minimum of 1 shard, so that’s a minimum of about $11 a month. If you’re sucking in data fairly constantly, this is well worth it. However if you’re tinkering around with a test IoT device, why pay money that you don’t have to? On the flip side API Gateway is charged per million API calls. Secondarily, I like working with APIs of my own design, rather than Amazon designed APIs. It means I can build lightweight interfaces on more platforms.

We’re going to start at the “bottom” and work our way upwards, so first DynamoDB.

Step #1 — Create a new table.
Primary Key = Hash & Range
Hash Name = device (Type string)
Range Name = timedate (Type number)
Screen Shot 2015-08-04 at 6.58.15 PM
Step #2 — Skip the global indexes.
We don’t need them for our small test.
Screen Shot 2015-08-04 at 7.04.31 PM
Step #3 — Provision Throughput.
I went with 5 read, 2 write. This costs ~$1.46 a month.
Screen Shot 2015-08-04 at 7.04.44 PM
Step #4 — Turn on alarms Screen Shot 2015-08-04 at 7.07.04 PM
Step #5 — Create! Screen Shot 2015-08-04 at 7.07.10 PM

With that DynamoDB is done and will create itself shortly. Since it’s a NoSQL database, we don’t need to worry about the data going into it at all. So the next step is creating our Lambda function which will take the API call data and push it into DynamoDB.

Step #1 — Create a new Lambda function.
Skip the “blueprint”
Name = OurBlogLambda
Runtime = Node.JS
Get the code from this gist.
Screen Shot 2015-08-04 at 7.20.41 PM
Step #2 — The rest of the Lambda answers.
Handler = index.handler (that’s the default)
Role = Select “Basic with DynamoDB”, this will spawn a new window to walk you through and create a new IAM role. When you’re done, you should have a role named something like “lambda_dynamo”.
Leave the advanced settings alone.
Screen Shot 2015-08-04 at 7.25.17 PM
Step #3 — Review and create Lambda function. Screen Shot 2015-08-04 at 7.27.30 PM
Step #4 — Test.
Click the test button and paste in the following JSON:

{
  "device": "test",
  "lat": "-1.1",
  "lon": "2.2"
}
Screen Shot 2015-08-04 at 7.35.59 PM
Step #5 — If everything went well, you’ll have a “SUCCESS”
If you want, you can also go check DynamoDB and you should see a record in your database now.
Screen Shot 2015-08-04 at 7.36.40 PM

With that, you’ve completed the Lambda setup process. As you should have been able to figure out, our data of choice to be submitted is latitude & longitude. You can very easily change this Lambda function to be any sort of data type and/or names. You can also add/subtract the number of items being submitted to Dynamo, in this function, at any time. So with everything working, now we move on to the API Gateway.

Step #1 — Create a new API. Screen Shot 2015-08-04 at 7.45.40 PM
Step #2 — Create a new Resource.
We’re going to name this “submitData” since it’s taking data in (to Lambda). Later you could create more resources for getting data out.

Update 2016-02-29 Per the comments it looks like this step is no longer required.

Screen Shot 2015-08-05 at 11.56.41 AM
Step #3 — Create a method.
We’re submitting data, so POST makes the most sense. Make sure you were still selecting your new “/submitdata” resource.
Screen Shot 2015-08-05 at 11.58.06 AM
Step #4 — Setup the POST to go to Lambda.
Click “POST” under “/submitdata” to the left. We’re using a Lambda function and if you start to type the name, it should auto-fill for you.
Screen Shot 2015-08-05 at 11.59.46 AM
Step #5 — Save & Accept permission change.
When you go to save your changes, AWS will confirm you want to give API Gateway permission to invoke Lambda.
Screen Shot 2015-08-05 at 12.01.22 PM
Step #6 — Confirm everything looks correct.
Your “/submitdata — POST — Method Execution” should look almost identical to my screenshot.
Screen Shot 2015-08-05 at 12.02.38 PM
Step #7 — Test it!
Click the test button and put the following code into the “Request body”:

{
  "device": "apitest",
  "lat": "-10.01",
  "lon": "20.02"
}

You should see a “SUCCESS” in the response body. You can also look at DynamoDB to confirm your row is in the database.

Screen Shot 2015-08-05 at 12.06.03 PM
Step #8 — Deploy API.
Click “Deploy API”.
Deployment Stage = New Stage
Stage name = testing
Screen Shot 2015-08-05 at 12.07.08 PM
Step #9 — Get the URL.
If everything saved previously you should now see ” Invoke URL: https://RANDOMVALUEHERE.execute-api.us-west-2.amazonaws.com/testing “. You’re almost done!
Screen Shot 2015-08-05 at 12.10.04 PM
Step #10 — Curl Test
On your command line try:
curl -H "Content-Type: application/json" -X POST -d "{\"device\":\"curl\",\"lat\":\"12.34\",\"lon\":\"45.67\"}" https://CHANGEME.execute-api.us-west-2.amazonaws.com/testing/submitdata
(make sure that the URL matches that which you got in the previous step).

You should see “SUCCESS”, and be able to see your data in DynamoDB.

Screen Shot 2015-08-05 at 12.13.37 PM

reinvent_launch-page_illustration_lambda
reinvent_launch-page_illustration_lambda

Congratulations, you’re all done! You can now accept data POST’ed to a REST API, fed into Lambda, and stored in DynamoDB. One caveat for you IoT people reading this tutorial though, AWS API Gateway is HTTPS ONLY. Unfortunately it doesn’t not operate on regular unencrypted HTTP. So if you’re trying to submit data directly from an Arduino, you’re out of luck. You’ll need to use a headend with more CPU power to submit to API Gateway.

Now take this skeleton and get fancy with it. Are you accepting data in one format (ex: Degrees centigrade) and need it in another (ex: Fahrenheit)? Rather than doing that in your device, Lambda can do the conversion before storage. In fact, Lambda can do a ton more (e.g. store data in S3, Launch EC2 instances, fire off messages in SNS), it can do basically anything the AWS SDK for NodeJS can do.

AWS, Code, IoT Tags:AWS, AWS API Gateway, AWS DynamoDB, AWS Kinesis, AWS Lambda, css, curl, education, internet of things, IoT, long post, nodejs, REST API, tutorial

Post navigation

Previous Post: AWS IoT Hack Series with MediaTek LinkIt One
Next Post: Lost some files in AWS Glacier? I found ’em

More Related Articles

Installing nginx 1.9.7 on Ubuntu 15.04 / 15.10 Tech
Hooray! For Today is IPv6 Launch Day! IPv6
Now AWS Certified Solutions Architect – Professional! AWS

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

August 2015
S M T W T F S
 1
2345678
9101112131415
16171819202122
23242526272829
3031  
« Jul   Sep »

amazon Android anime apache apple arduino asus averatec AWS Bad Company 2 cloud DIY eee fanime fanimecon github google iPad iphone IPv6 javascript kindle linkedin linux macbook air mass effect nodejs openvpn osx php review San Francisco security tutorial Ubuntu urban garden usb video wifi windows 7 winter urban garden wordpress xbox xbox 360 XboxLIVE

Copyright © 2022 Obviate.io

Powered by PressBook Premium theme