Posts

Update dynamodb items programatically

Today, I wanted to update all dynamodb items and couldn't find a handy example in AWS documentation.  Below is a handy script that I created to update all the dynamodb table rows with current time stamp using python from __future__ import print_function # Python 2/3 compatibility import boto3 import json import decimal from boto3.dynamodb.conditions import Key, Attr import datetime, time row = 0 table_name = 'table' dynamodb = boto3.resource('dynamodb') table = dynamodb.Table(table_name) # Helper class to convert a DynamoDB item to JSON. class DecimalEncoder(json.JSONEncoder): def default(self, o): if isinstance(o, decimal.Decimal): if o % 1 > 0: return float(o) else: return int(o) return super(DecimalEncoder, self).default(o) def updateItem(item): global row response = table.update_item( Key=item, UpdateExpression='SET #s = :updated', Expre
Post a Comment
Read more

Tracing Requests in AWS Serverless Applications

Image
Operating Serverless APIs using Amazon API Gateway, AWS Lambda and Microservices (running in ECS) can be bit challenging if logging is not given any forethought. Best practice is to decide in while designing the application what needs to be logged, how the log messages can be correlated and finally where is it should be sent for aggregation.  Its difficult to reconcile log events for a serverless API sent across multiple layers of application stack using CloudWatch log groups and log streams. Tracking down logs for a specific request or tailing request logs for a specific request can sometimes be a cumbersome experience.  Debugging for a specific issue or Tracing a specific request can be made easy by using the x-amzn-requestid and x-amzn-trace-id that comes for free when using AWS API Gateway. Its easy to leverage these Ids and keep track of your request.   In the following deployment diagram  How to capture the request ID for a particular API request: When the request
9 comments
Read more

Leveraging AWS Elasticsearch

Image
Recently I had an use case where I had to expose AWS DynamoDB search as an API.  I could have gone with the common approach of using AWS API Gateway integrating with AWS Lambda which runs the search against the  AWS DynamoDB  as   however this approach wouldn't be scalable under load.  Alternative to the above solution would be to use AWS  Elasticsearch Service .  ElasticSearch  is an open source product from Elastic that’s is designed to help us search in a way that is highly available and abstracted from datastore.  AWS  Elasticsearch  Service  is AWS hosted ElasticSearch that takes care of set-up and management of the back end server and provides us with an endpoint that we can get developing with.  AWS Elasticsearch Service makes it easy to deploy, secure, operate, and scale Elasticsearch for log analytics, full text search, application monitoring, and more.  AWS  Elasticsearch Service is a fully managed service that delivers Elasticsearch’s easy-to-use APIs and
2 comments
Read more

nginx on Docker

nginx [engine x] is an HTTP and reverse proxy server, a mail proxy server, and a generic TCP/UDP proxy server Docker  is an open platform for developers and sysadmins to build, ship, and run distributed applications, whether on laptops, data center VMs, or the cloud I’m going to do a quick run thru of building a web server that returns ‘hello world’ as html and json, using Nginx and Docker under the covers Make sure you have docker & nginx installed before proceeding further Run the following commands to set up files & folders  mkdir hello_world_docker_nginx cd hello_world_docker_nginx touch Dockerfile docker-compose.yml mkdir src conf touch src/index.html conf/nginx.conf After running the above commands, file structure would look like ├── Dockerfile ├── conf │   └── nginx.conf ├── docker-compose.yml └── src └── index.html copy the below contents in the mentioned files index.html : <!DOCTYPE html> <html> <head> <met
1 comment
Read more

What the hell is nodejs call back ?

Image
Callback is an asynchronous function. A callback function is called at the completion of a given task. Node makes heavy use of callbacks. All the APIs of Node are written in such a way that they support callbacks. Usually there are two way to write node functions - Blocking Code & Non-Blocking Code. Both of them have their own use cases. Non-Blocking code means the functions are executed parellelly (asynchronously) which is preferred when performance is a key factor (for example think of UI, where widgets gets loaded parellelly). Node scripts that use callbacks is hard to get right intuitively.  A lot of code ends up entagled and becomes hard to understand, debug. Typical sequential function using callback would look like same function when you want to execute parellely would look like Above scripts are example of creating your own functions however there are npm libraries https://www.npmjs.com/package/async
1 comment
Read more

Validate Cognito tokens in Kong

Assuming Kong environment is set up and operating as expected, this blog helps to Validate Cognito tokens in Kong.  JWT token issued by popular  identity  solutions such as Auth0 , Amazon Cognito etc., can be easily Authorized by kong.  Here are the steps to validate JWT token issued by Auth0 in Kong.  In this blog, I am going to focus on how to validate JWT token issued by Amazon Cognito.  Assuming that Amazon Cognito user pools are set up and operating as expected.  Every  successful authentication of a user, Amazon Cognito issues  ID Token Access Token ID token is represented as a JSON Web Key Token (JWT).  The token contains claims about the identity of the authenticated user. For example, it includes claims such as  name ,  family_name ,  phone_number , etc., For more information about standard claims, see the  OpenID Connect specification . Extract the public key from Amazon Cognito public endpoint using   https://cognito-idp.{region}.amazonaws.com/{userPoolId}/
3 comments
Location: Melbourne VIC, Australia
Read more