Load Balancing and Zero Downtime Update for the Digital Card Store on Amazon Web Services

In this post, I will show how to use AWS Elastic Load Balancing (ELB) and AWS Auto Scaling Groups (ASG) to provide load balancing and zero downtime update for our digital card store. 

In my last post, I have written about running a digital card store prototype on Amazon Web Services (AWS).  The application was built with Java Spring Boot framework. In the post, I have created one EC2 instance to run the application by using S3 and EC2 services.

For a prototype, runnning the application in one EC2 instance may be sufficient to show how it works. However, in a real world scenario, we should consider the disadvantages of running only one instance.

Disadvantages of Running One Instance

First let's review the mechanism:

1. We run one EC2 instance

2. We use the public ip address of the instance to access

Now, let's think about the disadvantages of this mechanism.

What if the instance is stopped by accident. Our application will not be available. Until we start the same instance or an another instance, users can not use the application.

Also we may stop the instance to update the application. During the update process, the application won't be servicing the requests.

We should consider another condition. If our application usage increases, one instance will not be sufficient to process all the requests efficiently. Requests will start to be delayed after some level and finally the application won't accept new connections anymore.

Considering the second point, the use of the public ip address (or the public host name) of the instance will be a problem when we launch another instance. The address of the new instance will be different. In this case we should notify our users about the address change. Again this is not a suitable solution for production usage.

Advantages of Running Multiple Instances

To address all these problems we should run multiple instances of the application. The mechanism is described below:

1. We run multiple EC2 instances

2. We use a load balancer to distribute the requests to the instances

3. We use the address of the load balancer to access the application

In this way:

1. The load is distributed to the instances, so each instance process only a part of the load.

2. The application will be available when one instance is shut down by accident or for update purposes.

3. Users are not affected by instance address changes because they are using the load balancer address, not the address of a specific instance.

Elastic Load Balancing and Auto Scaling Groups on AWS

In AWS, we use Elastic Load Balancing (ELB) service as load balancer. When we create an ELB, a public domain name is generated for the ELB. We use this ELB address to access to the application behind the ELB.

AWS provides auto scaling by Auto Scaling Groups (ASG). ASG enables us to dynamically scale your EC2 instances out or scale in according to some condition. For example, if cpu usage is over 90% for 5 minutes ASG can scale out by launching a new instance, and scale in by terminating one instance if cpu usage is below 10% for 10 minutes. AWS ASG enables us to define this conditions by AWS console or AWS CLI tools.

In addition to dynamic scaling, ASG provides manual scaling. This is useful for the situations that are specific to your account. For example, if your application is known to get less traffic from 6PM to 9AM and at weekends, you can make significant cost savings by reducing the instances with a script.

Providing Load Balancing And Zero Downtime Update

For this post, I will create an ELB and an ASG to provide load balancing and availability during the update process. This way we can update our application with zero downtime. Please note that while we can get this functionality with AWS CodeDeploy,  I especially use AWS Java SDK to access AWS ELB, ASG and EC2 services in this post.

I will use manual scaling feature of ASG to launch additional instances with the new application version and to close the old instances with the old application version. The process is below:

1. Change the application

2. Build the application

3. Upload the application to the S3 bucket

4. Create a new instance in ASG to get updated application

5. Close the instances that run the old application

Every ASG has min, max and desired instance count. For this post, I will create an ASG with min instance count and desired instance count is set to 2. Max instance count will be 4. With this settings, our application will run with 2 instance in normal time. Because the min instance count is set to 2, if one instance is terminated accidentally, ASG will launch a new instance and keep instance count at 2 and we will be sure that enough instance is running every time. Also ELB will provide load balancing by distributing the requests to instances in a round-robin fashion.

By setting the max instance to 4, we leave a room for update processes. During the application process, we will launch two new instance with the new version and instance count will increase to 4. This way our application will be available during the entire update process. After the update, instance count will decrease to 2 again.

The steps for the entire update process are below:

1. Application is running 2 instances before the update.

2. We launch 2 new instances by setting desired instance count to 4.

The instance count will increase to 4 and new instances will get the new application.

3. We wait for ELB to register new instances.

This is to make sure that before destroying old instances, ELB can use the new instances. The new instances is not registered very quickly. ELB does a few health checks to ensure they are started to serving requests.

4. Deregister old instances from ELB.

Before terminating old instances we should remove the old instances from ELB. Otherwise, ELB will try to use this instances and a few requests can get errors before unhealthy threshold ends.     

5. We set the desired instance count to 2 again to close the oldest instances.

While creating our ASG we specify termination policy as OldestIntance especially to ensure the new instances are not terminated during scaling in.

6. Now we have 2 instances that run the new application.

To follow and understand the process easily, I have built a Java Spring Boot application that show the application version and public ip address of the instance that process the request. Using the application we will see the ELB is dispatching the requests to different instances and we will notice the application versions. You can clone the application here.

Prerequisites

1. An AWS account

2. AWS CLI Tools

3. Git

4. Maven

5. Key pair, Security Group, IAM instance role and S3 bucket.

In my first post, I showed how to register and install AWS CLI Tools. I will use the key pair, security group, IAM instance role and S3 bucket that I have created in my previous post.

The steps to deploy and update the application are shown below.

1. Prepare the application

2. Create the ELB

3. Create Launch Configuration

4. Create the ASG to launch the application and see load balancing works

5. Change the application

6. Do zero downtime update

Let's start.

1. Prepare the application

Execute the git clone command to clone the application.

$ git clone https://github.com/ceyhunozgun/cardstore2

Package the application using the commands below.

$ cd cardstore2

$ mvn package

Maven will create the WAR file in the target directory.

$ ls target/*.war

target/cardstore-0.0.1-SNAPSHOT.war

To put the WAR to the S3 bucket, execute the following command. Replace cardstoredeploy with your S3 bucket name.

$ aws s3 cp target/cardstore-0.0.1-SNAPSHOT.war s3://cardstoredeploy/

upload: target\cardstore-0.0.1-SNAPSHOT.war to s3://cardstoredeploy/cardstore-0.0.1-SNAPSHOT.war

Now you should see the WAR in AWS S3 Console.

2. Create the ELB

Create the load balancer by executing the command below. Don't forget to change the availability zone and security group accordingly.

$ aws elb create-load-balancer --load-balancer-name CardStoreLB --listeners "Protocol=HTTP,LoadBalancerPort=8080,InstanceProtocol=HTTP,InstancePort=8080" --security-groups sg-8567a2ee --availability-zones eu-central-1a

{

    "DNSName": "CardStoreLB-XXXXX.eu-central-1.elb.amazonaws.com"

}

Note the created DNS name to access the application later. We should change the default health check settings for ELB to register new instances quickly.

$ aws elb configure-health-check --load-balancer-name CardStoreLB --health-check Target=HTTP:8080/,Interval=5,UnhealthyThreshold=2,HealthyThreshold=2,Timeout=2

3. Create the Launch Configuration

Create the launch configuration by executing the command below. The launch configuration will be used as a template by ASG when creating new instances. Don't forget to change the S3 bucket name in init script. Also change the security group parameter accourdingly.

$ aws autoscaling create-launch-configuration --launch-configuration-name CardStoreLC --key-name CardStoreKP --image-id ami-af0fc0c0 --instance-type t2.micro --user-data file://cardstore_ec2_init_script.txt --security-groups sg-8567a2ee  --iam-instance-profile S3-Admin-Role

4. Create the ASG to launch the application and see load balancing works

Create the auto scaling group by executing the command below. Note the --termination-policies "OldestInstance"  parameter. OldestInstance value is required to be sure that the instances with the new version of application is not terminated during scaling in. Don't forget to change the availability zone accordingly.

$ aws autoscaling create-auto-scaling-group --auto-scaling-group-name CardStoreASG --launch-configuration-name CardStoreLC --load-balancer-names CardStoreLB --min-size 2 --max-size 4 --termination-policies "OldestInstance"  --availability-zones eu-central-1a

After creating the the auto scaling group, ASG will launch two EC2 instances and register them to ELB. It will take some time to register the instances to ELB. After the instances are registered you can access the application at ELB DNS address.

You should see the home page of the application that shows the app version as V1 and the public ip address of the instance that process the request. Refresh the browser several times and you should see that ip address changes at each request. This shows that your ELB works.

Also you can execute the script below to see IP address changes. Remember to change ELB address. As you can see from the output requests are distributed to instances.

$ while true; do curl -s CardStoreLB-XXXX.eu-central-1.elb.amazonaws.com:8080|grep V; curl -s CardStoreLB-XXXX.eu-central-1.elb.amazonaws.com:8080|grep V; curl -s CardStoreLB-XXXX.eu-central-1.elb.amazonaws.com:8080|grep V; curl -s CardStoreLB-XXXX.eu-central-1.elb.amazonaws.com:8080|grep V; sleep 3; echo ""; done

        ##V1 35.158.36.111##

        ##V1 35.158.25.0##

        ##V1 35.158.36.111##

        ##V1 35.158.25.0##

        ##V1 35.158.36.111##

        ##V1 35.158.25.0##

        ##V1 35.158.36.111##

        ##V1 35.158.25.0##

5. Change the application

Open the IndexController.java source file using the command below and change the application version from "V1" to "V2".

$ notepad src/main/java/com/cardstore/controller/IndexController.java

6. Do zero downtime update

Now we are ready to build the applicaton, upload the WAR file to S3 and doing zero downtime upgrade by running the command below. Don't forget to replace the S3 bucket name in the buildAndUpdateApp.bat file.

$ start buildAndUpdateApp.bat

After the WAR file uploaded to S3 we can watch the progress by executing a command like below. At start you should see only the V1 in the output. After a while you should see V2 also, showing that new instances are launched and registered with ELB. And later you should only see V2, showing that old instances are terminated. Please replace ELB DNS name with your ELB's DNS name.

$ while true; do curl -s CardStoreLB-XXXXX.eu-central-1.elb.amazonaws.com:8080|grep V; curl -s CardStoreLB-XXXXX.eu-central-1.elb.amazonaws.com:8080|grep V; curl -s CardStoreLB-XXXXX.eu-central-1.elb.amazonaws.com:8080|grep V; curl -s CardStoreLB-XXXXX.eu-central-1.elb.amazonaws.com:8080|grep V; sleep 3; echo ""; done

        ##V1 35.158.36.111##

        ##V1 35.158.25.0##

        ##V1 35.158.36.111##

        ##V1 35.158.25.0##

        ##V1 35.158.36.111##

        ##V1 35.158.25.0##

        ##V1 35.158.36.111##

        ##V1 35.158.25.0##

...

        ##V1 35.158.36.111##

        ##V1 35.158.25.0##

        ##V1 35.158.36.111##

        ##V2 35.158.36.181##

        ##V1 35.158.25.0##

        ##V1 35.158.36.111##

        ##V2 35.158.36.181##

        ##V1 35.158.25.0##

        ##V2 35.158.36.181##

        ##V1 35.158.25.0##

        ##V1 35.158.36.111##

        ##V2 35.158.36.181##

        ##V1 35.158.25.0##

        ##V1 35.158.36.111##

        ##V2 35.158.36.181##

        ##V2 35.158.35.126##

...

        ##V2 35.158.36.181##

        ##V2 35.158.35.126##

        ##V2 35.158.36.181##

        ##V2 35.158.35.126##

        ##V2 35.158.36.181##

        ##V2 35.158.35.126##

        ##V2 35.158.36.181##

        ##V2 35.158.35.126##

The zero downtime update is done with my awsupdateasg application. This application is a Java program that use AWS Java SDK.

The output of awsupdateasg should look like below. Output show the instance id, instance state, instance health status, ASG state, public ip address and ELB state. At first new instances is not registered in ELB and their ELB state is shown as null.

A few seconds later, their state should turn to OutOfService showing that ELB is doing health checks before registering them.

Then their ELB state should turn to InService. At this point all 4 instances are in use.

And finally awsupdateasg deregisters old instances. Their state turns to OutOfService. And finally old intances are terminated as shown below.

Summary

In this post, I have used a Java program I developed, which uses AWS Java SDK to create a zero downtime update mechanism. You can access the code here. The program uses AWS EC2, ELB and ASG APIs.

As I use different AWS services in the next few weeks, I will try to write a blog about their usages.

As always, please remember to delete all the resources you have created to avoid any charges.

Running a digital card store on AWS

In my previous post, I've written about registering an AWS Free Tier account and about the things you should beware of.

As I promised in my previous post, I will show how to run a simple Java Spring Boot application on AWS.

As an example, I have developed a digital card store application using Spring Boot framework. Digital trade cards are the digital versions of classic trade cards. There are so popular digital cards that it is hard to believe a simple JPG file worth $225, $440 and even $1524.

So our application is a prototype digital card store that manages cards. Source code for the application can be found here. In the future posts, I will add various features that use different AWS services.

The prototype allows you  to add a new card, list existing cards and get details of a specific card using REST requests and responses.

Prerequisites

1. An AWS account

2. AWS CLI Tools

3. Git

4. Maven

In my previous post, I showed how to register and install AWS CLI Tools.

Steps

The steps to build and run the application are below.

1. Clone the application from GitHub

2. Build and test the application

3. Put the Spring Boot fat jar to S3.

4. Launch a new EC2 instance that pulls the jar from S3 and starts the application.

Let's start.

1. Clone the application from GitHub

Execute the git clone command to clone the application.

git clone https://github.com/ceyhunozgun/cardstore1

2. Build and test the application

cd cardstore1

mvn package

mvn spring-boot:run

After the application starts, direct your browser to http://localhost:8080

You should see the home page of the card store app.

In the home page you can create a new card and list the cards using the link below.

3. Put the Spring Boot Fat Jar to S3

When we package the application, Maven will create the fat jar file in the target directory.

$ ls target/*.jar

target/cardstore-0.0.1-SNAPSHOT.jar

Before copying the file to S3, we should create a S3 bucket. We can create a new bucket using AWS CLI.

$ aws s3 mb s3://cardstoredeploy

make_bucket: cardstoredeploy

To put the jar to the S3 bucket, execute the following command.

$ aws s3 cp target/cardstore-0.0.1-SNAPSHOT.jar s3://cardstoredeploy

upload: target\cardstore-0.0.1-SNAPSHOT.jar to s3://cardstoredeploy/cardstore-0.0.1-SNAPSHOT.jar

Now you should see the jar in AWS S3 Console.

4. Launch the EC2 Instance

In this step, we will use AWS CLI to create an EC2 instance. Before we create an EC2 instance we should do some preparations. The preparations can be done by AWS CLI, but to keep the things simple we will use the AWS Console.

To launch an EC2 instance we should do the following:

1. Create an IAM Role for EC2 service to read the jar file from S3. 

Although full access is not needed in this application, we will create a role with AmazonS3FullAccess policy to use it in future posts.

• Sign in to the AWS Console. Under AWS Services choose IAM. 
• Select Roles from the navigation pane and then click Create Role.
• Enter S3-Admin-Role as Role Name and click Next Step.
• Expand AWS Service Roles section and select  Amazon EC2. 
• Enter S3 to filter the policies and select the check box for AmazonS3FullAccess. 
• Click Next Step to review the role and then click Create Role.

2. Generate EC2 key pair that will be used to access the EC2 instance.

A key pair is required to create an EC2 instance. This key pair is normally used to connect to the instance using SSH. In this post we won't connect to the EC2 instance over SSH.

• At the AWS console choose EC2.  
• Select Key Pairs under Network & Security from the navigation pane. Click Create Key Pair. 
• Enter CardStoreKP as key pair name and click Create. 

After the key pair is created the private key file is automatically downloaded by your browser. Save this private key file in a safe place. This key file is required to connect to EC2 instances via SSH. You can find the detailed information here

3. Create a security group to allow inbound SSH and HTTP traffic (ports 22 and 8080)

Security groups are used to allow incoming and outgoing traffic. For our application we will create a security group to allow inbound access for SSH and HTTP protocols.

• In AWS Console, click Services and select EC2.  
• Under Network & Security, choose Security Groups. Click Create Security Group. 
• Enter CardStoreSG as name and 'Card Store Security Group to allow SSH and 8080 Http traffic' as description and leave the VPC as default. 
• Under Inbound rules click Add Rule. 
• Select SSH from Type combo. Choose Anywhere under Source. 
• Click Add Rule again and select Custom TCP Rule and enter 8080 in Port Range. Choose Anywhere under Source. 
• Click Create to create security group. 

All the outbound traffic is allowed by default.

Please note that to keep it simple, we have allowed any source. In real usage scenarios, you should limit the access accordingly.

Note Security group Id. It should be like ' sg-8567a2ee'.

4. Create the EC2 instance that gets the jar from S3 and runs it.

We will use AWS CLI to create an EC2 instance that runs Amazon Linux. To get the jar file from S3 and run the application when the instance starts, the EC2 init script mechanism is used. 

Init script is in cardstore1_ec2_init_script.txt file.

#!/bin/bash

yum install java-1.8.0 -y
yum remove java-1.7.0-openjdk -y

mkdir app

aws s3 cp --region eu-central-1 s3://cardstoredeploy/cardstore-0.0.1-SNAPSHOT.jar app/

java -jar app/cardstore-0.0.1-SNAPSHOT.jar

The current Amazon Linux comes with Java 7 installed. Because Spring Boot requires Java 8, first we install Java 8. After that we remove Java 7 to make Java 8 default. Once Java installation is done, we get the jar file from S3.

Please note that --region parameter should point to the region that your S3 bucket is created in. Finally, the application is launched by java command.

Now we are ready to launch the Amazon Linux EC2 instance. We create the EC2 instance with aws ec2 run-instances command.

$ aws ec2 run-instances --image-id ami-af0fc0c0 --count 1 --instance-type t2.micro --key-name CardStoreKP --user-data file://cardstore1_ec2_init_script.txt --associate-public-ip-address --security-group-ids sg-8567a2ee  --iam-instance-profile Name=S3-Admin-Role

The meaning of the parameters for the run-instances command is explained below.

--image-id ami-af0fc0c0

The Amazon Machine Image (AMI) id for current Amazon Linux operating system.

--count 1

We want to launch one instance.

--instance-type t2.micro

We use t2.micro to stay in the AWS Free Tier limits.

--key-name CardStoreKP

The key name of the key pair we have created before.

--user-data file://cardstore1_ec2_init_script.txt

The file to specify the commands that will be executed when the server starts.

--associate-public-ip-address

Required for instance to get a public ip address, so we can connect from outside.

--security-group-ids sg-8567a2ee

The id of the security group we have created before.

--iam-instance-profile Name=S3-Admin-Role

The name of the IAM role that will be used by instance. Required for instance to access S3 services in init script.

After this command, a new EC2 instance will be created. It will take some time to instance to be created and get an public ip address. To connect to instance, we will need the public ip address that will be assigned to the instance. We can get the public ip address by executing the describe-instances command.

$ aws ec2 describe-instances|grep PublicIpAddress

"PublicIpAddress": "35.157.111.179"

For my instance 35.157.111.179 was assigned. Your ip address will  be different. You should use the ip address assigned to your instance.

After the application started you can connect to the instance using your browser at http://<ip address>:8080

The home page of the application should look like the below.

You can enter the card details and click 'Create Card' button to create a new card. The id of the created card will be displayed.

You can use this id to get its details at http://<ip address>:8080/cards/<id>

And finally you can list all the cards by using http://<ip address>:8080/cards/

Summary

In this post, I showed how to run a simple Java Spring Boot application on AWS. Source code for the application can be found here.

In my next posts, I will show how to use other services on Amazon Web Services.