Implementing Object Persistence for Our Digital Cards with Amazon DynamoDB

In my previous post, I have written about load balancing our digital card store application and updating it with zero downtime.

The application was using an in-memory dummy Spring repository to have a quick start. But if we don't use a persistent repository, our cards will be lost when the EC2 instance is stopped. To provide persistence we can use Amazon DynamoDB.

Amazon DynamoDB is a NoSQL cloud database service. It provides a managed, scalable, schema-free data storage.

When we design scalable, high throughput applications, we should consider using NoSQL databases where they are appropriate.

DynamoDB Concepts

In DynamoDB, data is stored in the tables just like the tables in relational databases. A table holds the items which are similar to the rows in relational databases. Also the columns in relational databases resemble to the attributes in DynamoDB. Main difference between DynamoDB and relational databases is in the table columns. In relational databases, the schema of each table is fixed and pre-defined. In DynamoDB, only the primary key of the table is defined at creation time. This way, every item can have different number of attributes. We can think every item as a hash map with different keys and values. The picture below shows the structure of a DynamoDB table. For more information, see DynamoDB Getting Started Guide.

Every item is defined with its primary key. A primary key is consists of a hash key and an optional range key. Hash key is used to select the DynamoDB partition. Partitions are parts of the table data. Range keys are used to sort the items in the partition, if they exist.

When using DynamoDB, data modeling is very important. So we should pay attention to use cases of the application and the data model we create. For more information, see DynamoDB Best Practices.

Spring Data DynamoDB Module

Spring Data project has a module to provide automatic CRUD repository interface implementations that greatly reduces the effort required to use DynamoDB. In this post, I will use Spring Data DynamoDB module to easily access DynamoDB functionality. For more information, see Spring Data DynamoDB GitHub page.

The steps to add DynamoDB persistence to our card store application are below.

1. Configure DynamoDB

2. Create DynamoDB tables

3. Create Java entity classes

4. Create Spring Data repository interfaces

5. Implement the controllers

Let's start.

1. Configure DynamoDB

First, we add Maven dependencies to use AWS Java SDK for DynamoDB and Spring Data DynamoDB module as below.

             <dependency>

                    <groupId>com.amazonaws</groupId>

                    <artifactId>aws-java-sdk-dynamodb</artifactId>

                    <version>1.9.6</version>

             </dependency>

             <dependency>

                    <groupId>com.github.derjust</groupId>

                    <artifactId>spring-data-dynamodb</artifactId>

                    <version>4.5.0</version>

             </dependency>

Then we create a DynamoDBConfig class to configure DynamoDB. Amazon DynamoDB provides a local version to use at development time. While developing the application we can use the local DynamoDB endpoint. We can specify the value of amazon.dynamodb.endpoint variable in application.properties file. For more information, see Setting Up DynamoDB Local. To use real DynamoDB service use a blank value and set your region accordingly.

We specify our Java package of the repository interfaces with @EnableDynamoDBRepositories annotation.

@Configuration

@EnableDynamoDBRepositories(basePackages = "com.cardstore.dao")

public class DynamoDBConfig {

       @Value("${amazon.dynamodb.endpoint}")

       private String amazonDynamoDBEndpoint;

       @Bean

       public AmazonDynamoDB amazonDynamoDB() {

             AmazonDynamoDB amazonDynamoDB = new AmazonDynamoDBClient();

             if (amazonDynamoDBEndpoint != null && !amazonDynamoDBEndpoint.equals("")) {

                    amazonDynamoDB.setEndpoint(amazonDynamoDBEndpoint);

             } else

                    amazonDynamoDB.setRegion(Region.getRegion(Regions.EU_CENTRAL_1));

             return amazonDynamoDB;

       }

}

2. Create DynamoDB tables

To decide the structure of the tables, first we should consider the use cases and the requirements for our digital card store application.

In our digital card store application there must be users that own the cards. Let's list the requirements for users.

  1. Users should be able to register themselves with their user name, name, email address and password and system should assign an initial balance.

  2. Users should be able to login with their user name and password.

  3. Users should be able to logout.

The requirements for the cards are below.

  1. Users should be able to create a card with a name.

  2. Users should be able to list their cards.

  3. Users should be able to mark the cards as on sale with a price.

  4. Users should be able to list the cards that are on sale.

  5. Users should be able to buy a card.

With these requirements we can design the User and Card tables as below. imageURL attribute of Card table will be used in future posts.

After we decided to table structure, we can use AWS CLI to create the tables with the commands below. To use local DynamoDB, you can use endpoint-url parameter as --endpoint-url http://192.168.99.100:9000 .

aws dynamodb create-table --table-name User --attribute-definitions AttributeName=username,AttributeType=S --key-schema AttributeName=username,KeyType=HASH --provisioned-throughput ReadCapacityUnits=1,WriteCapacityUnits=1

aws dynamodb create-table --table-name Card --attribute-definitions AttributeName=owner,AttributeType=S AttributeName=name,AttributeType=S --key-schema AttributeName=owner,KeyType=HASH AttributeName=name,KeyType=RANGE --provisioned-throughput ReadCapacityUnits=1,WriteCapacityUnits=1

 3. Create Java entity classes

Once we created the tables, we can create Java entity classes User and Card. We use Spring Data DynamoDB module annotations to associate tables and keys to Java classes and their fields. @DynamoDBTable annotation shows the table name and @DynamoDBHashKey annotation shows the hash key of the table. If the Java field name and DynamoDB attribute name differs we can use @DynamoDBAttribute annotation for fields.

Spring Data DynamoDB module uses composite key classes if range key is used. @Id annotation is used to mark composite key field. In composiye key classes, fields are marked with @DynamoDBHashKey and @DynamoDBRangeKey annotations. Card class uses CardId composite key class as below.

@DynamoDBTable(tableName = "User")

public class User {

    @DynamoDBHashKey

    private String username;

    private String name;

    private String password;

    private String email;

    private double balance;

@DynamoDBTable(tableName = "Card")

public class Card implements Cloneable {

       @Id

       private CardId cardId;

      

       private String dateLoaded;

       private String imageUrl;

       private boolean inSale;

       private double price;

public class CardId implements Serializable {

       @DynamoDBHashKey

       private String owner;

      

       @DynamoDBRangeKey

       private String name;

4. Create Spring Data repository interfaces

After entity classes are defined we can create Spring Data repository interfaces to access DynamoDB tables. Spring Data module creates the implementation in run-time.

@EnableScan

public interface UserRepository extends CrudRepository<User, String> {

}

UserRepository interface extends Spring Data CrudReposity interface which provides basic CRUD methods.

@EnableScan

public interface CardRepository extends CrudRepository<Card, CardId> {

       List<Card> findByOwner(String owner);

      

       List<Card> findByInSale(@Param("inSale") boolean inSale);

}

CardRepository interface extends Spring Data CrudReposity interface for basic CRUD methods and adds two custom methods. findByOwner method will be used for finding the cards by their owner and findByInSale method will be used to find cards by inSale field. Spring Data module generates implementations of these methods by using method name convention for generating filter expressions.

5. Implement the controllers

Now both entity classes and repository clases are ready. We can implement our controller classes that manages users and cards.

UserController class includes user and session management methods. This controller class uses UserRepository class to manage users in DynamoDB.

@Controller

public class UserController {

       public static final String USER_KEY_FOR_SESSION = "USER";

       @Autowired

       UserRepository userRepository;

home method is used to direct user to login page if the user is not logged in. If the user is logged in the dashboard is shown.

@RequestMapping("/")

public String home(Map<String, Object> model, HttpSession session) {

       User user = userfromSession(session);

       if (user == null)

             return "index";

       else {

             // get up to date balance information from table, because balance

             // can be changed any time.

             User uptoDate = userRepository.findOne(user.getUsername());

             user.setBalance(uptoDate.getBalance());

             model.put("user", user);

             return "dashboard";

       }

}

registerUser method is used to register users. The initial balance is set to 100.

@RequestMapping(value = "/users", method = RequestMethod.POST)

@ResponseBody

public boolean registerUser(@RequestBody User user) {

       User previous = userRepository.findOne(user.getUsername());

       if (previous == null) {

             user.setBalance(100);

             userRepository.save(user);

       }

       return previous == null;

}

login method is used to log the users in by checking user's password and creating the http session.

@RequestMapping(value = "/login", method = RequestMethod.POST, produces = "text/plain")

@ResponseBody

public String login(@RequestBody User user, HttpServletRequest request) {

       String error = "None";

       User existing = userRepository.findOne(user.getUsername());

       boolean canLogin = existing != null && existing.getPassword().equals(user.getPassword());

       if (!canLogin)

             error = "User name and password mismatch.";

       else {

             HttpSession session = request.getSession(true);

             session.setAttribute(USER_KEY_FOR_SESSION, existing);

       }

       return error;

}

logout method is used to log the users out by invalidating the current http session.

@RequestMapping(value = "/logout", method = RequestMethod.POST)

@ResponseBody

public boolean logout(HttpServletRequest request) {

       HttpSession session = request.getSession(false);

       if (session != null)

             session.invalidate();

       return true;

}

userFromSession and loggedIn methods are used for session management.

public static User userfromSession(HttpSession session) {

       User user = (User) session.getAttribute(USER_KEY_FOR_SESSION);

       return user;

}

public static boolean loggedIn(HttpSession session) {

       return userfromSession(session) != null;

}

After UserController class, we can create CardController class to manage cards. This class uses both UserRepository and CardRepository classes to access both users and cards in DynamoDB.

@RestController

public class CardController {

       @Autowired

       CardRepository cardRepository;

      

       @Autowired

       UserRepository userRepository;

listCards method is used to list user's cards and list on sale cards.

@RequestMapping(value = "/cards", method = RequestMethod.GET)

public List<Card> listCards(@RequestParam(name = "inSale", required = false) boolean inSale, HttpSession session) {

       User user = UserController.userfromSession(session);

       if (inSale)

             return (List<Card>) cardRepository.findByInSale(inSale);

       else

             return (List<Card>) cardRepository.findByOwner(user.getUsername());

}

saveCard method is used to create a new cardc with a name.

@RequestMapping(value = "/cards", method = RequestMethod.POST)

public Card saveCard(@RequestBody Card card, HttpSession session) {

       User user = UserController.userfromSession(session);

       card.setOwner(user.getUsername());

       card.setDateLoaded(new SimpleDateFormat("yyyy-MM-dd").format(new Date()));

       cardRepository.save(card);

       return card;

}

getCard method is used to get details of a specific card.

// we used {name:.+} instead of {name} to get . in path like

// '/cards/ceyhun.ozgun'

@RequestMapping(value = "/cards/{name:.+}", method = RequestMethod.GET)

public Card getCard(@PathVariable("name") String name, HttpSession session) {

       User user = UserController.userfromSession(session);

       Card card = cardRepository.findOne(new CardId(user.getUsername(), name));

       return card;

}

sellCard method is used to put a specific card on sale with a price.

@RequestMapping(value = "/sell", method = RequestMethod.POST)

public boolean sellCard(@RequestBody Card card, HttpSession session) {

       User user = UserController.userfromSession(session);

       card.setOwner(user.getUsername());

       Card existing = cardRepository.findOne(new CardId(card.getOwner(), card.getName()));

       if (existing != null && !existing.isInSale()) {

             existing.setInSale(true);

             existing.setPrice(card.getPrice());

             cardRepository.save(existing);

             return true;

       }

       return false;

}

buyCard method is used to buy a card on sale. As the owner of the card changed, we must first delete the old card and save the card again with the new owner. Also we decrease the balance of the buyer and increase the balance of the seller.

@RequestMapping(value = "/buy", method = RequestMethod.POST)

public BuyResult buyCard(@RequestBody Card card, HttpSession session) {

       User user = UserController.userfromSession(session);

       Card cardToBuy = cardRepository.findOne(new CardId(card.getOwner(), card.getName()));

       if (cardToBuy == null || !cardToBuy.isInSale())

             return new BuyResult("Can't buy a non existing or not buyable card.");

       if (cardToBuy.getOwner().equals(user.getUsername()))

             return new BuyResult("Can't buy your own card.");

       // get latest balance from table, do not use the balance in the session

       User currentUser = userRepository.findOne(user.getUsername());

       if (currentUser.getBalance() < cardToBuy.getPrice())

             return new BuyResult("Can't buy a card with price " + cardToBuy.getPrice() + " with a balance " + currentUser.getBalance());

       Card newCard;

       try {

             newCard = (Card) cardToBuy.clone();

       } catch (CloneNotSupportedException e) {

             throw new RuntimeException("not expected");

       }

       newCard.setOwner(currentUser.getUsername());

       newCard.setInSale(false);

       // owner field is the hash key of the Card table, so we must delete the

       // old card and save the new card

       cardRepository.delete(cardToBuy);

       cardRepository.save(newCard);

       // transfer the price of the card from buyer to seller

       double newBalance = currentUser.getBalance() - cardToBuy.getPrice();

       currentUser.setBalance(newBalance);

       userRepository.save(currentUser);

       User seller = userRepository.findOne(cardToBuy.getOwner());

       seller.setBalance(seller.getBalance() + cardToBuy.getPrice());

       userRepository.save(seller);

       return new BuyResult(newBalance);

}

Now we are developed the server side of the application. To complete the application we should develop the web interfaces also. These interfaces are Login, Register User forms and the dashboard page. You can develop these interfaces according to the controller paths and entity attributes. The code can be found at GitHub.

Deploying the application on AWS

Before deploying the application to AWS, we must prepare IAM role the EC2 instances use. In my previous posts EC2 instances used an IAM role that have only S3 access policy. For this post we should give DynamoDB access to the IAM role we use. We can use AWS Console to create an IAM role named CardStoreRole that have AmazonS3FullAccess and AmazonDynamoDBFullAccess policies.

You can create a load balancer and auto scaling group to launch the application using the instructions on my previous posts.

Once the application launched you can use the application to register two different users. Then you can create one card with first user and buy the card with second user as shown in the pictures below.

Summary

In this post, I have shown how to use Amazon DynamoDB to provide object persistence in a Spring Boot application. The code can be found at my GitHub repository.

In my next post, I will show how to use Amazon Simple Queue Service and Amazon Simple Email Service to provide email activation for user registration.