Dynamic Data Processing Using Serverless Java With Quarkus on AWS Lambda (Part 1)

This step-by-step tutorial explores how Quarkus enables Java developers to implement serverless functions on AWS Lambda to process dynamic data on AWS DynamoDB.

Daniel Oh

CORE ·

May. 01, 23 · Tutorial

Like (4)

Save

6.32K Views

With the growth of the application modernization demands, monolithic applications were refactored to cloud-native microservices and serverless functions with lighter, faster, and smaller application portfolios for the past years. This was not only about rewriting applications, but the backend data stores were also redesigned in terms of dynamic scalability, high performance, and flexibility for event-driven architecture. For example, traditional data structures in relational databases started to move forward to a new approach that enables to storage and retrieval of key-value and document data structures using NoSQL databases.

However, faster modernization presents more challenges for Java developers in terms of steep learning curves about new technologies adoption and retaining current skillsets with experience. For instance, Java developers need to rewrite all existing Java applications to Golang and JavaScript for new serverless functions and learn new APIs or SDKs to process dynamic data records by new modernized serverless applications.

This article will take you through a step-by-step tutorial to learn how Quarkus enables Java developers to implement serverless functions on AWS Lambda to process dynamic data on AWS DynamoDB. Quarkus enables developers not only to optimize Java applications for superfast startup time (e.g., milliseconds) and tiny memory footprints (e.g., less than 100 MB) for serverless applications, but developers can also use more than XX AWS extensions to deploy Java applications to AWS Lambda and access AWS DynamoDB directly without steep learning curves.

Creating a New Serverless Java Project Using Quarkus

We’ll use the Quarkus command to generate a new project with required files such as Maven Wrapper, Dockerfiles, configuration properties, and sample code. Find more information about the benefits of the Quarkus command (CLI) here.

Run the following Quarkus command in your working directory.

    Shell
   
   quarkus create piggybank --java=17

You need to use the JDK 17 version since AWS Lambda currently supports JDK 17 as the latest version by default Java runtime (Corretto).

Let’s start Quarkus Live Coding, also known as quarkus dev mode, using the following command.

    Shell
   
   cd piggybank && quarkus dev mode

Developing Business Logic for Piggybank

Now let's add a couple of Quarkus extensions to create a DynamoDB entity and relevant abstract services using the following Quarkus command in the Piggybank directory.

    Shell
   
   quarkus ext add amazon-dynamodb resteasy-reactive-jackson

The output should look like this.

    Java
   
   [SUCCESS] ✅  Platform io.quarkus.platform:quarkus-amazon-services-bom has been installed
[SUCCESS] ✅  Extension io.quarkiverse.amazonservices:quarkus-amazon-dynamodb has been installed
[SUCCESS] ✅  Extension io.quarkus:quarkus-resteasy-reactive-jackson has been installed

Creating an Entity Class

You will create a new data model (entry.java) file to define Java attributes that map into the fields in DynamoDB. The Java class should look like the following code snippet (you can find the solution in the GitHub repository):

    Java
   
 

   @RegisterForReflection
public class Entry {

    public Long timestamp;
    public String accountID;
    ...

    public Entry() {}

    public static Entry from(Map<String, AttributeValue> item) {
        Entry entry = new Entry();
        if (item != null && !item.isEmpty()) {
            entry.setAccountID(item.get(AbstractService.ENTRY_ACCOUNTID_COL).s());
    ...
        }
        return entry;

    }
...
}
  

The @RegisterForReflectionannotation instructs Quarkus to keep the class and its members during the native compilation. Find more information here.

Creating an Abstract Service

Now you will create a new AbstractService.java file to consist of helper methods that prepare DynamoDB to request objects for reading and adding items to the table. The code snippet should look like this (find the solution in the GitHub repository):

    Java
   
 

   public class AbstractService {

    public String accountID;
    ...

    public static final String ENTRY_ACCOUNTID_COL = "accountID";
    ...

    public String getTableName() {
        return "finance";
    }

    protected ScanRequest scanRequest() {
        return ScanRequest.builder().tableName(getTableName())
            .attributesToGet(ENTRY_ACCOUNTID_COL, ENTRY_DESCRIPTION_COL, ENTRY_AMOUNT_COL, ENTRY_BALANCE_COL, ENTRY_DATE_COL, ENTRY_TIMESTAMP, ENTRY_CATEGORY).build();
    }
...
}
  

Adding a Business Layer for REST APIs

Create a new EntryService.java file to extend the AbstractService class that will be the business layer of your application. This logic will store and retrieve the entry data from DynamoDB synchronously. The code snippet should look like this (solution in the GitHub repository):

    Java
   
 

   @ApplicationScoped

public class EntryService extends AbstractService {

    @Inject
    DynamoDbClient dynamoDB;

    public List<Entry> findAll() {
        List<Entry> entries = dynamoDB.scanPaginator(scanRequest()).items().stream()
                .map(Entry::from)
                .collect(Collectors.toList());
        entries.sort((e1, e2) -> e1.getDate().compareTo(e2.getDate()));
        BigDecimal balance = new BigDecimal(0);
        for (Entry entry : entries) {
            balance = balance.add(entry.getAmount());
            entry.setBalance(balance);
        }
        return entries;
    }
...
}
  

Creating REST APIs

Now you'll create a new EntryResource.java file to implement REST APIs to get and post the entry data from and to DynamoDB. The code snippet should look like the below (solution in the GitHub repository):

    Java
   
 

   @Path("/entryResource")
public class EntryResource {
    SimpleDateFormat piggyDateFormatter = new SimpleDateFormat("yyyy-MM-dd+HH:mm");

    @Inject
    EntryService eService;

    @GET
    @Path("/findAll")
    public List<Entry> findAll() {
        return eService.findAll();
    }
...
}
  

Verify the Business Services Locally

First, we need to install a local DynamoDB that the piggy bank services access. There’re a variety of ways to stand up a local DynamoDB such as downloading an executable .jar file, running a container image, and deploying by Apache Maven repository. Today, you will use the Docker compose to install and run DynamoDB locally. Find more information here.

Create the following docker-compose.yml file in your local environment.

    YAML
   
 

   version: '3.8'
services:
  dynamodb-local:
    command: "-jar DynamoDBLocal.jar -sharedDb -dbPath ./data"
    image: "amazon/dynamodb-local:latest"
    container_name: dynamodb-local
    ports:
      - "8000:8000"
    volumes:
      - "./docker/dynamodb:/home/dynamodblocal/data"
    working_dir: /home/dynamodblocal
  

Then, run the following command-line command.

    Shell
   
   docker-compose up

The output should look like this.

    Shell
   
 

   [+] Running 2/2
 ⠿ Network quarkus-piggybank_default  Created                                                                                                                                           0.0s
 ⠿ Container dynamodb-local           Created                                                                                                                                           0.1s
Attaching to dynamodb-local
dynamodb-local  | Initializing DynamoDB Local with the following configuration:
dynamodb-local  | Port:    8000
dynamodb-local  | InMemory:    false
dynamodb-local  | DbPath:    ./data
dynamodb-local  | SharedDb:    true
dynamodb-local  | shouldDelayTransientStatuses:    false
dynamodb-local  | CorsParams:    null
dynamodb-local  | 
  

Creating an Entry Table Locally

Run the following AWS DynamoDB API command to create a new entry table in the running DynamoDB container.

    Shell
   
   aws dynamodb create-table --endpoint-url http://localhost:8000  --table-name finance  --attribute-definitions AttributeName=accountID,AttributeType=S  AttributeName=timestamp,AttributeType=N    --key-schema AttributeName=timestamp,KeyType=HASH AttributeName=accountID,KeyType=RANGE --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5  --table-class STANDARD

Adding DynamoDB Clients Configurations

DynamoDB clients are configurable in the application.properties programmatically. You also need to add to the classpath a proper implementation of the sync client. By default, the extension uses the java.net.URLConnection HTTP client.

Open the pom.xml file and copy the following dependency right after the quarkus-amazon-dynamodb dependency.

    XML
   
       <dependency>
      <groupId>software.amazon.awssdk</groupId>
      <artifactId>url-connection-client</artifactId>
    </dependency>

Then, add the following key and value to the application.properties to specify your local DynamoDB's endpoint.

    Java
   
   %dev.quarkus.dynamodb.endpoint-override=http://localhost:8000

Starting Quarkus Live Coding

Now you should be ready to verify the Piggybank application using Quarkus Dev mode and local DynamoDB.

Run the Quarkus Dev mode using the following Quarkus command.

    Shell
   
   quarkus dev

The output should end up this.

    Shell
   
 

   __  ____  __  _____   ___  __ ____  ______ 
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ 
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   

[io.quarkus] (Quarkus Main Thread) Profile dev activated. Live Coding activated.quarkus xx.xx.xx.) s2023-04-30 21:14:49,824 INFO  [io.quarkus] (Quarkus Main Thread) Installed features: [amazon-dynamodb, cdi, resteasy-reactive, resteasy-reactive-jackson, smallrye-context-propagation, vertx]
--

Tests paused
Press [r] to resume testing, [o] Toggle test output, [:] for the terminal, [h] for more options>
  

Run the following curl command to insert several expense items into the piggybank account (entry table).

    Shell
   
   curl -X POST http://localhost:8080/entryResource -H 'Content-Type: application/json' -d '{"accountID": "Food", "description": "Shrimp", "amount": "-20", "balance": "0", "date": "2023-02-01"}'
curl -X POST http://localhost:8080/entryResource -H 'Content-Type: application/json' -d '{"accountID": "Car", "description": "Flat tires", "amount": "-200", "balance": "0", "date": "2023-03-01"}'
curl -X POST http://localhost:8080/entryResource -H 'Content-Type: application/json' -d '{"accountID": "Payslip", "description": "Income", "amount": "2000", "balance": "0", "date": "2023-04-01"}'
curl -X POST http://localhost:8080/entryResource -H 'Content-Type: application/json' -d '{"accountID": "Utilities", "description": "Gas", "amount": "-400", "balance": "0", "date": "2023-05-01"}'

Verify the stored data using the following command.

    Shell
   
   curl http://localhost:8080/entryResource/findAll

The output should look like this.

    JSON
   
   [{"accountID":"Food","description":"Shrimp","amount":"-20","balance":"-30","date":"2023-02-01"},{"accountID":"Drink","description":"Wine","amount":"-10","balance":"-10","date":"2023-01-01"},{"accountID":"Payslip","description":"Income","amount":"2000","balance":"1770","date":"2023-04-01"},{"accountID":"Car","description":"Flat tires","amount":"-200","balance":"-230","date":"2023-03-01"},{"accountID":"Utilities","description":"Gas","amount":"-400","balance":"1370","date":"2023-05-01"}]

You can also find a certain expense based on accountID. Run the following curl command again.

    Shell
   
   curl http://localhost:8080/entryResource/find/Drink

The output should look like this.

    JSON
   
   {"accountID":"Drink","description":"Wine","amount":"-10","balance":"-10","date":"2023-01-01"}

Conclusion

You learned how Quarkus enables developers to write serverless functions that connect NoSQL databases to process dynamic data. To stand up local development environments, you quickly ran the local DynamoDB image using the docker-compose command as well. Quarkus also provide various AWS extensions including amazon-dynamodb to access the AWS cloud services directly from your Java applications. Find more information here.

In the next article, you’ll learn how to create a serverless database using AWS DynamoDB and build and deploy your local serverless Java functions to AWS Lambda by enabling SnapStart.

AWS Lambda Amazon DynamoDB Quarkus Java (programming language)

Opinions expressed by DZone contributors are their own.

Comments

Trending

Dynamic Data Processing Using Serverless Java With Quarkus on AWS Lambda (Part 1)

This step-by-step tutorial explores how Quarkus enables Java developers to implement serverless functions on AWS Lambda to process dynamic data on AWS DynamoDB.

Creating a New Serverless Java Project Using Quarkus

Developing Business Logic for Piggybank

Creating an Entity Class

Creating an Abstract Service

Adding a Business Layer for REST APIs

Creating REST APIs

Verify the Business Services Locally

Creating an Entry Table Locally

Adding DynamoDB Clients Configurations

Starting Quarkus Live Coding

Conclusion

Trending

Partner Resources