{{announcement.body}}
{{announcement.title}}

Microservice: Async Rest Client to DynamoDB using Spring Boot

DZone 's Guide to

Microservice: Async Rest Client to DynamoDB using Spring Boot

In this post, I will be exploring using asynchronous DynamoDB API and Spring Webflux by building a simple reactive REST application.

· Microservices Zone ·
Free Resource

Overview

Starting from Spring framework 5.0 and Spring Boot 2.0, the framework provides support for asynchronous programming, so does AWS SDK starting with 2.0 version.

In this post, I will be exploring using asynchronous DynamoDB API and Spring Webflux by building a simple reactive REST application. Let's say we need to handle HTTP requests for retrieving or storing some Event (id:string, body: string). The event will be stored in DynamoDB.

It might be easier to simply look at the code on Github and follow it there.

Maven Dependencies

Let's start with Maven dependencies for WebFlux and DynamoDB asynchronous SDK.

XML
 




xxxxxxxxxx
1
12


 
1
<dependencies>
2
    <dependency>
3
        <groupId>org.springframework.boot</groupId>
4
        <artifactId>spring-boot-starter-webflux</artifactId>
5
        <version>2.2.4.RELEASE</version>
6
    </dependency>
7
    <dependency>
8
        <groupId>software.amazon.awssdk</groupId>
9
        <artifactId>dynamodb</artifactId>
10
        <version>2.10.76</version>
11
    </dependency>
12
</dependencies>



DynamoDB Stuff

Spring Configuration

A simple config were we set up a connection to DynamoDB. For test purposes, we need to specify dynamoEndpoint but for real application, we need to specify the AWS region. DynamoEndpoint will point to the local DynamoDB instance which we will start during tests.

Java
 




xxxxxxxxxx
1
27


 
1
@Configuration
2
public class AppConfig {
3
    @Value("${aws.accessKey}")
4
    String accessKey;
5
 
           
6
    @Value("${aws.secretKey}")
7
    String secretKey;
8
 
           
9
    @Value("${dynamodb.endpoint:}")
10
    String dynamoEndpoint;
11
 
           
12
    @Bean
13
    AwsBasicCredentials awsBasicCredentials(){
14
        return AwsBasicCredentials.create(accessKey, secretKey);
15
    }
16
 
           
17
    @Bean
18
    DynamoDbAsyncClient dynamoDbAsyncClient(
19
      AwsBasicCredentials awsBasicCredentials
20
    ){
21
        DynamoDbAsyncClientBuilder clientBuilder = DynamoDbAsyncClient.builder();
22
        clientBuilder
23
                .credentialsProvider(StaticCredentialsProvider.create(awsBasicCredentials));
24
                if(!dynamoEndpoint.isEmpty()){
25
                    clientBuilder.endpointOverride(URI.create(dynamoEndpoint));
26
                }
27
        return clientBuilder.build();
28
    }
29
}



Application.yaml with connection details.

YAML
 




xxxxxxxxxx
1


1
aws:
2
  accessKey: any
3
  secretKey: any
4
dynamodb:
5
  endpoint: http://localhost:8000/



Reactive DynamoDB Service

Unfortunately, async version of AWS SDK doesn't have support for DynamoDBMapper yet (you can track mapper's readiness here), so table creation, sending requests and parsing responses need to be done by "low level" API.

So we need to create DynamoDbService for handling:

  • Creation table if it does not exists
  • Saving and retrieving event
Java
 




x


 
1
@Service
2
public class DynamoDbService {
3
 
           
4
    public static final String TABLE_NAME = "events";
5
    public static final String ID_COLUMN = "id";
6
    public static final String BODY_COLUMN = "body";
7
 
           
8
    final DynamoDbAsyncClient client;
9
 
           
10
    @Autowired
11
    public DynamoDbService(DynamoDbAsyncClient client) {
12
        this.client = client;
13
    }
14
 
           
15
    //Creating table on startup if not exists
16
    @PostConstruct
17
    public void createTableIfNeeded() throws ExecutionException, InterruptedException {
18
        ListTablesRequest request = ListTablesRequest
19
                .builder()
20
                .exclusiveStartTableName(TABLE_NAME)
21
                .build();
22
        CompletableFuture<ListTablesResponse> listTableResponse = client.listTables(request);
23
 
           
24
        CompletableFuture<CreateTableResponse> createTableRequest = listTableResponse
25
                .thenCompose(response -> {
26
                    boolean tableExist = response
27
                            .tableNames()
28
                            .contains(TABLE_NAME);
29
                    
30
                    if (!tableExist) {
31
                        return createTable();
32
                    } else {
33
                        return CompletableFuture.completedFuture(null);
34
                    }
35
                });
36
 
           
37
        //Wait in synchronous manner for table creation
38
        createTableRequest.get();
39
    }
40
 
           
41
    public CompletableFuture<PutItemResponse> saveEvent(Event event) {
42
        Map<String, AttributeValue> item = new HashMap<>();
43
        item.put(ID_COLUMN, AttributeValue.builder().s(event.getUuid()).build());
44
        item.put(BODY_COLUMN, AttributeValue.builder().s(event.getBody()).build());
45
 
           
46
        PutItemRequest putItemRequest = PutItemRequest.builder()
47
                .tableName(TABLE_NAME)
48
                .item(item)
49
                .build();
50
 
           
51
        return client.putItem(putItemRequest);
52
    }
53
 
           
54
    public CompletableFuture<Event> getEvent(String id) {
55
        Map<String, AttributeValue> key = new HashMap<>();
56
        key.put(ID_COLUMN, AttributeValue.builder().s(id).build());
57
 
           
58
        GetItemRequest getRequest = GetItemRequest.builder()
59
                .tableName(TABLE_NAME)
60
                .key(key)
61
                .attributesToGet(BODY_COLUMN)
62
                .build();
63
 
           
64
        return client.getItem(getRequest).thenApply(item -> {
65
            if (!item.hasItem()) {
66
                return null;
67
            } else {
68
                Map<String, AttributeValue> itemAttr = item.item();
69
                String body = itemAttr.get(BODY_COLUMN).s();
70
                return new Event(id, body);
71
            }
72
        });
73
    }
74
 
           
75
    private CompletableFuture<CreateTableResponse> createTable() {
76
        KeySchemaElement keySchemaElement = KeySchemaElement
77
                .builder()
78
                .attributeName(ID_COLUMN)
79
                .keyType(KeyType.HASH)
80
                .build();
81
 
           
82
        AttributeDefinition attributeDefinition = AttributeDefinition
83
                .builder()
84
                .attributeName(ID_COLUMN)
85
                .attributeType(ScalarAttributeType.S)
86
                .build();
87
        
88
        CreateTableRequest request = CreateTableRequest.builder()
89
                .tableName(TABLE_NAME)
90
                .keySchema(keySchemaElement)
91
                .attributeDefinitions(attributeDefinition)
92
                .billingMode(BillingMode.PAY_PER_REQUEST)
93
                .build();
94
 
           
95
        return client.createTable(request);
96
    }
97
}



Reactive REST Controller

A simple controller with the GET method for retrieving events by id and POST method for saving events in DynamoDB. We can do it in two ways — implement it with annotations or get rid of annotations and do it in a functional way. There is no performance impact, in almost most cases it is absolutely based on individual preference what to use.

Annotated Controllers

Java
 




xxxxxxxxxx
1
21


 
1
@RestController
2
@RequestMapping("/event")
3
public class AnnotatedController {
4
 
           
5
    final DynamoDbService dynamoDbService;
6
 
           
7
    public AnnotatedController(DynamoDbService dynamoDbService) {
8
        this.dynamoDbService = dynamoDbService;
9
    }
10
 
           
11
    @GetMapping(value = "/{eventId}", produces = MediaType.APPLICATION_JSON_VALUE)
12
    public Mono<Event> getEvent(@PathVariable String eventId) {
13
        CompletableFuture<Event> eventFuture = dynamoDbService.getEvent(eventId);
14
        return Mono.fromCompletionStage(eventFuture);
15
    }
16
 
           
17
    @PostMapping(consumes = MediaType.APPLICATION_JSON_VALUE)
18
    public void saveEvent(@RequestBody Event event) {
19
        dynamoDbService.saveEvent(event);
20
    }
21
}



Functional Endpoints

This is a lightweight functional programming model in which functions are used to route and handle requests. Here I created EventHandler just for the simplicity of reading the code.

Java
 




xxxxxxxxxx
1
35


 
1
@Configuration
2
public class HttpRouter {
3
 
           
4
    @Bean
5
    public RouterFunction<ServerResponse> eventRouter(DynamoDbService dynamoDbService) {
6
        EventHandler eventHandler = new EventHandler(dynamoDbService);
7
        return RouterFunctions
8
                .route(GET("/eventfn/{id}")
9
                        .and(accept(APPLICATION_JSON)), eventHandler::getEvent)
10
                .andRoute(POST("/eventfn")
11
                        .and(accept(APPLICATION_JSON))
12
                        .and(contentType(APPLICATION_JSON)), eventHandler::saveEvent);
13
    }
14
 
           
15
    static class EventHandler {
16
        private final DynamoDbService dynamoDbService;
17
 
           
18
        public EventHandler(DynamoDbService dynamoDbService) {
19
            this.dynamoDbService = dynamoDbService;
20
        }
21
 
           
22
        Mono<ServerResponse> getEvent(ServerRequest request) {
23
            String eventId = request.pathVariable("id");
24
            CompletableFuture<Event> eventGetFuture = dynamoDbService.getEvent(eventId);
25
            Mono<Event> eventMono = Mono.fromFuture(eventGetFuture);
26
            return ServerResponse.ok().body(eventMono, Event.class);
27
        }
28
 
           
29
        Mono<ServerResponse> saveEvent(ServerRequest request) {
30
            Mono<Event> eventMono = request.bodyToMono(Event.class);
31
            eventMono.map(dynamoDbService::saveEvent);
32
            return ServerResponse.ok().build();
33
        }
34
    }
35
}



Spring DynamoDB Integration Test

Maven Dependencies

For running integration test with DynamoDB we need DynamoDBLocal, which is not really the DynamoDB, but SQLite with implemented DynamoDB interfaces on top of it.

XML
 




xxxxxxxxxx
1
31


 
1
<dependency>
2
    <groupId>com.amazonaws</groupId>
3
    <artifactId>DynamoDBLocal</artifactId>
4
    <version>1.12.0</version>
5
    <scope>test</scope>
6
</dependency>
7
 
           
8
<build>
9
    <plugins>
10
        <plugin>
11
            <groupId>org.apache.maven.plugins</groupId>
12
            <artifactId>maven-dependency-plugin</artifactId>
13
            <version>2.10</version>
14
            <executions>
15
                <execution>
16
                    <id>copy</id>
17
                    <phase>test-compile</phase>
18
                    <goals>
19
                        <goal>copy-dependencies</goal>
20
                    </goals>
21
                    <configuration>
22
                        <includeScope>test</includeScope>
23
                        <includeTypes>so,dll,dylib</includeTypes>
24
                        <!--Keep an eye on output directory - it will be used for starting dynamodb-->
25
                        <outputDirectory>${project.basedir}/target/native-libs</outputDirectory>
26
                    </configuration>
27
                </execution>
28
            </executions>
29
        </plugin>
30
    </plugins>
31
</build>
32
 
           
33
<repositories>
34
    <repository>
35
        <id>dynamodb-local-oregon</id>
36
        <name>DynamoDB Local Release Repository</name>
37
        <url>https://s3-us-west-2.amazonaws.com/dynamodb-local/release</url>
38
    </repository>
39
</repositories>



DynamoDB server

Now we need to start DynamoDB before the test run. I prefer to do it as JUnit Class Rule, but we can also do it as a spring bean.

Java
 




xxxxxxxxxx
1
29


1
public class LocalDynamoDbRule extends ExternalResource {
2
 
           
3
    protected DynamoDBProxyServer server;
4
 
           
5
    public LocalDynamoDbRule() {
6
        //here we set the path from "outputDirectory" of maven-dependency-plugin
7
        System.setProperty("sqlite4java.library.path", "target/native-libs");
8
    }
9
 
           
10
    @Override
11
    protected void before() throws Exception {
12
        this.server = ServerRunner
13
            .createServerFromCommandLineArgs(new String[]{"-inMemory", "-port", "8000"});
14
        server.start();
15
    }
16
 
           
17
    @Override
18
    protected void after() {
19
        this.stopUnchecked(server);
20
    }
21
 
           
22
    protected void stopUnchecked(DynamoDBProxyServer dynamoDbServer) {
23
        try {
24
            dynamoDbServer.stop();
25
        } catch (Exception e) {
26
            throw new RuntimeException(e);
27
        }
28
    }
29
}



Running Test

Now we can create an integration test and test get event by id and save the event.

Java
 




xxxxxxxxxx
1
39


 
1
@RunWith(SpringRunner.class)
2
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
3
public class IntegrationTest {
4
 
           
5
    @ClassRule
6
    public static LocalDynamoDbRule dynamoDbRule = new LocalDynamoDbRule();
7
 
           
8
    @Autowired
9
    private WebTestClient webTestClient;
10
 
           
11
    @Test
12
    public void getEvent() {
13
        // Create a GET request to test an endpoint
14
        webTestClient
15
                .get().uri("/event/1")
16
                .accept(MediaType.APPLICATION_JSON)
17
                .exchange()
18
                // and use the dedicated DSL to test assertions against the response
19
                .expectStatus().isOk()
20
                .expectBody(String.class).isEqualTo(null);
21
    }
22
 
           
23
    @Test
24
    public void saveEvent() throws InterruptedException {
25
        Event event = new Event("10", "event");
26
        webTestClient
27
                .post().uri("/event/")
28
                .body(BodyInserters.fromValue(event))
29
                .exchange()
30
                .expectStatus().isOk();
31
        Thread.sleep(1500);
32
        webTestClient
33
                .get().uri("/event/10")
34
                .accept(MediaType.APPLICATION_JSON)
35
                .exchange()
36
                .expectStatus().isOk()
37
                .expectBody(Event.class).isEqualTo(event);
38
    }
39
}



Docker

Here we gonna prepare our application for running in docker, so it will be ready for deploying to AWS.

HINT: Starting from Java 10 you can specify how much memory JVM will use depending on container memory.-XX:MaxRAMPercentage=75.0 means JVM won't use more than 75% of a container memory.

Dockerfile

Dockerfile
 




xxxxxxxxxx
1
18


 
1
# Use our standard java12 baseimage
2
FROM openjdk:12-alpine
3
 
           
4
# Copy the artifact into the container
5
COPY target/dynamodb-spring-*-exec.jar /srv/service.jar
6
 
           
7
# Run the artifact and expose the default port
8
WORKDIR /srv
9
 
           
10
ENTRYPOINT [ "java", \
11
    "-XX:+UnlockExperimentalVMOptions", \
12
    "-XX:+ExitOnOutOfMemoryError", \
13
    "-XX:MaxRAMPercentage=75.0", \
14
    "-Djava.security.egd=file:/dev/./urandom", \
15
    "-jar", "service.jar", \
16
    "--spring.profiles.active=prod" ]
17
 
           
18
EXPOSE 8080



Building the docker container itself  docker build -t spring-dynamo.

Also, let's see what was generated by docker image ls.

Plain Text
 




xxxxxxxxxx
1


 
1
REPOSITORY          TAG                 IMAGE ID            CREATED              SIZE
2
spring-dynamo       latest              a974d880400e        About a minute ago   364MB
3
openjdk             12-alpine           0c68e7c5b7a0        12 months ago        339MB



Finally, our POC is ready!

Happy coding :)

Topics:
aws, dynamodb, java, microservice, spring, tutorial

Published at DZone with permission of Yegor Bondarenko . See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}