Microservice: Async Rest Client to DynamoDB using Spring Boot
In this post, I will be exploring using asynchronous DynamoDB API and Spring Webflux by building a simple reactive REST application.
Join the DZone community and get the full member experience.
Join For FreeOverview
Starting from Spring framework 5.0 and Spring Boot 2.0, the framework provides support for asynchronous programming, so does AWS SDK starting with 2.0 version.
In this post, I will be exploring using asynchronous DynamoDB API and Spring Webflux by building a simple reactive REST application. Let's say we need to handle HTTP requests for retrieving or storing some Event (id:string, body: string). The event will be stored in DynamoDB.
It might be easier to simply look at the code on Github and follow it there.
Maven Dependencies
Let's start with Maven dependencies for WebFlux and DynamoDB asynchronous SDK.
xxxxxxxxxx
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webflux</artifactId>
<version>2.2.4.RELEASE</version>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>dynamodb</artifactId>
<version>2.10.76</version>
</dependency>
</dependencies>
DynamoDB Stuff
Spring Configuration
A simple config were we set up a connection to DynamoDB. For test purposes, we need to specify dynamoEndpoint but for real application, we need to specify the AWS region. DynamoEndpoint will point to the local DynamoDB instance which we will start during tests.
xxxxxxxxxx
public class AppConfig {
("${aws.accessKey}")
String accessKey;
("${aws.secretKey}")
String secretKey;
("${dynamodb.endpoint:}")
String dynamoEndpoint;
AwsBasicCredentials awsBasicCredentials(){
return AwsBasicCredentials.create(accessKey, secretKey);
}
DynamoDbAsyncClient dynamoDbAsyncClient(
AwsBasicCredentials awsBasicCredentials
){
DynamoDbAsyncClientBuilder clientBuilder = DynamoDbAsyncClient.builder();
clientBuilder
.credentialsProvider(StaticCredentialsProvider.create(awsBasicCredentials));
if(!dynamoEndpoint.isEmpty()){
clientBuilder.endpointOverride(URI.create(dynamoEndpoint));
}
return clientBuilder.build();
}
}
Application.yaml with connection details.
xxxxxxxxxx
aws
accessKeyany
secretKeyany
dynamodb
endpointhttp//localhost8000/
Reactive DynamoDB Service
Unfortunately, async version of AWS SDK doesn't have support for DynamoDBMapper yet (you can track mapper's readiness here), so table creation, sending requests and parsing responses need to be done by "low level" API.
So we need to create DynamoDbService for handling:
- Creation table if it does not exists
- Saving and retrieving event
x
public class DynamoDbService {
public static final String TABLE_NAME = "events";
public static final String ID_COLUMN = "id";
public static final String BODY_COLUMN = "body";
final DynamoDbAsyncClient client;
public DynamoDbService(DynamoDbAsyncClient client) {
this.client = client;
}
//Creating table on startup if not exists
public void createTableIfNeeded() throws ExecutionException, InterruptedException {
ListTablesRequest request = ListTablesRequest
.builder()
.exclusiveStartTableName(TABLE_NAME)
.build();
CompletableFuture<ListTablesResponse> listTableResponse = client.listTables(request);
CompletableFuture<CreateTableResponse> createTableRequest = listTableResponse
.thenCompose(response -> {
boolean tableExist = response
.tableNames()
.contains(TABLE_NAME);
if (!tableExist) {
return createTable();
} else {
return CompletableFuture.completedFuture(null);
}
});
//Wait in synchronous manner for table creation
createTableRequest.get();
}
public CompletableFuture<PutItemResponse> saveEvent(Event event) {
Map<String, AttributeValue> item = new HashMap<>();
item.put(ID_COLUMN, AttributeValue.builder().s(event.getUuid()).build());
item.put(BODY_COLUMN, AttributeValue.builder().s(event.getBody()).build());
PutItemRequest putItemRequest = PutItemRequest.builder()
.tableName(TABLE_NAME)
.item(item)
.build();
return client.putItem(putItemRequest);
}
public CompletableFuture<Event> getEvent(String id) {
Map<String, AttributeValue> key = new HashMap<>();
key.put(ID_COLUMN, AttributeValue.builder().s(id).build());
GetItemRequest getRequest = GetItemRequest.builder()
.tableName(TABLE_NAME)
.key(key)
.attributesToGet(BODY_COLUMN)
.build();
return client.getItem(getRequest).thenApply(item -> {
if (!item.hasItem()) {
return null;
} else {
Map<String, AttributeValue> itemAttr = item.item();
String body = itemAttr.get(BODY_COLUMN).s();
return new Event(id, body);
}
});
}
private CompletableFuture<CreateTableResponse> createTable() {
KeySchemaElement keySchemaElement = KeySchemaElement
.builder()
.attributeName(ID_COLUMN)
.keyType(KeyType.HASH)
.build();
AttributeDefinition attributeDefinition = AttributeDefinition
.builder()
.attributeName(ID_COLUMN)
.attributeType(ScalarAttributeType.S)
.build();
CreateTableRequest request = CreateTableRequest.builder()
.tableName(TABLE_NAME)
.keySchema(keySchemaElement)
.attributeDefinitions(attributeDefinition)
.billingMode(BillingMode.PAY_PER_REQUEST)
.build();
return client.createTable(request);
}
}
Reactive REST Controller
A simple controller with the GET method for retrieving events by id and POST method for saving events in DynamoDB. We can do it in two ways — implement it with annotations or get rid of annotations and do it in a functional way. There is no performance impact, in almost most cases it is absolutely based on individual preference what to use.
Annotated Controllers
xxxxxxxxxx
("/event")
public class AnnotatedController {
final DynamoDbService dynamoDbService;
public AnnotatedController(DynamoDbService dynamoDbService) {
this.dynamoDbService = dynamoDbService;
}
(value = "/{eventId}", produces = MediaType.APPLICATION_JSON_VALUE)
public Mono<Event> getEvent( String eventId) {
CompletableFuture<Event> eventFuture = dynamoDbService.getEvent(eventId);
return Mono.fromCompletionStage(eventFuture);
}
(consumes = MediaType.APPLICATION_JSON_VALUE)
public void saveEvent( Event event) {
dynamoDbService.saveEvent(event);
}
}
Functional Endpoints
This is a lightweight functional programming model in which functions are used to route and handle requests. Here I created EventHandler just for the simplicity of reading the code.
xxxxxxxxxx
public class HttpRouter {
public RouterFunction<ServerResponse> eventRouter(DynamoDbService dynamoDbService) {
EventHandler eventHandler = new EventHandler(dynamoDbService);
return RouterFunctions
.route(GET("/eventfn/{id}")
.and(accept(APPLICATION_JSON)), eventHandler::getEvent)
.andRoute(POST("/eventfn")
.and(accept(APPLICATION_JSON))
.and(contentType(APPLICATION_JSON)), eventHandler::saveEvent);
}
static class EventHandler {
private final DynamoDbService dynamoDbService;
public EventHandler(DynamoDbService dynamoDbService) {
this.dynamoDbService = dynamoDbService;
}
Mono<ServerResponse> getEvent(ServerRequest request) {
String eventId = request.pathVariable("id");
CompletableFuture<Event> eventGetFuture = dynamoDbService.getEvent(eventId);
Mono<Event> eventMono = Mono.fromFuture(eventGetFuture);
return ServerResponse.ok().body(eventMono, Event.class);
}
Mono<ServerResponse> saveEvent(ServerRequest request) {
Mono<Event> eventMono = request.bodyToMono(Event.class);
eventMono.map(dynamoDbService::saveEvent);
return ServerResponse.ok().build();
}
}
}
Spring DynamoDB Integration Test
Maven Dependencies
For running integration test with DynamoDB we need DynamoDBLocal, which is not really the DynamoDB, but SQLite with implemented DynamoDB interfaces on top of it.
xxxxxxxxxx
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>DynamoDBLocal</artifactId>
<version>1.12.0</version>
<scope>test</scope>
</dependency>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.10</version>
<executions>
<execution>
<id>copy</id>
<phase>test-compile</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<includeScope>test</includeScope>
<includeTypes>so,dll,dylib</includeTypes>
<!--Keep an eye on output directory - it will be used for starting dynamodb-->
<outputDirectory>${project.basedir}/target/native-libs</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<repositories>
<repository>
<id>dynamodb-local-oregon</id>
<name>DynamoDB Local Release Repository</name>
<url>https://s3-us-west-2.amazonaws.com/dynamodb-local/release</url>
</repository>
</repositories>
DynamoDB server
Now we need to start DynamoDB before the test run. I prefer to do it as JUnit Class Rule, but we can also do it as a spring bean.
xxxxxxxxxx
public class LocalDynamoDbRule extends ExternalResource {
protected DynamoDBProxyServer server;
public LocalDynamoDbRule() {
//here we set the path from "outputDirectory" of maven-dependency-plugin
System.setProperty("sqlite4java.library.path", "target/native-libs");
}
protected void before() throws Exception {
this.server = ServerRunner
.createServerFromCommandLineArgs(new String[]{"-inMemory", "-port", "8000"});
server.start();
}
protected void after() {
this.stopUnchecked(server);
}
protected void stopUnchecked(DynamoDBProxyServer dynamoDbServer) {
try {
dynamoDbServer.stop();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
Running Test
Now we can create an integration test and test get event by id and save the event.
xxxxxxxxxx
(SpringRunner.class)
(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class IntegrationTest {
public static LocalDynamoDbRule dynamoDbRule = new LocalDynamoDbRule();
private WebTestClient webTestClient;
public void getEvent() {
// Create a GET request to test an endpoint
webTestClient
.get().uri("/event/1")
.accept(MediaType.APPLICATION_JSON)
.exchange()
// and use the dedicated DSL to test assertions against the response
.expectStatus().isOk()
.expectBody(String.class).isEqualTo(null);
}
public void saveEvent() throws InterruptedException {
Event event = new Event("10", "event");
webTestClient
.post().uri("/event/")
.body(BodyInserters.fromValue(event))
.exchange()
.expectStatus().isOk();
Thread.sleep(1500);
webTestClient
.get().uri("/event/10")
.accept(MediaType.APPLICATION_JSON)
.exchange()
.expectStatus().isOk()
.expectBody(Event.class).isEqualTo(event);
}
}
Docker
Here we gonna prepare our application for running in docker, so it will be ready for deploying to AWS.
HINT: Starting from Java 10 you can specify how much memory JVM will use depending on container memory.-XX:MaxRAMPercentage=75.0 means JVM won't use more than 75% of a container memory.
Dockerfile
xxxxxxxxxx
# Use our standard java12 baseimage
FROM openjdk:12-alpine
# Copy the artifact into the container
COPY target/dynamodb-spring-*-exec.jar /srv/service.jar
# Run the artifact and expose the default port
WORKDIR /srv
ENTRYPOINT [ "java", \
"-XX:+UnlockExperimentalVMOptions", \
"-XX:+ExitOnOutOfMemoryError", \
"-XX:MaxRAMPercentage=75.0", \
"-Djava.security.egd=file:/dev/./urandom", \
"-jar", "service.jar", \
"--spring.profiles.active=prod" ]
EXPOSE 8080
Building the docker container itself docker build -t spring-dynamo.
Also, let's see what was generated by docker image ls.
xxxxxxxxxx
REPOSITORY TAG IMAGE ID CREATED SIZE
spring-dynamo latest a974d880400e About a minute ago 364MB
openjdk 12-alpine 0c68e7c5b7a0 12 months ago 339MB
Finally, our POC is ready!
Happy coding :)
Published at DZone with permission of Yegor Bondarenko. See the original article here.
Opinions expressed by DZone contributors are their own.
Comments