Sticky Sessions With Apache APISIX — The Demo
When it comes to sticky sessions, if necessary, you should replicate the data to other upstreams because this one might go down. This post illustrates it with a demo.
CORE
·
Join the DZone community and get the full member experience.
Join For FreeLast week, we described the concept behind sticky sessions: you forward a request to the same upstream because there's context data associated with the session on that node. However, if necessary, you should replicate the data to other upstreams because this one might go down. In this post, we are going to illustrate it with a demo.
The Overall Design
Design options are limitless. I'll keep myself to a familiar stack, the JVM. Also, as mentioned in the previous post, one should only implement sticky sessions with session replication. The final design consists of two components: an Apache APISIX instance with sticky sessions configured and two JVM nodes running the same application with session replication.
The application uses the following:
| Dependency | Description |
|---|---|
| Spring Boot | Eases the usage of Spring libraries |
| Spring MVC | Allows offering HTTP endpoints |
| Thymeleaf | View technology |
| Spring Session | Offers an abstraction over session replication |
| Hazelcast (embedded) | Implements session replication |
| Spring Security | Binds an identity to a user session |
The design looks like the following:
app1 and app2 are two instances of the same app; I didn't want to overcrowd the diagram with redundant data.
The Heart of the Application
The heart of the application is a session-scoped bean that wraps a counter, which can only be incremented:
@Component
@SessionScope
public class Counter implements Serializable { //1
private int value = 0;
public int getValue() {
return value;
}
public void incrementValue() {
value++;
}
}- Necessary for Hazelcast serialization to work
We can use this bean in the controller:
@Controller
public class IndexController {
private final Counter counter;
public IndexController(Counter counter) { //1
this.counter = counter;
}
@GetMapping("/")
public String index(Model model) { //2
counter.incrementValue();
model.addAttribute("counter", counter.getValue());
return "index";
}
}2. Inject the session-scoped bean in the singleton controller thanks to Spring's magic
3. When we send a GET request to the root, increment the counter value and pass it to the model
Finally, we display the bean's value on the Thymeleaf page:
<!DOCTYPE html>
<html xmlns:th="http://www.thymeleaf.org" lang="en">
<body>
<div th:text="${counter}">3</div>Configuring Spring Session with Hazelcast
Spring Session offers a filter that wraps the original HttpServletRequest to override the getSession() method. This method returns a specific Session implementation backed by the implementation configured with Spring Session, in our case, Hazelcast.
We need only a few tweaks to configure Spring Session with Hazelcast.
First, annotate the Spring Boot application class with the relevant annotation:
@SpringBootApplication
@EnableHazelcastHttpSession
public class SessionApplication {
...
}Hazelcast requires a specific configuration as well. We can use XML, YAML, or code.
Since it's a demo, I can choose whatever I want, so let's code it. Spring Boot requires either an Hazelcast object or a configuration object. The latter is enough:
@Bean
public Config hazelcastConfig() {
var config = new Config();
var networkConfig = config.getNetworkConfig();
networkConfig.setPort(0); //1
networkConfig.getJoin().getAutoDetectionConfig().setEnabled(true); //2
var attributeConfig = new AttributeConfig() //3
.setName(HazelcastIndexedSessionRepository.PRINCIPAL_NAME_ATTRIBUTE)
.setExtractorClassName(PrincipalNameExtractor.class.getName());
config.getMapConfig(HazelcastIndexedSessionRepository.DEFAULT_SESSION_MAP_NAME) //3
.addAttributeConfig(attributeConfig)
.addIndexConfig(new IndexConfig(
IndexType.HASH,
HazelcastIndexedSessionRepository.PRINCIPAL_NAME_ATTRIBUTE
));
var serializerConfig = new SerializerConfig();
serializerConfig.setImplementation(new HazelcastSessionSerializer()) //3
.setTypeClass(MapSession.class);
config.getSerializationConfig().addSerializerConfig(serializerConfig);
return config;
}- Choose a random port to avoid port conflict
- Allow Hazelcast to search for other instances and automatically form a cluster. It's going to be necessary when deployed as per our design
- Copy-pasted from the Spring Session documentation
Configuring Spring Security
Most Spring Session examples somehow use Spring Security, and though it's not strictly necessary, it makes the design easier. I want to explain why first.
One can think about sessions as a gigantic hash table. In regular applications, the key is the JSESSIONID cookie value, the value, another hash table of session data. However, the JSESSIONID is specific to the node. The app will give a different JSESSIONID if one uses another node. Since the key is different, there's no way to access the session data, even if the session data is shared across nodes. To prevent this loss, we need to come up with a different key. Spring Security allows using a principal (or the login name) as the session data key.
Here's how I set up a basic Spring Security configuration:
@Bean
public SecurityFilterChain securityFilterChain(UserDetailsService service, HttpSecurity http) throws Exception {
return http.userDetailsService(service) //1
.authorizeHttpRequests(authorize -> authorize.requestMatchers(
PathRequest.toStaticResources().atCommonLocations()) //2
.permitAll() //2
.anyRequest().authenticated() //3
).formLogin(form -> form.permitAll()
.defaultSuccessUrl("/") //4
).build();
}- The default in-memory user details service doesn't allow custom user details classes. I had to provide my own.
- Allow everybody to access static resources at "common" locations
- All other requests must be authenticated
- Allow everybody to access the authentication form
- Redirect to the root if successful, which maps the above controller
Putting Our Design to the Test
Beside the counter, I want to display two additional pieces of data: the hostname and the logged-in user.
For the hostname, I add the following method to the controller:
@ModelAttribute("hostname")
private String hostname() throws UnknownHostException {
return InetAddress.getLocalHost().getHostName();
}Displaying the logged-in user requires an additional dependency:
<dependency>
<groupId>org.thymeleaf.extras</groupId>
<artifactId>thymeleaf-extras-springsecurity6</artifactId>
</dependency>On the page, it's straightforward:
<!DOCTYPE html>
<html xmlns:th="http://www.thymeleaf.org"
xmlns:sec="http://www.thymeleaf.org/extras/spring-security"> <!--1-->
<body>
<td sec:authentication="principal.label">Me</td> <!--2-->- Add the
secnamespace. It's not necessary but may help the IDE to help you - Require the underlying
UserDetailimplementation to have agetLabel()method
Last but not least, we need to configure Apache APISIX with sticky sessions, as we saw last week:
routes:
- uri: /*
upstream:
nodes:
"webapp1:8080": 1
"webapp2:8080": 1
type: chash
hash_on: cookie
key: cookie_JSESSIONID
#ENDHere's the design implemented on Docker Compose:
services:
apisix:
image: apache/apisix:3.3.0-debian
volumes:
- ./apisix/config.yml:/usr/local/apisix/conf/config.yaml:ro
- ./apisix/apisix.yml:/usr/local/apisix/conf/apisix.yaml:ro #1
ports:
- "9080:9080" #2
depends_on:
- webapp1
- webapp2
webapp1:
build: ./webapp
hostname: webapp1 #3
webapp2:
build: ./webapp
hostname: webapp2 #3- Use the previous configuration file
- Only expose the API Gateway to the outside world
- Set the hostname to display it on the page
We can log in using one of the two hard-coded accounts. I'm using john, with password john and label John Doe. Notice that Apache APISIX sets me on a node and keeps using the same if I refresh.
We can try to log in with the other account (jane/jane) from a private window and check the counter starts from 0.
Now comes the fun part. Let's stop the node which should be hosting the session data, here webapp2 and refresh the page:
docker compose stop webapp2We can see exciting things in the logs. Apache APISIX can no longer find the webapp2, so it forwards the request to the other upstream that it knows, webapp1.
- The request is still authenticated; it goes through Spring Security
- The framework gets the principal out of the request
- It queries Spring Session
- And gets the correct counter value that Hazelcast replicated from the other node
The only side-effect is an increased latency because of Apache APISIX timeout. It's 5 seconds by default, but you can configure it to a lower value if needed.
When we start webapp2 again, everything works as expected again.
Conclusion
In this post, I described a possible setup for sticky sessions with Apache APISIX and replication involving the Spring ecosystem and Hazelcast. Many other options are available depending on your stack and framework of choices.
The complete source code for this post can be found on GitHub.
To go further:
Published at DZone with permission of Nicolas Fränkel, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
Comments