Creating a Cluster File Storage With ActiveJ
In this tutorial we'll create a cluster storage with three partitions that communicate via TCP connection.
Join the DZone community and get the full member experience.Join For Free
In this tutorial we will create a cluster storage with three servers that communicate via TCP protocol. There also will be a cluster client. Each server will have an HTTP GUI server for simplicity of use and debugging. The cluster storage will support file uploads with automatic repartitioning according to the provided rule and replication count.
- ActiveFS - a lightweight abstraction for managing distributed file storage.
- ActiveInject - dependency injection library.
- ActiveJ HTTP - high-performance HTTP clients, servers and servlets.
- ActiveJ Launchers - a tool for managing application lifecycle.
- ActiveJ Eventloop - efficient management of asynchronous operations.
- ActiveJ Config - a useful extension for properties files.
ActiveFS is one of the most important technologies in this tutorial. It provides a lightweight abstraction on top of common file operations like upload, download, append, list, copy, move, delete, and others. It allows operations with local, remote or distributed file storage. ActiveFS is a one of the ActiveJ Java platform technologies.
Add all the required Maven dependencies to your project:
activej-launchers-fs module will import all the required ActiveJ technologies that were mentioned above.
The first thing we need to do is to create a launcher class BasicClusterServerLauncher for our servers. We’ll need the following instances:
- local ActiveFS,
- AsyncHttpServer for GUI that will simplify working with your cluster storage,
- utils for repartitioning management like task schedulers, ClusterRepartitionController, and FsPartitions for tracking alive partitions and their statuses.
The partitions will communicate via TCP protocol, while GUI server will use HTTP protocol.
Now we can move on to creating a client launcher ClusterTcpClientLauncher. We need to provide a task scheduler to detect dead partitions, AsyncHttpServer for GUI, remote ActiveFS, and FsPartitions for managing partitions.
We also need an instance of ClusterActiveFs class, an ActiveFs implementation that operates on other partitions as a cluster and contains some redundancy and fail-tolerance capabilities.
Here’s the architecture of our distributed P2P storage:
You can create as many partitions as you wish, cluster client is optional as you can create an alternative client implementation that supports ActiveFS protocol.
Let’s launch three partitions on different ports. For this purpose you need to create three Run/Debug configurations for ClusterTcpServerLauncher and provide the following VM options:
Next, provide the following VM options for ClusterTcpClientLauncher:
The storage will run with the following configurations:
Launch all the created configurations and go to
127.0.0.1:8080 in your browser to work with the storage. Switch between ports (
8083) to check your partitions’ GUI.
You can try to upload files, create directories, kill some partitions to check repartitioning.
Published at DZone with permission of Valeria Listratova. See the original article here.
Opinions expressed by DZone contributors are their own.