Processing Files Sequentially in Mule

In this article, I am going to show how we can process files in a sequence with File Connector in Mule.

Problem Statement

A local file system folder contains text files where the filenames are padded with sequence numbers such as file101.txt, file100.txt, and so on. We are supposed to append the contents of these files into a target file according to the ascending order of the sequence numbers associated with the filenames.

Pre-requisites

• Anypoint Studio 6+
• Mule ESB 3.8

Solution

• Create a Mule project with Anypoint Studio. Here, it was named sequential-file-processing.
• Create a folder named input in src/main/resources. The input folder contains the text files to be processed in ascending order.
• Create a folder named output in src/main/resources. The output folder contains the target file.
• In order to process files in an order, we need to create a Java class implementing java.util.Comparator interface. Create the Java class in src/main/java. In the article, it was named FilenameComparator. The purpose of this java class is to order the filenames in ascending order of the sequence numbers associated with the filenames so that the File inbound endpoint can read the files in that order.

package file;

import java.io.File;
import java.util.Comparator;

public class FilenameComparator implements Comparator<Object> {

    @Override
    public int compare(Object o1, Object o2) {

        File file1 = (File) o1;
        File file2 = (File) o2;

        int index1 = Integer.parseInt(file1.getName().substring(4, file1.getName().indexOf(".")));
        int index2 = Integer.parseInt(file2.getName().substring(4, file2.getName().indexOf(".")));

        if (index1 == index2) {
            return 0;
        } else if (index1 > index2) {
            return 1;
        } else {
            return -1;
        }
    }
}

• Create a Global Mule Configuration Element  for File  connector and configure as per the following figure.

• Drag the File endpoint from the palette and place it on the canvas and configure the properties as per figures below.

• Drag another File endpoint from the palette and place it in the process area of the flow and configure it as per the following figure. The purpose of this endpoint is to append the contents of the text files into the target file in ascending order.

• Configure the flow’s Processing Strategy to synchronous.

Complete XML Code

<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:tracking="http://www.mulesoft.org/schema/mule/ee/tracking" xmlns:file="http://www.mulesoft.org/schema/mule/file" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
http://www.mulesoft.org/schema/mule/ee/tracking http://www.mulesoft.org/schema/mule/ee/tracking/current/mule-tracking-ee.xsd">
    <file:connector name="SequentialFile" autoDelete="true" outputAppend="true" streaming="true" validateConnections="true" doc:name="File"/>
    <flow name="sequential-file-peocessingFlow" processingStrategy="synchronous">
        <file:inbound-endpoint path="src/main/resources/input" connector-ref="SequentialFile" responseTimeout="10000" comparator="file.FilenameComparator" doc:name="File">
            <file:filename-regex-filter pattern="^.*\.txt$" caseSensitive="true"/>
        </file:inbound-endpoint>
        <file:outbound-endpoint path="src/main/resources/output" outputPattern="output.txt" connector-ref="SequentialFile" responseTimeout="10000" doc:name="File"/>
    </flow>
</mule>

Result

Create the following line sequential text files in the src/main/resources/input  folder.

Run the mule application and open the output.txt  file from src/main/resources/output folder. The file contents should be as per the figure below.

The result depicted in the above figure assures that the input files were processed in the ascending order of the filename sequence numbers.