MuleSoft: Parse a Flat File to Copybook

Due to his success on modernizing legacy applications, Max’s boss contacted him again, asking his assistance to modernize a COBOL application. This is another main application in his company, which has been running for years and is still reliable. Without breaking the current production line, Max has to develop a new integration point to extract the potential value.

Max started preparing this integration by collecting the data format/structure, continued with the data transformation, and then stored the result. After a quick assessment, he determined that the input data is in the form of a Flat File, and the expected result is a Copybook.

He realizes that the Copybook format is still in early access. With a little doubt, he just keeps going!

Preparing the Data Definition/Schema

As the initial step, Max creates the Flat File definition file and save it as book.ffd in the folder src/main/resources.

form: FLATFILE
name: 'flatFile'
segments:
- id: 'Book'
  tag: '1'
  name: Book Collection
  values:
  - { name: 'title', type: String, length: 32 }
  - { name: 'author', type: String, length: 32 }
  - { name: 'amount', type: Integer, length: 5 }

Then, he continues the next definition file for Copybook. This file is named book.cpy and stored in the same folder.

       01  BOOK-RECORD.
           03  BOOK-KEY.
               05  BOOK-ISBN         PIC X(10).
           03  BOOK-TITLE            PIC X(32).
           03  BOOK-AUTHOR           PIC X(32).
           03  BOOK-CATEGORY         PIC X(10).
           03  BOOK-STOCK            PIC X(5).

Creating the Configuration File

To simulate the transformation process, Max creates the following flow:

• HTTP: As the message source, it listens for the user request.
• Transform message: The transformer responsible for transforming the Flat File to Copybook.
• Byte array to string: Transforms the transformation result as a string.
• File: Stores the transformation result to the src/main/resources/output folder and saves it as bookOrder.cpy.

Defining the Transformation

After opening the Transform Message properties, Max goes to the Input section and starts  defining the transformation:

1. Click the  Define metadata link, a new pop-up window is displayed.
2. Click Add to create a new metadata type.
3. Set the Type id to bookFlat and click Create type to confirm.
4. Set the Type with FLATFILE.
5. Select the Flat File schema file (book.ffd) from src/main/resources folder by clicking "…" (browse).
6. The message structure will be automatically populated.
7. Finish the first definition by clicking Select.

Next, he defines the Output metadata by repeating steps slightly similar to those above.

1. Click Define metadata link.
2. Click Add to create a new metadata type.
3. Set the Type id with bookCpy and click Create type to confirm.
4. Set the Type with COPYBOOK (Early Access).
5. Select the copybook file (book.cpy) to import by clicking "…" (browse).
6. The message structure will be automatically populated.
7. Finish this definition by clicking Select.

Then, he starts the mapping by dragging and dropping the associated fields from Input to the Output section. This action will trigger an automatic DataWeave expression creation. Later, he can modify or enhance the initial expression as required.

And eventually, he finalizes the mapping by enhancing the DataWeave expression to this:

%dw 1.0
%output text/plain schemaPath = "bookCpy.ffd" , segmentIdent = "BOOK-RECORD"
---
payload map ((payload01 , indexOfPayload01) -> {
    BOOK-KEY: {
        BOOK-ISBN: "000-" ++ indexOfPayload01
    },
    BOOK-TITLE: payload01.title,
    BOOK-AUTHOR: payload01.author,
    BOOK-CATEGORY: "Science",
    BOOK-STOCK: payload01.amount as :string
})

Running the Transformation

Prior to the live testing, Max prepares the data as an input value. He takes a simple example from the previous integration:

Mule in Action                  David Dossot & John D Emic         50
Undisturbed REST                Mike Stowe                        100

Then he starts the Mule application and executes a simple HTTP Post request from a REST Client application, Postman.

He waits for the transformation process for a while and gets a new file:

000-0     Mule in Action                  David Dossot & John D Emic      Science   50   
000-1     Undisturbed REST                Mike Stowe                      Science   100  

With a big smile on his face, Max successfully parsed a Flat File to Copybook!