DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Last call! Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • An Introduction to Object Mutation in JavaScript
  • Generic and Dynamic API: MuleSoft
  • Understanding the Differences Between Repository and Data Access Object (DAO)
  • JSON-Based Serialized LOB Pattern

Trending

  • Stateless vs Stateful Stream Processing With Kafka Streams and Apache Flink
  • *You* Can Shape Trend Reports: Join DZone's Software Supply Chain Security Research
  • AI-Based Threat Detection in Cloud Security
  • Performance Optimization Techniques for Snowflake on AWS
  1. DZone
  2. Data Engineering
  3. Data
  4. MuleSoft: Map a Complex Object to DataWeave With a Fixed-Width Format

MuleSoft: Map a Complex Object to DataWeave With a Fixed-Width Format

Learn the steps needed (and the reasons why they're needed) to update a legacy app involving mapping complex objects to DataWeave with fixed widths.

By 
Sulthony H user avatar
Sulthony H
·
Aug. 10, 16 · Tutorial
Likes (4)
Comment
Save
Tweet
Share
23.3K Views

Join the DZone community and get the full member experience.

Join For Free

Max is more than happy because his legacy application can now be modernized. It can integrate with other modern applications, so he wants to do the same thing for another app. Currently, this application stores the data in a fixed-width format.

At the beginning, there is no significant impediment to achieving his plan. There is plenty of straightforward documentation. He started with a simple integration by receiving data in JSON form and transformed it to a fixed width.

Map a Simple Object to DataWeave With a Fixed Width

Here's our example:

[
    {
        "title": "Mule in Action",
        "author": "David Dossot & John D Emic",
        "amount": 50
    }
]


Here's the configuration flow to receive that input via HTTP POST:

Image title


Flat File Definition:

form: FIXEDWIDTH
name: 'flatFile'
values: 
- { name: 'title', type: String, length: 32 }
- { name: 'author', type: String, length: 32 }
- { name: 'amount', type: Integer, length: 5 }


DataWeave Code:

%dw 1.0
%output text/plain schemaPath = "simple.ffd"
---
payload map {
    title: $.title,
    author: $.author,
    amount: $.amount
}


And the result:

Mule in Action                  David Dossot & John D Emic         50


If he changes the input, by adding more JSON data, for example, he still gets the expected result.

[
    {
        "title": "Mule in Action",
        "author": "David Dossot & John D Emic",
        "amount": 50
    },
    {
        "title": "Undisturbed REST",
        "author": "Mike Stowe",
        "amount": 100
    }
]


The result will be updated accordingly:

Mule in Action                  David Dossot & John D Emic         50
Undisturbed REST                Mike Stowe                        100


Map a Complex Object to DataWeave With a Fixed Width

This is where Max gets frustrated. He followed everything in the documentation. He implemented all the suggested solutions from his favorite developer forum. He modified the Flat File Definition structure with every possible format. But nothing's worked. He's still unable to get the expected result when transforming a complex object.

Here is the example of input data:

{
    "books": [
        {
            "title": "Mule in Action",
            "author": "David Dossot & John D Emic",
            "amount": 50
        },
        {
            "title": "Undisturbed REST",
            "author": "Mike Stowe",
            "amount": 100
        }
    ],
    "otherPublications": {
        "pamphlet": 75
    }
}


Flat File Definition:

form: FIXEDWIDTH
structures:
- id: 'publication'
  name: publication
  tagStart: 0
  data:
  - { idRef: 'books', count: '>1'}
  - { idRef: 'otherPublications', count: 1 }
segments:
- id: 'books'
  name: books
  tag: 'books>'
  values:
  - { name: 'title', type: String, length: 32 }
  - { name: 'author', type: String, length: 32 }
  - { name: 'amount', type: Integer, length: 5 }
- id: 'otherPublications'
  name: otherPublications
  tag: 'others>'
  values:
  - { name: 'pamphlet', type: Integer, length: 5 }


DataWeave Code:

%dw 1.0
%output text/plain structureIdent = "publication" , schemaPath = "complex.ffd"
---
{
    books: payload.books map ({
        title: $.title,
        author: $.author,
        amount: $.amount
    }),
    otherPublications: {
        pamphlet: payload.otherPublications.pamphlet
    }
}


But no matter what he does, he only gets this:

others>   75


Apparently, the books data is not recognized by DataWeave. He then tried to modify the definition. But it still returns the same result. Rereading the documentation still didn't help him.

In the midst of his despair, he tried to replace the DataWeave code with the example input data. Obviously, he gets the same result.

But what if he modified the DataWeave code with a direct transformation, for the books data in particular?

%dw 1.0
%output text/plain structureIdent = "publication" , schemaPath = "complex.ffd"
---
{
    books: [{
        title: payload.books[0].title,
        author: payload.books[0].author,
        amount: payload.books[0].amount
    },{
        title: payload.books[1].title,
        author: payload.books[1].author,
        amount: payload.books[1].amount
    }],
    otherPublications: {
        pamphlet: payload.otherPublications.pamphlet
    }
}


He got it!

books>Mule in Action                  David Dossot & John D Emic         50
books>Undisturbed REST                Mike Stowe                        100
others>   75


But be careful, this is not a best practice, and it's prone to error.

When he reverts back to the previous DataWeave code, he'll get the incorrect result. Why can this happen? Especially whereas both codes produce the same JSON data.

This could be a bug in DataWeave...

However, he then found a tricky solution to solve this issue.

  1. Ensure the books collection is an Array by adding a bracket: [ ... map ...]
  2. Remove the additional bracket using flatten, because the map itself already returns an Array.

Hence, he creates the following code to map a complex object to DataWeave with a fixed width:

%dw 1.0
%output text/plain structureIdent = "publication" , schemaPath = "complex.ffd"
---
{
    books: flatten [payload.books map ({
        title: $.title,
        author: $.author,
        amount: $.amount
    })],
    otherPublications: {
        pamphlet: payload.otherPublications.pamphlet
    }
}


Finally, Max gets the expected result. He solves a tricky problem with a tricky solution. And again, he successfully modernized another legacy application!

Object (computer science) Data (computing) MuleSoft

Opinions expressed by DZone contributors are their own.

Related

  • An Introduction to Object Mutation in JavaScript
  • Generic and Dynamic API: MuleSoft
  • Understanding the Differences Between Repository and Data Access Object (DAO)
  • JSON-Based Serialized LOB Pattern

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!