DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Coding
  3. Languages
  4. The Power of Transformation, Part I: An Introduction to DataWeave

The Power of Transformation, Part I: An Introduction to DataWeave

MuleSoft's DataWeave allows you to perform a variety of transformations, whether you're working with HTTP, JSON, or SOAP XML.

Harish Kumar user avatar by
Harish Kumar
·
Feb. 15, 17 · Opinion
Like (5)
Save
Tweet
Share
5.44K Views

Join the DZone community and get the full member experience.

Join For Free

When you work with the integration of different systems, sooner or later, you will have to transform data. In today’s world, very few systems talk in the same data language. For example, you may have an HTTP server that understands JSON, but it needs to talk to a server that only understands SOAP XML. Even if it understands JSON, the chances of having same JSON schema or format is very slim, so in these cases, you need to do transform the data.

Mule provides a very powerful tool to do these kinds of transformations: DataWeave.

DataWeave (DW)

DW is a new data transformation engine of Mule. DW is the replacement to DataMapper. You may have used DataMapper before; it provided a graphical drag and drop approach to data transformation. Please keep in mind that from Mule 3.7, Mule runtime hasn't had DataMapper and if you want to use it, you need to separately provide the dependencies. From Mule v4, there will be no longer support for DataMapper! So, I highly encourage everyone to either move to DW or convert your DM.

Canonical Data Format

Before we embark on the journey to understand DW, we need to understand that DW is designed in canonical data format standards. In simple terms, canonical data format is a way in which we represent any kind of data or object in a standard format. Any expression we write in DW will be in canonical format.

There are only three data types that can result from a DataWeave expression:

  1. Simple types (like strings and numbers).
  2. Arrays.
  3. Objects.

Each of these can be expressed literally as below:

Simple Type Literal “This is simple type”
“This too”
2016
Array Literal [1,2,3,4]
[“One”,”Two”,”Three”]
Object Literal {
“company” : “IT Ventures”,
“employees”: [
{“firstName”:”John”, “lastName”:”Doe”},
{“firstName”:”Anna”, “lastName”:”Smith”},
{“firstName”:”Peter”, “lastName”:”Jones”}
]
}

DataSense and DataWeave

When we work with DW, we get to use the powerful abilities of DataSense. With DataSense, you get suggestions and views of data with which you are working. You should also be mindful of the mime types of the Message data that you have to transform. When this is not set explicitly at design time and you use any of your connectors, this will default to application/Java.

For HTTP requests and responses, Mule will look at the content type header and set it accordingly. You should set the mime type explicitly when you are using the set variable and set payload processors. Failing to set the mime type explicitly for XML content will result in it being interpreted as a mere Java string by the engine.

mimetypeVariable Reference Expression

Understanding DW is all about understanding how DW engine normalizes your input data and how it is represented by the engine internally whenever references are made to variables, session variables, inbound properties, outbound properties, etc. All these expressions convert the incoming data into the canonical DW format that we discussed above. So, thinking in terms of DW simple types, arrays, and objects will help to write better transformation.

Below is the sample of XML conversion to DW literals. If we change the output type to application/DW, we can see that the conversion is done by the engine of the input XML. Here is how all the elements of XML are normalized as a key:value pair in the object:

dw_literal

In the case of complex element types, the normalized value is itself an object and repeating values are normalized as repeating key:value pairs.

Anypoint Studio

Anypoint Studio supports DW development with a DW editor. In DW editor, you will find three panes.

On the left, DataSense displays the structure of the incoming message along with any example of data you have provided.

On the right pane, the expected outgoing structure is displayed as per DataSense. It will also display the design time result of the transformation that constantly refreshes itself as the expression of transformation is written.

dwsample

The middle pane is where you write your transformation logic. It is divided into two sections. The header is where the declaration of mime type of transformation output is done, along with some reusable functions and global variables as also namespaces for XML use cases. The mandatory part of the header is the declaration of the DW version (i.e., %dw 1.0 and the output mime type).

Below the dotted line is where you write the transformation expressions. Only one expression is needed for this. There is a number of expression types, but the majority of the transformations can be done with semi-literal expressions of DW objects.

The same DW transformer can produce multiple outputs from the same incoming message. This can be achieved by clicking the plus circle at the bottom right of the transformation pane. With this, you will get another transformation pane in which you can select the output from the drop-down at the top of the transformation pane. The important thing to note is that you can use different flow vars or specify different targets for your transformation.

Output Rendering

The DW engine separates the actual transformation process within the canonical format from the final rendering of the same in the output mime type you defined in the header.

Irrespective of how complex your expression is, if it’s an object with deep-nested structure and expressions and operators which combine expressions, all of these must be executed first and return their value before the outer expression returns its value. It is this final value that is rendered in the output mime type.

Be aware of the choice of mime type, as there will be some limitations associated with them. Arrays will not be rendered in XML. Choose to repeat keys instead. Likewise, repeating keys cannot be rendered in JSON. Generate arrays instead.

To render an object to XML in line with the rule that XML documents may only contain one root element, the object may only contain one key:value pair. Its value can, as we have seen, itself be an object of any complexity.

Conclusion

This was an introduction to DW. I will be creating further videos and blogs related to DataWeave to go more in details of various ways to use DW in your transformation.

Let me know if you have found this blog helpful and informative.

Data (computing) Object (computer science) XML

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Best Practices for Writing Clean and Maintainable Code
  • The 31 Flavors of Data Lineage and Why Vanilla Doesn’t Cut It
  • What Is Policy-as-Code? An Introduction to Open Policy Agent
  • Web Application Architecture: The Latest Guide

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: