Comparing Two Arrays Using Dataweave

While Dataweave 2.0 provides several out-of-the-box functions to transform the data, sometimes we need to orchestrate more than one such function to achieve the desired result. One such thing we often encounter is comparing two arrays and finding out the differences. Let's check out the different scenarios and see how we can achieve them.

Scenario 1: Comparing Two Arrays With Identical Structure

In this case, both arrays will have the same fields. If we want to see the items which are present in array 1 but not in array 2, we can use the "--" function.

Array 1 - defined as variable 'list1' :

[
	{
		"city": "Rotterdam",
		"country": "Netherlands"
	},
	{
		"city": "Amsterdam",
		"country": "Netherlands"
	},
	{
		"city": "Utrecht",
		"country": "Netherlands"
	}
]

Array 2 - defined as variable 'list2':

[
	{
		"city": "Rotterdam",
		"country": "Netherlands"
	},
	{
		"city": "Amsterdam",
		"country": "Netherlands"
	},
	{
		"city": "Eindhoven",
		"country": "Netherlands"
	}
]

Items present in list1 but not in list2:

vars.list1 -- vars.list2

The output of the above operation:

[
  {
    "city": "Utrecht",
    "country": "Netherlands"
  }
]

While the above scenario can be done easily with the default Dataweave operator, let's dive into the complex situation of comparing dissimilar arrays.

Scenario 2: Comparing Two Arrays With Different Structures

Array 1 - defined as variable 'list3':

[
	{
		"city": "Rotterdam",
		"country": "Netherlands"
	},
	{
		"city": "Amsterdam",
		"country": "Netherlands"
	},
	{
		"city": "Utrecht",
		"country": "Netherlands"
	}
]

Array 2 - defined as variable - 'list4':

[
	{
		"city": "Rotterdam",
		"country": "Netherlands",
		"province": "South Holland"
	},
	{
		"city": "Amsterdam",
		"country": "Netherlands",
		"province": "North Holland"
	},
	{
		"city": "Eindhoven",
		"country": "Netherlands",
		"province": "North Brabant"
	}
]

As we can see, the variable list4 has an extra field, 'province', which is not present in the variable list3. So, it won't be possible to compare both arrays with the default functions. 

In this case, below code snippet could help us to compare based on the identical fields between the two arrays:

vars.list3 -- (vars.list4 map {city: $.city, country: $.country})

The output of the above step:

[
	{
		"city": "Utrecht",
		"country": "Netherlands"
	}
]

If we want to find the elements in list4 but not in list3,

((list4 map {city: $.city, country: $.country}) -- list3) map (obj, ind) -> (list4 filter ($.city == obj.city and $.country == obj.country))[0]

Which gives the output as:

[
  {
    "city": "Eindhoven",
    "country": "Netherlands",
    "province": "North Brabant"
  }
]

Let's see what we are doing with the above expression:

(list4 map {city: $.city, country: $.country})

In the above step, we are trying to convert list4 into the same structure as list3.

((list4 map {city: $.city, country: $.country}) -- list3)

After converting the arrays into the same structure, we are finding out items that are in list4 but not in list3.

((list4 map {city: $.city, country: $.country}) -- list3) map (obj, ind) -> (list4 filter ($.city == obj.city and $.country == obj.country))[0]

After finding out the elements which are in list4 but not in list3, we are using that list to match with the elements in list4, so we get the complete structure of list4 only for the different items.