In the last article, you can find the idea of how to introduce field filtering for your APIs. My simplified Content Filter allows for providing negative or positive filtering. The first one tells which fields Filter removes from the target response. The latter one tells which fields should be available in the response. Today I show you how I have implemented this using DataWeave language.

Filterable DataWeave Module

Okay, our input is straightforward. The Content Filter has two input arguments, like in the diagram below

Filter Parameter

As you remember, we should be able to do positive or negative filtering. We can filter many fields that are in a comma-separated form. In order to distinguish negative filtering, each field should have a minus prefix like -type. In the case of positive filtering, no prefix is required.

I have also introduced *all keyword as a default one to represent the idea of all fields. We can combine it with both filters. Below you can see two ways of removing single type property from the object:

x

*all,-type

-type

Filtered Entity

The second argument for our Content Filter is the entity on which the filter will be applied. We should be either an object or an array.

Below you find the code implementation in DataWeave.

DataWeave module:

You can reuse your DataWeave code by either creating a custom module or a mapping file. This is applied only to Mule 4. For DataWeave 1.0 we have readUrl function to read DataWeave external file.

Filterable Module

Parsing the Filter

First, we need to parse the field list. As you can see in the code below, we split the string by comma and remove the *all string if it exists.

xxxxxxxxxx

fun parseFieldsFilter(fieldsList) =

    do {

     var fields = fieldsList default "" splitBy ","  filter (value) -> (value != "*all")

     ---

     {

        fields: fields,

        "type": fields match {

            case items if sizeOf (items) == 0 -> "*"

            case items if items every ($ startsWith "-") -> "-"

            else -> "+"

        }

     }

    }

As a result, we receive an object with the fields array and the type of filtering. Here is an example: 

xxxxxxxxxx

*all,-type,-name,-surname

After applying parseFieldsFilter function, we receive: 

xxxxxxxxxx

{

   "fields": ["-type", "-name", "-surname"],

   "type": "-"

}

For the specified example, we know that we should apply negative filtering and remove fields type, name, and surname from the output.

Filter by Match

We need to apply the filter on the supplied object. Three cases can appear:

1. Nothing to filter, all fields should be return
2. Remove specified fields from the output
3. Persist only fields specified

xxxxxxxxxx

fun filterItem(payload, filter) = 

    filter."type" match {

        case "*" -> payload

        case "-" -> negativeFilter(payload, filter)

        case "+" -> positiveFilter(payload, filter)

        else -> payload

    }

Function filterItem accepts payload and filter. Using the match function, we either do the negative, positive filtering, or nothing at all — more about the match you can find here.

Filter on Either Object or Array

Last one thing to consider. Content Filter should be able to apply the filter on both Object and Array types. Below you can find a code snippet with the function filterBy.

xxxxxxxxxx

fun filterItem(payload, filter) = 

    filter."type" match {

        case "*" -> payload

        case "-" -> negativeFilter(payload, filter)

        case "+" -> positiveFilter(payload, filter)

        else -> payload

    }

One more time, I have used the match function to verify the incoming type. As you can see in the case of an Object, I call the filterItem function. In the case of an Array, I iterate over each element and apply the same function. Other input types the function ignores.

Apply the Logic

I have encapsulated the filtering logic within the module. Below you can see a DataWeave snippet applying the filtering logic on the payload. The list of fields to filter is coming from the field query parameters.

%dw 2.0

output application/json

import filterBy from modules::FilterFieldsModule

---

filterBy(payload, attributes.queryParams.fields)

Source Code:

Source Code is available at my GitHub account here. If you have any comments or questions regarding the code don’t hesitate to write to me.

Summary

As I have stated in my previous blog post, this filtering logic may be to complex for some scenarios. However, it can be extended or simplified easily. The design and implementation, in my opinion, is in the middle – no overcomplicated. I hope that it helps to introduce filtering in your project.

Do you have any other design or implementation? Share it with the community. 

Cheers!