Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Integrating Mule With data.world to Monitor Bigfoot

DZone's Guide to

Integrating Mule With data.world to Monitor Bigfoot

Using Mule to integrate with data.world will give you access to a data resource repository for various uses. Maybe you can even find a certain elusive creature...

· Integration Zone
Free Resource

Modernize your application architectures with microservices and APIs with best practices from this free virtual summit series. Brought to you in partnership with CA Technologies.

Mule ESB helps us integrate with different systems. In this example, we will see how to connect to data.world, a data resource repository, and retrieve some data. Let’s integrate Mule with data.world to monitor Bigfoot sightings around us.

What Is Data.world?

data.world is a data resource repository that contains probably thousands of datasets. Many of these datasets are public and shared with public domain license.

There are two options to connect to data.world and both uses dwsql to retrieve data -

  1. JDBC Driver - data.world has a JDBC driver that can be used in any JVM based tool.
  2. Query API - It also exposes Query APIs that allows us to run SQL statements to retrieve data.

NOTE: The dataset chosen is for demo purposes only, I have not validated the dataset. So, if it says Bigfoot is not in your state but tomorrow he knocks your door, don’t blame Mule or me for not alerting you!

Mule Integration

We will integrate Mule with data.world using both options and build an HTTP endpoint to query the number of sightings of Bigfoot in the requested state.

Prerequisite

  1. Mule Anypoint Studio and access to Mule Resources.

  2. To access data.world, you will need to register on their site and get your API token at https://data.world/settings/advanced.

Using JDBC

This approach is similar to integrating any other traditional database.

  • Add a Maven dependency to your project -

<dependency>
    <groupId>world.data</groupId>
    <artifactId>dw-jdbc</artifactId>
    <version>0.4.1</version>
</dependency>
  • Create database connection config -

<db:generic-config name="Generic_Database_Configuration"
  driverClassName="world.data.jdbc.Driver"
  url="jdbc:data:world:sql:timothyrenner:bfro-sightings-data;user=manikmagar;password=abcdefghi"
  doc:name="Generic Database Configuration"/>
  • URL structure :

URL:  jdbc:data:world:sql:[dataset owner userid]:[dataset id]
User: your data.world username
Password: your data.world API token
  • Integration flow:

HTTP listener path: http://localhost:8081/data/jdbc?state=Delaware

It should output: Number of Bigfoot sightings in Delaware: 5

<flow name="mule-demo-data-worldFlow" initialState="started">
       <http:listener config-ref="HTTP_Listener_Configuration" path="/data/jdbc" doc:name="HTTP"/>
       <set-variable value="#[message.inboundProperties.'http.query.params'.state]" variableName="state" doc:name="Variable" />
      <db:select config-ref="Generic_Database_Configuration"
      doc:name="Database" >
              <db:parameterized-query><![CDATA[SELECT * FROM bfro_reports where state = #[flowVars.state]]]></db:parameterized-query>
      </db:select>
    <set-payload value="#['Number of Bigfoot sightings in '+ flowVars.state +':' + payload.size()]" doc:name="Set Payload"/>
        <logger message="#[payload ]" level="INFO" doc:name="Logger"/>
</flow>

Using Query API

  • HTTP Request config for data.world API endpoint -

<http:request-config name="DW_HTTPS_Request_Configuration"
  protocol="HTTPS"
  port="443"
  host="query.data.world"
  basePath="/sql"
  doc:name="HTTP Request Configuration"/>

basePath must be /sql to run SQL queries.

  • Integration flow:

HTTP listener path: http://localhost:8081/data/api?state=Delaware

It should output: Number of Bigfoot sightings in Delaware: 5

<flow name="mule-demo-data-worldFlow1">
        <http:listener config-ref="HTTP_Listener_Configuration" path="/data/api" doc:name="HTTP"/>
        <set-variable value="#[message.inboundProperties.'http.query.params'.state]" variableName="state" doc:name="Variable" />
        <set-payload value="#[&quot;SELECT * FROM bfro_reports where state = '&quot;+ flowVars.state + &quot;'&quot;]"
          doc:name="Set Payload" mimeType="text/plain"/>
        <http:request config-ref="DW_HTTPS_Request_Configuration"
          path="/timothyrenner/bfro-sightings-data" method="GET" doc:name="HTTP">   (1)
            <http:request-builder>
                <http:query-param paramName="query" value="#[payload]"/>
                <http:header headerName="accept" value="application/json"/>
                <http:header headerName="authorization" value="Bearer abcdefgh"/> (2)
            </http:request-builder>
        </http:request>
        <byte-array-to-string-transformer mimeType="application/json" doc:name="Byte Array to String"/>
        <json:json-to-object-transformer returnClass="java.util.HashMap" doc:name="JSON to Object"/>
        <set-payload value="#['Number of Bigfoot sightings in '+ flowVars.state +':' + payload.results.bindings.size()]" doc:name="Set Payload"/>
        <logger message="#[payload]" level="INFO" doc:name="Logger"/>
    </flow>
  1. Path format is /[dataset owner userid]/[dataset id]

  2. Bearer should have your API token, e.g. abcdefgh

Conclusion

Integrations are easy!

References

  1. Data World integrations - https://data.world/integrations/java

  2. Data World API Doc - https://dwapi-query.api-docs.io/v0/sql/sql-query
  3. Bigfoot Dataset - https://data.world/timothyrenner/bfro-sightings-data
  4. Mule Database Connector - https://docs.mulesoft.com/mule-user-guide/v/3.8/database-connector
  5. Mule Http Connector - https://docs.mulesoft.com/mule-user-guide/v/3.8/http-connector

The Integration Zone is proudly sponsored by CA Technologies. Learn from expert microservices and API presentations at the Modernizing Application Architectures Virtual Summit Series.

Topics:
mule esb ,integration ,data repository ,mule

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}