DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Software Design and Architecture
  3. Integration
  4. Integrating Big Data Platforms With Bedrock: REST API

Integrating Big Data Platforms With Bedrock: REST API

With the robust REST API that's built into Bedrock, integrations are possible with virtually any product, and Bedrock can be used without the UI.

Adam Diaz user avatar by
Adam Diaz
·
Feb. 09, 17 · Opinion
Like (1)
Save
Tweet
Share
3.32K Views

Join the DZone community and get the full member experience.

Join For Free

In conversations about the Bedrock Data Lake Management Platform, the most common question we hear is, “Can Bedrock integrate with Product X?” It's almost immediately followed by, “Can I use Bedrock without the UI?” The answer to both questions is yes — and it’s all made possible by the robust REST API that is built into Bedrock.

Screen Shot 2017-02-01 at 12.17.29 PM.png

This image displays an example of the interactive API documentation that is built on Swagger. Although classic documentation formats are available via PDF, you’ll want to check this out. The interactive documentation expedites the learning curve for making basic calls and crafting the correct input syntax for the API you are trying to use.

Understanding API Calls Through the Command Line

For example, you may want to understand variables in the Bedrock Namespace. Much like environmental variables in a Linux shell, Bedrock has a Namespace used to pass variables to different parts of the product. The command line below is a great example of what this will look like.

[bedrock@hostname]$ curl -D token.json -H "Content-Type: application/json" \
-X POST -d '{"username":"YYYYY","password":"XXXXX"}' \
'http://localhost:8080/bedrock-app/services/rest/admin/getUserRole'
 
[bedrock@hostname]$ JSESSIONID=$(cat /path/to/token/token.json | grep
JSESSIONID: | cut -d' ' -f 2)
 
[bedrock@hostname]$ curl -H "JSESSIONID: $JSESSIONID" -X POST --header
'Content-Type: application/json' --header 'Accept: application/json' -d '{}'
'http://localhost:8080/bedrock-app/services/rest/workflows/search?projectIds=9&
api_key=saveEntityInstance'
 
{
 "responseMessage": "success",
 "restUri": null,
 "result": {
   "currentPage": 1,
   "totalRecords": 2,
   "workFlowDetails": [
     {

 
          "CSVValues":
"59,Automated_Ingest_DQ_TM_noWM,unassigned,admin,11/18/2016
12:36:37,admin,11/22/2016 17:19:51",
       "category": "unassigned",
       "createdBy": "admin",
       "createdDate": "11/18/2016 12:36:37",
       "modifiedBy": "admin",
       "modifiedDate": "11/22/2016 17:19:51",
       "wfId": 59,
       "wfName": "Automated_Ingest_DQ_TM_noWM"
     },
     {
       "CSVValues": "62,Automated_Ingest_DQ_TM_TT,unassigned,admin,11/18/2016

16:22:33,admin,11/21/2016 21:46:07",
       "category": "unassigned",
       "createdBy": "admin",
       "createdDate": "11/18/2016 16:22:33",
       "modifiedBy": "admin",
       "modifiedDate": "11/21/2016 21:46:07",
       "wfId": 62,
       "wfName": "Automated_Ingest_DQ_TM_TT"
     }
   ]
 },

"page": null
}

For project ID 9 (AKA Trouble Tickets in this system), there are two known workflows with IDs 59 and 62.

Practicing API Calls With the Swagger UI

Most users need some assistance when using a new API. It’s really helpful to have a way to practice iterating over the creation of a correct call before putting it into your process. In the interactive Swagger documentation, that same call would look something like the diagram below:

Screen Shot 2017-02-01 at 12.26.11 PM.pngUnder the workflow page, we navigate to the search API and then fill in the correct values. In this case, passing just a project ID is enough. This corresponds to the concept of projects in Bedrock. These can be listed out via the “Administration” menu under “Projects” or via another API call to obtain all project IDs.

curl -H "JSESSIONID: $JSESSIONID" -X GET --header 'Accept: application/json'
'http://localhost:8080/bedrock-app/services/rest/projects?api_key=saveEntityInstance'

RESPONSE BODY TRUNCATED:

{
 "responseMessage": "SUCCESS",
 "restUri": "/projects",
 "result": [
   {
     "projectId": 10,
     "projectName": "Taxi Data",
     "createdBy": "admin",
     "createdDateTime": 1479150403000,
     "canManageUserRoles": true,
     "isDefaultProject": false,
     "roleNames": [
       "Project Administrator"
     ]
   },
   {
     "projectId": 9,
     "projectName": "Trouble Tickets",
     "createdBy": "admin",
     "createdDateTime": 1478618016000, …

The final screen will show a response body along with the response header and response code (not shown here). This is a quick and easy way to interrogate and learn the REST API for Bedrock. There are actions for controlling the entire system. In fact, this is how the web interface for Bedrock operates.

API REST Web Protocols Big data

Published at DZone with permission of , DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • How Observability Is Redefining Developer Roles
  • What Is a Kubernetes CI/CD Pipeline?
  • How to Create a Real-Time Scalable Streaming App Using Apache NiFi, Apache Pulsar, and Apache Flink SQL
  • 7 Awesome Libraries for Java Unit and Integration Testing

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: