DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Last call! Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • How To Base64 Encode or Decode Content Using APIs in Java
  • How To Get Cell Data From an Excel Spreadsheet Using APIs in Java
  • Effective Java Collection Framework: Best Practices and Tips
  • The Generic Way To Convert Between Java and PostgreSQL Enums

Trending

  • Medallion Architecture: Why You Need It and How To Implement It With ClickHouse
  • A Complete Guide to Modern AI Developer Tools
  • Event-Driven Architectures: Designing Scalable and Resilient Cloud Solutions
  • Automatic Code Transformation With OpenRewrite
  1. DZone
  2. Data Engineering
  3. Data
  4. JBang: How to Script With Java for Data Import From an API

JBang: How to Script With Java for Data Import From an API

Faster development time in Java by writing standard syntax and skipping the project setup with JBang scripting to pull data from an API.

By 
Jennifer Reif user avatar
Jennifer Reif
DZone Core CORE ·
Nov. 03, 23 · Tutorial
Likes (3)
Comment
Save
Tweet
Share
7.8K Views

Join the DZone community and get the full member experience.

Join For Free

It's right in the middle of the busy conference season, and I was prepping for an upcoming conference talk.

As I often do, I went to Neo4j Aura to spin up a free database and use Cypher with APOC to import data from an API, but this API requires a header, and the APOC procedure that adds headers to a request is blocked by security in Aura. Hmm, I needed a new route.

I decided to try JBang, which is a tool for scripting with Java. I had heard about it but hadn't tried it yet. It's pretty cool, so I wanted to share my onboarding.

What Is JBang?

Java developers have lamented the lack of a scripting language for Java for years. JBang solves this problem. I found an excellent overview of JBang from a post on InfoQ (Scripting Java with a jBang).

JBang provides a way of running Java code as a script...[It] is a launcher script, written in bash and powershell, that can discover or download a JVM, and then (down)load the Java script given in an argument. The implementation of JBang is a Java JAR archive, which it then launches to execute further commands.

JBang can run jsh or java files; the latter is a standard Java class with a main() method. However, unlike JShell, comments at the top of JBang allow dependencies to be automatically downloaded and set up on the classpath. JShell allows adding JARs to the classpath at launch, but any (recursive) dependencies have to be added manually.

JBang seems like a nicer alternative to either using a full-fledged Java project or a Linux script. Let's get a bit more detail about the data API we will pull from before we dive into writing the script!

Setup: Install/Download

First, we need to install JBang from the download page. I had to find the download for my operating system and then choose an install type. Since I use SDKMan to manage my Java versions, I installed JBang with SDKMan, too.

Shell
 
sdk install jbang


Several IDEs have plugins for JBang, as well, including IntelliJ. The IntelliJ plugin seems to have several nice features, including import suggestions. However, I had trouble utilizing it from an existing project or randomly created script, but I had to create a separate project initialized with JBang. I probably need to play with this a bit more since it would simplify the import problem (discussed in a bit). Anyway, I decided to mess with the plugin later and just use the command line for now.

API Details

I wanted to import data for traveling with pets, and the Yelp Fusion API was one that I knew I wanted to use. This was also the one that required a header on the request, which led me down the path toward JBang in the first place.

The Yelp API has a really useful playground where I could test a few requests before I started writing the script. I also used the playground to verify syntax and get sample code for an API call in Java.

Write the Script

In the playground, you can choose the endpoint you want to hit, any parameters, as well as the language you want to use to make the request. I chose Java and the parameters I knew I needed, and it gave me the following code:

Java
 
OkHttpClient client = new OkHttpClient();

Request request = new Request.Builder()
  .url("https://api.yelp.com/v3/businesses/search?location=" + location + "&categories=" + category + "&attributes=dogs_allowed&limit=50&sort_by=distance")
  .get()
  .addHeader("accept", "application/json")
  .addHeader("Authorization", "Bearer " + yelpApiKey)
  .build();

Response response = client.newCall(request).execute();


Now, I tweaked the code a bit above to use placeholder variables for location, category, and yelpApiKey so that I could pass in arbitrary values later. The code sample from the playground auto-includes your API token, so I copied/pasted the block above into my JBang script, and then I needed to go back and add dependencies.

This was where JBang was a little less convenient and where an IDE plugin might come in handy. I had to go to Maven Central and search for the dependencies I needed. There isn't an auto-import, which makes sense since we don't have a dependency manager like Maven or Spring that could potentially search dependencies for useful import suggestions.

I also wanted to pull pet travel data from several (of the many) categories Yelp offers. Since there is a high request limit but a smaller result limit, I decided to hit the endpoint for each category independently to retrieve the maximum results for each category. I also wanted a parameter for the location so that I could pull data for different cities. Finally, I needed a file to output the results so that I wouldn't have to hit the API each time I might want to load the data. I added the following variables to the script:

Java
 
filename = "yelpApi.json";
String[] yelpCategories = {"active","arts","food","hotelstravel","nightlife","pets","restaurants","shopping"};
String location = "New%20York%20City";


Last but not least, I needed to create the JSON object to format and hold the results and then write that to the JSON file.

Java
 
try {
    JSONObject json = new JSONObject();
    JSONArray jsonArray = new JSONArray();
    String jsonData = "";
    OkHttpClient client = new OkHttpClient().newBuilder().connectTimeout(20, TimeUnit.SECONDS).build();

    for (String category : yelpCategories) {
        <API call>

        jsonData = response.body().string();
        JSONObject obj = new JSONObject(jsonData);
        JSONArray array = obj.getJSONArray("businesses");
        JSONObject place = new JSONObject();

        int n = array.length();
        for (int i = 0; i < n; ++i) {
            place = array.getJSONObject(i);

            if (!place.isEmpty()) {
                json.append(category, place);
            }
        }
    }

    FileWriter myWriter = new FileWriter(filename);
    myWriter.write(json.toString(4));
    myWriter.close();
    System.out.println("Successfully wrote to Yelp file.");
} catch (IOException e) {
    e.printStackTrace();
}


Following this, I needed a few more import statements. You might notice that I added a connect timeout to the request. This is because the servers for one of the APIs were sometimes a bit sluggish, and I decided to wrap the other API calls with the same timeout protection to prevent the script from hanging or erroring out.

The full version of the code is available on GitHub.

Running the Script

To run, we can use the command `jbang` plus the name of the script file. So our command would look like the following:

Shell
 
jbang travelPetDataImport.java


This will run the script and output the results to the file we specified. We can check the file to make sure the data was written as we expected.

Wrap Up!

I was really impressed and happy with the capabilities and simplicity of JBang! It provided a straightforward way to write a script using the same Java syntax I'm comfortable with, and it was easy to get started. Next time, I'd like to figure out the IDE plugin so that I can hopefully take advantage of import suggestions and other efficiencies available.

I'm looking forward to using JBang more in the future!

Resources

  • GitHub repository: Accompanying code for this blog post
  • Website: JBang
  • Documentation: JBang
  • Data: Yelp Fusion API
API Data (computing) Java (programming language)

Opinions expressed by DZone contributors are their own.

Related

  • How To Base64 Encode or Decode Content Using APIs in Java
  • How To Get Cell Data From an Excel Spreadsheet Using APIs in Java
  • Effective Java Collection Framework: Best Practices and Tips
  • The Generic Way To Convert Between Java and PostgreSQL Enums

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!