DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
View Events Video Library
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Integrating PostgreSQL Databases with ANF: Join this workshop to learn how to create a PostgreSQL server using Instaclustr’s managed service

Mobile Database Essentials: Assess data needs, storage requirements, and more when leveraging databases for cloud and edge applications.

Monitoring and Observability for LLMs: Datadog and Google Cloud discuss how to achieve optimal AI model performance.

Automated Testing: The latest on architecture, TDD, and the benefits of AI and low-code tools.

Related

  • Build Quicker With Zipper: Building a Ping Pong Ranking App Using TypeScript Functions
  • Stateless vs. Stateful Widgets: Make the Right Choice for Your Flutter App
  • Configuring SSO Using WSO2 Identity Server
  • Deploy MuleSoft App to CloudHub2 Using GitHub Actions CI/CD Pipeline

Trending

  • API Design
  • Creating a High-Performance DevOps Toolchain
  • AWS Amplify: A Comprehensive Guide
  • What Is a Flaky Test?

How to Combine Spark with Heroku for Fast, Simple App Deployment

Gen Furukawa user avatar by
Gen Furukawa
·
Apr. 17, 15 · Interview
Like (0)
Save
Tweet
Share
6.29K Views

Join the DZone community and get the full member experience.

Join For Free

The Spark web framework promises to make distributed data management more efficient. The Heroku service facilitates testing and deploying your apps. Put the two together, and you get app development at Internet speed.

It didn't take long for the Spark web framework for distributed data management to capture the attention of app developers. Spark was originally conceived as a way to offer "a simple API for distributed data processing in general-purpose programming languages (Java, Python, Scala)," as Reynold Xin explains in a March 26, 2015, post on Opensource.com.

To date, Spark is known primarily as a faster, simpler, and more efficient alternative to Hadoop's MapReduce. In a March 17, 2015, article in InfoWorld, Platfora executive Peter Schlampp writes that MapReduce can be difficult to use even for data scientists. By contrast, Spark features a tool for accelerated queries, a machine learning library, a graph processing engine, a streaming analytics engine, and other advanced analytics "right out of the box."

Xin credits the Spark API's functional transformations on distributed collections of data, or Resilient Distributed Datasets (RDDs), as the reason the framework can express tasks in dozens of lines of code that previously required thousands of lines. In an attempt to deliver the power of distributed processing to a wider audience, a new DataFrame API has been created for Spark as an extension to the original RDD API.

DataFrames are able to scale from kilobytes of data on a single PC to petabytes in a large cluster. They support many different data formats and storage systems, and they feature APIs for Java, Python, and Scala, with an API for R in development via SparkR. The Spark SQL Catalyst optimizer delivers state-of-the-art code generation and optimization.

young=users[users.age < 21]

This example illustrates how to use DataFrames to manipulate the demographic data of a large group of users. Source: Opensource.com

Teaming Spark and Heroku to simplify and speed up Java app development

In an attempt to take some of the sting out of developing and testing Java apps, Arjun Surendra created a tutorial that demonstrates how to use Heroku to develop and deploy a Java app to the Internet in under five minutes. Surendra uses the netbeans API, but the technique works with any IDE or text editor. It requires JDK7 in addition to Spark.

You start by creating a simple Java Maven project that includes App.java, home.ftl.html, login.ftl.html, log4j.properties, pom.xml, and system.properties. Finally, you add the Procfile.

web: java $JAVA_OPTS - jar target/hellowworld-1.0

Developing and deploying a Java app to the Internet via Heroku uses a straightforward Procfile. Source: Arjun Surendra

Once you're ready to deploy the app to Heroku, the Heroku tool belt installs the required git version control. You then create the Heroku repository, enter your Heroku credentials, and push the code to the Heroku owned git repo, where it is built and deployed automatically.

The Heroku Dev Center provides information on using Heroku with Spring MVC Hibernate, as well as with the Play Framework, and with Node.js. Another useful resource is the Java on Heroku Forum, which includes a post describing how to troubleshoot application errors related to Procfile, system.properties, pom.xml, and other project components.

For efficient, centralized management of your apps, databases, IT operations, and business services in real time, sign up for the Happy Apps application-management service. Happy Apps lets you set up rules so you are alerted via SMS and email whenever incidents or specific events occur. You can group and monitor multiple apps, databases, web servers, and app servers. In addition to an overall status, you can view the status of each individual group member.

Happy Apps is the only app-management service to support SSH and agent-based connectivity to all your apps on public, private, and hybrid clouds. The service provides dependency maps for determining the impact your IT systems will have on other apps. All checks performed on your apps are collected in easy-to-read reports that can be analyzed to identify repeating patterns and performance glitches over time. Visit the Happy Apps site to sign up for a free trial.

app

Opinions expressed by DZone contributors are their own.

Related

  • Build Quicker With Zipper: Building a Ping Pong Ranking App Using TypeScript Functions
  • Stateless vs. Stateful Widgets: Make the Right Choice for Your Flutter App
  • Configuring SSO Using WSO2 Identity Server
  • Deploy MuleSoft App to CloudHub2 Using GitHub Actions CI/CD Pipeline

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends: