Over a million developers have joined DZone.

Google's Big Data Dataflow and Pub/Sub Become Generally Available

Alongside the announcement of Dataflow and Pub/Sub's new general availability, Google Cloud Platform also integrated with Cloudera's Director 1.5.

· Big Data Zone

Learn how you can maximize big data in the cloud with Apache Hadoop. Download this eBook now. Brought to you in partnership with Hortonworks.

Google's entire suite of big data tools has now become generally available. Yesterday, Google revealed on its Cloud Platform Blog that Dataflow and Pub/Sub, two of its data analysis products current on their Google Cloud Platform, would have general availability.

Previously in beta, Dataflow was built with MapReduce, FlumeJava and MillWheel in mind. It aims to handle the "complexity of developing separate systems for batch and streaming data sources by providing a unified programming model." Dataflow offers batch and streaming processing of big data.

Google's Dataflow

Cloud Pub/Sub analyzes big data streams in real-time in addition to integrating services and applications. It boasts a single API and claims to be "cost-effective," fast and scalable.

This has been a big week for the tech giant. In addition to the announcement that it was shifting some of its endeavors under the guise of Alphabet, Google named Sundar Pichai as its newest CEO. However, former CEO Larry Page isn't going too far -- he's taking charge as CEO of Alphabet. Alphabet is going to be Google's "parent company," overseeing the development of the Google X lab, Calico, Fiber and Nest. Google's "core businesses, such as search, ads, maps, Android, YouTube and 'related technical infrastructure,'" will stay under the Google name.

The same day as its Pub/Sub and Dataflow announcements, Google Cloud became one of the first to integrate with the new Cloudera Director 1.5. Announced on Cloudera's blog, Cloudera Director 1.5 is touted as the "integrated solution for deploying and managing enterprise-grade Hadoop in cloud environments." Google Cloud joined with Cloudera via Director's open API. Cloudera also honed its production-grade features, including "enabling high availability for clusters and Kerberos integration for security."

In addition, as reported by ZDNet's Rachel King, Cloudera's Hadoop is now Google Cloud Platform certified.

Hortonworks DataFlow is an integrated platform that makes data ingestion fast, easy, and secure. Download the white paper now.  Brought to you in partnership with Hortonworks

Topics:
google cloud platform ,hadoop ,big data

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

SEE AN EXAMPLE
Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.
Subscribe

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}