Turnkey Real-Time Stream Processing for Enterprises

It was great talking to Kostas Tzoumas, Founder of data Artisans and one of the original creators of Apache Flink®, about their new dA Platform 2. Featuring the Application Manager, the new release productionizes stream processing and enables companies to provide real-time data applications as a centralized enterprise service. dA Platform 2 reduces the manpower, cost, and effort required to provide a reliable and high-impact stream processing platform across an organization. 

Apache Flink is a powerful and expressive system for processing streams of data at massive scale. Its rich API, stateful event-at-a-time processing, low latency, and high-throughput performance make it the best choice for streaming data applications. Global companies such as Netflix, Uber, Alibaba, Capital One, Cisco, ING, and King use Flink as the stream processing platform of choice for large-scale stateful applications, including real-time analytics, search and content ranking, and fraud/anomaly/threat detection. 

Alibaba has used Apache Flink to improve search recommendations based on trend and personal data. The relevant and timely recommendation resulted in a 30% increase in conversion versus previous Singles' Day sales.

Uber is migrating to Flink as a company-wide service given the amount of data they use to drive marketing, prevent suspicious activities, and live GPS streams that provide more accurate estimated time of arrival (ETA).

Analysts predict the streaming analytics market will reach $13.7 billion USD by 2021, as companies across industries are adopting streaming data applications for fraud detection, sales and marketing management, predictive asset maintenance, risk management, and operations management, among other use cases. Because Apache Flink processes data in real-time and can be applied to unbounded datasets, it is quickly emerging as the stream processing engine of choice for streaming data applications. 

"The ability to react to data in real-time is mission-critical for today’s enterprises, and Flink is being rapidly adopted for large-scale stateful applications that can process data instantly,” says Tzoumas. “dA Platform 2 makes it possible for enterprises to run streaming applications in production across their organization quickly and efficiently. We’re excited to preview this new technology with the community at Flink Forward Berlin and showcase true turnkey stream processing for enterprises of all shapes and sizes.” 

Highlights of dA Platform 2

New to dA Platform 2 is the Application Manager, which streamlines the process of deploying and maintaining real-time streaming applications in production, greatly reducing the time-to-market and personnel required for businesses to realize value from streaming. It integrates with an organization’s existing streaming data sources, developer workflows, service deployment architecture, and logging and metrics infrastructure to be the nexus of all streaming data processing applications within the organization. dA Platform enables an organization to realize the full potential of streaming data processing with Flink and to focus on application development rather than the supporting infrastructure. The product solves many of the process and operational challenges around managing a fleet of Flink applications.

Key highlights include:

• Orchestrates the lifecycle of Flink applications through development cycles.

• Integrates with logging and metrics systems to help engineers during debugging and development.

• Integrates with container platforms like Kubernetes for dynamic resource allocation.

• Automates common operations like redeploying an updated Apache Flink application or initiating a savepoint.

• Maintains records of all user actions and automatic processes for later auditing.