3 Core Competencies of Digital: Cloud, Big Data, and Intelligent Middleware

“Ultimately, the cloud is the latest example of Schumpeterian creative destruction: creating wealth for those who exploit it and leading to the demise of those that don’t.” — Joe Weiman, author of Cloudonomics: The Business Value of Cloud Computing.

The Cloud as a Venue for Digital Workloads

As 2017 begins, it can safely be said that no industry leader questions the existence of the new digital economy or the fact that every firm out there needs to create a digital strategy. Myriad organizations are taking serious business steps to making their platforms highly customer-centric via a renewed operational metrics focus. They are also working on creating new business models using their analytics investments. Examples of these verticals include banking, insurance, telecom, healthcare, energy, etc.

As a general trend, the digital economy brings immense opportunities while exposing firms to risks, as well. Customers now demanding highly contextual products, services, and experiences — all accessible via an easy API (Application Programming Interfaces).

Big Data Analytics (BDA) software revenues will grow from nearly $122B in 2015 to more than $187B in 2019, according to Forbes.  At the same time, it is clear that exploding data generation across the global economy has become a clear and present business phenomenon. Data volumes are rapidly expanding across industries. However, while the production of data itself has increased, it is also driving the need for organizations to derive business value from it. As IT leaders know well, digital capabilities need low cost yet massively scalable and Agile information delivery platforms — which only cloud computing can provide.

For a more detailed technical overview, see here.

Big Data and Big Data Analytics Drive Consumer Interactions 

The onset of digital architectures in enterprise businesses implies the ability to drive continuous online interactions with global consumers, customers, clients, or patients. The goal is not just to provide engaging visualization but also to personalize services clients care about across multiple channels of interaction. The only way to attain digital success is to understand your customers at a micro level while constantly making strategic decisions on your offerings to the market. Big Data has become the catalyst in this massive disruption as it can help business in any vertical solve their need to understand their customers better and perceive trends before the competition does. Big Data thus provides the foundational platform for successful business platforms.

The three key areas where Big Data and cloud computing intersect are:

1. Data science and exploration.
2. ETL, data backups, and data preparation.
3. Analytics and reporting.

Big Data drives business use cases in Digital in myriad ways. Key examples include the following. 

1. Obtaining a real-time single view of an entity — typically, a customer across multiple channels, product silos, and geographies.
2. Customer segmentation by helping businesses understand their customers down to the individual micro level as well as at a segment level.
3. Customer sentiment analysis by combining internal organizational data, clickstream data, and sentiment analysis with structured sales history to provide a clear view into consumer behavior.
4. Product recommendation engines that provide compelling personal product recommendations by mining real-time consumer sentiment, product affinity information with historical data.
5. Market basket analysis, observing consumer purchase history and enriching this data with social media, web activity, and community sentiment regarding past purchase and future buying trends.

Further, digital implies the need for sophisticated, multifactor business analytics that need to be performed in near real-time on gigantic data volumes. The only deployment paradigm capable of handling such needs is cloud computing, whether public or private. Cloud was initially touted as a platform to rapidly provision compute resources. Now, with the advent of digital technologies, the cloud and Big Data will combine to process and store all this information. According to the IDC, by 2020, spending on cloud-based Big Data analytics will outpace on-premise by a factor of 4.5. 

Intelligent Middleware Provides Digital Agility

Digital applications are applications modular, flexible, and responsive to a variety of access methods — mobile and non-mobile. These applications are also highly process-driven and support the highest degree of automation. The need of the hour is to provide enterprise architecture capabilities around designing flexible digital platforms that are built around the efficient usage of data, speed, agility, and a service oriented architecture. The choice of open source is key, as it allows for a modular and flexible architecture that can be modified and adopted in a phased manner (as you will shortly see).

The intention in adopting a SOA (or even a microservices) architecture for digital capabilities is to allow lines of business the ability to incrementally plug-in lightweight business services like customer onboarding, electronic patient records, performance measurement, trade surveillance, risk analytics, claims management, etc.

Intelligent middleware adds significant value in six specific areas.

1. Supports a high degree of process automation and orchestration, thus enabling the rapid conversion of paper-based business processes to a true digital form in a manner that lends itself to continuous improvement and optimization.
2. Business rules help by adding a high degree of business flexibility and responsiveness.
3. Native mobile applications enable platforms to support a range of devices and consumer behavior across those front ends.
4. Platforms as a Service engines that enable rapid application and business capability development across a range of runtimes and container paradigms.
5. Business process integration engines which enable rapid application and business capability development
6. Middleware brings the notion of DevOps into the equation. Digital projects bring several technology and culture challenges that can be solved by a greater degree of collaboration, continuous development cycles, and new toolchains — without giving up proven integration with existing (or legacy) systems.

Intelligent middleware not only enables automation and orchestration but also provides an assembly environment to string different (micro)services together. Finally, it also enables less technical analysts to drive application lifecycle as much as possible.

Further, digital business projects call out for mobile native applications — which a forward-looking middleware stack will support. Middleware is a key component for driving innovation and improving operational efficiency.

5 Key Business Drivers for Combining Big Data, Intelligent Middleware, and the Cloud

The key benefits of combining the above paradigms to create new digital applications are the following.

1. Enable Elastic Scalability Across the Digital Stack

Cloud computing can handle the storage and processing of any amount of data and any kind of data. This calls for the collection and curation of data from dynamic and highly distributed sources such as consumer transactions, B2B interactions, machines such as ATMs and geo-location devices, click streams, social media feeds, server and application log files, multimedia content such as videos, etc. It needs to be noted that data volumes here consist of multi-varied formats and differing schemas, transport protocols, and velocities. Cloud computing provides the underlying elastic foundation to analyze these datasets.

2. Support Polyglot Development, Data Science and Visualization

Cloud technologies are polyglot in nature. Developers can choose from a range of programming languages (Java, Python, R, Scala, C#, etc.) and development frameworks (such as Spark and Storm). Cloud offerings also enable data visualization using a range of tools from Excel to BI Platforms.

3. Reduce Time to Market for Digital Business Capabilities

Enterprises can avoid time-consuming installation, setup, and other upfront procedures. This can help deploy Hadoop in the cloud without buying new hardware or incurring other up-front costs. On the same vein, even Big Data analytics should be able to support self-service across the lifecycle — from data acquisition, preparation, analysis, and visualization.

4. Support a Multitude of Deployment Options (Private/Public/Hybrid Cloud)

A range of scenarios for product development, testing, deployment, backup, or cloud-bursting is efficiently supported in pursuit of cost and flexibility goals.

5. Fill the Talent Gap

Open-source technology is the common thread across cloud, Big Data, and middleware. The hope is that the ubiquity of open source will serve as a critical level in enabling the filling up of the IT-business skills scarcity gap.

As opposed to building standalone or one-off business applications, a digital platform mindset is a more holistic approach capable of producing higher rates of adoption — and thus revenues. Platforms abound in the web-scale world at companies like Apple, Facebook, Google, etc. Digital applications are constructed like lego blocks and they reuse customer and interaction data to drive cross-sell and upsell among different product lines. The key components here are to ensure that one starts off with products with high customer attachment and retention. While increasing brand value, it is key to ensure that customers and partners can also collaborate in the improvements in the various applications hosted on top of the platform.