Over a million developers have joined DZone.

Artificial Intelligence: The Cornerstone of Big Data

DZone 's Guide to

Artificial Intelligence: The Cornerstone of Big Data

Do you believe the hype that AI is the next big thing? Read on to help you make your decision and learn how big data has led to the rise of AI.

· AI Zone ·
Free Resource

Rumor has it that artificial intelligence is the next big industrial application coming out of big data, after Internet of Things. Do you agree? Let’s take a look at the market together and examine the evolution of data processing and business intelligence.

The Last 10 Years

The past decade has seen the rise of some great new IT technologies including cloud computing, blockchain, and big data. Among them, big data analytics has been viewed as just a "marketing thing" generating a lot of buzz. It is now becoming a standard across industries. Even more, big data analytics is now a de facto influencer of business decisions.

According to a study lead by EMC and Cap Gemini, 65% of big companies have estimated that they are taking the risk to become irrelevant if they do not adopt adequate new data analytics solutions to support their modern data platform. IDC confirms this with their predictions that Big Data annual spending will reach $48.6 billion in 2019.

Studies also revealed that every day, 2.5 quintillion bytes of data are being created worldwide. It is shocking to think that up to 90% of existing data was created in the last two years. Whatever study you reference, the estimate is that up to 80% of that owned data is unstructured. This gigantic amount of crude data needs to be refined in order to free all its potential. Otherwise, it is considered useless.

Legacy Process Improvements With Big Data

Now that companies handle their legacy IT processes, CRM, business intelligence, sales forecasting, logistical processing, product management, and so much more, big data analytics is opening up new opportunities and wins for the company.

A growing number of businesses want to better understand the personality and the emotions of their customers. They want to share their data with as many people on their staff as possible. The objective is to explore news markets, find new opportunities, invent new business strategies, and ultimately produce more value and profits.

So how can you take better advantage of your data? How can you discover new models within your disparate heterogeneous data? How can you gain new experiences in gathering, analyzing, and harnessing the data extracted or derived from multiple sources in real-time? Stephen Hawking has the answer for you:

Humans, limited by slow biological evolution, couldn’t compete with and would be superseded by AI. 

This year, two researchers working on the topics of algorithms and theory, distributed systems, and parallel computing and machine intelligence succeeded in teaching two neuronal networks how to “talk” to each other. The intention was that the interaction between the first two shall not be understood by a third one “without being taught specific algorithms for these purposes.”

Around the same time, Google DeepMind AlphaGo beat for the first time one of the best Go players in the world. Go is a board game offering more possible positions than there are atoms in the universe. It was not supposed to be possible for a machine to beat a human. According to Elon Musk, “experts in the field thought AI was 10 years away from achieving this.”

But how do data management, exploration and analysis, and artificial intelligence fit together? We have to first understand that the neuronal network chatting experience revealed that the AI system was “effective in making sense of metadata and in-traffic analysis,” which is all about grabbing and understanding data.

So, what does that mean?

Artificial intelligence appears to be the ultimate way to derive intelligence from data, outperforming BI, “classic” big data analytics, and even machine learning.

Defining Artificial Intelligence

Artificial intelligence can simply be described as hardware and software systems combined to simulate human intelligence. To do so, they need to be able to ingest information and determine which information is the best to use. AI makes up rules, that set operational processes that lead to conclusions. AI can correct itself in order to progress and ultimately define and pursue your business goals. It’s not only about analyzing data but also about interpreting it. AI does not only predict insights but it also provides personalized recommendations. AI is a system that learns in order to improve itself — it interacts with humans and learns to understand human feelings.

According to Wikipedia, optical character recognition (i.e. when your phone's camera can read a sign and translate it with an app for example) “is no longer perceived as an exemplar of AI, having become a routine technology.” However, beating a Go champion or driving a car implies that the system has the ability to learn and to solve problems. For the record, to compete with a human brain, IBM’s Watson combines AI and analytical software and relies on a supercomputer — super, as in approximately 3,000 processors connected to 15 TB of RAM.

The Future of Artificial Intelligence

Analysts are announcing the coming of the fourth platform. It will include new technologies that will become standard within the next decade. They predict major evolutions in both technical and business areas. They expect the acceleration of innovations, the evolution of the way enterprises interact with their customers and partners, and digital transformation. According to them, AI, IoT, and robotics are on their way to becoming mainstream.

By 2019, 40% of digital transformation initiatives and 100% of IoT initiatives will be supported by AI capabilities. 

Artificial intelligence is no longer the dream of science fiction novelists writing about conquests of far-away planets — it's now being embraced by "traditional" businesses. How do you think do search engines serve you so well? What kind of algorithm is running optimizations for your map applications to give you the best route to your destination? How do you think big logistic companies like FedEx and UPS are routing your parcel all over the world?

They are all leveraging the teachings of AI studies.

Gartner estimates that by 2020, AI will serve over 50% of all analytical interactions. Reaching AI is a natural path coming from the BI and data analytics journey. Today, AI is able to "see" (face recognition) and to hear you, understand you, and propose solutions to your questions (Siri). It also includes natural language processing, deep learning, and predictive analytics. Artificial intelligence empowers the data intelligence IT market. AI will be the next major innovation after cloud and big data.

Artificial Intelligence Today

Stanford researchers have drawn up a list of domains in which industries will progress in leaps and bounds by 2030.

The first is transportation. In October 2016, a truck drove itself from Fort Collins through Denver to Colorado Springs. Someday, self-driven cars will emerge as the obviously safer way to solve the personal and industrial transportation problem.

On the basis of data collected from personal monitoring devices, applications, and file records, healthcare is evolving to improve the quality of hospital care from the intake to the release of patients. A legendary example of AI is when a Gartner analyst was unexpectedly asked during a live AI demo whether he was stressed out. The AI suggested that he join his hotel’s fitness facility. 

Education is also a big domain in which AI could bring a lot of value. NLP and ML can provide personalization. This has already been tested in universities and is expected to expand significantly in the years ahead.

Technologies Leading to AI

In order for AI-based solutions to keep their promises, many areas need to be investigated deep further including large-scale machine learning, deep learning, NLP, neuromorphic computing, and IoT.

Natural language processing (NLP) is providing systems that can be used in banking, insurance, retail, etc. that are designed to process discussions with customers or patients through interactive dialog, not just reacting to preloaded models. They can “remember” you, evaluate your emotional state, “understand” what you are saying, and even predict your intentions.

We estimate the robots and AI solutions market will grow to $153 billion by 2020. (Bank of America Merrill Lynch)

In AI for CRM, SFDC laid the foundations for a journey from classic computing to AI. They explain that the four key challenges for adopting AI are data, expertise, infrastructure, and context. The book claims that there are three things needed for AI adoption:

  • Data models to intelligently classify, process, and analyze data.
  • Raw data to feed the models so they can keep improving.
  • Processing power to drive fast, efficient computing.

While we can discuss the expertise required, the complexity of the models, and the processing power that one needs to gather, data is what we need at the core. When you interact with Siri, for example, the text extracted from what you said during your “request” with Siri is recorded. It feeds the AI systems and is stored and analyzed to improve its capabilities. Artificial intelligence systems need as much of your data as you can provide it with. It never stops learning.

Source: AirTick video YouTube snapshot.

Existing AI Applications and Platforms

UC Berkeley researchers created a smartphone app (MyShake) able to detect earthquakes. Algorithms compile and analyze data gathered from smartphones, and then send a warning with earthquake-estimated details like locations, time, and magnitude.

AirTick (see the video snapshot above) is an artificial intelligence project built by a team at Nanyang Technological University in Singapore. The tool analyzes a lot of photos to estimate air quality.

Today, the sum of Internet users, mobile users, and IoT devices is around 15 billion — and this number just keeps growing. Analysts estimate that the adoption of AI technologies could lead enterprises to up to 30% productivity gain. IDC predicts that by 2020, the digital world should be made of up to 25 billion intelligent systems surfing on 50 trillion Go of data.

The big data analytics footprint is facing uninterrupted growth and is becoming the new standard. Likewise, the volume of data is ever-growing — whether it's coming from sensors, classic transactional data, or social media — with a speed that is outpacing our current technology. Artificial intelligence can offer businesses new and unforeseen efficiencies. It can improve industrial processes and human interactions. Ultimately, it can manage tasks that are manual right now, allowing humans to focus more on tasks that need human intelligence or intuition.

“Upgrading” data intelligence platforms with advanced analytics and AI processes is no longer an option. In many cases, AI is not presented as yet another suite of solutions but as the evolution of technologies that make better existing ones (i.e. Watson, Microsoft Cognitive Services, Google’s AI platform, Salesforce’s Einstein, Oracle’s Adaptive Intelligent Applications). These platforms are leading the way to cognitive data analytics, machine intelligence, and intelligent cloud applications building the fourth platform ecosystem.

What about you? Are you considering AI for your job or business? I would be happy to read your thoughts on our technological evolution.

Shout out to Niki for your help. Thank you so much for your support.

ai ,big data analytics ,machine learning

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}