What Is Deep Learning?
Deep learning is just very huge artificial neural networks capable of using much more volumes of data to learn with increasing performance.
Join the DZone community and get the full member experience.Join For Free
This term "deep learning" has been on fire for past two decades. Every machine learning enthusiast wants to work on it and many big companies are already making an impact in the data science field by exploring it — for example, the Google Brain project from Google or DeepFace from Facebook.
The reason is simple:
"For most flavors of the old generations of learning algorithms, performance will plateau. Deep learning is the first class of algorithms that is scalable. Performance just keeps getting better as you feed them more data."
This means that the performance of other algorithms stays constant but the performance of deep learning increases. According to a talk given by Andrew Ng from Coursera, the core of deep learning is that we now have enough fast computers and data to train our algorithms. Deep learning comes into focus when we have to consider very large data that can't be fully utilized by any other algorithms.
So, we know now why deep learning is so famous and is more effective than any other algorithm. But wait... do we really know what it is? Let's try to find out.
What Is It?
In my previous article, I explained the difference between artificial intelligence, machine learning, and deep learning. Now, let's find out more about deep learning.
Simple googling the keyword "deep learning" will yield you this definition:
Deep learning is a subset of machine learning in Artificial Intelligence (AI) that has networks which are capable of learning unsupervised from data that is unstructured or unlabeled. Also known as Deep Neural Learning or Deep Neural Network.
Deep learning is an algorithm for machine learning (supervised). The base for deep learning is artificial neural networks (ANN).
An ANN (artificial neural network) is an algorithm that mimics the workings of the human brain to process information.
We all know that the human mind is capable of processing large amounts of data, as it deals with the real environment with lots of variables and processes that information to get some knowledge out of it. The human brain does this using neural networks. The term "neural network" implies a network of billions of neurons connected with each other. These neurons forms layers and these layers are then connected to each other to process the data through each layer for necessary computing.
ANNs do the same. They are made of perceptrons that mimic the behavior of a neuron. These perceptrons then form layers. With these layers interconnected, we get an artificial neural network. Usually, we work with two layers or three layers while working with ANN. We'll get to know more about artificial neural networks in further articles.
Now, let's come back to the topic. Deep learning is just an extension of ANNs. Deep learning and ANNs both work on perceptrons and are made of multiple layers that consist of perceptrons. So, what is the difference between them? The key difference here is that we go deeper into ANNs. That means that we use more layers than usual in an ANN to achieve deep neural networks and we call it deep learning.
The Advantage of Layers
So, we use more layers in a multi-layered neural network to make it a deep neural network. What benefit comes from that?
Let's take an example of typical image recognition. In a neural net, we can add the first layer, which might learn simple components of an image, like edges. Adding another layer will make our net learn more complicated features that are combinations of edges, shapes, etc. After that, one more layer can learn combinations of shapes. For example, it could learn facial features if performing face detection, or it might learn components of vehicles (like wheels, headlights etc.) in a vehicle recognition task. Hence, with each layer, we are adding more computation power to our network.
So, deep neural networks are neural networks that may process more complex data to find more information in it.
Why Call It Deep Learning?
ANNs have been in machine learning world since 1950. Why did they get popular only recently and why are they now being used by the name deep learning instead of ANN?
A few reasons are lack of input data, lack of processing power, and inefficient use of the algorithm.
In a co-authored article titled Reducing the Dimensionality of Data With Neural Networks, Geoffrey Hinton, a pioneer in the field of artificial neural networks, says:
"It has been obvious since the 1980s that backpropagation through deep autoencoders would be very effective for nonlinear dimensionality reduction, provided that computers were fast enough, data sets were big enough, and the initial weights were close enough to a good solution. All three conditions are now satisfied."
So, to differentiate between the old neural network and new and powerful feedforward neural networks, the term deep learning is used.
Now, we know that deep learning is just very huge artificial neural networks capable of using much more volumes of data to learn with increasing performance. So, the requirement here is the ANN. We'll try to understand ANNs in the following blog.
I hope this article has cleared up the definition of deep learning for you and has helped you understand more about it.
Published at DZone with permission of Anuj Saxena, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.