What Is Deep Learning?

eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

If your organization is using artificial intelligence (AI) and machine learning (ML) on any kind of widespread basis, chances are good that you have some deep learning projects in the works. Interest in deep learning has spiked recently, and it has become a critical enabler in many different industries.

Industry reports reflect this skyrocketing interest in deep learning. To look a few years back, a 2018 report titled How Companies Are Putting AI to Work through Deep Learning found that only 28 percent of enterprises surveyed were using deep learning. But the more recent AI Adoption in the Enterprise 2021 survey found that the percentage of respondents using deep learning had more than doubled to 67 percent.

You can also see this increasing use of deep learning reflected in spending. Gartner doesn’t break out spending on deep learning specifically, but it forecasts that the total AI market will reach $62.5 billion in 2022. That’s a 21.3% increase from the $51.5 billion spent in 20221.

Grand View Research says that just the deep learning portion of the AI market was worth $34.8 billion in 2021. And it estimates that spending will grow by more than 34.3 percent per year between 2022 and 2030.

Clearly, deep learning is becoming big business. But what exactly is deep learning?

Also see: Top AI Software 

Deep Learning, Machine Learning and Artificial Intelligence

Before you can understand deep learning, you need to understand two foundational concepts: artificial intelligence and machine learning.

Artificial intelligence encompasses all the technologies that allows machines to think like humans. It includes capabilities like understanding and speaking human languages, describing the contents of an image, ascertaining a speaker’s emotional state, and learning new concepts.

Machine learning is a subset of artificial intelligence. It refers to all the technologies that allow computers to learn something new without being explicitly programmed.

Deep learning is a subset of machine learning. It refers to ML that takes place on artificial neural networks.

Also see: The Future of Artificial Intelligence

What Is Deep Learning?

Several different organizations have offered definitions of deep learning. But many of these definitions are difficult to understand unless you have a background in data science.

For example, Gartner says, “Deep learning, a variant of machine learning algorithms, uses multiple layers of algorithms to solve problems by extracting knowledge from raw data and transforming it at every level.”

IBM offers a slightly more comprehensible definition:

Deep learning attempts to mimic the human brain—albeit far from matching its ability—enabling systems to cluster data and make predictions with incredible accuracy. Deep learning is a subset of machine learning, which is essentially a neural network with three or more layers. These neural networks attempt to simulate the behavior of the human brain—albeit far from matching its ability—allowing it to “learn” from large amounts of data.

In essence, deep learning represents an attempt to allow computers to learn the same way that human babies do. When a baby is born, it knows nothing. Babies use the networks of neurons in their brains to take in information about the world around them and make sense of it, slowly coming to conclusions about the world.

Deep learning systems rely on artificial neural networks designed to be very similar to the neurons in an infant’s brains. These networks include multiple layers that allow the system to process and re-process data until it learns the important characteristics of the data it is analyzing.

Also see: How AI is Altering Software Development with AI-Augmentation 

Types of Deep Learning

You can organize the different kinds of deep learning into three different categories depending on the type of data they use:

1. Supervised deep learning relies on tagged data. In this kind of deep learning, you feed the data model a lot of different data and tell the model what it is. The computer then learns on its own which characteristics cause data to fall into one category or another so that it can extrapolate on its own. The human equivalent is adults pointing at different objects and telling a baby the name of those objects. Eventually, the baby learns on its own what makes a “dog” different from a “bottle” or a “book.”
2. Unsupervised deep learning relies on untagged data. Essentially, the system learns by mimicry. One example that may be familiar are the deep learning systems that take in examples of human art and then generate their own artwork. The human equivalent is a baby learning to say sounds like “mamamama” or “dadadada” by mimicking the adults it hears. Over time and with reinforcement the sounds turn into recognizable words that have meaning.
3. Semi-supervised deep learning involves a combination of tagged and untagged data. It’s probably the closest to how actual babies learn, with some explicit training combined with a lot of observation and mimicry.

Each of the kinds of deep learning has its own pros and cons. Supervised learning generates the fastest and most accurate results, but it requires a lot of work up front on the part of the people operating the system. Unsupervised learning doesn’t require the same level of setup, but it’s less reliable and takes a long time. Semi-supervised is in between the two—requiring much less setup than fully supervised learning, while generating significantly more reliable results than unsupervised learning.

Also see: Top Digital Transformation Companies

Deep Learning Architectures

The artificial neural networks that deep learning relies on can take many different forms. Fully explaining these would require an entire book, but here are some short descriptions of some of the most common deep learning architectures:

• Convolutional neural networks are probably the most widely used deep learning architecture, particularly for image processing, but they require graphics processing units (GPUs) and advanced processing capabilities to perform the complicated calculations required.
• Feedforward neural networks were the first type of artificial neural network; they feed data in one direction without any loops or cycles.
• Radial basis function neural networks are a type of feedforward neural network that include an input layer, a radial basis function activation layer, and an output layer.
• Recurrent neural networks follow a temporal sequence and are particularly useful for speech and handwriting analysis.
• Kohonen self-organizing neural networks are useful for creating feature maps from unsupervised data.
• Modular neural networks include a series of independent neural networks that can be joined together.

Data scientists are developing new types of architecture, as well as variants of the existing kinds, all the time, so the list is constantly growing and changing.

Also see: Best Machine Learning Platforms 

Deep Learning Use Cases

Deep learning is most useful for very complex problems with a lot of different variables. Some of the most common use cases include the following:

• Natural language processing allows computer systems to understand and generate human speech. It can include voice-to-text, text-to-voice, machine translation, tagging, named entity recognition, and sentiment analysis. It enables applications like digital assistants (like Siri and Alexa), chatbots, spam detection, social media analytics, and many more. Deep learning helps natural language processing engines improve over time by identifying and mimicking patterns in human speech.
• Image processing is one of the most common uses for deep learning. It enables a wide array of different applications, including biometrics and facial recognition, analyzing medical scans, autonomous vehicles, identifying faulty parts on an assembly line, image sharpening, and even the filters popular in social media and video call software.
• Fraud prevention becomes much more accurate when organizations use deep learning techniques to identify anomalous patterns in sales and financial data. Financial institutions, retailers, law enforcement, transportation and other organizations use deep learning to quickly identify and halt fraudulent activity.
• Cybersecurity tools have traditionally lagged behind cybercriminals, who are always developing new techniques to circumvent existing prevention measures. Deep learning techniques allow cybersecurity software to identify expected and unexpected patterns in network traffic, making it possible to detect and prevent brand new attacks that no one has ever seen before.
• Drug development becomes much faster when researchers employ deep learning techniques. During the coronavirus pandemic, scientists trained models on biochemical datasets and then used the resulting algorithms to identify drugs that could potentially treat the illness. These same techniques could speed development of pharmaceuticals for a variety of different diseases.
• Climate science models involve a huge number of variables—making it an ideal candidate for deep learning approaches. These techniques are helping scientists refine their models and make more accurate forecasts.
• Video games are making extensive use of deep learning techniques. Deep learning allows the developers to create more lifelike characters and animations, to enable better audio interactions, and to develop bots whose abilities are on par with the best human players.
• Predictive analytics has become a critical tool to enable businesses to create strategies and optimize their operations. While not all predictive analytics requires the use of deep learning techniques, applying deep learning in situations with high numbers of variables and large sets of historical data can yield excellent results.

Deep Learning Tools and Services

Because deep learning requires extensive compute, graphics processing, memory, and storage capabilities, many deep learning projects run either on high-performance computing systems or in the public cloud. Most of the major public cloud vendors have platform as a service (PaaS) offerings that support deep learning. Options include:

• AWS SageMaker
• Google Cloud AI and Machine Learning Services
• Microsoft Azure Machine Learning
• IBM Watson Studio
• Oracle Machine Learning

Other popular deep learning tools include the following:

• Caffe
• Caffe2
• Deeplearning4J
• Gluon
• H20.ai
• Keras
• MATLAB
• PyTorch
• TensorFlow
• /Theano

The Future of Deep Learning

Looking ahead, the future of deep learning is tied closely to the outlook for AI as a whole. Many analysts say that artificial intelligence is at something of a crossroads currently. Enterprises have invested heavily in these technologies, but AI initiatives aren’t always achieving the desired results.

“The AI software market is picking up speed, but its long-term trajectory will depend on enterprises advancing their AI maturity,” explains Alys Woodward, senior research director at Gartner.

“Successful AI business outcomes will depend on the careful selection of use cases,” adds Woodward. “Use cases that deliver significant business value, yet can be scaled to reduce risk, are critical to demonstrate the impact of AI investment to business stakeholders.”

The advancement of deep learning also depends heavily on the development of faster hardware that can quickly process ever-larger datasets. Some researchers believe that the development of quantum computing systems will eventually enable deep learning systems with capabilities that we can’t even imagine today.

Until those systems arrive, look for researchers to continue refining their deep learning techniques and scaling deep learning systems to handle more data and find the answers to more questions.