AI technology can be divided into 3 layers. While we will likely see incremental improvements in algorithm design and application of those algorithms to specific domains, step change is possible in computing if breakthroughs can be achieved in quantum computing.
- Algorithms that enable machine decision making including artificial neural networks (ANN), Bayesian inference and evolutionary computing. ANNs can be categorized by their depth (i.e. number of layers) and structure (i.e. how nodes are connected). Most recent progress in AI was due to deep neural networks (also called deep learning).
- Computing technology to run those algorithms: Computing is key in AI, advances in computing power enabled the wave of AI commercialization thanks to deep learning since the 2010s.
- Application of those algorithms to specific domains including reinforcement learning, computer vision, machine vision, natural language processing (NLP), recommendation systems.
Algorithms are like recipes for AI systems to learn from data or to make decisions. Advances in algorithm design are key for advancement for AI.
Artificial Neural Networks (ANNs)
The neural network is a popular machine learning technique that is inspired by the human brain and the neural network in our brains.
These networks consist of interconnected artificial neurons. Every neuron processes the input data with a predefined mathematical function and produces and output which becomes the input for other neurons. An ANN can be divided between the input, hidden and output layers. With the given input data, the coefficients between neurons are computed to achieve accurate outputs at the end.
Geoffrey Hinton, one of the pioneers in deep learning explained artificial neural networks in quite understandable terms during a recent interview:
You have relatively simple processing elements that are very loosely models of neurons. They have connections coming in, each connection has a weight on it, and that weight can be changed through learning. And what a neuron does is take the activities on the connections times the weights, adds them all up, and then decides whether to send an output. If it gets a big enough sum, it sends an output. If the sum is negative, it doesn’t send anything. That’s about it. And all you have to do is just wire up a gazillion of those with a gazillion squared weights, and just figure out how to change the weights, and it’ll do anything. It’s just a question of how you change the weights.
Data scientists use this technology in decision-making, improved forecasting, image recognition, and robotics.
ANNs can be deep or shallow and can have different structures
ANNs by depth
Shallow networks, which were the focus of significant research in 50s and 60s could not learn complex tasks. For example, the book Perceptron by Marvin Minsky and Seymour Papert famously proved that a single layer of perceptrons (neurons) could not even learn simple logical functions like XOR. At the time, machines were not powerful enough to train deep networks and this finding reduced research focus on shallow ANNs.
In 2010s, with the availability of cheap computing power and GPUs, scientists were able to train deep networks in reasonable time frames. Since then, research in AI has focused on deep ANNs (i.e. deep learning).
ANNs by structure
Deep networks can be built in various architectures. Below, we explain some of the most popular/promising architectures.
Recurrent Neural Networks
Recurrent neural networks are recursive neural networks (i.e. networks where same set of weights are applied recursively to inputs) built to operate with a sequence of inputs. Recurrent neural networks outperform other current approaches especially in Natural Language Processing (NLP) tasks where prediction depends on previous inputs. For example, Android started relying on them by 2015 for text-to-speech synthesis. Feel free to check out Andrej Karpathy’s , Sr. Director of AI at Tesla, article on RNNs applied to various tasks from writing Shakespeare style articles to Linux source code.
Capsule networks aim to mimic the human brain more closely. These networks consist of capsules and, each capsule includes a set of neurons. Each neuron in the capsule handles a specific feature while capsules work simultaneously. This enables capsule networks to do tasks in parallel.
A popular example of capsule networks is face recognition. Capsule networks are built to hold inner information in memory. For example, capsule networks are superior to other current popular approaches in distinguishing examples like the two below images.
Geoffrey Hinton, one of the pioneers of deep learning, states that capsule networks cuts error rates by 45% compared to previous AI algorithms.
AI requires computing power for learning. Without powerful computing technologies, AI agents may not give intended results for businesses. Thus, developers aim to create stronger computing technologies to improve AI performance. To do that, they may construct new technologies or develop current hardware that can handle more complicated jobs.
These advances mostly enable incremental enhancement. Yet, quantum computing is a game-changer that can bring a step-change. Feel free to read our future of AI article for more on quantum computing and other computing trends relevant for AI.
Application of AI Algorithms to Specific Domains
Reinforcement Learning (RL)
In real life, learning is dynamic. We make experiments, observe results and make new experiments. Reinforcement learning tackles interactive learning environments with AI algorithms. With reinforcement learning, the AI agent interacts with its environment and takes consecutive actions to maximize its gains.
Unlike traditional learning, RL doesn’t look for any patterns to take action. Instead, it generates numerical values as rewards for desired outcomes and makes sequential decisions to maximize its total reward. In this process, AI agents keep exploring and updating their beliefs just as humans learn from their experience in the real world.
The most well-known RL example is Google’s DeepMind AlphaGo, which has defeated the world’s number one Go player Ke Jie in two consecutive games. Similarly, RL is widely used in robotics as it requires goal based exploration as well.
Computer vision includes the techniques of perceiving and distinguishing images with computers. Improving the image quality, image matching, object recognition, and image reconstruction are all subcategories of computer vision. The main goal of this technology is to make computers understand and be able to implement human visual perception.
The use cases include tumor detection in healthcare, tracking UAVs in the military, emotion detection, and car plate recognition. Unlocking our phones with face recognition is another daily-life example of computer vision.
Natural Language Processing (NLP)
Natural Language Processing (NLP) is concerned with how AI agents perceive human languages. This technology involves text and speech recognition, natural language understanding, generation, and translation.
NLP is currently used in chatbots, cybersecurity, article summarization, instant translation, spam detection, and information extraction.
Social media is a common NLP use case for information extraction. For example, Cambridge Analytica relied on a few pieces of structured data like voters’ likes for its now infamous segmentation of the American voters. Users’ posts contain far more detailed data on those users’ preferences which can be analyzed for better segmentation. This is one of the reasons why data privacy is increasing in importance with more restrictive legislation passed over time.
Recommendation systems aim to predict the users’ future preferences based on their previous information. This information can include general information, photos you like, or items you have purchased. These systems enable users to have personalized experiences and allow them to discover new things that interest them.
The most common examples are Amazon, Netflix, and Spotify which recommends you books, movies, and music based on your usage history.
If you wonder how you can apply AI to mobile applications, you can read this article.
To learn more about how these AI technologies will evolve in the future, feel free to read our article on the future of AI.