Natural Language Processing- Part 1
Let’s say you heard about Natural Language Processing (NLP), some technology that can magically understand human language as it is spoken and written (referred to as natural language). Maybe you were using Siri or Google Assistant, reading a sentiment analysis of tweets, or using machine translation, and you wondered how it is possible to achieve something so complex.
Now, you want to know more about NLP, but you do not know where to start.
Before you dive in, please note the goal of this post is to give an overview of NLP and reading materials and resources to get more knowledge about NLP. The list below is just a very general starting point (and likely incomplete).
Basic programming skills (e.g., in Python) are recommended.
Natural Language Processing(NLP), Artificial Intelligence, Machine Learning: What’s the Difference?
Before beginning the NLP journey, we often see these 3 terms. One may get wires crossed when trying to differentiate between the three.
The first thing to know is that NLP and machine learning are both subsets of Artificial Intelligence.
Artificial Intelligence(AI) is an umbrella term for machines that can simulate human intelligence. It is a science that studies the human brain and uses that outcome as a study basis for intelligent systems. It is not related only to computer science but also to mathematics, psychology, philosophy, etc. This covers a wide range of applications, from speech recognition to manufacturing and drone robots to self-driving cars.
Natural Language Processing (NLP) deals with how computers understand and translate the human language. With NLP, machines can make sense of written or spoken text and perform tasks like translation, keyword extraction, topic classification, and more.
But to automate these processes and deliver accurate responses, you’ll need machine learning.
Machine learning (ML) is a subset of AI at its most basic is the practice of using algorithms to parse data, learn from it, and then decide or predict something in the world. So rather than hand-coding software routines with a specific set of instructions to accomplish a particular task, the machine is trained using large amounts of data and algorithms that give it the ability to learn how to perform the task.
AI-powered chatbots, for example, use NLP to interpret what users say and what they intend to do and machine learning to automatically deliver more accurate responses by learning from past interactions.
What is Natural Language Processing?
Natural language processing, or NLP, is a field concerned with enabling machines to understand human language.
“The goal of this new field is to get computers to perform useful tasks involving human language, tasks like enabling human-machine communication, improving human-human communication, or simply doing useful processing of text or speech.” (Jurafsky, Manning 2011)
It is a field of research that sits at the intersection of Computer science, linguistics, and artificial intelligence. NLP involves taking the naturally written or naturally spoken language and process it with machines to automatically complete some task or make it easier for a human to do.
The evolution of natural language processing
NLP draws from various disciplines, including computer science and computational linguistics dating back to the mid-20th century. Its evolution included the following major milestones:
- The origin of NLP can be traced way back to the 1940s. When the soldiers needed to understand messages in another language, NLP was invented. However, it was called Machine Translation (MT) back then. It converted one human language into another. ( for example, Russian to English).
- Alan Turing developed the Turing Test in the 1950s to determine whether or not a computer is truly intelligent. He published an article titled “Computing Machinery and Intelligence,” which proposed what is now called the Turing test as a criterion of intelligence. The test involves automated interpretation and the generation of natural language as the criterion of intelligence.
Turing test — developed by Alan Turing in 1950, is a test of a machine’s ability to exhibit intelligent behaviour equivalent to, or indistinguishable from, that of a human.
- NLP was largely rules-based from the 1950s to the late 1980s, using handcrafted rules developed by linguists to determine how computers process language.
- Starting in the late 1980s, there was a revolution in NLP with the introduction of machine learning(ML) algorithms for language processing. This was due to both the steady increase in computational power (see Moore’s law) and the gradual lessening of the dominance of Chomskyan theories of linguistics (e.g., transformational grammar), whose theoretical underpinnings discouraged the sort of corpus linguistics that underlies the ML approach to language processing. Some of the earliest-used ML algorithms, such as decision trees, produced systems of hard if-then rules similar to existing hand-written rules.
- Natural language processing saw dramatic growth after the 2000s in popularity as a term. With advances in computing power, natural language processing has also gained numerous real-world applications.
- In the 2010s, deep learning(DL) took over, and deep neural network-style ML methods became widespread in natural language processing. Results showed that such techniques could achieve state-of-the-art results in many natural language tasks, such as language modeling, parsing, and others.
Today, approaches to NLP involve a combination of classical linguistics and statistical methods. Popular techniques include using word embeddings to capture semantic properties of words and an increase in end-to-end learning of a higher-level task (e.g., question answering).
Where is NLP currently?
With the power of DL models, we can process text(understood as sequences of words or sequences of characters that form the basis of human language), time series (weather data), and sequence data in general.
I believe today, we are only limited to our limitations for applications in this technology. The same goes for most AI subfields but not all. Below is the list of the few most obvious applications:
- Text classification(such as email spam filtering)
- Timeseries classification(such as stock market value prediction or weather forecast).
- Sequence-to-sequence learning, such as decoding a French sentence into English
The field of NLP has undergone many transformations over the years, and by far, the most noticeable was from 2012(after the DL boom) till now, which has brought a few foundational concepts that culminated with many years of research.
What is natural language processing used for?
Some of the main functions that natural language processing algorithms perform are:
- Search engines autocomplete users’ searches as they type them and predict what information or website they’re ultimately seeking.
- Speech recognition algorithms interpret voice commands to provide information or take action with virtual assistants like Amazon’s Alexa, Apple’s Siri, or Microsoft’s Cortana.
- Chatbots carry out a natural conversation for an extended period though this is seldom done convincingly today. They are nevertheless helpful for relatively linear conversations on narrow topics, such as the routine components of some customer service phone calls.
Studying the fundamentals is vital in learning about any subject you are studying. I will encourage you to start your NLP journey will the below courses as they cover a lot of the fundamental topics you will require for NLP.
†Courses
- Dan Jurafsky & Chris Manning: Natural Language Processing( Introductory Video Series)
- Stanford CS224d: Deep Learning for Natural Language Processing -(Deep learning, Advanced ML algorithms, and Neural Network architectures for NLP)
- Coursera: Introduction to Natural Language Processing — (NLP course offered by the University of Michigan)
†Blogs
†Bonus
Here are a few other resources and projects that could help you to stay informed about the field of NLP:
