Introduction to Unsupervised Learning
What do you do when your dataset doesn’t have any labels? Unsupervised learning is a group of machine learning algorithms and approaches that work with this kind of “no-ground-truth” data.
This post will walk through what unsupervised learning is, how it’s different than most machine learning, some challenges with implementation, and provide some resources for further reading.
What is Unsupervised Learning?
Unsupervised machine learning cannot be directly applied to a regression because it is unknown what the output values could be, therefore making it impossible to train the algorithm how you normally would.
The easiest way to understand what’s going on here is to think of a test. When you took tests in school, there were questions and answers; your grade was determined by how close your answers were to the actual ones (or the answer key). But imagine if there was no answer key, and there were only questions. How would you grade yourself?
Now apply this framework to machine learning. Traditional datasets in ML have labels (think: the answer key), and follow the logic of “X leads to Y.” For example: we might want to figure out if people with more Twitter followers typically make higher salaries. We think that our input (Twitter followers) might lead to our output (salary), and we try to approximate what that relationship is.
The stars are data points, and machine learning works on creating a line that explains how the input and outcomes are related. But in unsupervised learning, there are no outcomes! We’re just looking to analyze in the input, which is our Twitter followers. There is no salary, or Y, involved at all. Just like there not being an answer key for the test.
Maybe we don’t have access to salary data, or we’re just interested in different questions. It doesn’t matter! The important thing is that there is no output to match to, and no line to draw that represents a relationship.
So what exactly is the goal of unsupervised learning then? What do we do when we only have input data without labels?
Types of Unsupervised Learning
Clustering
Any business needs to focus on understanding customers: who they are and what’s driving their purchase decisions?
You’ll usually have different groups of users that can be split across a few criteria. These criteria can be as simple, such as age and gender, or as complex as persona and purchase process. Unsupervised learning can help you accomplish this task automatically.
Clustering algorithms will run through your data and find these natural clusters if they exist. For your customers, that might mean one cluster of 30-something artists and another of millennials who own dogs. You can typically modify how many clusters your algorithms looks for, which lets you adjust the granularity of these groups. There are a few different types of clustering you can utilize:
- K-Means Clustering – clustering your data points into a number (K) of mutually exclusive clusters. A lot of the complexity surrounds how to pick the right number for K.
- Hierarchical Clustering – clustering your data points into parent and child clusters. You might split your customers between younger and older ages, and then split each of those groups into their own individual clusters as well.
- Probabilistic Clustering – clustering your data points into clusters on a probabilistic scale.
These variations on the same fundamental concept might look something like this in code:
#Import the KMeans package from Scikit Learn
from sklearn.cluster import KMeans
#Grab the training data
x = os.path(‘train’)
#Set the desired number of clusters
k = 5
#Run the KMeans algorithm
kmeans = KMeans(n_clusters=k).fit(x)
#Show the resulting labels
kmeans.labels_Any clustering algorithm will typically output all of your data points and the respective clusters to which they belong. It’s up to you to decide what they mean and exactly what the algorithm has found. As with much of data science, algorithms can only do so much: value is created when humans interface with outputs and create meaning.
Data Compression
Even with major advances over the past decade in computing power and storage costs, it still makes sense to keep your data sets as small and efficient as possible. That means only running algorithms on necessary data and not training on too much. Unsupervised learning can help with that through a process called dimensionality reduction.
Dimensionality reduction (dimensions = how many columns are in your dataset) relies on many of the same concepts as Information Theory: it assumes that a lot of data is redundant, and that you can represent most of the information in a data set with only a fraction of the actual content. In practice, this means combining parts of your data in unique ways to convey meaning. There are a couple of popular algorithms commonly used to reduce dimensionality:
- Principal Component Analysis (PCA) – finds the linear combinations that communicate most of the variance in your data.
- Singular-Value Decomposition (SVD) – factorizes your data into the product of three other, smaller matrices.
These methods as well as some of their more complex cousins all rely on concepts from linear algebra to break down a matrix into more digestible and informatory pieces.
Reducing the dimensionality of your data can be an important part of a good machine learning pipeline. Take the example of an image-centerpiece for the burgeoning field of computer vision. We outlined here how big a dataset of images can be and why. If you could reduce the size of your training set by an order of magnitude, that will significantly lower your compute and storage costs while making your models run that much faster. That’s why PCA or SVD are often run on images during preprocessing in mature machine learning pipelines.
Generative Models
Generative models are a class of unsupervised learning models in which training data are given and new samples are generated from the same distribution. These models must discover and efficiently learn the essence of the given data to try to generate similar data. The long term benefit of this type of model is its ability to automatically learn the features of the given data.
A common example of generative models is an image dataset. Given a set of images, a generative model could generate a set of images similar to the given set.
Unsupervised Deep Learning
Unsurprisingly, unsupervised learning has also been extended to neural nets and deep learning. This area is still nascent, but one popular application of deep learning in an unsupervised fashion is called an Autoencoder.
Autoencoders follow the same philosophy as the data compression algorithms above––using a smaller subset of features to represent our original data. Like a Neural Net, an Autoencoder uses weights to try and mold the input values into a desired output; but the clever twist here is that the output is the same thing as the input! In other words, the Autoencoder tries to figure out how to best represent our input data as itself, using a smaller amount of data than the original.
Autoencoders have proven useful in computer vision applications like object recognition, and are being researched and extended to domains like audio and speech.
Challenges in Implementing Unsupervised Learning
In addition to the regular issues of finding the right algorithms and hardware, unsupervised learning presents a unique challenge: it’s difficult to figure out if you’re getting the job done or not.
In supervised learning, we define metrics that drive decision making around model tuning. Measures like precision and recall give a sense of how accurate your model is, and parameters of that model are tweaked to increase those accuracy scores. Low accuracy scores mean you need to improve, and so on.
Since there are no labels in unsupervised learning, it’s near impossible to get a reasonably objective measure of how accurate your algorithm is. In clustering for example, how can you know if K-Means found the right clusters? Are you using the right number of clusters in the first place? In supervised learning we can look to an accuracy score; here you need to get a bit more creative.
A big part of the “will unsupervised learning work for me?” question is totally dependent on your business context. In our example of customer segmentation, clustering will only work well if your customers actually do fit into natural groups. One of the best (but most risky) ways to test your unsupervised learning model is by implementing it in the real world and seeing what happens! Designing an A/B test–with and without the clusters your algorithm outputted–can be an effective way to see if it’s useful information or totally incorrect.