From Supervised Learning to Reinforcement Learning

I am continuing to organize the material from the previous class. This time, I also looked up a few points that were not covered in the lecture. Copying the outline would have been quick, but understanding each term before writing it down has slowed me down considerably. There is still a lot left to review, so I am taking it one section at a time.

Machine learning finds relationships and patterns in data, but not every model learns in the same way. The main categories differ in whether a person provides the correct answer, whether the training task is created from the data itself, or whether an action is evaluated by its outcome.

Learning type	Input	Answer	What it learns	Example
Supervised	Data	Human-provided	Relationship between input and answer	House prices, spam
Unsupervised	Data	None	Structure within the data	Customer groups, dimension reduction
Self-supervised	Raw data	Generated from the data	Predicting one part from another	Masked words
Reinforcement	Environment state	Action outcome	Better action selection	Games, robots

The terms data and raw data in this table do not refer to different file formats. The same image or sentence can be used for supervised learning when a person supplies a label, or for unsupervised learning when no answer is supplied and the goal is to examine the structure of the data. The self-supervised case needs a more specific example, which I cover below.

Supervised Learning Connects Inputs to Answers

Supervised learning uses input data together with known answers. When training a model to distinguish cats from dogs, each image comes with a cat or dog label. The model learns the relationship between features in the image and the supplied answer.

Supervised learning is usually divided into regression and classification according to what must be predicted.

Type	Prediction target	Relationship the model learns	Example
Regression	A continuous number	Relationship between inputs and a numeric value	Predicting price from house size
Classification	A predefined class	Boundaries and probabilities between classes	Spam detection, disease screening

Regression predicts a continuous value such as a house price or temperature. The word itself did not make much sense to me, so I looked into where it came from.

Francis Galton was a nineteenth-century British researcher who studied physical traits and heredity statistically. While comparing the heights of parents and their children, he observed that the children of very tall parents were, on average, shorter than their parents, while the children of very short parents were, on average, taller. He called this tendency for an extreme measurement to move toward the population average regression toward the mean.

This does not mean every child must be closer to the average. It describes an average tendency observed across groups. Galton represented the relationship between parent and child height numerically and with a fitted line, using parent height to explain the average height of their children. The term regression remained attached to this type of relationship analysis and later broadened to methods that relate inputs to continuous outcomes.

That is why machine-learning problems such as predicting a house price from its size and room count, or tomorrow’s temperature from current weather measurements, are called regression. Finding that connection finally made the term less arbitrary to me.

Classification predicts one of a predefined set of categories. A label may be stored as a number such as 0 or 1, but that does not turn the problem into numeric prediction. If 0 means normal and 1 means defective, the numbers represent classes, so the task is classification.

Unsupervised Learning Finds Structure Without Answers

Unsupervised learning uses input data without known answers. In a customer dataset, each customer might have features such as purchase frequency, average spending, and preferred products. Customers with similar values can be treated as similar records. Looking at the distribution means checking where values are concentrated and which records are far apart. Groups and recurring patterns that emerge without predefined answers are what structure within the data means here.

Clustering places similar records into groups. A customer-clustering task can compare purchase frequency, average spending, and frequently purchased products. No one first labels each record as a loyal customer or a discount-sensitive customer. The groups are formed from similarity in the data. This is different from supervised classification, where the model learns to reproduce classes that were already defined.

Transformation and dimension reduction are also common unsupervised tasks. A dimension is one value needed to describe a single record. If one customer is represented by age, purchase frequency, and total spending, that record has three dimensions. Having ten thousand customers does not create ten thousand dimensions. The number of recorded features per customer determines the dimensionality.

Suppose each customer has 30 features, including age, recent visits, purchase frequency, total spending, average order value, and the proportion of purchases made with a discount. With ten thousand customers, the shape changes like this:

Before: 10,000 customers × 30 features per customer
After:  10,000 customers × 2 or 5 new features per customer

The number of customers stays the same. Only the number of values used to represent each customer decreases. Features that move together, such as purchase frequency and total spending, can be combined with other purchasing patterns into a smaller set of new axes. Reducing the data to two axes makes it possible to place each customer on a two-dimensional chart and inspect where customers with similar behavior gather.

Self-Supervised Learning Creates Answers from the Data

Self-supervised learning creates a prediction task from information already present in the original data instead of requiring a person to attach each label. Masking part of a sentence is one representative method.

Original sentence: The weather is nice today.
Model input:       The [masked word] is nice today.
Word to predict:   weather

Before sending the sentence to the model, the training pipeline hides the word weather and keeps the original word as the answer. The model predicts the missing word from the surrounding context, and the training process compares that prediction with the stored word to calculate the error. The original sentence supplies the answer without a person creating a separate answer sheet for it.

Raw data does not mean a different kind of data from the data used in supervised or unsupervised learning. It means the sentence or image before a person has attached a separate label. Predicting the next word in a sentence or reconstructing a hidden part of an image follows the same general idea.

Self-supervised learning is sometimes discussed within the broader category of unsupervised learning. It is also often separated because it creates an explicit prediction task from the data rather than only searching for groups or patterns.

Reinforcement Learning Uses Actions and Outcomes

In reinforcement learning, an agent is the program that chooses an action, while the environment is the game or physical setting in which it acts. The environment state is the information available for judging the current situation. In a game, that could include the character’s position, remaining health, and nearby obstacles. For a robot, it could include sensor readings and joint positions.

The agent observes the current state and selects an action such as moving or attacking. The environment returns the next state and a score for the result of that action. Reinforcement learning calls this score a reward. A policy is the rule the agent uses to select an action from the current state, and training improves the policy to increase the reward accumulated over time.

Observe the current state
  → choose an action
  → receive the next state and reward
  → update the policy from the experience
  → repeat

In a game, the model may choose movement or an attack from the current screen and receive points or the final win or loss as feedback. It is not given the correct action for every moment. Instead, it learns which sequences of decisions lead to better long-term outcomes.

This is often compared with learning through trial and error, but it is not the same as the full process of human learning. Reinforcement learning defines states, actions, rewards, and policies mathematically, then optimizes accumulated reward within that setup. Human behavior involves far more than those elements.

Writing this out took me beyond the simple distinction between learning with and without an answer sheet. Supervised, unsupervised, self-supervised, and reinforcement learning differ in how the learning problem is formed and what information is used to improve the model. I still have a pile of lecture notes left, but at least this part is now organized in my own words.