Code › ai-engineering-study

[AI] Weekly Paper #1: Linear Algebra, EDA, and Data Preprocessing

The first weekly paper on the mathematical language of machine learning and the preparation of data for analysis

codeit-ai-sprintcodeit-ai-main-courseweekly-paperaibootcampai-engineeringlinear-algebraedadata-preprocessingmachine-learninglearning

This is my first weekly paper. I used what I learned in class as the starting point, then looked up the parts I wanted to understand more clearly. The paper covers three questions: why machine learning needs linear algebra, what exploratory data analysis does, and how missing, duplicate, and outlier values can be handled.

1. What is linear algebra, and why is it necessary for machine learning?

Wikipedia’s overview of linear algebra describes a field concerned with linear equations, linear maps, vector spaces, and their representation through matrices.

Linear algebra studies linear equations and transformations using objects such as vectors and matrices.

Source: Wikipedia, Linear algebra

A system of linear equations can be written in a compact form:

Ax=bAx = b

In this expression, the matrix AA represents the coefficients, the vector xx contains the unknown values, and the vector bb contains the results. Linear algebra provides the rules for working with these objects and for describing linear transformations between vector spaces.

For this paper, I am less interested in the full mathematical formalism than in how these ideas are used in machine learning. Going further into vector spaces, bases, and transformations would quickly become a separate topic.

Linear algebra acts as a basic language for representing data and performing machine-learning computations. Tabular data can be stored as a matrix whose rows are samples and whose columns are features. Images can be represented as arrays of pixel values, and text can be converted into vectors or matrices after an encoding step.

Training these models often requires large matrix multiplications. Linear regression expresses predictions through vector and matrix operations, while optimization methods such as gradient descent update a vector of model parameters. The data, the model parameters, and many of the calculations between them are therefore expressed with the same mathematical tools.

2. What is exploratory data analysis?

Exploratory data analysis, usually shortened to EDA, was developed to examine what data can reveal before relying only on formal statistical hypothesis testing.

EDA examines a dataset through summaries and visualizations to discover its structure, relationships, and unusual values.

Source: Wikipedia, Exploratory data analysis

When I receive a dataset, EDA is the process of taking it apart and checking its characteristics. I can inspect distributions, draw graphs, compare variables, and calculate statistical relationships. These steps help explain the phenomenon represented by the data and reveal patterns that are difficult to notice from raw rows and columns alone.

Data preprocessing, visualization, and statistical methods all take part in this process. Missing values or extreme observations may become visible during exploration, while a chart or correlation analysis can suggest a relationship that deserves a more focused test.

The distinction became easier to remember through an analogy. Formal hypothesis testing resembles a trial because it evaluates a claim using established procedures. EDA resembles a detective’s investigation because it examines the available evidence first and uses the findings to form possible explanations. After the data has been laid out and explored, a hypothesis can emerge from what was found.

3. How should missing, duplicate, and outlier values be handled?

  • A missing value is a value that was not collected or recorded for some reason.
  • A duplicate value is the same observation appearing more than once in the data.
  • An outlier is an observation that is unusually high or low compared with the overall distribution.

These imperfections need to be addressed before analysis or model training, but there is no single treatment that works for every dataset.

Missing values can be removed or imputed. Removing every row with a missing value may reduce the dataset enough to weaken the analysis, so the amount of available data and the reason for the missingness should be checked first. Simple imputation can replace numerical values with a mean or median and categorical values with a mode. Other approaches estimate a missing value from related variables or, in time-series data, fill it with a preceding or following observation.

Duplicate records are generally removed when they represent repeated copies rather than legitimate repeated events. Leaving unintended duplicates in the dataset can give those observations more influence than they should have.

Outliers require additional judgment because an extreme value is not automatically an error. Depending on the analysis, a skewed numerical variable can be transformed with a logarithm, extreme values can be capped at an upper or lower boundary, or a small number of invalid observations can be removed. The treatment should follow the meaning of the data rather than the size of the number alone.

When numerical features need to be put on a comparable scale, Min-Max transformation or Z-score standardization can also be used. Min-Max transformation maps the minimum value to 0 and the maximum value to 1:

x′=x−xminxmax−xminx' = \frac{x-x_{min}}{x_{max}-x_{min}}

Z-score standardization subtracts the mean from each value and divides the result by the standard deviation. After this transformation, the transformed data has a mean of 0 and a standard deviation of 1:

z=x−μσz = \frac{x-\mu}{\sigma}

I wrote separately about the difference between these transformations and why feature scaling matters before applying regularization such as Ridge or Lasso.