📕 Vector Projection is a simple yet elegant concept we see often in machine linear algorithms, most notably Support Vector Machines. In this tutorial, we dive deeper into Vector Projections, understanding the intuition behind the formula and how it works.
This short tutorial is split into the following sections
In every vector projection, it involves two components
For this example, we will project the green vector on the red vector.
When you project the green vector on the red vector, the output of the scalar projection is what is highlighted.
The vector projection is the vector from the origin to the tip of what is highlighted (where the black line forms a 90 degree angle with the red)
Vector Projections are as simple as that! Note that the highlighted vector projection can simply be calculated if we can find the magnitude from the origin to where the black vector meets the red vector!
Using trigonometry and linear algebra, we can easily calculate the vector projection. As we know from trigonometry,
$$ \begin{equation}cos(\theta) = \frac{adjacent}{hypotenuse} \end{equation} $$
We also know from linear algebra, that given two vectors, say $U$ and $X$
$$ \begin{equation} cos(\theta) = \frac{U \cdot X}{|U||X|} \end{equation} $$
Given these two formulas, we can rearrange $(1)$ so that it becomes the following
$$ \begin{equation} \frac{adjacent}{hypotenuse} = \frac{U \cdot X}{|U||X|} \end{equation} $$
If we use the picture with the added markers on the right for reference, we can use equation $(3)$ and write out the formula for the scalar projection (the scalar projection is the magnitude of the adjacent side)
$$ \frac{|AC|}{\cancel{|AB|}} = \frac{AB \cdot AC}{\cancel{|AB|}|AC|} $$
As we are solving for $AC$, we can cancel the $AB$ term. As a result, the scalar projection becomes a simple formula.
$$ \begin{equation}{|AC|} = \frac{AB \cdot AC}{|AC|}\end{equation} $$