Here, we have two vectors, . They exist in the plane deﬁned by which is a two dimensional space (unless a and b point in the same direction).
Provided , .
Thus, the component of in the direction of is given by
The component of orthogonal (perpendicular) to is given by
Given , we can use and to represent the projection of vector onto and .
Given with linearly independent columns and vector :
Given with linearly independent columns, there exists a matrix with mutually orthonormal columns and upper triangular matrix such that . The vector that is the best solution (in the linear least-squares sense) to is given by
An algorithm for computing the QR factorization is given by
Any matrix can be written as the product of three matrices, the Singular Value Decomposition (SVD): $$A = U \Sigma V^T$$ where
If we partition
where and have columns and is , then is the “best” rank-k approximation to matrix B. So, the “best” rank-k approximation is given by the choices and .