$K < m$
Randomly pick K training examples
Set $\mu_1, \ldots, \mu_K$ equal to these K examples.
How to fix Local Optima
Data Compression
Visualization
Say we want to reduce a 2D data to 1D:
A more formal description is:
More general:
PCA is not linear regression
Sigma = (1/m)*(X'X)
[U, S, V] = svd(Sigma);
svd
: singular value decompositionU,S,V
: Matrix
[U, S, V] = svd(Sigma);
Ureduce = U(:, 1:k);
z = Ureduce'*x;
svd
function([U, S, V] = svd(Sigma)
), we get S, which is a diagonal matrix, and $$\frac{\frac{1}{m} \sum_{i=1}^m \lVert x^{(i)} - x_{\text{approx}^{(i)}} \rVert ^2}{\frac{1}{m} \sum_{i=1}^m \lVert x^{(i)} \rVert ^2} = 1 - \frac{\sum_{i=1}^k S_{ii}}{\sum_{i=1}^n S_{ii}} \le 0.01$$