Week 10 - Vector Spaces, Orthogonality, and Linear Least-Squares
- Solving underdetermined systems
- Orthogonal Vectors & Orthogonal Spaces
- Approximating a Solution
- Refers
- Words
1. Solving underdetermined systems
- Important attributes of a linear system and associated matrix A:
- (example:)
- The row-echelon form of the system.
- The pivots.
- the first nonzero entry in each row: 1, 2.
- The free variables.
- the columns that has no pivots:
- The dependent variables.
- the columns that has pivots:
- A specific solution.
- Often called a particular solution.
- The most straightforward way is to set the free variables equal to zero
- =>
- =>
- =>
- A basis for the null space.
- Often called the kernel of the matrix.
- =>
- So the basic for
- A general solution.
- Often called a complete solution.
- given by:
- A basis for the column space, .
- Often called the range of the matrix.
- equal to the number of dependent variables.
- The columns that have pivots in them are linearly independent. The corresponding columns in the original matrix are also linearly independent:
- A basis for the row space, .
- The row space is the subspace of all vectors that can be created by taking linear combinations of the rows of a matrix.
- List the rows that have pivots in the row echelon form as column vectors:
- Notice these are the first and third row of A.
- The dimension of the row and column space.
- = number of pivots
- = 2
- The rank of the matrix.
- = number of pivots
- = 2
- The dimension of the null space.
- = the number of non-pivots columns
- = 2
2. Orthogonal Vectors & Orthogonal Spaces
- Vectors x and y are considered to be orthogonal (perpendicular) if they meet at a right angle:
2.1. Normal Vector
- The normal vector, often simply called the "normal," to a surface is a vector which is perpendicular to the surface at a given point.
- For example:
- Define the plane as format:
- Vector is normal to the plane.
- Vector is pointing to the plane.
- Vector is pointing to the plane.
- Then should be on the plane and perpendicular to
- Then
- So we can use to represent the plane.
Cross Product
- the cross product or vector product is a binary operation on two vectors in three-dimensional space () and is denoted by the symbol ×.
- Given two linearly independent vectors a and b, the cross product, a × b, is a vector that is perpendicular to both a and b and thus normal to the plane containing them.
- Because and
- So we can use vectors a and b to get n. ()
Visualizing a column space as a plane in R3
- For example:
- .
- rref: reduced row-echelon form.
- Define is the normal vector to , And vector is point to the surface. Then:
- Use cross product, we get
- <=>
2.2. Orthogonal Spaces
- Definition: Let be subspaces. Then and are said to be orthogonal iff and implies . Denoted by
- Definition: Given subspace , the set of all vectors in that are orthogonal to is denoted by (pronounced as “V-perp”).
2.3. Fundamental Spaces
- Recall some definitions. Let and have k pivots. Then:
- Column space: .
- dimension: k
- Null space: .
- dimension: n - k
- is vector
- Row space: .
- dimension: k
- Left null space: .
- dimension: m - k
- is vector
- Column space: .
Theorem: Let . Then:
- .
- every can be written as where and .
- is a one-to-one, onto mapping from to .
- is orthogonal to and the dimension of equals , where is the dimension of .
- For example:
- ,
- =>
- =>
3. Approximating a Solution
- Find a line to interpolate these points:
- Clearly, there is no line could go through all these points, then what is the best approximation?
- Set
- We've learned before that has a solution iff . In other words, b is in the plane of .
- So, here we are solving .
Set the projection of b = ,
We can get
- where for all .
- Also => . So, (same as ), which means
- Rewrite it, we get .
- This is known as the normal equation associated with the problem .
- Although is nonsingular, then
- And the vector closest to is given by
- This shows that if A has linearly independent columns, then is the vector in the columns space closest to b. This is the projection of b onto the column space of A.
- And the “best” solutionis known as the “linear least-squares” solution.
4. Refers
- https://en.wikipedia.org/wiki/Row_and_column_spaces
- https://www.khanacademy.org/math/linear-algebra/vectors-and-spaces/dot-cross-products/v/defining-a-plane-in-r3-with-a-point-and-normal-vector
- https://en.wikipedia.org/wiki/Normal_(geometry))
- http://mathworld.wolfram.com/NormalVector.html
- https://en.wikipedia.org/wiki/Cross_product
5. Words
- orthogonality [,ɔ:θɔɡə'næləti] n. [数] 正交性;相互垂直