Orthogonal Projections

Let be a subspace. ${\bf P} \in \mathbb{R}^{n \times n}$ is the orthogonal projection onto $S$ if $range({\bf P}) = S$, ${\bf P}^2 = {\bf P}$ and ${\bf P}^T = {\bf P}$.

1. Show the following:
   1. If ${\bf x} \in \mathbb{R}^n$ and ${\bf P}$ is an orthogonal projection on to $S$ ( ${\bf Px} \in S)$, then $({\bf I} - {\bf P})$ is an orthogonal projection onto $S^\perp$ ( $({\bf I} - {\bf P}){\bf x} \in S^\perp$) where $S^\perp$ is the orthogonal complement of $S$.
   2. The orthogonal projection onto a subspace is unique.
   3. If ${\bf v} \in \mathbb{R}^n$, then ${\bf P} = {\bf v}{\bf v}^T/{\bf v}^T{\bf v}$ is the orthogonal projection onto $S = span\{{\bf v}\}$.
   4. If the columns of ${\bf V} = \left[ {\bf v_1},\ldots,{\bf v_k}\right]$ are an orthonormal basis for $S$, then ${\bf V}{\bf V}^T$ is the unique orthonormal projection onto $S$.
2. Suppose that the SVD of ${\bf A}$ is ${\bf A} = {\bf U}{\bf\Sigma}{\bf V}^T$ and $rank({\bf A}) = r$. If we have the ${\bf U}$ and ${\bf V}$ partitionings:
        
   Then, show that
   1. ${\bf V_r}{\bf V_r}^T =$ projection onto $null({\bf A})^\perp = range({\bf A}^T)$
   2. ${\bf\tilde{V}_r}{\bf\tilde{V}_r}^T =$ projection onto $null({\bf A})$
   3. ${\bf U_r}{\bf U_r}^T =$ projection onto $range({\bf A})$
   4. projection onto $range({\bf A})^\perp = null({\bf A}^T)$
3. Let ${\bf v} \in \mathbb{R}^n$ be non-zero. The $n \times n$ matrix
   $${\bf P} = {\bf I} - 2\frac{{\bf v}{\bf v}^T}{{\bf v}^T{\bf v}}$$
   is the Householder reflection discussed in class. Show that:
   1. ${\bf P}$ is an orthogonal projection.
   2. When a vector ${\bf v} \in \mathbb{R}^n$ is multiplied by ${\bf P}$, it is reflected in the hyperplane .
   3. If
      $${\bf v} = {\bf x} \pm \Vert{\bf x}\Vert _2 {\bf e_1}$$
      Then,