Next: Orthogonal Projections
Up: CSL361 Problem set 5:
Previous: SVD
- Consider the least-squares problem:
Find the least-squares solution to the
set of
equations
, where and
Show that the following constitute a solution:
- Find the SVD
- Set
- Find the vector defined by
,
where is the diagonal entry of
- The solution is
- Consider the least-squares problem:
Find the general least-squares solution to the
set of
equations
, where and
Show that the following constitute a solution:
- Find the SVD
- Set
- Find the vector defined by
,
for
, and otherwise.
- The solution of minimum norm
is
- The general solution is
where
are the last columns of .
- Given a square diagonal matrix , its pseudo-inverse is
defined as the diagonal matrix such that
For an
matrix with with
,
its pseudo-inverse is defined as
Show that the least-squares solution to an
system of
equations
of rank is given by
(pseudo-inverse).
In the case of a deficient-rank system,
is the solution that minimizes
.
- Show that if is an
matrix of rank ,
then
and, in general,
a least-squares solution can be obtained by solving
the normal equations
- Weighted least-squares: Let be a positive
definite matrix. Then the -norm is defined as
. The
weighted least-squares problem is one of minimizing
. The most common weighting is
when is diagonal. Show that weigthed least-sqaures
solution can be obtained by solving:
- Consider the constrained least squares problem:
Given of size
, find that
minimizes
subject to
.
Show that the solution is given by the last column of
where
is the
SVD of .
- Consider the following constrained least-squares problem:
Given an
matrix with ,
find the vector that minimizes
subject to
and
.
Show that a solution is given as:
- If has fewer rows than columns, then add
rows to to make it square. Compute the
SVD
. Let
be the matrix obtained from after
deleting the first columns where is the number of
non-zero entries in
.
- Find the solution to minimization of
subject to
.
- The solution is obtained as
.
- Consider the following constrained least-squares problem:
Given an
matrix with ,
find the vector that minimizes
subject to
and
.
Show that a solution is given as:
- Compute the
SVD
. Let
- Let be the matrix of first columns of
where
.
- Find the solution to minimization of
subject to
.
- The solution is obtained as
.
Next: Orthogonal Projections
Up: CSL361 Problem set 5:
Previous: SVD
Subhashis Banerjee
2005-10-03