Let be symmetric and positive definite matrices, and let . Consider the quadratic function for and a descent method to approximate the solution of :
Define the concept of steepest descent and show how to compute the optimal stepsize
Optimal step size
Choose such that is minimized i.e.
Setting the above expression equal to zero gives the optimal :
Note that since is symmetric
Formulate the steepest descent (or gradient method) method and write a pseudocode which implements it.
Note that . Then the minimal is given by
Let be a preconditioner of . Show how to modify the steepest descent method to work for and write a pseudocode. Note that may not be symmetric. (Hint: proceed as with the conjugate gradient method).
Since is symmetric, positive definite, where is upper triangular (Cholesky Factorization).
is positive definite:
since positive definite