Math Problem Statement

Let A \in \mathbb{R}^{n \times n} be a constant matrix and b \in \mathbb{R}^{n} be a constant vector. Let z \in \mathbb{R}^{n}. Consider the function g(z) defined as g(z)=\frac{1}{2}|A z-b|{2}^{2}+\frac{1}{2}|z|{2}^{2} Consider the problem of finding the minimum of g(z) using gradient descent. Derive the optimal step size \alpha_{k} to be used at iteration number k of the gradient descent scheme assuming estimate z_{k-1} from the previous iteration.

Solution

Ask a new question for Free

By Image

Drop file here or Click Here to upload

Math Problem Analysis

Mathematical Concepts

Gradient Descent
Matrix Operations
Quadratic Forms

Formulas

Gradient of a quadratic function

Theorems

-

Suitable Grade Level

Advanced Undergraduate