Math Problem Statement

Reformulation of a nonsmooth problem. This is the first part of exercise 12.5 in Nocedal [1]. See section A.1 in the Appendix for the definition of the l∞l_{\infty}l∞​ norm. Also, see our lecture notes from Nov 1.

Let v:Rn→Rmv : \mathbb{R}^n \rightarrow \mathbb{R}^mv:Rn→Rm be a smooth vector function and consider the unconstrained optimization problem

min⁡∥v(x)∥∞\min |v(x)|_{\infty}min∥v(x)∥∞​

Reformulate this problem as a smooth constrained optimization problem.

References

Jorge Nocedal and Stephen J. Wright. Numerical Optimization. Springer, second edition, 2006.

Solution

To reformulate the unconstrained optimization problem

minv(x)\min \|v(x)\|_{\infty}

as a smooth constrained optimization problem, we start by understanding the ll_{\infty} norm. The ll_{\infty} norm of a vector v(x)Rmv(x) \in \mathbb{R}^m is defined as:

v(x)=maxivi(x)\|v(x)\|_{\infty} = \max_{i} |v_i(x)|

where vi(x)v_i(x) denotes the ii-th component of the vector v(x)v(x).

Reformulation Steps

  1. Introduce Auxiliary Variable: Let tt be a new variable that will represent the maximum absolute value of the components of v(x)v(x). Hence, we introduce the constraint:

    vi(x)t,i=1,,m|v_i(x)| \leq t, \quad \forall i = 1, \ldots, m

  2. Formulate the Constrained Problem: The original problem can now be reformulated as:

    mint\min t

    subject to the constraints:

    vi(x)ti=1,,m|v_i(x)| \leq t \quad \forall i = 1, \ldots, m

    This formulation maintains smoothness since v(x)v(x) is smooth, and the constraints are also well-defined.

Final Formulation

The smooth constrained optimization problem can be stated as:

\text{minimize} & \quad t \\ \text{subject to} & \quad |v_i(x)| \leq t, \quad \forall i = 1, \ldots, m \end{align*}$$ This approach effectively captures the original unconstrained problem using smooth constraints, thus facilitating the use of optimization techniques that rely on constrained formulations. If you have any questions or need further details on this topic, feel free to ask! ### Additional Questions for Exploration: 1. How does the choice of the $$l_{\infty}$$ norm affect the optimization landscape? 2. What methods can be employed to solve the resulting constrained optimization problem? 3. How would you reformulate this problem if $$v(x)$$ were not smooth? 4. What are the implications of introducing auxiliary variables in optimization? 5. Can this formulation be extended to other norms, such as $$l_1$$ or $$l_2$$? ### Tip: When working with constrained optimization problems, ensure that you understand the properties of the constraints and their implications on the feasibility and optimality of solutions.

Ask a new question for Free

By Image

Drop file here or Click Here to upload

Math Problem Analysis

Mathematical Concepts

Optimization
Nonsmooth Analysis
Vector Functions

Formulas

l∞ norm: ∥v(x)∥∞ = max_i |v_i(x)|
Constrained Problem: min t subject to |v_i(x)| ≤ t

Theorems

Properties of the l∞ norm
Constrained Optimization Theorems

Suitable Grade Level

Graduate Level