We are tasked with proving the equivalence between a function being convex and satisfying the inequality

$f(\lambda x_1 + (1 - \lambda) x_2) \le \lambda f(x_1) + (1 - \lambda) f(x_2)$

for all $x_1, x_2 \in \mathbb{R}^n$ and $\lambda \in [0, 1]$. To do this, we need to prove both directions of the implication:

(a) "If" Part:

Assume that the inequality

$f(\lambda x_1 + (1 - \lambda) x_2) \le \lambda f(x_1) + (1 - \lambda) f(x_2)$

holds for all $x_1, x_2 \in \mathbb{R}^n$ and $\lambda \in [0, 1]$. We need to show that $f$ is convex, meaning the epigraph of $f$, the set $S = \{ (x, t) \in \mathbb{R}^{n+1} : f(x) \le t \}$, is convex.

Proof:

1. Take any two points $(x_1, t_1)$ and $(x_2, t_2)$ in the epigraph $S$, so that: $f(x_1) \le t_1 \quad \text{and} \quad f(x_2) \le t_2.$

2. For any $\lambda \in [0, 1]$, define the point $x = \lambda x_1 + (1 - \lambda) x_2$ and $t = \lambda t_1 + (1 - \lambda) t_2$.

3. By our assumption, we know that: $f(x) = f(\lambda x_1 + (1 - \lambda) x_2) \le \lambda f(x_1) + (1 - \lambda) f(x_2).$

4. Since $f(x_1) \le t_1$ and $f(x_2) \le t_2$, we have: $f(x) \le \lambda t_1 + (1 - \lambda) t_2 = t.$

5. Therefore, the point $(x, t)$ lies in the epigraph $S$, i.e., $(x, t) \in S$.

6. This shows that the epigraph is convex, and by definition, this implies that $f$ is convex.

(b) "Only If" Part:

Now assume that $f$ is convex, meaning the epigraph $S = \{ (x, t) \in \mathbb{R}^{n+1} : f(x) \le t \}$ is convex. We need to show that $f$ satisfies the inequality:

$f(\lambda x_1 + (1 - \lambda) x_2) \le \lambda f(x_1) + (1 - \lambda) f(x_2)$

for all $x_1, x_2 \in \mathbb{R}^n$ and $\lambda \in [0, 1]$.

Proof:

1. Let $x_1, x_2 \in \mathbb{R}^n$, and set $t_1 = f(x_1)$ and $t_2 = f(x_2)$. Thus, the points $(x_1, t_1)$ and $(x_2, t_2)$ lie in the epigraph $S$, i.e., $f(x_1) \le t_1$ and $f(x_2) \le t_2$.

2. Since $S$ is convex, the point $(\lambda x_1 + (1 - \lambda) x_2, \lambda t_1 + (1 - \lambda) t_2)$ must also lie in $S$. This means: $f(\lambda x_1 + (1 - \lambda) x_2) \le \lambda t_1 + (1 - \lambda) t_2.$

3. Since $t_1 = f(x_1)$ and $t_2 = f(x_2)$, we have: $f(\lambda x_1 + (1 - \lambda) x_2) \le \lambda f(x_1) + (1 - \lambda) f(x_2).$

Thus, the inequality is satisfied, completing the proof.

Conclusion:

We have shown both directions of the equivalence:

• If the inequality holds, $f$ is convex.
• If $f$ is convex, the inequality holds.

Therefore, the function $f$ is convex if and only if

$f(\lambda x_1 + (1 - \lambda) x_2) \le \lambda f(x_1) + (1 - \lambda) f(x_2)$

for any $x_1, x_2 \in \mathbb{R}^n$ and $\lambda \in [0, 1]$.

---

Would you like further clarifications or have any specific questions on this proof?

Here are some related questions you might find interesting:

1. What is the definition of a convex function in terms of its second derivative?
2. Can a function be convex if it is not differentiable? If so, give an example.
3. What are the conditions under which a function is concave?
4. How does the concept of convexity relate to optimization problems?
5. What is the role of convexity in machine learning, especially in loss functions?

Tip: Remember that convexity is crucial in optimization because any local minimum of a convex function is also a global minimum, making it easier to find optimal solutions.