Convexity arguments in optimisation almost always reduce to the same three
inequalities about secant slopes. I have derived them from scratch at least four
times, which is three times too many, so the derivation lives here now.
Statement
Let f be convex on an interval I, and let x<y<z be points of I. Then
1
y−xf(y)−f(x)≤z−xf(z)−f(x)≤z−yf(z)−f(y)
Lemma 1 — secant slopes of a convex function are non-decreasing in both endpoints.
The middle term is what makes this useful: it says the average slope over the
whole interval lies between the two partial average slopes.
Proof
Write y as a convex combination of the endpoints. Since x<y<z,
2
y=(1−t)x+tz,t=z−xy−x∈(0,1)
By convexity of f:
3
f(y)≤(1−t)f(x)+tf(z)
Subtract f(x) from both sides and use tf(z)−tf(x)=t(f(z)−f(x)):
4
f(y)−f(x)≤t[f(z)−f(x)]
Dividing by y−x=t(z−x), which is strictly positive, gives the left
inequality of Lemma 1:
5
y−xf(y)−f(x)≤z−xf(z)−f(x)
For the right inequality of Lemma 1, multiply out the target inequality by the
positive quantity (z−x)(z−y) and collect terms; what is left is
6
f(y)≤z−xy−xf(z)+z−xz−yf(x)
which is convexity again, with the weights summing to one. So the two
inequalities are the same statement read in opposite directions, and no separate
argument is needed. □
Why it is worth writing down
The lemma is trivial. Its value is that it converts "by convexity" — which is
where hand-waving usually starts — into a specific inequality you can point at.
Most of the arguments I have watched fall apart did so because a step assumed
this lemma without stating it, and then applied it to a function that was not
convex.