User:IssaRice/Extreme value theorem

From Machinelearning

Working through the proof in Pugh's book by filling in the parts he doesn't talk about.

For x∈[a,b], define Vx=f([a,x])={f(t):a≤t≤x} to be the values that f takes on as the input ranges from a to x (inclusive).

Let M=sup{f(x):a≤x≤b}=supVb (this number exists by the boundedness theorem) and X={x∈[a,b]:supVx<M}.[note 1]

Our goal now is to find some x such that f(x)=M. If f(a)=M this is easy.

So now suppose f(a)<M. Then a∈X. We already know that X is bounded above, for instance by the number b. We can thus take the least upper bound of X, say c=supX. We already know f(c)≤M, so if we can just eliminate the possibility that f(c)<M, we will be done.

So suppose f(c)<M. We want to find M′<M such that f(t)<M′ for all t∈[a,c]. That would mean that supVc≤M′<M. To do this, we split the interval into two parts. Choose ϵ>0 with ϵ<M−f(c).[note 2] By continuity at c, there exists a δ>0 such that |t−c|<δ implies |f(t)−f(c)|<ϵ. So now pick a point like c−δ/2, and split the interval into [a,c−δ/2] and [c−δ/2,c].

  • Since c−δ/2<c, there exists x∈X such that c−δ/2<x (otherwise c−δ/2 would be a smaller upper bound for X). So supVc−δ/2≤supVx<M. This means that f(t)≤supVc−δ/2<M for all t∈[a,c−δ/2].
  • But now if t∈[c−δ/2,c], then |t−c|<δ, so |f(t)−f(c)|<ϵ. This means f(t)<f(c)+ϵ<M.

Now we can choose M′=max{supVc−δ/2,f(c)+ϵ}. Then whatever t∈[a,c] happens to be, we can say f(t)≤M′.

If c<b then by continuity we can find points t to the right of c where supVt<M, which contradicts the fact that c is an upper bound of such points.

Therefore, c=b, which implies that M=supVb=supVc<M, a contradiction. So the assumption that f(c)<M was false, and we conclude f(c)=M.

Notes

  1. ↑ If we had used "≤" in the definition of X, then when we take the supremum we would just end up with b, regardless of where f achieves the maximum.
  2. ↑ It is important here that ϵ does not equal M−f(c); choosing this ϵ would be too weak and we would not be able to conclude supVc<M, rather only that supVc≤M.