User:IssaRice/Formalizing adjustment of epsilons

From Machinelearning

I want to elaborate on a thing that shows up when proving part (a) of https://taoanalysis.wordpress.com/2020/05/24/exercise-5-6-1/

Actually, a similar thing appears in Tao's original proof (Proposition 5.5.12) and the situation is simpler there (fewer variables to deal with), so let's start there.

Tao's proof

In the first part of the proof, we suppose , and let . Then we show the bound . Now we say that since , we can choose small enough that .

What's so sneaky about this? It's the fact that we say "let " and then later adjust the value of . Typically in math, when we say "let ", we are allowing the value to be arbitrary, i.e. potentially adversarially chosen. We can't let the adversary choose something, then later change our mind and say "wait a minute, let me pick it instead"!

So what is going on, and why is the argument correct?

I'll use quantifiers so we can be really clear about what's bound and what's free.

We want to show , because this will show that , which will show that isn't an upper bound of .

What we have are:

(1)
(2)

So now what we do is use (2) to obtain a particular . In order to use (1), we actually need to to show , but we only have .

So actually, we want to show

(3)

To do this, use (2) to obtain such that . If then we are done. If , then so , which means we can use 1/2. In either case we have found a number that works, so we have (3).

So now, use (3) to obtain some . By (1), we have . Since , we can apply this implication to obtain . By (3), we know so we have which is what we wanted.

So what was the issue? It was that in the first part of the proof, we were allowing to be arbitrary so that we could show the implication (1). Once we showed that, however, we want to close that context so that we could start a new context with some particular . In first-order logic, these aren't the same , but when writing out proofs we aren't too careful about this.

The general case