Equivalence of random coinflips view and minimal programs view

From Machinelearning
Revision as of 00:23, 16 April 2019 by IssaRice (talk | contribs)

Two of the several ways of viewing the deterministic variant of Solomonoff induction are:

  • The "random coinflips view": Given a monotone Turing machine, we feed the machine with random data (such as from unbiased coinflips). The probability of string x is then the probability that the machine outputs something starting with x. This one is somewhat hard to think about, because one usually thinks of a Turing machine as e.g. a Python interpreter that takes Python programs. If one feeds a Python interpreter with randomness, one is likely to end up running a horrible mess that isn't even syntactically valid. I guess the reply here is something like (1) if a program isn't even syntactically valid, it just results in a program with no output; (2) using an encoding, one can filter out all invalid programs so that each input into the UTM does something; (3) more generally, thinking of "coinflips" is meant to suggest the idea that one has no information about what the program looks like.
  • The "minimal programs view": Given a monotone Turing machine T, the minimal programs producing x are those programs p such that T(p)=x* (i.e. T when run with p produces an output starting with x) and such that no proper prefix of p produces x (i.e. if qp then T(q)x*). This view then says that to get the probability of x, we add up all the minimal programs weighted by length as follows: for each program p we assign the weight 2|p|, where |p| is the length of p in bits. Thus the measure is p2|p|, where the sum is taken over all minimal programs p producing x. The idea with the 2|p| is that there are 2n programs of length n, so given just the information that p has length n, the probability that we have a specific program is 1/2n=2n.

These two views turn out to be equivalent in the sense that .