Abstracting sensory data in Solomonoff induction

From Machinelearning
Jump to: navigation, search

One potential misconception one might have about Solomonoff induction is the degree to which one "abstracts" the sensory information that Solomonoff induction is "allowed to use".

(Eliezer brings this up a couple of times in his Solomonoff dialogue)

Examples:

  • If one is predicting whether the sun will rise tomorrow using Solomonoff induction, the sensory stream one uses is not just a string of 1s and 0s for "sun rose" vs "sun didn't rise", i.e. the sensory stream is not just 1, 1, 1, 1, ... This would be abstracting out the sensory data too much. Rather, one is allowed to use all the rich information one has acquired during the course of one's life, including the brightness of the sun on all previous days, experimental results of all physics experiments one has performed, the image of one eating one's own breakfast on every day, and so forth. (example due to eliezer)
  • If one is predicting the sequence of balls drawn out of a barrel, then the sensory data one has access to is not merely the sequence BWBW... of the balls, but also all prior information one has, e.g. the fact that the balls are drawn from a barrel, the sequences of draws one has seen from previous barrels one has seen, etc. (example due to eliezer)
  • Solomonoff induction ignores the element of time, which might seem to limit the phenomena about which Solomonoff induction can make accurate predictions. For instance, if a program immediately prints out 1, 2, 3, 4, then takes an hour to print 5, this would seem to rule out simple programs such as for (i = 1 to infinity) { print i; }. And yet, Solomonoff induction would just see "1, 2, 3, 4, 5" so wouldn't be able to rule out this simple program. Again the problem is that one is abstracting out the sensory data too much. In this example, if a human is allowed to use pauses in the output, then so should Solomonoff induction. The non-abstracted/raw sensory stream would then look like 1, 2, 3, 4, NULL, …, NULL, 5, which allows Solomonoff induction to rule out the simple program.