Variants of Solomonoff induction

From Machinelearning
Revision as of 03:29, 31 March 2019 by IssaRice (talk | contribs)

This page lists some variants of Solomonoff induction.

For determinism, I think "deterministic" is the same as "Solomonoff prior" and "stochastic" is the same as "universal mixture".

For discrete vs continuous, I think this just means whether the prior we define is over finite strings or over infinite sequences (where we want to know the probability of an infinite sequence starting with a given finite string).

Source Formula Determinism Type of machine used Discrete vs continuous Notes
LessWrong Wiki[1] where is the set of self-delimiting programs Deterministic Page doesn't say, but uses self-delimiting programs and it's discrete, so prefix Turing machine? Discrete because of the rather than
Scholarpedia discrete universal a priori probability[2] where the sum is over halting programs deterministic? prefix Turing machine discrete
Scholarpedia continuous universal a priori probability[2] where the sum is over minimal programs deterministic? Monotone Turing machine Continuous
Sterkenburg (p. 22)[3] where is a finite string, is the set of all halting (valid) inputs of length to the reference machine , is the set of all halting (valid) inputs of length that output something starting with deterministic universal Turing machine (no restrictions on prefix-free-ness) discrete?
Sterkenburg (p. 24)[3] where is the shortest program such that (i.e. the shortest program that causes the reference machine to output and halt) deterministic universal Turing machine, universal prefix machine (to get a probability distribution) discrete? this formula does not define a probability distribution over strings because the sum of probabilities does not converge
Sterkenburg (p. 25)[3] where is the set of all programs of length such that begins with deterministic universal Turing machine is divergent even for a single , so this is not actually a workable version, but is intended as a stepping stone
Sterkenburg (p. 26)[3] deterministic universal Turing machine, universal prefix machine (to get a probability distribution) The use of the is a hack to get the sum to converge

References

  1. https://wiki.lesswrong.com/wiki/Solomonoff_induction
  2. 2.0 2.1 Marcus Hutter; Shane Legg; Paul M.B. Vitanyi. "Algorithmic probability". Scholarpedia. 2007.
  3. 3.0 3.1 3.2 3.3 Tom Florian Sterkenburg. "The Foundations of Solomonoff Prediction". February 2013.