Summary table of probability terms
This page is a summary table of probability terms.
|Borel subsets of the reals|
|A Borel set|
|Events or measurable sets|
|Probability measure||or or|
|Probability triple or probability space|
|Distribution||or or or or or|
|Induced probability space|
|Cumulative distribution function or CDF|
|Probability density function or PDF|
|Preimage of random variable||but all we need is|
|Function of a random variable, where|
|Expected value of|
|Utility function||I think this is what the type must be, based on how it's used. But we usually think of the utility function as assigning numbers to outcomes; but if that is so, it must be a random variable! What's up with that?|
|Expected utility of||is indeed a random variable, so the type check passes.|
All the utility stuff isn't really related to machine learning. It's more related to the decision theory stuff I'm learning. I'm putting it here for now for convenience but might move it later.
TODO add "probability distribution over S" and "probability distribution on A" 
Li and Vitanyi (An Introduction to Kolmogorov Complexity and Its Applications, p. 19) calls the probability measure on a probability distribution over S (the sample space).
TODO: add probability mass function (defined only for discrete random variables)
Let be a probability space.
- Given a random variable , we can compute its distribution . How? Just let
- Given a random variable, we can compute the probability density function. How?
- Given a random variable, we can compute the cumulative distribution function. How?
- Given a distribution, we can retrieve a random variable. But this random variable is not unique? This is why we can say stuff like "let ".
- Given a distribution , we can compute its density function. How? Just find the derivative of . (?)
- Given a cumulative distribution function, we can compute the random variable. (Right?)
- Given a probability density function, can we get everything else? Don't we just have to integrate to get the cdf, which gets us the random variable and the distribution?
- Given a cumulative distribution function, how do we get the distribution? We have , which gets us some of what the distribution maps to, but is bigger than this. What do we do about the other values we need to map? We can compute intervals like . And we can apparently do the same for unions and limiting operations.
Philosophical details about the sample space
Given a random variable and any reasonable predicate about , we can replace with its extension for some . And from then on, we can write as . In other words, we can just work with Borel sets of the reals (measuring them with the distribution) rather than the original events (measuring them with the original probability measure). Where did go? , so you can write using . But once you already have , you don't need to know what is.
- 254A, Notes 0: A review of probability theory and 275A, Notes 0: Foundations of probability theory by Terence Tao
- Basic Random Variable Concepts by Kenneth Kreutz-Delgado
- Various questions on Mathematics Stack Exchange:
- Tim Gowers: