Quantities defined for a random variable that depend only on the distribution
Some quantities defined for a random variable, such as expected value and entropy (as well as quantities defined in terms of those, such as variance), depend only on the distribution of the random variable (and not, say, on the sample space). This means that a different random variable with the same distribution will have the same quantity.
We can think of such quantities as functions depending on a distribution, rather than a function of the random variable. For example, some texts define entropy for a list of probabilities instead(? in addition?) of a random variable with that distribution (i.e. ), denoting the entropy as rather than .
If it doesn't "really matter" how such quantities are defined, why do we so often define them for a random variable rather than distribution? I'm not sure, but I think one reason is that a random variable takes real values, so it is possible to easily define derivative quantities using arithmetic operations, e.g. . a natural way to define a distribution (for finite sets) is as a function . so the expected value is . but now what if we want to express the idea of shifting all the x's by ? we couldn't just say ; we would have to reach into the domain of p and shift stuff there. so something like " such that ". this gives the desired distribution, but it's very cumbersome to write down. now a clever person might say "ah, but so we could call it " but of course this doesn't work because p need not be injective.
Entropy seems to be one of the rare cases where the quantity depends only on the probabilities, not the values of the random variable. i.e. in , there are no s appearing in the expression; when they do appear, as in , they are always "wrapped around" the probability. This means that instead of specifying a distribution as a function , we can get away with specifying a list