User:IssaRice/Pareto distribution

From Machinelearning


I still don't know how this is derived, but here's how to make sense of the 80/20 rule:

What does it mean to say that the top 20% own 80% of the wealth? It means that cxf(x)dxxmxf(x)dx=0.80 where we choose the cutoff c according to cf(x)dx=0.20.

Why? cf(x)dx is the fraction of people with wealth above the cutoff c; we're just integrating the pdf of the distribution.

cxf(x)dx is the amount of wealth owned by the people with wealth above c. This is because at each wealth level x, such people are a fraction f(x)dx of people, and they each own wealth x, so the wealth owned by them is xf(x)dx. Now just sum over all of them from c to infinity.

The above can also be expressed as E[XX>c]E[X]=0.80 where the cutoff c is chosen by P(X>c)=0.20.

Using α=(log5)/(log4) (the specific value of alpha needed to make the 80/20 rule hold) and xm=1 in the Pareto distribution, we get c=10.201/α=10.20(log4)/(log5). The mean is αα17.213 [1] and cxf(x)dx5.77 [2]. Sure enough, 5.77/7.213 is very close to 0.80.

Another way to look at this is to look at the inverse CDF, which takes the percentile of wealth to the actual value of the wealth. We should get 10.21cdf101cdf1=0.8 [3].

deriving the scaled pareto

from the graph of the inverse cdf, it's clear that the y axis isn't what we want (it doesn't correspond to income). it seems natural to scale the whole graph in this setting, so given the inverse cdf x=1(1F)1/α, scale this by a constant β>0 to get x=β(1F)1/α. Solving this for F gives F=1βαxα and differentiating wrt x gives the pdf f(x)=βααxα+1.

since the y-axis of the inverse cdf was shifted uniformly, the "80/20"-ness of the distribution doesn't change.

three ways to picture the pareto distribution

  • pdf
  • inverse cdf; examples: [4], [5]
  • lorenz curve; examples: [6]