Foundations of probability · Section 3 of 3
Random variables
Turning outcomes into numbers; the Poisson law as a limit of binomials.
about 20 min
Start from a problem
A call centre has customers. Each of them calls within a given hour with probability , independently of the others. How many calls arrive in that hour? The number is random, of course; the real question is what the probability of calls is. With only lines open, how likely is it that they are not enough?
Doing it honestly involves numbers like , frighteningly large. Yet people noticed long ago that whenever you count how often a rare event happens (misprints on a page, calls in an hour, photons hitting a detector in a second), the distribution has nearly the same shape, and it depends only on the average count. Why?
To answer that, we first have to turn "the outcome of an experiment" into "a number". That is what a random variable does.
Play first, read later. The bars show how many of independent trials succeed, each with probability ; the dots show another distribution that depends only on the average count . Keep and drag from to . How do the bars and dots end up related? Then press "Sample": the row of cells in the middle is the complete outcome of one run, and the number of red cells is the number counted in that run.
- binomial B(n, p)
- Poisson(λ)
Peak analysis idea from: github.com/huzhuofan1020-svg/EXFB
Counting simulation idea from: github.com/huzhuofan1020-svg/PoissonDist
What did you observe?Play first, then answer. Read the explanation only after you have thought it through.
- 1.Keep λ = 3 and set n to 10, then 100, then 1000. How do the bars and dots come together? By what factor does the distance d shrink each time?
- 2.Fix n = 20 and compare λ = 1 with λ = 10. The n is the same; why is one approximation good and the other poor?
- 3.Press “tied peak” (n = 9, λ = 4.5). How many bars are tallest? In general, where is the most likely value?
The larger and the smaller the success probability , the closer the bars come to the dots, until only seems to matter; the plot below them suggests the gap shrinks roughly like . The dots follow , which suggests
Before we can say what it means for one distribution to approach another, we must say what "a number" and "a distribution" are.
Definitions
Let be a probability space. A function is called a random variable. If takes only finitely or countably many values, it is discrete.
A random variable is neither random nor a variable: it is a perfectly definite function, and all the randomness lives in the outcome . In the experiment, the row of cells is one outcome (each of the trials a success or a failure), and is the number of red cells. For every real ,
The probability mass function of a discrete random variable is the function , where ranges over the possible values of .
Its values are non-negative and add up to : the events for different are pairwise disjoint and together make up , so countable additivity gives
To write down the distribution of a number of successes, we extend independence from two events to many: are mutually independent if for every choice of some of them, the probability that all the chosen ones occur equals the product of their probabilities.
Run trials whose successes are mutually independent, each succeeding with probability . The number of successes has the binomial distribution, written .
If , then
Take one particular outcome: successes exactly at trials and failures at the other . By independence (replacing some independent events by their complements keeps them independent), its probability is , whichever trials the successes fall on. Different outcomes are disjoint, and there are ways to choose which trials succeed; add them up. By the binomial theorem
Where is the tallest bar? Compare neighbouring terms.
Let and . Then is largest at ; if is a whole number, and are tied for the maximum.
For , since
The ratio exceeds exactly when , that is, when . So the mass function rises while , falls once , and two neighbours are equal where . Write . If is not a whole number, the peak is at ; if it is, then and and share the peak.
Since , the most likely value is within of , but it need not equal ; indeed need not be a whole number, while only takes whole values. The small triangle in the experiment marks , and "tied peak" shows two equally tall bars.
Let . A random variable with values and
has the Poisson distribution with parameter , written
It is a genuine mass function: by the Taylor expansion of ,
Theorems and proofs
Let and (for ). For each fixed ,
Split the expression into three factors:
using
because as , which is just the derivative of at . Since is continuous,
Every step uses " fixed": if grew with , need not tend to . So the theorem says that each bar converges separately. The below the experiment measures the gap between the whole mass functions, and there is a stronger, more useful version for it.
The quantity in the experiment,
is the total variation distance between the two distributions. One can show that it equals the largest possible difference between and over all sets . In other words, if you answer any question about the count with the Poisson law instead of the binomial one, you are off by at most . In 1960 Le Cam proved
and the trials need not share one success probability: with probabilities , still . On the log–log plot this bound is a line of slope (the dashed line in the experiment), and the true runs parallel to it: ten times as many trials, a tenth of the error.
Large alone is not enough; must be small. Keep : with () the distance is about , with () about . can never exceed and is much narrower than . The next section shows that the binomial variance is and the Poisson variance is : they differ by the factor . The Poisson approximation is about rare events: each unlikely on its own, but with many chances.
Same distribution does not mean equal. Toss a fair coin times. The number of heads and the number of tails both follow , exactly the same distribution, yet unless the split is exactly to they are never equal. A distribution records the probability of each value, not which produced it. Likewise is not a number but an event, .
Applications
The number of calls is , with . With the Poisson law,
against the exact . With only lines, roughly one hour in twelve runs short; with , , about one hour in eighty. Neither nor appears in the calculation, only their product : double the customers and halve each one's chance of calling, and the answer barely moves.
people form pairs, and each pair shares a birthday with probability . The number of pairs that share is a count of many rare events, approximately with . Hence
exactly the exponential that the probability spaces section obtained from . At ,
The number of photons reaching a detector in one second is
(The second step puts .) So
In the plotter, draw and the constant . At the first is only , at it is , at it is : the key step in the proof of the Poisson limit theorem is this curve approaching this horizontal line.
Further: waiting times and densities
The same idea answers another question: how long until the next call? Let calls arrive at an average rate of per hour. Cut into short pieces; each contains a call with probability about , so by the Poisson limit theorem the number of calls in is approximately . The waiting time exceeds exactly when no call arrives in :
can be any real number in , and for every particular , so a mass function is useless here. We need another description.
The distribution function of a random variable is . If there is a non-negative function such that for every real
then is called continuous and is its probability density.
Then
The probability spaces section assumed finite or countable, where every subset can be an event. A random variable with continuous values such as needs an uncountable , and then not every subset can sensibly be given a probability: is defined only on a large enough family of sets, one that contains all intervals. The rigorous treatment is measure theory, which this path does not go into; this section only needs probabilities of intervals, where the issue never arises.
The binomial peak sits near , the Poisson peak near . The precise version of this "centre" is the mean, and of the "width" the variance: the subject of the next section, mean and variance (in progress). There we will also see that the covariance of two random variables is an inner product, so the Cauchy–Schwarz inequality gives at once.
Binomial peak analysis idea from: github.com/huzhuofan1020-svg/EXFB
Poisson counting simulation idea from: github.com/huzhuofan1020-svg/PoissonDist
Exercises
0 of 4 done
✓ All exercises in this section are done.
Discussion
Found a mistake, have a question or a suggestion? You can also write to 620026600@qq.com
No comments yet. Share your understanding, ask a question, or point out an error on this page.
Log in to join the discussion