<<

. 4
( 6)



>>


15. Assume that p and q are independent statements relative to a
given measure. Prove that each of the following pairs of state-
ments are independent relative to this same measure.

(a) p and ¬q.
(b) ¬q and p.
(c) ¬p and ¬q
110 CHAPTER 4. PROBABILITY THEORY

16. Prove that for any three statements p, q, and r,

Pr[p § q § r] = Pr[p] · Pr[q|p] · Pr[r|p § q].

17. A coin is thrown twice. Let p be the statement “Heads turns up
on the ¬rst toss” and q the statement “Heads turns up on the
second toss”. Show that it is possible to assign a measure to the
possibility space {HH, HT, TH, TT} so that these statements are
not independent.

18. A multiple-choice test question lists ¬ve alternative answers, of
which just one is correct. If a student has done the homework,
then he or she is certain to identify the correct answer; otherwise,
the student chooses an answer at random. Let p be the statement
“The student does the homework” and q the statement “The
student answers the question correctly”. Let Pr[p] = a.

(a) Find a formula for Pr[p|q] in terms of a.
(b) Show that Pr[p|q] ≥ Pr[p] for all values of a. When does the
equality hold?

19. A coin is weighted so that heads has probability .7, tails has
probability .2, and it stands on edge with probability .1. What
is the probability that it does not come up heads, given that it
does not come up tails?
1
[Ans. .]
8

20. A card is drawn at random from a deck of playing cards. Are the
following pairs of statements independent?

(a) p: A jack is drawn. q: A black card is drawn.
(b) p: An even numbered heart is drawn. q: A red card smaller
than a ¬ve is drawn.
21. A simple genetic model for the color of a person™s eyes is the
following: There are two kinds of color-determining genes, B and
b, and each person has two color-determining genes. If both are
b, he or she has blue eyes; otherwise he or she has brown eyes.
Assume that one-quarter of the people have two B genes, one-
quarter of the people have two b genes, and the rest have one B
gene and one b gene.
111
4.6. FINITE STOCHASTIC PROCESSES

(a) If a person has brown eyes, what is the probability that he
or she has two B genes?
Assume that a child™s mother and father have brown eyes
and blue eyes, respectively.
(b) What is the probability that the child will have brown eyes?
(c) If the child has brown eyes, what is the probability that the
father has two B genes?
1
[Ans. .]
2

22. Three red, three green, and three blue balls are to be put into
three urns, with at least two balls in each urn. Then an urn is
selected at random and two balls withdrawn.

(a) How should the balls be put in the urns in order to maximize
the probability of drawing two balls of di¬erent color? What
is the probability?
[Ans. 1.]
(b) How should the balls be put in the urns in order to maximize
the probability of withdrawing a red and a green ball? What
is the maximum probability?
7
[Ans. .]
10



4.6 Finite stochastic processes
We consider here a very general situation which we will specialize in
later sections. We deal with a sequence of experiments where the out-
come on each particular experiment depends on some chance element.
Any such sequence is called a stochastic process. (The Greek word
“stochos” means “guess”.) We shall assume a ¬nite number of exper-
iments and a ¬nite number of possibilities for each experiment. We
assume that, if all the outcomes of the experiments which precede a
given experiment were known, then both the possibilities for this ex-
periment and the probability that any particular possibility will occur
would be known. We wish to make predictions about the process as a
whole. For example, in the case of repeated throws of an ordinary coin
we would assume that on any particular experiment we have two out-
1
comes, and the probabilities for each of these outcomes is 2 regardless
112 CHAPTER 4. PROBABILITY THEORY




Figure 4.3: ™¦

of any other outcomes. We might be interested, however, in the proba-
bilities of statements of the form, “More than two-thirds of the throws
result in heads”, or “The number of heads and tails which occur is the
same”, etc. These are questions which can be answered only when a
probability measure has been assigned to the process as a whole. In this
section we show how a probability measure can be assigned, using the
given information. In the case of coin tossing, the probabilities (hence
also the possibilities) on any given experiment do not depend upon the
previous results. We will not make any such restriction here since the
assumption is not true in general.
We shall show how the probability measure is constructed for a par-
ticular example, and the procedure in the general case is similar. We
assume that we have a sequence of three experiments, the possibilities
for which are indicated in Figure 4.3. The set of all possible outcomes
which might occur on any of the experiments is represented by the set
{a, b, c, d, e, f }. Note that if we know that outcome b occurred on the
¬rst experiment, then we know that the possibilities on experiment two
are {a, e, d}. Similarly, if we know that b occurred on the ¬rst experi-
ment and a on the second, then the only possibilities for the third are
{c, f }. We denote by pa the probability that the ¬rst experiment re-
sults in outcome a, and by pb the probability that outcome b occurs in
the ¬rst experiment. We denote by pb,d the probability that outcome
d occurs on the second experiment, which is the probability computed
on the assumption that outcome b occurred on the ¬rst experiment.
Similarly for pb,a ,pb,e ,pa,a ,pa,c . We denote by pbd,c the probability that
outcome c occurs on the third experiment, the latter probability being
113
4.6. FINITE STOCHASTIC PROCESSES

computed on the assumption that outcome b occurred on the ¬rst ex-
periment and d on the second. Similarly for pba,c ,pba,f , etc. We have
assumed that these numbers are given and the fact that they are proba-
bilities assigned to possible outcomes would mean that they are positive
and that pa + pb = 1, pb,a + pb,e + pb,d = 1, and pbd,a + pbd,c = 1, etc.
It is convenient to associate each probability with the branch of
the tree that connects to the branch point representing the predicted
outcome. We have done this in Figure 4.3 for several branches. The
sum of the numbers assigned to branches from a particular branch point
is 1, e.g., pb,a + pb,e + pb,d = 1.
A possibility for the sequence of three experiments is indicated by a
path through the tree. We de¬ne now a probability measure on the set
of all paths. We call this a tree measure. To the path corresponding to
outcome b on the ¬rst experiment, d on the second, and c on the third,
we assign the weight pb ·pb,d ·pbd,c . That is the product of the probabilities
associated with each branch along the path being considered. We ¬nd
the probability for each path through the tree.
Before showing the reason for this choice, we must ¬rst show that it
determines a probability measure, in other words, that the weights are
positive and the sum of the weights is 1. The weights are products of
positive numbers and hence positive. To see that their sum is 1 we ¬rst
¬nd the sum of the weights of all paths corresponding to a particular
outcome, say b, on the ¬rst experiment and a particular outcome, say
d, on the second. We have
pb · pb,d · pbd,a + pb · pb,d · pbd,c = pb · pb,d [pbd,a + pbd,c ] = pb · pb,d .
For any other ¬rst two outcomes we would obtain a similar result.
For example, the sum of the weights assigned to paths corresponding
to outcome a on the ¬rst experiment and c on the second is pa · pa,c .
Notice that when we have veri¬ed that we have a probability measure,
this will be the probability that the ¬rst outcome results in a and the
second experiment results in c.
Next we ¬nd the sum of the weights assigned to all the paths cor-
responding to the cases where the outcome of the ¬rst experiment is
b. We ¬nd this by adding the sums corresponding to the di¬erent pos-
sibilities for the second experiment. But by our preceding calculation
this is
pb · pb,a + pb · pb,e + pb · pb,d = pb [pb,a + pb,e + pb,d ] = pb .
Similarly, the sum of the weights assigned to paths corresponding
to the outcome a on the ¬rst experiment is pa . Thus the sum of all
114 CHAPTER 4. PROBABILITY THEORY

weights is pa + pb = 1. Therefore we do have a probability measure.
Note that we have also shown that the probability that the outcome of
the ¬rst experiment is a has been assigned probability pa in agreement
with our given probability.
To see the complete connection of our new measure with the given
probabilities, let Xj = z be the statement “The outcome of the jth
experiment was z”. Then the statement [X1 = b § X2 = d § X3 = c] is
a compound statement that has been assigned probability pb · pb,d · pbd,c .
The statement [X1 = b§X2 = d] we have noted has been assigned prob-
ability pb · pb,d and the statement [X1 = b] has been assigned probability
pb . Thus
pb · pb,d · pbd,c
Pr[X3 = c|X2 = d § X1 = b] = = pbd,c ,
pb · pb,d
pb · pb,d
Pr[X2 = d|X1 = b] = = pb,d .
pb
Thus we see that our probabilities, computed under the assumption
that previous results were known, become the corresponding condi-
tional probabilities when computed with respect to the tree measure.
It can be shown that the tree measure which we have assigned is the
only one which will lead to this agreement. We can now ¬nd the proba-
bility of any statement concerning the stochastic process from our tree
measure.

Example 4.11 Suppose that we have two urns. Urn 1 contains two
black balls and three white balls. Urn 2 contains two black balls and
one white ball. An urn is chosen at random and a ball chosen from this
urn at random. What is the probability that a white ball is chosen?
A hasty answer might be 1 since there are an equal number of black
2
and white balls involved and everything is done at random. However,
it is hasty answers like this (which is wrong) which show the need for
a more careful analysis.
We are considering two experiments. The ¬rst consists in choosing
the urn and the second in choosing the ball. There are two possibilities
for the ¬rst experiment, and we assign p1 = p2 = 1 for the probabilities
2
of choosing the ¬rst and the second urn, respectively. We then assign
p1,W = 3 for the probability that a white ball is chosen, under the
5
assumption that urn 1 is chosen. Similarly we assign p1,B = 2 , p2,W = 3 ,
1
5
p2,B = 2 . We indicate these probabilities on their possibility tree in
3
Figure 4.4. The probability that a white ball is drawn is then found
115
4.6. FINITE STOCHASTIC PROCESSES




Figure 4.4: ™¦

from the tree measure as the sum of the weights assigned to paths which
lead to a choice of a white ball. This is 1 · 5 + 1 · 1 = 15 .
3 7
™¦
2 23



Example 4.12 Suppose that a drunkard leaves a bar which is on a
corner which he or she knows to be one block from home. He or she is
unable to remember which street leads home, and proceeds to try each
of the streets at random without ever choosing the same street twice
until he or she goes on the one which leads home. What possibilities
are there for the trip home, and what is the probability for each of
these possible trips? We label the streets A, B, C, and Home. The
possibilities together with typical probabilities are given in Figure 4.5.
The probability for any particular trip, or path, is found by taking the
™¦
product of the branch probabilities.


Example 4.13 Assume that we are presented with two slot machines,
A and B. Each machine pays the same ¬xed amount when it pays o¬.
1
Machine A pays o¬ each time with probability 2 , and machine B with
1
probability 4 . We are not told which machine is A. Suppose that we
choose a machine at random and win. What is the probability that
we chose machine A? We ¬rst construct the tree (Figure 4.6) to show
the possibilities and assign branch probabilities to determine a tree
measure. Let p be the statement “Machine A was chosen” and q be
the statement “The machine chosen paid o¬”. Then we are asked for

Pr[p § q]
Pr[p|q] =
Pr[q]
116 CHAPTER 4. PROBABILITY THEORY




Figure 4.5: ™¦




Figure 4.6: ™¦
117
4.6. FINITE STOCHASTIC PROCESSES

The truth set of the statement p § q consists of a single path which has
1
been assigned weight 4 . The truth set of the statement q consists of
two paths, and the sum of the weights of these paths is 1 · 2 + 1 · 1 = 8 .
1 3
2 24
Thus Pr[p|q] = 2 . Thus if we win, it is more likely that we have machine
3
A than B and this suggests that next time we should play the same
machine. If we lose, however, it is more likely that we have machine
B than A, and hence we would switch machines before the next play.
™¦
(See Exercise 9.)

Exercises
1. The fractions of Republicans, Democrats, and Independent voters
in cities A and B are

City A: .30 Republican, .40 Democratic, .30 Independent;

City B: .40 Republican, .50 Democratic, .10 Independent.

A city is chosen at random and two voters are chosen succes-
sively and at random from the voters of this city. Construct a
tree measure and ¬nd the probability that two Democrats are
chosen. Find the probability that the second voter chosen is an
Independent voter.

[Ans. .205; .2.]

2. A coin is thrown. If a head turns up, a die is rolled. If a tail
turns up, the coin is thrown again. Construct a tree measure to
represent the two experiments and ¬nd the probability that the
die is thrown and a six turns up.

3. An athlete wins a certain tournament if he or she can win two
consecutive games out of three played alternately with two oppo-
nents A and B. A is a better player than B. The probability of
2
winning a game when B is the opponent 3 . The probability of
1
winning a game when A is the opponent is only 3 . Construct a
tree measure for the possibilities for three games, assuming that
he or she plays alternately but plays A ¬rst. Do the same assum-
ing that he or she plays B ¬rst. In each case ¬nd the probability
that he or she will win two consecutive games. Is it better to play
two games against the strong player or against the weaker player?
118 CHAPTER 4. PROBABILITY THEORY

10 8
[Ans. ;; better to play strong player twice.]
27 27


4. Construct a tree measure to represent the possibilities for four
throws of an ordinary coin. Assume that the probability of a
head on any toss is 1 regardless of any information about other
2
throws.

5. A student claims to be able to distinguish beer from ale. The
student is given a series of three tests. In each test, the student
is given two cans of beer and one of ale and asked to pick out
the ale. If the student gets two or more correct we will admit
the claim. Draw a tree to represent the possibilities (either right
or wrong) for the student™s answers. Construct the tree measure
which would correspond to guessing and ¬nd the probability that
the claim will be established if the student guesses on every trial.

6. A box contains three defective light bulbs and seven good ones.
Construct a tree to show the possibilities if three consecutive
bulbs are drawn at random from the box (they are not replaced
after being drawn). Assign a tree measure and ¬nd the probabil-
ity that at least one good bulb is drawn out. Find the probability
that all three are good if the ¬rst bulb is good.

119 5
[Ans. ; .]
120 12


7. In Example 4.12, ¬nd the probability that the drunkard reaches
home after trying at most one wrong street.

8. In Example 4.13, ¬nd the probability that machine A was chosen,
given that we lost.

9. In Example 4.13, assume that we make two plays. Find the prob-
ability that we win at least once under the assumption

(a) That we play the same machine twice.
19
[Ans. .]
32

(b) That we play the same machine the second time if and only
if we won the ¬rst time.
20
[Ans. .]
32
119
4.6. FINITE STOCHASTIC PROCESSES

10. A chess player plays three successive games of chess. The player™s
psychological makeup is such that the probability of winning a
1
given game is ( 2 )k+1 , where k is the number of games won so far.
1
(For instance, the probability of winning the ¬rst game is 2 , the
probability of winning the second game if the player has already
won the ¬rst game is 1 , etc.) What is the probability that the
4
player will win at least two of the three games?

11. Before a political convention, a political expert has assigned the
following probabilities. The probability that the President will
be willing to run again is 1 . If the President is willing to run, the
2
President and his or her Vice President are sure to be nominated
and have probability 3 of being elected again. If the President
5
1
does not run, the present Vice President has probability 10 of
being nominated, and any other presidential candidate has prob-
ability 1 of being elected. What is the probability that the present
2
Vice President will be re-elected?

13
[Ans. .]
40


12. There are two urns, A and B. Urn A contains one black and one
red ball. Urn B contains two black and three red balls. A ball is
chosen at random from urn A and put into urn B. A ball is then
drawn at random from urn B.

(a) What is the probability that both balls drawn are of the
same color?
7
[Ans. .]
12

(b) What is the probability that the ¬rst ball drawn was red,
given that the second ball drawn was black?
2
[Ans. .]
5

Supplementary exercises.

13. Assume that in the World Series each team has probability 1 of2
winning each game, independently of the outcomes of any other
game. Assign a tree measure. (See Section ?? for the tree.) Find
the probability that the series ends in four, ¬ve, six, and seven
games, respectively.
120 CHAPTER 4. PROBABILITY THEORY

14. Assume that in the World Series one team is stronger than the
other and has probability .6 for winning each of the games. Assign
a tree measure and ¬nd the following probabilities.

(a) The probability that the stronger team wins in 4, 5, 6, and
7 games, respectively.
(b) The probability that the weaker team wins in 4, 5, 6, and 7
games, respectively.
(c) The probability that the series ends in 4, 5, 6, and 7 games,
respectively.
[Ans. .16; .27; .30; .28.]
(d) The probability that the strong team wins the series.
[Ans. .71.]

15. Redo Exercise 14 for the case of two poorly matched teams, where
the better team has probability .9 of winning a game.

[Ans. (c).66;.26;.07;.01; (d).997.]

16. In the World Series from 1905 through 1965 (excluding series of
more than seven games) there were 11 four-game, 14 ¬ve-game,
13 six-game, and 20 seven-game series. Which of the assumptions
in Exercises 13, 14, 15 comes closest to predicting these results?
Is it a good ¬t?

[Ans. .6; No.]

17. Consider the following assumption concerning World Series: Ninety
per cent of the time the two teams are evenly matched, while 10
per cent of the time they are poorly matched, with the better
team having probability .9 of winning a game. Show that this
assumption comes closer to predicting the actual outcomes than
those considered in Exercise 16.

18. We are given three coins. Coin A is fair while coins B and C are
loaded: B has probability .6 of heads and C has probability .4 of
heads. A game is played by tossing a coin twice starting with coin
B. If a head is obtained, B is tossed again, otherwise the second
coin to be tossed is chosen at random from A and C.
121
4.7. BAYES™S PROBABILITIES

(a) Draw the tree for this game, assigning branch and path
weights.
(b) Let p be the statement “The ¬rst toss results in heads” and
let q be the statement “The second toss results in heads”.
Find Pr[p], Pr[q], Pr[q|p].
[Ans. .6; .54; .6.]
19. A and B play a series of games for which they are evenly matched.
A player wins the series either by winning two games in a row,
or by winning a total of three games. Construct the tree and the
tree measure.
(a) What is the probability that A wins the series?
(b) What is the probability that more than three games need to
be played?
20. In a room there are three chests, each chest contains two drawers,
and each drawer contains one coin. In one chest each drawer
contains a gold coin; in the second chest each drawer contains a
silver coin; and in the last chest one drawer contains a gold coin
and the other contains a silver coin. A chest is picked at random
and then a drawer is picked at random from that chest. When
the drawer is opened, it is found to contain a gold coin. What is
the probability that the other drawer of that same chest will also
contain a gold coin?
2
[Ans. .]
3



4.7 Bayes™s probabilities
The following situation often occurs. Measures have been assigned in
a possibility space U . A complete set of alternatives, p1 , p2 , . . . , pn has
been singled out. Their probabilities are determined by the assigned
measure. (Recall that a complete set of alternatives is a set of state-
ments such that for any possible outcome one and only one of the
statements is true.) We are now given that a statement q is true. We
wish to compute the new probabilities for the alternatives relative to
this information. That is, we wish the conditional probabilities Pr[pj |q]
for each pj . We shall give two di¬erent methods for obtaining these
probabilities.
122 CHAPTER 4. PROBABILITY THEORY

The ¬rst is by a general formula. We illustrate this formula for
the case of four alternatives: p1 , p2 , p3 , p4 . Consider Pr[p2 |q]. From the
de¬nition of conditional probability,
Pr[p2 § q]
Pr[p2 |q] = .
Pr[q]
But since p1 , p2 , p3 , p4 are a complete set of alternatives,
Pr[q] = Pr[p1 § q] + Pr[p2 § q] + P r[p3 § q] + Pr[p4 § q].
Thus
Pr[p2 § q]
Pr[p2 |q] = .
Pr[p1 § q] + Pr[p2 § q] + P r[p3 § q] + Pr[p4 § q]
Since Pr[pj § q] = Pr[pj ]Pr[q|pj ], we have the desired formula
Pr[p2 ]Pr[q|p2 ]
Pr[p2 |q] = .
Pr[p1 ]Pr[q|p1 ] + Pr[p2 ]Pr[q|p2 ] + Pr[p3 ]Pr[q|p3 ] + Pr[p4 ]Pr[q|p4 ]
Similar formulas apply for the other alternatives, and the formula gen-
eralizes in an obvious way to any number of alternatives. In its most
general form it is called Bayes™s theorem.

Example 4.14 Suppose that a freshman must choose among mathe-
matics, physics, chemistry, and botany as his or her science course. On
the basis of the interest he or she expressed, his or her adviser assigns
probabilities of .4, .3, .2 and .1 to the student™s choosing each of the
four courses, respectively. The adviser does not hear which course the
student actually chose, but at the end of the term the adviser hears
that he or she received an A in the course chosen. On the basis of the
di¬culties of these courses the adviser estimates the probability of the
student getting an A in mathematics to be .1, in physics .2, in chemistry
.3, and in botany .9. How can the adviser revise the original estimates
as to the probabilities of the student taking the various courses? Using
Bayes™s theorem we get
(.4)(.1) 4
Pr[The student took math|The student got an A] = =
(.4)(.1) + (.3)(.2) + (.2)(.3) + (.1)(.9) 25
Similar computations assign probabilities of .24, .24, and .36 to the
other three courses. Thus the new information, that the student re-
ceived an A, had little e¬ect on the probability of having taken physics
or chemistry, but it has made mathematics less likely, and botany much
™¦
more likely.
123
4.7. BAYES™S PROBABILITIES

It is important to note that knowing the conditional probabilities
of q relative to the alternatives is not enough. Unless we also know the
probabilities of the alternatives at the start, we cannot apply Bayes™s
theorem. However, in some situations it is reasonable to assume that
the alternatives are equally probable at the start. In this case the
factors Pr[p1 ], . . . , Pr[p4 ] cancel from our basic formula, and we get the
special form of the theorem:
If Pr[p1 ] = Pr[p2 ] = Pr[p3 ] = Pr[p4 ] then

Pr[q|p2 ]
Pr[p2 |q] = .
Pr[q|p1 ] + Pr[q|p2 ] + Pr[q|p3 ] + Pr[q|p4 ]


Example 4.15 In a sociological experiment the subjects are handed
four sealed envelopes, each containing a problem. They are told to open
one envelope and try to solve the problem in ten minutes. From past
experience, the experimenter knows that the probability of their being
able to solve the hardest problem is .1. With the other problems, they
have probabilities of .3, .5, and .8. Assume the group succeeds within
the allotted time. What is the probability that they selected the hardest
problem? Since they have no way of knowing which problem is in which
envelope, they choose at random, and we assign equal probabilities to
the selection of the various problems. Hence the above simple formula
applies. The probability of their having selected the hardest problem
is
.1 1
=.
.1 + .3 + .5 + .8 17
™¦

The second method of computing Bayes™s probabilities is to draw a
tree, and then to redraw the tree in a di¬erent order. This is illustrated
in the following example.

Example 4.16 There are three urns. Each urn contains one white
ball. ln addition, urn I contains one black ball, urn II contains two,
and urn III contains 3. An urn is selected and one ball is drawn. The
probability for selecting the three urns is 6 , 1 , and 3 , respectively. If
1 1
2
we know that a white ball is drawn, how does this alter the probability
that a given urn was selected?
First we construct the ordinary tree and tree measure, in Figure
4.7.
124 CHAPTER 4. PROBABILITY THEORY




Figure 4.7: ™¦




Figure 4.8: ™¦
125
4.7. BAYES™S PROBABILITIES




Figure 4.9: ™¦

Next we redraw the tree, using the ball drawn as stage 1, and the
urn selected as stage 2. (See Figure 4.8.) We have the same paths as
before, but in a di¬erent order. So the path weights are read o¬ from
the previous tree. The probability of drawing a white ball is
1 1 1 1
++ =.
12 6 12 3
This leaves the branch weights of the second stage to be computed. But
this is simply a matter of division. For example, the branch weights
for the branches starting at W must be 4 , 2 , 1 to yield the correct
11
4
path weights. Thus, if a white ball is drawn, the probability of having
1
selected urn I has increased to 4 , the probability of having picked urn
1
III has fallen to 4 , while the probability of having chosen urn II is
™¦
unchanged (see Figure 4.9).
This method is particularly useful when we wish to compute all the
conditional probabilities. We will apply the method next to Example
4.14. The tree and tree measure for this example in the natural order is
shown in Figure 4.10. In that ¬gure the letters M, P, C, and B stand
for mathematics, physics, chemistry, and botany, respectively.
The tree drawn in reverse order is shown in Figure 4.11. Each
path in this tree corresponds to one of the paths in the original tree.
Therefore the path weights for this new tree are the same as the weights
assigned to the corresponding paths in the ¬rst tree. The two branch
weights at the ¬rst level represent the probability that the student
receives an A or that he or she does not receive an A. These probabilities
are also easily obtained from the ¬rst tree. In fact,
Pr[A] = .04 + .06 + .06 + .09 = .25
and
Pr[¬A] = 1 ’ Pr[A] = .75.
126 CHAPTER 4. PROBABILITY THEORY




Figure 4.10: ™¦




Figure 4.11: ™¦
127
4.7. BAYES™S PROBABILITIES




Figure 4.12: ™¦

We have now enough information to obtain the branch weights at
the second level, since the product of the branch weights must be the
path weights. For example, to obtain pA,M we have
.25 · pA,M = .04; pA,M = .16.
But pA,M is also the conditional probability that the student took
math given that he or she got an A. Hence this is one of the new
probabilities for the alternatives in the event that the student received
an A. The other branch probabilities are found in the same way and
represent the probabilities for the other alternatives. By this method
we obtain the new probabilities for all alternatives under the hypothesis
that the student receives an A as well as the hypothesis that the student
does not receive an A. The results are shown in the completed tree in
Figure 4.12.

Exercises
1. Urn I contains 7 red and 3 black balls and urn II contains 6 red
and 4 black balls. An urn is chosen at random and two balls are
drawn from it in succession without replacement. The ¬rst ball
is red and the second black. Show that it is more probable that
urn II was chosen than urn I.
2. A gambler is told that one of three slot machines pays o¬ with
probability 1 , while each of the other two pays o¬ with probability
2
1
.
3
128 CHAPTER 4. PROBABILITY THEORY

(a) If the gambler selects one at random and plays it twice, what
is the probability that he or she will lose the ¬rst time and
win the second?
25
[Ans. .]
108

(b) If the gambler loses the ¬rst time and wins the second, what
is the probability he or she chose the favorable machine?
9
[Ans. .]
25


3. During the month of May the probability of a rainy day is .2. The
Dodgers win on a clear day with probability .7, but on a rainy
day only with probability .4. If we know that they won a certain
game in May, what is the probability that it rained on that day?

1
[Ans. .]
8


4. Construct a diagram to represent the truth sets of various state-
ments occurring in the previous exercise.

5. On a multiple-choice exam there are four possible answers for
each question. Therefore, if a student knows the right answer, he
or she has probability 1 of choosing correctly; if the student is
1
guessing, he or she has probability 4 of choosing correctly. Let us
further assume that a good student will know 90 per cent of the
answers, a poor student only 50 per cent. If a good student has
the right answer, what is the probability that he or she was only
guessing? Answer the same question about a poor student, if the
poor student has the right answer.

11
[Ans. ; .]
37 5


6. Three economic theories are proposed at a given time, which ap-
pear to be equally likely on the basis of existing evidence. The
state of the American economy is observed the following year,
and it turns out that its actual development had probability .6 of
happening according to the ¬rst theory; and probabilities .4 and
.2 according to the others. How does this modify the probabilities
of correctness of the three theories?
129
4.7. BAYES™S PROBABILITIES

7. Let p1 , p2 , p3 , and p4 be a set of equally likely alternatives. Let
Pr[q|p1 ] = a, Pr[q|p2 ] = b, Pr[q|p3 ] = c, Pr[q|p4 ] = d. Show that if
a + b + c + d = 1, then the revised probabilities of the alternatives
relative to q are a, b, c, and d, respectively.

8. In poker, Smith holds a very strong hand and bets a considerable
amount. The probability that Smith™s opponent, Jones, has a
better hand is .05. With a better hand Jones would raise the bet
with probability .9, but with a poorer hand Jones would raise
only with probability .2. Suppose that Jones raises, what is the
new probability that he or she has a winning hand?
9
[Ans. .]
47


9. A rat is allowed to choose one of ¬ve mazes at random. If we know
that the probabilities of his or her getting through the various
mazes in three minutes are .6, .3, .2, .1, .1, and we ¬nd that the
rat escapes in three minutes, how probable is it that he or she
chose the ¬rst maze? The second maze?
63
[Ans. ; .]
13 13


10. Three men, A, B, and C, are in jail, and one of them is to be
hanged the next day. The jailer knows which man will hang, but
must not announce it. Man A says to the jailer, “Tell me the
name of one of the other two who will not hang. If both are to go
free, just toss a coin to decide which to say. Since I already know
that at least one of them will go free, you are not giving away
the secret.” The jailer thinks a moment and then says, “No, this
would not be fair to you. Right now you think the probability
1
that you will hang is 3 , but if I tell you the name of one of the
1
others who is to go free, your probability of hanging increases to 2 .
You would not sleep as well tonight.” Was the jailer™s reasoning
correct?

[Ans. No.]

11. One coin in a collection of 8 million coins has two heads. The
rest are fair coins. A coin chosen at random from the collection
is tossed ten times and comes up heads every time. What is the
probability that it is the two-headed coin?
130 CHAPTER 4. PROBABILITY THEORY

12. Referring to Exercise 11, assume that the coin is tossed n times
and comes up heads every time. How large does n have to be
to make the probability approximately 1 that you have the two-
2
headed coin?

[Ans. 23.]

1
13. A statistician will accept job a with probability 2 , job b with
probability 1 , and job c with probability 6 . In each case he or
1
3
she must decide whether to rent or buy a house. The probabilities
1 2
of buying are 3 if he or she takes job a, 3 if he or she takes job
b, and 1 if he or she takes job c. Given that the statistician buys
a house, what are the probabilities of having taken each job?

[Ans. .3; .4; .3.]

14. Assume that chest X-rays for detecting tuberculosis have the fol-
lowing properties. For people having tuberculosis the test will
detect the disease 90 out of every 100 times. For people not hav-
ing the disease the test will in 1 out of every 100 cases diagnose
the patient incorrectly as having the disease. Assume that the
incidence of tuberculosis is 5 persons per 10,000. A person is
selected at random, given the X-ray test, and the radiologist re-
ports the presence of tuberculosis. What is the probability that
the person in fact has the disease?


4.8 Independent trials with two outcomes
In the preceding section we developed a way to determine a probabil-
ity measure for any sequence of chance experiments where there are
only a ¬nite number of possibilities for each experiment. While this
provides the framework for the general study of stochastic processes,
it is too general to be studied in complete detail. Therefore, in proba-
bility theory we look for simplifying assumptions which will make our
probability measure easier to work with. It is desired also that these as-
sumptions be such as to apply to a variety of experiments which would
occur in practice. In this book we shall limit outselves to the study of
two types of processes. The ¬rst, the independent trials process, will
be considered in the present section. This process was the ¬rst one to
be studied extensively in probability theory. The second, the Markov
131
4.8. INDEPENDENT TRIALS WITH TWO OUTCOMES

chain process, is a process that is ¬nding increasing application, par-
ticularly in the social and biological sciences, and will be considered in
Section 4.13.
A process of independent trials applies to the following situation.
Assume that there is a sequence of chance experiments, each of which
consists of a repetition of a single experiment, carried out in such a way
that the results of any one experiment in no way a¬ect the results in
any other experiment. We label the possible outcome of a single ex-
periment by a1 , . . . , ar . We assume that we are also given probabilities
p1 , . . . , pr for each of these outcomes occurring on any single experi-
ment, the probabilities being independent of previous results. The tree
representing the possibilities for the sequence of experiments will have
the same outcomes from each branch point, and the branch probabili-
ties will be assigned by assigning probability pj to any branch leading
to outcome aj . The tree measure determined in this way is the measure
of an independent trials process. In this section we shall consider the
important case of two outcomes for each experiment. The more general
case is studied in Section 4.11.
In the case of two outcomes we arbitrarily label one outcome “suc-
cess” and the other “failure”. For example, in repeated throws of a
coin we might call heads success, and tails failure. We assume there is
given a probability p for success and a probability q = 1 ’ p for failure.
The tree measure for a sequence of three such experiments is shown in
Figure 4.13. The weights assigned to each path are indicated at the end
of the path. The question which we now ask is the following. Given an
independent trials process with two outcomes, what is the probability
of exactly x successes in n experiments? We denote this probability by
f (n, x; p) to indicate that it depends upon n, x, and p.
Assume that we had a tree for this general situation, similar to the
tree in Figure 4.13 for three experiments, with the branch points labeled
S for success and F for failure. Then the truth set of the statement
“Exactly x successes occur” consists of all paths which go through x
branch points labeled S and n ’ x labeled F . To ¬nd the probability
of this statement we must add the weights for all such paths. We are
helped ¬rst by the fact that our tree measure assigns the same weight
to any such path, namely px q n’x . The reason for this is that every
branch leading to an S is assigned probability p, and every branch
leading to F is assigned probability q, and in the product there will
be x p™s and (n ’ x) q™s. To ¬nd the desired probability we need only
¬nd the number of paths in the truth set of the statement “Exactly x
132 CHAPTER 4. PROBABILITY THEORY




Figure 4.13: ™¦

successes occur”. To each such path we make correspond an ordered
partition of the integers from 1 to n which has two cells, x elements in
the ¬rst and n ’ x in the second. We do this by putting the numbers
of the experiments on which success occurred in the ¬rst cell and those
for which failure occurred in the second cell. Since there are n such
x
partitions there are also this number of paths in the truth set of the
statement considered. Thus we have proved:
In an independent trials process with two outcomes the
probability of exactly x successes in n experiments is given by

n x n’x
f (n, x; p) = pq .
x


Example 4.17 Consider n throws of an ordinary coin. We label heads
1
“success” and tails “failure”, and we assume that the probability is 2
for heads on any one throw independently of the outcome of any other
throw. Then the probability that exactly x heads will turn up is

1 n 1n
f (n, x; ) = ( ).
2 x2

For example, in l00 throws the probability that exactly 50 heads will
133
4.8. INDEPENDENT TRIALS WITH TWO OUTCOMES

turn up is
1 100 1 100
f (100, 50; ) = () ,
2 50 2
which is approximately .08. Thus we see that it is quite unlikely that
exactly one-half of the tosses will result in heads. On the other hand,
suppose that we ask for the probability that nearly one-half of the tosses
will be heads. To be more precise, let us ask for the probability that
the number of heads which occur does not deviate by more than l0 from
1
50. To ¬nd this we must add f (100, x; 2 ) for x = 40, 41, . . . , 60. If this
is done, we obtain a probability of approximately .96. Thus, while it
is unlikely that exactly 50 heads will occur, it is very likely that the
number of heads which occur will not deviate from 50 by more than l0.
™¦

Example 4.18 Assume that we have a machine which, on the basis
of data given, is to predict the outcome of an election as either a Re-
publican victory or a Democratic victory. If two identical machines are
given the same data, they should predict the same result. We assume,
however, that any such machine has a certain probability q of reversing
the prediction that it would ordinarily make, because of a mechanical
or electrical failure. To improve the accuracy of our prediction we give
the same data to r identical machines, and choose the answer which
the majority of the machines give. To avoid ties we assume that r is
odd. Let us see how this decreases the probability of an error due to a
faulty machine.
Consider r experiments, where the jth experiment results in success
if the jth machine produces the prediction which it would make when
operating without any failure of parts. The probability of success is
then p = 1 ’ q. The majority decision will agree with that of a per-
fectly operating machine if we have more than r/2 successes. Suppose,
for example, that we have ¬ve machines, each of which has a proba-
bility of .1 of reversing the prediction because of a parts failure. Then
the probability for success is .9, and the probability that the majority
decision will be the desired one is
f (5, 3; 0.9) + f (5, 4; 0.9) + f (5, 5; 0.9)
which is found to be approximately .991 (see Exercise 3).
Thus the above procedure decreases the probability of error due to
machine failure from .1 in the case of one machine to .009 for the case
™¦
of ¬ve machines.
134 CHAPTER 4. PROBABILITY THEORY

Exercises
1. Compute for n = 4, n = 8, n = 12, and n = 16 the probability of
1
obtaining exactly 2 heads when an ordinary coin is thrown.

[Ans. .375; .273; .226; .196.]

2. Compute for n = 4, n = 8, n = 12, and n = 16 the probability
that the fraction of heads deviates from 2 by less than 1 .
1
5


[Ans. .375; .711, .854; .923.]

3. Verify that the probability .991 given in Example 4.18 is correct.

4. Assume that Peter and Paul match pennies four times. (In match-
ing pennies, Peter wins a penny with probability 1 , and Paul wins
2
1
a penny with probability 2 .) What is the probability that Peter
wins more than Paul? Answer the same for ¬ve throws. For the
case of 12,917 throws.
511
[Ans. ; ; .]
16 2 2


5. If an ordinary die is thrown four times, what is the probability
that exactly two sixes will occur?

6. In a ten-question true-false exam, what is the probability of get-
ting 70 per cent or better by guessing?
11
[Ans. .]
64


7. Assume that, every time a batter comes to bat, he or she has
probability .3 for getting a hit. Assuming that hits form an in-
dependent trials process and that the batter comes to bat four
times, what fraction of the games would he or she expect to get
at least two hits? At least three hits? Four hits?

[Ans. .348; .084; .008.]

8. A coin is to be thrown eight times. What is the most probable
number of heads that will occur? What is the number having
the highest probability, given that the ¬rst four throws resulted
in heads?
135
4.8. INDEPENDENT TRIALS WITH TWO OUTCOMES

9. A small factory has ten workers. The workers eat their lunch at
one of two diners, and they are just as likely to eat in one as in the
other. If the proprietors want to be more than .95 sure of having
enough seats, how many seats must each of the diners have?

[Ans. Eight seats.]

10. Suppose that ¬ve people are chosen at random and asked if they
favor a certain proposal. If only 30 per cent of the people favor
the proposal, what is the probability that a majority of the ¬ve
people chosen will favor the proposal?

11. In Example 4.18, if the probability for a machine reversing its
answer due to a parts failure is .2, how many machines would
have to be used to make the probability greater than .89 that the
answer obtained would be that which a machine with no failure
would give?

[Ans. Three machines.]

12. Assume that it is estimated that a torpedo will hit a ship with
1
probability 3 . How many torpedos must be ¬red if it is desired
that the probability for at least one hit should be greater than
.9?

13. A student estimates that, if he or she takes four courses, he or
she has probability .8 of passing each course. If he or she takes
¬ve courses, he or she has probability .7 of passing each course,
and if he or she takes six courses he or she has probability .5 for
passing each course. The student™s only goal is to pass at least
four courses. How many courses should he or she take for the
best chance of achieving this goal?

[Ans. 5.]

Supplementary exercises.

14. In a certain board game players move around the board, and each
turn consists of a player™s rolling a pair of dice. If a player is on
the square Park Bench, he or she must roll a seven or doubles
before being allowed to move out.
136 CHAPTER 4. PROBABILITY THEORY

(a) What is the probability that a player stuck on Park Bench
will be allowed to move out on the next turn?
1
[Ans. .]
3

(b) How many times must a player stuck on Park Bench roll
before the chances of getting out exceed 3 .
4

[Ans. 4.]

15. A restaurant orders ¬ve pieces of apple pie and ¬ve pieces of
cherry pie. Assume that the restaurant has ten customers, and
3
the probability that a customer will ask for apple pie is 4 and for
1
cherry pie is 4 .

(a) What is the probability that the ten customers will all be
able to have their ¬rst choice?
(b) What number of each kind of pie should the restaurant order
if it wishes to order ten pieces of pie and wants to maximize
the probability that the ten customers will all have their ¬rst
choice?

16. Show that it is more probable to get at least one ace with 4 dice
than at least one double ace in 24 throws of two dice.

17. A thick coin, when tossed, will land “heads” with a probability of
5 5
, “tails” with a probability of 12 , and will land on edge with a
12
1
probability of 6 . If it is tossed six times, what is the probability
that it lands on edge exactly two times?

[Ans. .2009.]

18. Without actually computing the probabilities, ¬nd the value of x
for which f (20, x; .3) is largest.
2
19. A certain team has probability of winning whenever it plays.
3

(a) What is the probability the team will win exactly four out
of ¬ve games?
80
[Ans. .]
243

(b) What is the probability the team will win at most four out
of ¬ve games?
137
4.9. A PROBLEM OF DECISION

211
[Ans. .]
243

(c) What is the probability the team will win exactly four games
out of ¬ve if it has already won the ¬rst two games of the
¬ve?
4
[Ans. .]
9




4.9 A problem of decision
In the preceding sections we have dealt with the problem of calculating
the probability of certain statements based on the assumption of a given
probability measure. In a statistics problem, one is often called upon
to make a decision in a case where the decision would be relatively
easy to make if we could assign probabilities to certain statements, but
we do not know how to assign these probabilities. For example, if a
vaccine for a certain disease is proposed, we may be called upon to
decide whether or not the vaccine should be used. We may decide that
we could make the decision if we could compare the probability that
a person vaccinated will get the disease with the probability that a
person not vaccinated will get the disease. Statistical theory develops
methods to obtain from experiments some information which will aid
in estimating these probabilities, or will otherwise help in making the
required decision. We shall illustrate a typical procedure.
Smith claims to have the ability to distinguish ale from beer and
has bet Jones a dollar to that e¬ect. Now Smith does not mean that
he or she can distinguish beer from ale every single time, but rather a
1
proportion of the time which is signi¬cantly greater than 2 .
Assume that it is possible to assign a number p which represents
the probability that Smith can pick out the ale from a pair of glasses,
one containing ale and one beer. We identify p = 1 with having no
2
1 1
ability, p > 2 with having some ability, and p < 2 with being able to
distinguish, but having the wrong idea which is the ale. If we knew
the value of p, we would award the dollar to Jones if p were ¤ 1 , and
2
1
to Smith if p were > 2 . As it stands, we have no knowledge of p and
thus cannot make a decision. We perform an experiment and make a
decision as follows.
Smith is given a pair of glasses, one containing ale and the other
beer, and is asked to identify which is the ale. This procedure is re-
peated ten times, and the number of correct identi¬cations is noted. If
138 CHAPTER 4. PROBABILITY THEORY

the number correct is at least eight, we award the dollar to Smith, and
if it is less than eight, we award the dollar to Jones.
We now have a de¬nite procedure and shall examine this procedure
both from Jones™s and Smith™s points of view. We can make two kinds of
errors. We may award the dollar to Smith when in fact the appropriate
1
value of p is ¤ 2 , or we may award the dollar to Jones when the
1
appropriate value for p is > 2 There is no way that these errors can
be completely avoided. We hope that our procedure is such that each
bettor will be convinced that, if he or she is right, he or she will very
likely win the bet.
1
Jones believes that the true value of p is 2 . We shall calculate the
probability of Jones winning the bet if this is indeed true. We assume
that the individual tests are independent of each other and all have the
1
same probability 2 for success. (This assumption will be unreasonable if
the glasses are too large.) We have then an independent trials process
with p = 1 to describe the entire experiment. The probability that
2
Jones will win the bet is the probability that Smith gets fewer than
eight correct. From the table in Figure 4.14 we compute that this
probability is approximately .945. Thus Jones sees that, if he or she
is right, it is very likely that he or she will win the bet.
Smith, on the other hand, believes that p is signi¬cantly greater
than 1 . If Smith believes that p is as high as .9, we see from Figure
2
4.14 that the probability of Smith™s getting eight or more correct is
.930. Then both parties will be satis¬ed by the bet.
Suppose, however, that Smith thinks the value of p is only about
.75. Then the probability that Smith will get eight or more correct and
thus win the bet is .526. There is then only an approximately even
chance that the experiment will discover Smith™s abilities, and Smith
probably will not be satis¬ed with this. If Smith really thinks his or her
ability is represented by a p value of about 3 , we would have to devise
4
a di¬erent method of awarding the dollar. We might, for example,
propose that Smith win the bet if he or she gets seven or more correct.
3
Then, if Smith has probability 4 of being correct on a single trial, the
1
probability that Smith will win the bet is approximately .776. If p = 2
the probability that Jones will win the bet is about .828 under this new
arrangement. Jones™s chances of winning are thus decreased, but Smith
may be able to convince him or her that it is a fairer arrangement than
the ¬rst procedure.
In the above example, it was possible to make two kinds of er-
rors. The probability of making these errors depended on the way we
139
4.9. A PROBLEM OF DECISION




Figure 4.14: ™¦
140 CHAPTER 4. PROBABILITY THEORY

designed the experiment and the method we used for the required de-
cision. In some cases we are not too worried about the errors and can
make a relatively simple experiment. In other cases, errors are very im-
portant, and the experiment must be designed with that fact in mind.
For example, the possibility of error is certainly important in the case
that a vaccine for a given disease is proposed, and the statistician is
asked to help in deciding whether or not it should be used. In this case
it might be assumed that there is a certain probability p that a person
will get the disease if not vaccinated, and a probability r that the per-
son will get it if he or she is vaccinated. If we have some knowledge
of the approximate value of p, we are then led to construct an experi-
ment to decide whether r is greater than p, equal to p, or less than p.
The ¬rst case would be interpreted to mean that the vaccine actually
tends to produce the disease, the second that it has no e¬ect, and the
third that it prevents the disease; so that we can make three kinds of
errors. We could recommend acceptance when it is actually harmful,
we could recommend acceptance when it has no e¬ect, or ¬nally we
could reject it when it actually is e¬ective. The ¬rst and third might
result in the loss of lives, the second in the loss of time and money of
those administering the test. Here it would certainly be important that
the probability of the ¬rst and third kinds of errors be made small. To
see how it is possible to make the probability of both errors small, we
return to the case of Smith and Jones.

Suppose that, instead of demanding that Smith make at least eight
correct identi¬cations out of ten trials, we insist that Smith make at
least 60 correct identi¬cations out of 100 trials. (The glasses must now
be very small.) Then, if p = 1 , the probability that Jones wins the
2
bet is about .98; so that we are extremely unlikely to give the dollar to
1
Smith when in fact it should go to Jones. (If p < 2 it is even more likely
that Jones will win.) If p > 1 we can also calculate the probability that
2
Smith will win the bet. These probabilities are shown in the graph in
Figure 4.15. The dashed curve gives for comparison the corresponding
probabilities for the test requiring eight out of ten correct. Note that
3
with 100 trials, if p is 4 , the probability that Smith wins the bet is
1
nearly 1, while in the case of eight out of ten, it was only about 2 .
Thus in the case of 100 trials, it would be easy to convince both Smith
and Jones that whichever one is correct is very likely to win the bet.

Thus we see that the probability of both types of errors can be made
small at the expense of having a large number of experiments.
141
4.9. A PROBLEM OF DECISION




Figure 4.15: ™¦

Exercises
1. Assume that in the beer and ale experiment Jones agrees to pay
Smith if Smith gets at least nine out of ten correct.

(a) What is the probability of Jones paying Smith even though
Smith cannot distinguish beer and ale, and guesses?

[Ans. .011.]

(b) Suppose that Smith can distinguish with probability .9. What
is the probability of not collecting from Jones?

[Ans. .264.]

2. Suppose that in the beer and ale experiment Jones wishes the
probability to be less than .1 that Smith will be paid if, in fact,
Smith guesses. How many of ten trials must Jones insist that
Smith get correct to achieve this?

3. In the analysis of the beer and ale experiment, we assume that
the various trials were independent. Discuss several ways that
error can enter, because of the nonindependence of the trials, and
how this error can be eliminated. (For example, the glasses in
which the beer and ale were served might be distinguishable.)

4. Consider the following two procedures for testing Smith™s ability
to distinguish beer from ale.
142 CHAPTER 4. PROBABILITY THEORY

(a) Four glasses are given at each trial, three containing beer and
one ale, and Smith is asked to pick out the one containing
ale. This procedure is repeated ten times. Smith must guess
correctly seven or more times. Find the probability that
Smith wins by guessing.

[Ans. .003.]

(b) Ten glasses are given to Smith, and Smith is told that ¬ve
contain beer and ¬ve ale, and asked to name the ¬ve that
contain ale. Smith must choose all ¬ve correctly. Find the
probability that Smith wins by guessing.

[Ans. .004.]

(c) Is there any reason to prefer one of these two tests over the
other?

5. A testing service claims to have a method for predicting the order
in which a group of freshmen will ¬nish in their scholastic record
at the end of college. The college agrees to try the method on
a group of ¬ve students, and says that it will adopt the method
if, for these ¬ve students, the prediction is either exactly correct
or can be changed into the correct order by interchanging one
pair of adjacent students in the predicted order. If the method is
equivalent to simply guessing, what is the probability that it will
be accepted?

1
[Ans. .]
24


1
6. The standard treatment for a certain disease leads to a cure in 4
of the cases. It is claimed that a new treatment will result in a
cure in 3 of the cases. The new treatment is to be tested on ten
4
people having the disease. If seven or more are cured, the new
treatment will be adopted. If three or fewer people are cured, the
treatment will not be considered further. If the number cured
is four, ¬ve, or six, the results will be called inconclusive, and
a further study will be made. Find the probabilities for each of
these three alternatives under the assumption ¬rst, that the new
treatment has the same e¬ectiveness as the old, and second, under
the assumption that the claim made for the treatmnent is correct.
143
4.10. THE LAW OF LARGE NUMBERS

7. Three students debate the intelligence of Springer spaniels. One
claims that Springers are mostly (say 90 per cent of them) in-
telligent. A second claims that very few (say 10 per cent) are
intelligent, while a third one claims that a Springer is just as
likely to be intelligent as not. They administer an intelligence
test to ten Springers, classifying them as intelligent or not. They
agree that the ¬rst student wins the bet if eight or more are in-
telligent, the second if two or fewer, the third in all other cases.
For each student, calculate the probability that he or she wins
the bet, if he or she is right.

[Ans. .930, .930, .890.]

8. Ten students take a test with ten problems. Each student on each
1
question has probability 2 of being right, if he or she does not
cheat. The instructor determines the number of students who get
each problem correct. If instructor ¬nds on four or more problems
there are fewer than three or more than seven correct, he or she
considers this convincing evidence of communication between the
students. Give a justi¬cation for the procedure. [Hint: The table
in Figure 4.14 must be used twice, once for the probability of fewer
than three or more than seven correct answers on a given problem,
and the second time to ¬nd the probability of this happening on
four or more problems.]


4.10 The law of large numbers
In this section we shall study some further properties of the indepen-
dent trials process with two outcomes. In Section 4.8 we saw that the
probability for x successes in n trials is given by

n x n’x
f (n, x; p) = pq .
x

In Figure 4.16 we show these probabilities graphically for n = 8 and
p = 3 . In Figure 4.17 we have done similarly for the case of n = 7 and
4
3
p = 4.
We see in the ¬rst case that the values increase up to a maximum
value at x = 6 and then decrease. In the second case the values increase
up to a maximum value at x = 5, have the same value for x = 6, and
144 CHAPTER 4. PROBABILITY THEORY




Figure 4.16: ™¦




Figure 4.17: ™¦
145
4.10. THE LAW OF LARGE NUMBERS

then decrease. These two cases are typical of what can happen in
general.
Consider the ratio of the probability of x + 1 successes in n trials to
the probability of x successes in n trials, which is
n
px+1 q n’x’1 n’x p
x+1
·.
=
n x+1 q
px q n’x
x


This ratio will be greater than one as long as (n ’ x)p > (x + 1)q
or as long as x < np ’ q. If np ’ q is not an integer, the values
n x n’x
pq increase up to a maximum value, which occurs at the ¬rst
x
integer greater than np ’ q, and then decrease. In case np ’ q is an
integer, the values n px q n’x increase up to x = np ’ q, are the same
x
for x = np ’ q and x = np ’ q + 1, and then decrease.
Thus we see that, in general, values near np will occur with the
largest probability. It is not true that one particular value near np is
highly likely to occur, but only that it is relatively more likely than
a value further from np. For example, in 100 throws of a coin, np =
1
100 · 2 = 50. The probability of exactly 50 heads is approximately .08.
The probability of exactly 30 is approximately .00002.
More information is obtained by studying the probability of a given
deviation of the proportion of successes x/n from the number p; that
is, by studying for > 0,
x
’ p| < ].
Pr[|
n
For any ¬xed n, p, and , the latter probability can be found by
adding all the values of f (n, x; p) for values of x for which the inequality
p ’ < x/n < p + is true. In Figure 4.18 we have given these
probabilities for the case p = .3 with various values for and n. In the
¬rst column we have the case = .1. We observe that as n increases,
the probability that the fraction of successes deviates from .3 by less
than .1 tends to the value 1. In fact to four decimal places the answer
is 1 after n = 400. In column two we have the same probabilities for
the smaller value of = .05. Again the probabilities are tending to 1
but not so fast. In the third column we have given these probabilities
for the case = .02. We see now that even after 1000 trials there is still
a reasonable chance that the fraction x/n is not within .02 of the value
of p = .3. It is natural to ask if we can expect these probabilities also
to tend to 1 if we increase n su¬ciently. The answer is yes and this is
146 CHAPTER 4. PROBABILITY THEORY




Figure 4.18: ™¦
147
4.10. THE LAW OF LARGE NUMBERS




Figure 4.19: ™¦

<<

. 4
( 6)



>>