Bayes’ Theorem
Conditional probabilities
Overview
In this article you are going to learn about how to calculate the probability of one event given the occurrence of another event. Let’s get started!
Introduction
You will notice that many marathon winners are from Kenya. Can we quantify this? In other words, what is the probability that a marathon winner is from Kenya? We can use Bayes’ theorem to compute the likelihood. The calculation will involve conditional probabilities. So let’s discuss that concept first.
Overview
- First, we will discuss conditional probability.
- Next, we will illustrate two ways of calculating the probability that a person is both a marathon winner and from Kenya.
- Finally, we will use simple algebra to compute the quantity we want, i.e. the probability that a person is from Kenya, given that he is a marathon winner.
Conditional Probabilities
If we want to know the probability of some event A given some other event B, we can consider the following relationship:
The probability of A and B both being true can be expressed as
P(A and B) = P(B)*P(A given B)
Instead of given you will usually see the symbol “|” instead, i.e.
P(A and B) = P(B)*P(A|B)
In other words, the just multiply the probability of B times the probability of A given B. This is known as conditional probability.
Alternatively, we could equivalently compute it as follows:
P(A and B) = P(A)*P(B|A)
For example:
Suppose each colored spot represents a person. The probability that one of these 35 people is both a marathon winner (M) and comes from Kenya (K) can be calculated as:
P(M and K) = P(K) * P(M|K)
since that is the probability that someone is from Kenya times the probability that someone won a marathon given that they are from Kenya.
If we plug in the actual numbers, we would get:
P(K) * P(M|K) = ((6+2)/35) * (2/(2+6)) = 8/35 * 2/8 = 2/35
As we discussed above, we could equivalently express the same overall figure in this alternative arrangement:
P(M and K) = P(M) * P(K|M)
since that is the probability that someone won a marathon times the probability that someone is from Kenya given that he won a marathon.
Once again, if we plug in the actual numbers, we would get:
P(M) * P(K|M) = ((3+2)/35) * (2/(2+3)) = 5/35 * 2/5 = 2/35
Do you see how simple this is?
Now, comes the “trick” which comes in handy:
Notice that
P(M) * P(K|M) = P(M and K) = P(K) * P(M|K)
So, we can just divide both sides by P(M), let’s say, to obtain the following:
P(K|M) = P(K) * P(M|K)/P(M)
That is Bayes theorem! You now can calculate the conditional probability on the left hand side even if all you know (or can estimate) the information on the right hand side.
I hope you can see that this result generalizes to almost any application. For instance, what are the odds that
This comes in handy when you receive new data (evidence) and on that basis you want to adjust your predictions.
The purpose of this article was just to show with an example how the formula is derived. If you have any feedback on how this or future articles can be improved, I would appreciate you adding to the comments section. Thank you!
