Chris                               Wiggins                                             associate        professor                      applied             mathematics                                at Columbia University                  explanation.

A patient goes to see a doctor. The doctor performs a test with 99 percent reliability--that is, 99 percent of people who are sick test positive and 99 percent of the healthy people test negative. The doctor knows that only 1 percent of the people in the country are sick. Now the question is: if the patient tests positive, what are the chances the patient is sick?

More generally, Bayes's theorem is used in any calculation in which a "marginal" probability is calculated (e.g., p(+), the probability of testing positive in the example) from likelihoods (e.g., p(+|s) and p(+|h), the probability of testing positive given being sick or healthy) and prior probabilities (p(s) and p(h)): p(+)=p(+|s)p(s)+p(+|h)p(h). Such a calculation is so general that almost every application of probability or statistics must invoke Bayes's theorem at some point. In that sense Bayes's theorem is at the heart of everything from genetics to Google, from health insurance to hedge funds. It is a central relationship for thinking concretely about uncertainty, and--given quantitative data, which is sadly not always a given--for using mathematics as a tool for thinking clearly about the world.

The intuitive answer is 99 percent, but the correct answer is 50 percent, and Bayes's theorem gives us the relationship between what we know and what we want to know in this problem. What we are given--what we know--is p(+|s), which a mathematician would read as "the probability of testing positive given that you are sick"; what we want to know is p(s|+), or "the probability of being sick given that you tested positive." The theorem itself reads p(s|+)=p(+|s)p(s)/p(+), although what Reverend Bayes, who lived from 1702 to 1761, actually said was something simpler. Bayes stated the defining relationship expressing the probability you test positive AND are sick as the product of the likelihood that you test positive GIVEN that you are sick and the "prior" probability that you are sick (that is, the probability the patient is sick, prior to specifying a particular patient and administering the test).

Rather than relying on Bayes's math to help us with this, let us consider another illustration. Imagine that the above story takes place in a small country, with 10,000 people. From the prior p(s)=0.01, we know that 1 percent, or 100 people, are sick, and 9,900 are healthy. If we administer the test to everyone, the most probable result is that 99 of the 100 sick people test positive. Since the test has a 1 percent error rate, however, it is also probable that 99 of the healthy people test positive. Now if the doctor sends everyone who tests positive to the national hospital, there will be an equal number of healthy and sick patients. If you meet one, even though you are armed with the information that the patient tested positive, there is only a 50 percent chance this person is sick.

Now imagine the doctor moves to another country, performing the same test, with the same likelihood (p(+|s)) and, for that matter, the same success rate for healthy people, which we might call p(-|h), "the probability of scoring negative given that one is healthy." In this country, however, we suppose that only one in every 200 people is sick. If a new patient tests positive, it is actually more probable that the patient is healthy than sick. The doctor needs to update the prior. (The correct probability is left as a homework assignment for the reader.)

The importance of accurate data in quantitative modeling is central to the subject raised in the question: using Bayes's theorem to calculate the probability of the existence of God. Scientific discussion of religion is a popular topic at present, with three new books arguing against theism and one, University of Oxford professor Richard Dawkins's book The God Delusion, arguing specifically against the use of Bayes's theorem for assigning a probability to God's existence. (A Google news search for "Dawkins" turns up 1,890 news items at the time of this writing.) Arguments employing Bayes's theorem calculate the probability of God given our experiences in the world (the existence of evil, religious experiences, etc.) and assign numbers to the likelihood of these facts given existence or nonexistence of God, as well as to the prior belief of God's existence--the probability we would assign to the existence of God if we had no data from our experiences.
Dawkins's argument is not with the veracity of Bayes's theorem itself, whose proof is direct and unassailable, but rather with the lack of data to put into this formula by those employing it to argue for the existence of God. The equation is perfectly accurate, but the numbers inserted are, to quote Dawkins, "not measured quantities but & personal judgments, turned into numbers for the sake of the exercise."

Note that although this is receiving much attention now, quantifying one's judgments for use in Bayesian calculations of the existence of God is not new. Richard Swinburne, for example, a philosopher of science turned philosopher of religion (and Dawkins's colleague at Oxford), estimated the probability of God's existence to be more than 50 percent in 1979 and, in 2003, calculated the probability of the resurrection [presumably of both Jesus and his followers] to be "something like 97 percent."

 

(Swinburne assigns God a prior probability of 50 percent since there are only two choices: God exists or does not. Dawkins, on the other hand, believes "there's an infinite number of things that some people at one time or another have believed in ... God, Flying Spaghetti Monster, fairies, or whatever," which would correspondingly lower each outcome's prior probability.)

 

In reviewing the history of Bayes's theorem and theology, one might wonder what Reverend Bayes had to say about this, and whether Bayes introduced his theorem as part of a similar argument for the existence of God. But the good reverend said nothing on the subject, and his theorem was introduced posthumously as part of his solution to predicting the probability of an event given specific conditions.

 

In fact, while there is plenty of material on lotteries and hyperbolic logarithms, there is no mention of God in Bayes's "Essay towards Solving a Problem in the Doctrine of Chances," presented after his death to the Royal Society of London in 1763
(and available online at www.stat.ucla.edu/history/essay.pdf ).

One primary scientific value of Bayes's theorem today is in comparing models to data and selecting the best model given those data. For example, imagine two mathematical models, A and B, from which one can calculate the likelihood of any data given the model (p(D|A) and p(D|B)). For example, model A might be one in which spacetime is 11-dimensional, and model B one in which spacetime is 26-dimensional. 

Once I have performed a quantitative measurement and obtained some data D, one needs to calculate the relative probability of the two models: p(A|D)/p(B|D). 

Note that just as in relating p(+|s) to p(s|+), I can equate this relative probability to p(D|A)p(A)/p(D|B)p(B). To some, this relationship is the source of deep joy; to others, maddening frustration.

The source of this frustration is the unknown priors, p(A) and p(B). What does it mean to have prior belief about the probability of a mathematical model? Answering this question opens up a bitter internecine can of worms between "the Bayesians" and "the frequentists," a mathematical gang war which is better not entered into here. To oversimplify, "Bayesian probability" is an interpretation of probability as the degree of belief in a hypothesis; "frequentist probability is an interpretation of probability as the frequency of a particular outcome in a large number of experimental trials. In the case of our original doctor, estimating the prior can mean the difference between more-than-likely and less-than-likely prognosis. In the case of model selection, particularly when two disputants have strong prior beliefs that are diametrically opposed (belief versus nonbelief), Bayes's theorem can lead to more conflict than clarity.