Friday, January 25, 2013

Loftus's Outsider Test for Faith highlights this frequently asked question about probabilities

I've written a little bit about John Loftus's Outsider Test for Faith before a few times (indeed, these are the most popular posts on my blog by a considerable margin), using one interpretation of Bayesian reasoning to illustrate how it works. (If you're not reading his writing by now, let me remind you that you should read John Loftus's writing.) I've been reading up about Bayesian reasoning more since those posts and have some things to say about it--I'm not sure anymore that that is the best way to use Bayes's theorem--but that's not for here. Soon, I promise....

Anyway, I stumbled across something on Twitter earlier, where another nonbeliever was attempting to strike home a point that rests at the center of Loftus's construction of the Outsider Test of Faith. His comment amounts to the frequently asked (or commented upon) question: "Given the wide variety of religions in existence, which is more probable, that your particular religion is the one right one, or that none of them are?" This, for those who have read Loftus, is the question that sits at the center of the first step of the Outsider Test for Faith, and it has been remarked upon by a large number of very considered folks.

As a mathematician, I'm tempted to play with it because of the word problem that it is. Of course, as a mathematician, I am going to vastly oversimplify the problem, but to justify that decision, I will also qualify that I'm doing it and carefully construct why it's not such an unreasonable thing to do in the right context.

My oversimplification of the question simply will suppose the outsider's position in the Outsider Test for Faith: "I don't believe any of the world religions at present. Therefore, without more information, I will evaluate them as separate explanatory hypotheses about the world, each independent and approximately equally likely to be true (before considering any evidence)." This is the starting point in the OTF investigation, using one of the most damaging (and probably violence-inspiring) facts about the various religions: there are a lot of them that appear to make roughly analogous claims about the world.

To get a bit more into this, I will presume that there is some number N of world religions and consider an uninformed background in the attempt to answer the question: Which is more likely, that your particular religion is the one right one, or that none of them are?

Given N equally likely to be true, independent hypotheses, the probability that any particular one of them is true while all the rest are false is a bit lower than we might normally expect. It seems like the probability would simply be 1/N, but because of the requirement that all of the other religions are false as well, it's a bit worse than that naive approximation. Indeed, the formula that provides this probability is:


The second of those two factors, meaning the complicated one, is the probability that all the other hypotheses are simultaneously wrong, and the naive approximation makes the first factor (the probability that without any additional information, one of the N is the true one--which actually presumes that there is a true one, a problem this more complex expression does not have).

On the other hand, the likelihood that none of them is true is given by the expression:


With these tools, we can actually address the question under these oversimplifying assumptions. In general, what we immediately can see from a straightforward application of algebra is the answer to the question at hand: the probability that none of the hypotheses are true is (N-1) times the probability that some specific hypothesis, but no others, is true.

For different values of N, this is fairly interesting to see in numbers:
  • For N=2 (Christianity versus Islam, say), we get that the probability that one particularly is true while the other is false is 1/4 (25%), which is the same as the probability that neither is true.
  • For N=3 (throw in Judaism, e.g.), we get that the probability that one particularly is true while the others are false is 4/27 (14.8%), while the probability that all three are false is twice that, 8/27 (29.6%).
  • For N=20 (the number of major religions on earth), we get that the probability that one particularly is true while the others are false is 1.89%, while the probability that all twenty are false is 35.8% (nineteen times as likely).
  • For N=40000 (the approximate number of Christian denominations at present*--something an outsider has to choose in coming to Christianity), we get that the probability that one particularly is true while the others are false is a mere 0.000919% (about one in 109,000), while the probability that all of them are false is 36.8%.
  • If we consider all of the religions that ever were, it gets worse. If we consider all the religions that ever will be, worse still. If we consider all of the religions that possibly could be, even worse still, at least in terms of what this is measuring (see below).
The trend is very important and clear. While we see (if we look closer) that the probability that none of them is true approaches a particular value near 37% (the actual value as N gets arbitrarily large gets arbitrarily close to 1-(1/e), for those that know what the transcendental constant e is), the probability that some particular one is true while the rest are false plummets like roughly 1/(3N) (really, it's the same factor: nearer 0.37(1/N)). Admittedly, this is only a good approximation when N is relatively large, but a fair amount of argumentation (given religious diversity, past and present) is required to justify that N isn't a lot bigger than most of us would toss out at first guess.

This reads like a lot of gobbledegook, to be real about it, but what it means is that as the number of competing religions goes up, from the outside, the likelihood that any particular one of them is the One True Faith plummets like a stone while the likelihood that none of them are true rocks it steady at a little better than a third.

* Given the enormous disparity between the Christianities of today and the Christianities of yesteryear, it being hard to even imagine Inquisition-period Christians not burning most modern Christians alive for their heresies--this number of present denominations is rather a low-ball estimate of the proper number of Christianities that should be considered for this kind of analysis to be fair. Furthermore, if we throw in the total number of divergent sects of all of the varying religions, even if we limit our attention to the present, this number is easily justifiably significantly larger. Adding in all present and historical religions (say within the last few thousand years) makes it very difficult to make any argument that the real number that should be used here is any less than 100,000 total faiths, without even taking into account future or never-to-be-thought-of faiths.

In terms of Loftus's OTF, what this tells us is that we're perfectly justified in assigning an even lower prior probability, from the outsider's perspective, to any given religious faith than we might naively expect. Just in terms of what religions there are, we can see that an outsider is perfectly justified assigning a prior that is 1/3 of what he might ordinarily choose, supposing at least that such an outsider is open to the reality that there are a lot of religions out there.

(Just for fun, in essence, if we consider all of the billions and billions of religions that ever were, are, will be, or could possibly be even if the won't necessarily be, we can be justified in assuming a prior probability of validity of any particular one that is astonishingly small while being quite comfortable that, as as the prior to a competing hypothesis, there is pretty close to a 36.8% chance that "none" is the reality.)

This is not insubstantial. In a more cautious Bayesian analysis than I wrote about before, even doing the religious a great service by saying that their religious claims are as they say--highly likely to predict the world as we see it--while sticking to our guns that a non-religious (naturalistic) interpretation is equally good, the odds that "no religion is valid" is true as compared to "this particular religion is true" from an outsider's perspective look like
  • About 19 to 1 in favor of "no religion is valid" assuming only 20 major world religions as equal contenders, and
  • About 40,000 to 1 in favor of "no religion is valid" in the ~40,000 denominations situation. 
These odds aren't nail-in-the-coffin long, but given that they concede an enormous amount of the argument, in terms of apologetic concession, this is a pretty stout statement in terms of how unlikely any religion is to be true given how many of them there are in the world (particularly since distributions appear mostly to be geographic accidents of birth than anything else).

Indeed, even if we concede more to the apologists and say that their nay-saying about science might be right, say that the naturalism approach is only half as good as the religious argument at explaining our evidence (even though we have lots of good reasons to believe it is the other way around), just the existence of 20 major religions puts the odds at 10 to 1 against any particular one being true, and the 40,000 denominations argument gives us odds near 20,000 to 1 against any particular one of them being the right one. Realize for a moment how much this gives up to the apologist and look at those odds again! Even if we give them a huge part of their argument, the sheer existence of so many other faiths making analogous claims still puts their odds sharply out of their favor. Another way to say this is that even if we give apologists for some religion a great deal of their argument, an enormous amount of their work in establishing their case is still ahead of them!

Truly, there is a lot of weight in expecting to get this question wrong by picking any particular faith.

Not so fast!

To be totally fair, as I mentioned, this is oversimplified. Indeed, religions are a bit more complex than this. First of all, this construction ignores the fact that the religions themselves already (sometimes) posit mutual exclusion. For example, it is not possible for any pair of Islam, Judaism, and Christianity to be true. Indeed, it is not true for any of the One True Faiths, as I call those three, to be true while any other faith is true. On the other hand, faiths like Hinduism and Buddhism are much more inclusive. This complicates things if we want real numbers on this question, but from the perspective of the Outsider Test for Faith, particularly in assigning a prior probability for a bit of Bayesian reasoning on it, it doesn't matter much. The particulars can be swept up by considering the evidence.

For example, the first numerical example I give above suggests "Christianity versus Islam, say" and identifies a 1/4 chance that either particularly is true and a 1/4 chance that neither is true (therefore with a 1/4 chance that both are true). Both cannot be true, though, and so this analysis is not accurate to the reality of mutually exclusive religious claims--although as I noted, this could be weighed as evidence instead of as part of the prior background knowledge about the faiths, in which case the existence of the other in the world actually speaks against the validity of either!

Furthermore, as I noted above, long odds against don't really matter against really good evidence. The (incredibly tiresome) arguments around the burden of proof and the war on confirmation bias must continue then. If the evidence were extremely good for the validity of some branch of Christianity, for example, long prior odds wouldn't matter against the weight of that evidence (above, this situation would translate to the evidence being vastly better predicted by that particular theistic hypothesis as opposed to others). So, still, this isn't a nail-in-the-coffin kind of argument against believing in some religion--even if it is a god damned strong one that says, "but if you do, you're very probably wrong about which one."

Edit (26 Jan 2013): To see  more about how significantly this is an oversimplification, please see my comment on the matter below. Link.

Now, a really savvy reader here will notice that the construction I give, with lots and lots of religions, essentially gives a ~37% chance that no religion is true and therefore a ~63% chance that some religion is true. A dishonest one will attempt to read this as an argument that there is a ~63% chance that God exists, but that's bogus. Why? A couple of reasons.

First, not all religions have to posit that a God exists (indeed, some extant one don't!). In fact, it may be possible to imagine a godless version of each religion that exists, at least for many religions, akin to Jewish Atheism. Second, this analysis doesn't examine evidence except the number of religions, which is only one piece of data regarding that question.

A truly dishonest reader will conclude at this point that surely it suggests then that there's a ~63% chance that some religion is true. It doesn't. Again, this is merely an examination before considering the evidence. Do bear in mind, though, that even if it did create this argument (again, it doesn't), absolutely nothing guarantees that the hypothetical true religion even exists in the world yet (or ever will). Therefore, to take this ~37% thing as an argument for any religion in general lives at the height of academic dishonesty only succeeded by taking the next dishonest step and jumping from some religion to this religion--which is exactly what this argument destroys our confidence in doing.

What good is it, then?

The real value here is in terms of highlighting the power of the Outsider Test for Faith without even having to consider how an outsider is likely to evaluate evidence as compared to an insider (afflicted as insiders are with the cognitive bias we call "faith").

In particular, it takes a big, bold, and bright highlighter to the fact that the distribution of world religions is an enormously heavy and hard argument against the validity of any particular one of them, especially given the sheer number of them that exist--a number that seems to grow, not shrink, with time (which is even more evidence against them).


If you enjoy my writing, you can read more of it in my first book, God Doesn't; We Do: Only Humans Can Solve Human Challenges. If you choose to pick it up, I thank you for your support of myself, my family, and indie authors in general.


  1. Thanks for working through those examples.
    Even without the numbers we can see religion diverges over time, while science tends to converge over time.

    Even Philosophy discards bad ideas over time.

    1. Indeed.
      Incidentally, I spent way too much time and effort last night after posting this thinking about what the case would be if we "normalized" for the additional information that "trueness" in a religion must be mutually exclusive. That renormalization is relatively easy to do (and makes an even stronger case for "they're probably all wrong"), but it highlighted an interesting problem with this oversimplified construction that I didn't notice at first. Of course, since I know it's oversimplified and don't intend to use it as a knockout punch of any kind, it doesn't really matter.

      Notably, the above construction contains the "issue" that if we were in the state where N=1, i.e. only one extant religion, we would be able to conclude that there's a 100% chance that it's true as the construction is given (1/N chance of truth before considering evidence). Obviously, this cannot be accurate. Indeed, this number cannot meaningfully be more than 50% in a state of absolute not-knowledge (half "religion is true" and half "religion is false").

      It's very complicated to try to wheedle out better numbers than this, though, partly because (a) some but not all extant and hypothetical religions posit mutual exclusion, and (b) "none is true" being considered a viable position in the case of N=1 implies it should be in N>1, but that gets weird because then it is being considered as a position and as the null position at the same time. Not good.

      Truly, it's even worse than that because it seems plausible to suggest that "all true versus all false" is a bit too binary for such complex structures for understanding the world. It seems far more likely that each religion is some P_[religion] percent valid in explaining the world. Assessing these numbers accurately is probably impossible and definitely highly contentious.

      That's all a long-winded way of admitting that if this is to be a strong argument, instead of a cursory one regarding the evaluation of the prior in an attempt to do Bayesian reasoning, this problem is a *whole lot* harder than I've made it look here.

      I'm not in any way lying to say that I lost a lot of sleep over this last night.