Thursday, December 6, 2012

Continuing my Bayesian argument--the role of evidence

In my last post, I referenced how John Loftus's Outsider Test for Faith should have the impact of severely denting the expectation that any particular religion is true, and I did so in a way that allows us to make simple arguments that involve no real examination of any evidence outside of the distribution and general concept of the types of claims religions make. In other words, that was a completely a priori investigation, if you will, following the original definition for the OTF that Loftus provides in Chapter 4 of his Why I Became an Atheist: A Former Preacher Rejects Christianity. I don't think that's enough, nor do I think it's the fullness of what the OTF will enable, as I've mentioned, but more importantly, I think that as it stands, what I wrote yesterday is open pretty wide to a superficially meaningful (but empty in the depths) challenge.

The challenge:

The challenge is superficially meaningful because what I offered yesterday is essentially a priori, which most of us who think honestly about things will admit is a pretty damned bad way of going about understanding what is real and what isn't.

The challenge that I should be able to expect, if I was to offer that which I posted last night as if it were the final word on this analysis (which I didn't even do last night if one reads the end of it or the previous Bayes-theorem-based posts), is essentially that the argument I put forth applies to essentially anything where there is sufficient diversity, and thus it undermines the ability to believe in anything.

Let me use an example that is meaningfully enough controversial to make it feel palpable: the common claim by martial artists that "my martial art is the best martial art." Let's ignore the full reality of this here by being intentionally vague about "best," being that different martial arts could be best for different people in different situations, yadda, yadda.

An astute challenger would note that there are thousands of martial arts in the world, and so by the kind of a priori analysis I offered last night, no martial artist familiar with this kind of Bayesian reasoning could possibly accept the claim that theirs is the best (which the vast majority of them will think--again, vague on the word "best" here on purpose). Thus, any martial artist who takes the Outsider Test for Bestness is going to find herself in serious doubt about the claim of practicing the "best" martial art.

Maybe this is not all bad, but to discuss that is beside the point. The point--that makes this a superficially valid challenge--is that it would imply that there are people who are practicing the best martial art, which probably does exist in a meaningful way once the metric for martial art quality is defined, who will be seriously misled to believe that they are very unlikely to be practicing the best martial art.

To reframe it in terms of religions, the challenge to the OTF analysis I offered last night would be "with such an a priori evaluation, it is entirely plausible that someone following the true religion would be misled to believe they are not," which given the consequences that loving Gods offer us in those religions is a pretty raw deal to stick someone with.

The problem with this challenge:

Such a challenge compares apples to oranges. Observe that within martial arts, once we define a metric to measure martial arts with, there is no doubt that one (or a few) martial arts will actually be best. Note that this metric would constitute the evidence that would then be analyzed and fed into Bayes's theorem, which we've already seen will return a reasonable degree of certainty even against a low prior probability with sufficient evidence, e.g. here the number of points on some agreed-upon metric of quality. There is a guarantee that there will be some art or group of arts that score maximally on this metric. This is key.

There is absolutely no guarantee that any religion is true--a plain fact that extends not only to every religion but also to every conceivable religion (hence my claim that the prior in this religion question should be zero, almost surely--we've found an infinite sample space here, my friends, and this does not apply to every conceivable martial art because we're talking about the best among the finitely many that exist not the best as in the best-possible art). All of them could be false. To wit, there is, in fact, no evidence that any of the religions are true, particularly since there is no evidence that the God that sits at the center of them actually exists.

This challenge is common because it feels a bit like the rug has been pulled out from under religious belief (attn, theists: this is how all a priori arguments feel, including the bogus ones you make for your God or religion--a point that shouldn't be lost here). Loftus does a good job defending it, though, with his "religious diversity thesis," which essentially illustrates that the religion that someone ends up with overwhelmingly has the appearance of being arbitrary instead of based upon the potential validity of the religion (say, an accident of geography, like being born to certain parents or in a place where a certain brand of missionary will come before the others). Note that this does not apply to the example of the martial arts, which should attract people specifically because they appear to be the best available avenue to some set of goals.

This too is important--in that there is no credible evidence for the validity of any of the religions, it is very difficult (and a requirement of the believers) to establish that one religion might be more true than another. Indeed, that's nonsense because "more true" is rather a bogus term. This a huge departure from the martial arts example, which assumes some kind of metric could be agreed upon for the evaluation of "best."

Thus, the contention that "this a priori argument with the OTF via Bayes's theorem could apply the same way to anything, and thus this is nonsense," is complete rubbish.

What's up with the a priori OTF thing, then?

It seems rather clear that the goal of the a priori assessment available through the OTF, since it is a priori, is to get the cracks in the religious belief that would prevent someone from evaluating the evidence fairly, in Loftus's terminology to get someone to re-evaluate their "control beliefs" to allow a new perspective to take root. The goal, then, of that simple first-order evaluation would be (possibly) to reset the prior for a new evidence-based analysis and (definitely) to allow the person taking the test to evaluate the evidence with less bias, ideally with none. If we view religions as what they are, frameworks through which people interpret the world, the OTF has the goal of helping someone break free of the framework that is coloring their understanding of the evidence.

--This too is likely to be grounds for an objection: that the skeptic's perspective is itself a framework and therefore cannot be said automatically to be better than a religious one. Loftus handles this objection neatly on pp. 72-73 of the original Why I Became an Atheist, justifying the position of skeptical agnosticism as the proper default, a point of view that is generally agreed upon more widely. Without any evidence other than the existence of several competing religions, the OTF gives us good reason to believe that a skeptical approach is warranted, and that's exactly what the a priori evaluation tells us.--

The role of evidence

Evidence is everything in a Bayesian analysis. Indeed, as I have already discussed, if someone approaches Bayes's theorem with a complete punt on evidence, they are left with no new information. Their game is set entirely by their prior probability, and this is exactly what we mean by the term "closed minded." Their mind is made up, so let them be unable to defend their position. Bayes's theorem is of no use to them, though, because Bayes's theorem is specifically concerned with determining how evidence affects our decision making.

By far the most weighty effect of the Outsider Test for Faith is the fact that it will cause the evidence to be evaluated differently. When someone is colored by a particular religious bias, it is likely that the consequents will be chosen to reflect the idea that "my religion explains the world better than anything else could." As we saw, holding this position puts the entire process in the favor of the believer in a Bayesian analysis. A significant part of the point of viewing the religion from the outside is to remove that bias.

This is where the hammer falls, though, and it falls very hard. When viewed from the outside, the likelihood that the evidence is explained by a religion, for example, that says "whatever you pray for in faith, you shall receive" simply is not going to stand up. All of the evidence in the world points to "no credible evidence for prayer ever." That's only one example. The success at the biological theory of evolution to explain biodiversity, the success of physics to explain the workings of nature, the success of all of the sciences to make useful predictions about the world--none of which any religion has succeeded in, makes for another enormously weighty example. There are thousands upon thousands more too, some far heavier than that one, which I've already mentioned (like that slavery would have been allowed by a benevolent, loving God). 

Again, then, nearly every outsider would strongly agree that the relevant numbers should be very close to zero (I say zero, almost surely), in the case of how well the evidence would be explained if the religion were true, and very close to one (I say one, almost surely), in the case of how well the evidence could be explained by other explanations. Hell, this could be the very definition of an outsider!

The proof is in the pudding as well, revealing the arbitrary nature with which religions are accepted: the numbers that a Christian would assign to Islam or Hinduism are going to be very nearly the same numbers a Muslim or a Hindu would assign to Christianity, and thus these numbers are going to give a good sense of how an outsider would evaluate them. Indeed, a good way to go about the OTF would be to do the same kind of analysis for some other religion and then to realize those numbers apply to your own from the outside!

I say this specifically to avoid the allegation that I think a conclusion can be drawn without examining much evidence. I do not think that. I think that a significant argument already exists against every religion without having to look at any more evidence than the sheer number of them and usual ways by which one becomes a believer. Indeed, I think that argument is sufficient to warrant taking an outsider's view of the evidences of the world, as opposed to an insider's view, and I think that the OTF successfully puts us in the proper position both to accept that and to actively begin to do it. Then, the real OTF can take place, looking at the evidence with the Jesus-colored (e.g.) glasses removed. Once done, I also think the evidence is sufficient to literally destroy credence in any prior save the willfully obstinate position of being 100% sure (which I think the a priori argument reveals for being exactly what it is).

Editorial Note (1/10/2013): Richard Carrier has been kind enough to point out that my use of "a priori" in this post is not technically correct. I've decided to leave the terminology as it is with this note, because it is more genuine to do so. Where I've said "a priori" here, it is more proper to say "assuming only background knowledge," i.e. prior to examining the evidence. A fully a priori statement would not even assume background knowledge. For a proper treatment, readers are encouraged to check out Carrier's note in John Loftus's The End of Christianity.


If you enjoy my writing, you can read more of it in my first book, God Doesn't; We Do: Only Humans Can Solve Human Challenges. If you choose to pick it up, I thank you for your support of myself, my family, and indie authors in general.


  1. A couple topics to comment on:


    Before I had really gotten into Bayesian reasoning, I had already adopted an informal version of the philosophy of pragmatism, specifically epistemological pragmatism. It had taken me many years to finally arrive at this version, starting from literally the age of 12, when I first started to read about philosophy.

    So, I was quite pleased when, after learning about Bayes' Theorem, I found that the work I'd put into whittling down my personal philosophy to pragmatism had not been in vain. My version of pragmatism is like an informal description of Bayesian-style reasoning. I'm hoping one day to get to the point where I can re-formulate it as an actual application of Bayesian hypothesis testing. But in the meantime, you may find it very useful for its ability to put the same kinds of arguments you made above (about the success of science vs. the failure of faith) into a very intuitive language. I use it regularly, and I'm rather pleased with the results (if I do say so myself ;-) ). Here's probably the most accessible version of what I've written on it so far:

    Odds form of Bayes' Theorem

    There is an extremely useful way to formulate Bayes' Theorem in the form of odds (rather than probabilities):

    O(H|E.B) = O(H|E.B) * L(E|H.B)

    Where O(X|B) is the standard definition of odds in probability theory: O(X|Y.B) = P(X|Y.B) / P(~X|Y.B)

    And L(E|H.B) is called the 'likelihood ratio', which is not to be confused with an odds ratio, and is defined as: L(E|H.B) = P(E|H.B) / P(E|~H.B)

    The likelihood ratio serves as a multiplier, proportional to the 'strength' of the hypothesis (H) for predicting E, 'over' the strength of the alternative hypotheses (~H) in predicting E.

    Thinking about BT in terms of odds has a lot of advantages, primarily that it is very intuitive, once you get a handle on what odds are and how they behave compared to probabilities, and also because fairly quick and simple BT calculations can be done one after the other, each likelihood ratio (for each new piece of evidence) bumping the odds in favour of H up or down depending on how well H predicts it vs. ~H.

    Ian Pollock at Rationally Speaking has a couple decent posts going into more detail: Odds again: Bayes made usable, and Why we should use odds, all the time

    Additionally, if you use 'log odds', you end up with even nicer 'addition of evidence', which is really cool, but it helps a lot to become familiar with the more basic odds form first, IMO.

    One of the side effects of putting things in odds form is that absolute certainty and falsity become more obviously inappropriate (leading to divisions by zero), which I think is a nice touch for people new to probability theory. This is even more obvious in the log odds form, where absolute certainty is +oo and absolute falsity is -oo, which most people recognize as extremely unusual conditions which should usually be avoided.

    1. I should add (now that I've read it), your nice piece on pragmatism reminds me of something witty a sharp friend of mine said the other day when we were discussing the "non-overlapping magisteria" nonsense: "There is only one magisterium: reality."

      Technically, I agree in a sense and disagree in another. I think it is meaningful to talk about realms that are entirely abstract, thus not reality, and that constructions of great use to reality can be pulled from there (or not). In that sense, I think there are other "magisteria," but I don't think that they matter much unless they provide value against reality (either practical value in terms of solving problems or in terms of simply being fun, which solves the problem of wanting something to think about, etc.).

    2. Interesting, that's a neat way of thinking about it. I usually refer to it simply as 'imagination', but your point is a good one: that 'imagination' can be used not only for fun, but also constructively, whether as an abstract design space, or for evaluating thought-experiments, etc. These imagined ideas/concepts/scenarios/artworks/whatever can then be 'brought forth' into reality by the person either expressing them abstractly, or by creating some concrete artifact or some physical process. (Of course, all of this is obvious to people in everyday life, but it's not so easy to get philosophy to correctly model everyday life!)