Overview (4) – Reasoning with uncertainty

We can be sure about very little.

"'We reason rashly and at random,' says Timaeus in Plato, 'because our judgements, like ourselves, have in them a large element of chance.'"

Montaigne, Essays, Book One: Chapter 50, above p 130.

Uncertainty can increase the closer something is scrutinised. Our quotidian lives would be impossibly disrupted if we indulged in a Plato-like thoughtfulness, so we normally come relatively untroubled to our settled beliefs. Forensic fact-finding is fraught with difficulties that it is necessary to ignore if decisions are to be reached. There are even studies that suggest that the demeanour of a witness is not a reliable guide to his truthfulness; you may as well toss a coin. Is the only instruction a judge can sensibly give a jury the familiar exhortation to "decide what evidence you accept and what you reject" and "use your common sense"?

Some efforts to increase the accuracy of judicial fact-finding have called in aid mathematics. This approach can be given a label of convenience, but the danger is that doing so will suggest that these efforts are just passing fads. Peter Tillers, an academic from the USA, calls them the New Evidence Scholarship, and identifies three generations of this:

"The first generation of the New Evidence Scholarship emphasized the heuristic uses of mathematical analysis of evidence; it emphasized that numbers (especially as used in probability theory) could illuminate the logic and structure of factual inference in general and of particular problems of factual inference.

"The second generation of the New Evidence Scholarship focused on mathematically-laden problems of scientific evidence (e.g., DNA evidence) and on problems of factual inference that seem tractable to statistical analysis.

"The third generation of the New Evidence Scholarship (NES) also uses mathematical argument and analysis. But this variant of NES does not require or expect consumers of mathematical analysis to do computations. Instead, NES-3rd uses mathematics and computations to develop tools for deliberation about inference, tools that do not require or expect the user of the tool to do computations."

The growing recognition of the pitfalls of ignoring the logic of mathematics, and particularly the logic of probabilities, coupled with appreciation of the impracticality of requiring judges and jurors to apply complex formulae in their deliberations, is the unifying theme of this scholarship.

Each of these applications of mathematical reasoning is exemplified by cases discussed here. Sometimes mathematics is ignored when it would have been of assistance, as in Brown v Attorney-General 6 March 2005 and also Wi v R mentioned in update to note for 4 July 2008. Refinement of DNA techniques has been accompanied by recognition of increased risks of contamination and of falsely positive results: R v Hoey 2 January 2008. Applications of Bayes' Theorem have pointed to the correct way for probabilistic evidence to be given, and also to how reasoning with conditional probabilities can be fallacious: R v Bain 8 June 2009. I have collected some observations on reasoning with probabilities in a draft paper available here.

Some criticism of the use of mathematical reasoning is advanced on the basis that, since the values of the variables are a matter for dispute, the equation containing them is of no use. More moderate criticism, of the same kind, acknowledges that the equations may have limited use, although it makes too much of claimed difficulties in calculating the probability values. The latest developments in this branch of evidence are directed at meeting the points made by such critics.

I give the last words to clever old Montaigne:

" The uncertainty of my judgement is so evenly balanced in most cases that I would willingly refer the decision to a throw of the dice; and, after giving much thought to our human weakness, I observe that even sacred history has left us examples of this custom of leaving chance and fortune to make the decision in matters of doubt: 'and the lot fell upon Matthias' [Acts I, 26]."

Montaigne, Essays, Book Two: Chapter 17, above p 216.