The prosecutor’s fallacy is an error in reasoning that can be very difficult to notice. It came to attention recently in what is now our leading case on costs in criminal cases: R v Lyttle [2022] NZCA 52.
The Court dealt with appeals from three separate trials, and the one relevant here is R v S, the discussion of which begins at [105].
The issue was whether injuries had been caused by assault or by accident.
There was no evidence indicating guilt other than expert medical opinion evidence. The Crown called five experts, the defence two.
Inevitably, the experts’ evidence was all based on studies of similar injuries where the causes were known [165].
Usually, experts give their evidence in the form of a likelihood ratio: for example, the injuries are much more likely to have been caused by assault than by accident. The cases for the prosecution and the defence in the trial are compared in that way.
Although it is not obvious here, it seems that the experts, as summarised from [116], were in effect saying (for the Crown): the injuries here were very much more likely to have been caused by assault than by accident.
In calling attention to the prosecutor’s fallacy, the Court was in effect saying that the probability of seeing the injuries if they were caused by assault is not the same as the probability of assault, given the injuries.
That indeed is so, as is obvious if we consider a simple example of the same error: the probability of seeing that an animal has four legs, given that it is a sheep, is not the same as the probability of seeing that an animal is a sheep, given that it has four legs.
There are two separate things under examination in these cases: injuries by assault and injuries by accident; sheep and quadrupeds.
The seriousness of this error depends on context. If you are talking about all the sheep and all the quadrupeds in the world, it is a very serious error. But if you mean just those sheep and those chickens in that field, the error will not matter. If the context tends to exclude innocent explanations it is like the field, but with greater recognition of innocent explanations it becomes more like the world. In a trial the context of the case provides what can be called the prior probabilities of guilt and innocence.
In R v S the context was that of a situation where accidental fall was possible, and where the defendant was of good character, well spoken of as a caring person, and a person of whom months of covert audio surveillance after the event revealed nothing to raise any suspicion.
If we pretend that the combined likelihood ratios of the experts' evidence strongly favoured guilt, say only one chance in a hundred of innocence, that would in mathematical terms be 99 to 1. But without any other evidence of guilt, and to the contrary, every indication of innocence, the prior likelihood of guilt may be 1 in 99.
Combined, the priors and the likelihood ratio on these hypothetical figures would be 1. This is the ratio of the probability, on all the evidence in the case, of guilt to the probability on the same evidence of innocence. This means a one in two probability of guilt, or 0.5, which is not enough to meet the criminal standard of proof. Nor even the civil standard, which requires more than 0.5.
At [166] the Court summarised the approach that the prosecutor had suggested at trial. This seems to have involved the sheep/quadruped fallacy: a low probability of getting the evidence of the injuries if the defendant was innocent does not in context mean a low probability of innocence given the evidence of injuries.
All the evidence in combination had to be considered when assessing the likelihood of guilt.
One lesson of the appeal in R v S is that lawyers need to be trained in reasoning with conditional probabilities. The major hurdle is that people who become lawyers have mostly been taught at an impressionable age that they are no good at maths. People who become lawyers do, however, tend to be good at logical reasoning, so let’s not despair.
Another lesson is that rare events happen, and apparently overwhelming statistical evidence is only about probabilities, not about the actuality in a particular case. This seems to be the point that was made by the witness whose comment is noted at [127]. Indeed, in law facts are only ever established to some degree of probability, expressed in words as on the balance of probabilities or beyond reasonable doubt.
The jury in R v S were unable to agree on a verdict, and the Crown elected not to seek a retrial, as noted at [112], so the charge (manslaughter) was dismissed, which is equivalent to an acquittal.
If you are interested in this sort of thing, there is more on this site: just enter "Bayes" in the search box.
