Thursday, April 19, 2012

My 500th posting: Probative value and independent evidence

Probative value is the tendency that an item of evidence has to prove an issue in a case. Does the measure of that tendency depend on other independent evidence in the case?

For example, consider these facts on a charge of murder:

"[2] ... A hair found on the deceased's thumbnail had been subjected to mitochondrial DNA testing. The results of that testing showed two things: first, that the appellant could have been the donor of the hair and, second, how common the DNA profile found in the hair was in the community. This second aspect of the results was expressed in evidence both as a frequency ratio [sometimes called a "random occurrence ratio" or a "frequency estimate"] and as an exclusion percentage. The expert who had conducted the test gave evidence to the effect that one in 1,600 people in the general population (which is to say the whole world) would be expected to share the DNA profile that was found in the hair (a frequency ratio) and that 99.9 per cent of people would not be expected to have a DNA profile matching that of the hair (an exclusion percentage)."
The case, Aytugrul v R [2012] HCA 15 (18 April 2012) concentrated on whether expressing the statistics as an exclusion percentage was improperly prejudicial. It was argued for the appellant that to say 99.9% of people wouldn't match suggests that it is 99.9% likely that the appellant was guilty.

In the circumstances of this appeal it was unanimously held that there was no improper prejudice because the jury had been carefully directed on how they should reason. Determining whether there is improper prejudice requires having regard to the whole of the evidence [30].

The question I started with did not need to be answered in this case because the appeal turned on whether there was illegitimate prejudice, but Heydon J considered it in some detail [41]-[65], especially in relation to whether giving evidence of one way of expressing the scientific results rendered an alternative and equivalent way redundant and inadmissible.

Evidence about the matching of samples is like all other evidence: it raises the question, given the existence of this evidence, what is the probability that the defendant is guilty? This probability of guilt is the ultimate issue and involves consideration of all the admissible evidence in the case that the fact-finder accepts as true. The probative value of an item of evidence is its tendency to prove an issue, and where that is the ultimate issue, the question arises as to whether probative value should be assessed by considering the item of evidence in isolation, or whether its probative value should be considered in the context of other evidence.

Logicians would be isolationists but with some qualification. Independent evidence is irrelevant but dependent evidence is relevant to the probative value of an item of evidence, as is contextual evidence.

Usually evidence of the sort of match in Aytugrul would be given in terms such as: "whatever the probability that the defendant is guilty based on the other evidence in the case, this evidence increases that probability by 1600 times." Sometimes the same thing might be said in words rather than numbers, for example by saying the evidence strongly supports the proposition that the defendant is guilty (see [32]).

I am assuming that the ratio would result in a figure of 1600, although that is not clear. There might (here I am not referring to the particular facts of this case) be innocent explanations for the evidence of a match, even if the defendant was guilty: the test result might have been a false positive so that in reality it was someone else's hair, or the defendant's hair might have been transferred there when the police had custody of the body, or there might have been an error in handling scene exhibits and a mix-up between what was found on the victim and medical samples from the defendant. The probability of those sorts of errors, although normally small (but if larger they could make the test result irrelevant - see my concluding comment below), should be taken into account. In that sense some other evidence in the case could be relevant to the probative value of the evidence of a match. But that other evidence would not be independent of the existence of the hair.

Independent evidence, such as evidence of the time at which the defendant and the victim were together, and the time of death, does not affect the probative value of the evidence of the match between the hair and the defendant. It does not affect the numerator or the denominator of the likelihood ratio. Those are based on assumptions, or "givens": that the prosecution hypothesis of guilt is true, and that the defence hypothesis of innocence is true.

I infer from the extract from Dr Buckleton's evidence at [16], where he mentioned the lesser of two evils, that he would prefer not to put his evidence as either a random occurrence ratio or an exclusion percentage. The reference in the judge's summing up to Dr Buckleton's evidence of expecting the occurrence of that sort of hair in the relevant population of innocent people as between one in 50 people and one in 100 or less, seems to be a reference to the denominator of the likelihood ratio being between 0.01 and 0.02. Dr Buckleton usually gives his results as a likelihood ratio following the Bayesian approach, and he supports the "Bayes and the Law" site. Plainly there was a discrepancy between the experts in Aytugrul as to the statistical values. In any event, in this case Dr Buckleton said that the test results did not indicate a match: Aytugrul v R [2010] NSWCCA 272 at [55]. The implication of that evidence seems to have been lost in this case.

Significantly, the High Court recognised that the question of how this sort of evidence should be expressed is a question of psychology upon which the Court would need further evidence before making a legal rule about that [22]-[24]. (I recommend to anyone interested in this sort of thing the Nobel laureate psychologist Daniel Kahneman's "Thinking, Fast and Slow".) The Court did not say that evidence of this kind must be given in the form of a likelihood ratio, and it specifically did not lay down a rule that it could never be given as an exclusion percentage [21].

Note the difference between the probative value of evidence and the effect of its combination with other evidence in the case on the ultimate probability of guilt. In BBH v R, discussed here on 31 March 2012, it was this latter combination that was relevant in considering the Pfennig requirement.

There is a need for experts to agree on how to express their conclusions when giving evidence, and for legal professionals and judges to become familiar with correct reasoning with probabilities. In the NSWCCA in this case (see above link) McLellan CJ at CL, whose judgment is well worth reading for its references to specialist articles - including a paper by Kahneman - on the effects on juries of different ways of expressing statistical results, held that the trial was unfair, notwithstanding that the verdict was, on the evidence, not unreasonable, because the jury may have been led to think that the exclusion percentage was the same as the percentage likelihood of the defendant being guilty. The other judges in that court regarded the statistical evidence as strongly supportive of guilt and that was sufficient, there being no identifiable unfair prejudice to the defendant (Simpson J at [193]-[195]).

Indeed, as Bayesians know, if the other evidence in the case only established guilt to a probability of 0.1, the hair match evidence would increase that to 0.99. The same is true without any other evidence, but obviously there have to be other proven facts to give the match evidence context in relation to the offence charged. It is also appropriate to talk about groups of facts which together have probative value in relation to an issue, but their individual probative value is different from their collective probative value. Again, context is relevant, and it may be inappropriate to consider the probative value of a fact in isolation.

So what would have been unfair prejudice here? If the jury had not found as a fact that the hair had the DNA profile that the prosecution claimed it had, the other evidence about the frequency of a match would be irrelevant. To find as a fact that the hair had the alleged DNA profile, the jury would have to be satisfied about that to the standard of the balance of probabilities. Most subsidiary facts - that is, facts that are not elements of the relevant offence - only need to be proved to that standard, although there is an argument, popular - to the point of being law - in Australia, that critical facts have to be proved beyond reasonable doubt. Be that as it may, unfair prejudice would have existed if the match frequency evidence could have distracted the jury from its proper assessment of whether the hair did indeed have the DNA profile claimed for it.

Friday, April 06, 2012

The transcendent civility of verbal eunuchs

The case with the most amusing facts in a long time is Doré v Barreau du Québec, 2012 SCC 12 (22 March 2012). A judge was rude to counsel and counsel was rude to the judge. Both received reprimands from their respective disciplinary bodies.

The odd thing about this case is that whereas the judge was rude to counsel in open court, counsel was only rude to the judge in a private letter to him.

Another thing, not quite "odd" but at least strange, is that after setting up a fine sort of conceptual structure for the review of administrative decisions when Charter rights are involved, the Court's application of it to the facts is shrouded in mystery. The answer pops out, but because the private nature of counsel's letter was not taken into account we can't be sure why it didn't go the other way.

Valuable aspects of the judgment concern the duties of judges and of counsel as far as behaviour in court is concerned:

Judges must show respect for officers of the court (counsel), they must not be impatient and they have a duty to listen calmly to the parties and to counsel. They must respect the dignity of every individual who argues a case. Comments must not be immoderate. [14]

Counsel are bound by rules of professional conduct, and these include a requirement to behave respectfully and not undermine the processes of the court or the dignity of the judiciary. But at the same time counsel have rights of free speech, and a role in ensuring the accountability of the judiciary [64], and the protected tenure enjoyed by judges increases the threshold for the lawyers' expressive rights [65]. The balance between criticism and upholding dignity is fact-dependent and a discretionary administrative exercise. Criticism is measured against "the public's reasonable expectations of a lawyer's professionalism" [69] and must not overstep 'the generally accepted norms of moderation and dignity" [70].

The Supreme Court agreed with the conclusion of the disciplinary tribunal that had censured counsel.

In reality, the policy of discouraging a flood of vigorous personal correspondence between bar and bench may well have been the elephant in the courtroom.

"Lawyers potentially face criticisms and pressures on a daily basis. They are expected by the public, on whose behalf they serve, to endure them with civility and dignity. This is not always easy where the lawyer feels he or she has been unfairly provoked, as in this case. But it is precisely when a lawyer's equilibrium is unduly tested that he or she is particularly called upon to behave with transcendent civility. On the other hand, lawyers should not be expected to behave like verbal eunuchs. They not only have a right to speak their minds freely, they arguably have a duty to do so. But they are constrained by their profession to do so with dignified restraint." [68]
In this case one is tempted to suggest that the proper outcome would be to gather the judge and the lawyer together, get them to laugh a little, and send them off for brain scans, as ill-temper can be a symptom of tumours.