*/
Avoid the pitfalls of probabilistic reasoning and examine expert evidence with more confidence: Colin McCaul QC introduces new guidance from the Inns of Court College of Advocacy and the Royal Statistical Society
‘No practising lawyer should underestimate the difficulty involved in preparing and mounting an effective challenge to a well-prepared expert’s evidence by cross-examination, even when assisted by his or her client’s own expert.
With the power which an expert has to influence the decision of a fact-finding tribunal, whether judge or jury, goes responsibility. As some controversial cases have shown, the abuse by an expert of the power which he or she is given can cause serious harm and injustice.’
Lord Hodge, Justice of The Supreme Court, October 2017
Such have been the advances in advocacy training that most advocates nowadays will do a decent job of examining a witness of fact either in chief or by way of cross-examination. A thorough knowledge of the facts of the case, combined with careful preparation, will give the advocate the confidence and ability to bring out truth and to expose untruths.
When it comes to expert evidence, the landscape changes. Not only does the subject matter comprise material that often constitutes difficult ground for the advocate, but the experts’ opinions are themselves underpinned by what often appears to be impenetrable material. Many such experts have considerable presence and speak with great authority. A combination of all these factors provides the potential for the perfect storm whereby flawed opinion evidence is delivered to the court, it is not tested in a way that would unearth those flaws and a judgment is given based on that flawed evidence.
The Inns of Court College of Advocacy (ICCA) has drawn upon the expertise of seasoned, senior practitioners to create generic guidance for the assistance of all advocates involved in the preparation, admission and examination of expert evidence in any court or tribunal. This advice is compiled in the paper: Guidance on the preparation admission and examination of expert evidence. It has been distilled from principles and practices which are now well established across all areas of legal practice – civil, family, and criminal law, and covers all types of tribunal.
The guidance reminds advocates that, in addition to the relevant rules of court by which experts are bound, their expert might also be constrained by professional rules of practice or case-specific literature relevant to their own discipline. It is the responsibility of advocates to satisfy themselves that their expert is up-to-date and fully conversant with current regulations, research and good practice. Beware the expert who has retired from active practice.
To identify the central issues in dispute and expose the strengths and weaknesses of an opposing expert, early consultation with one’s own expert must be well-prepared and organised. The guidance contains much advice on this topic. Advocates must ensure that:
The importance of fully understanding every part of the evidence cannot be overstated. Without such an understanding, the advocate does not have the confidence or armoury with which to confront an expert witness. Add to this the fact that it is (rightly) drummed into the advocate from an early stage that no question should be asked to which he or she does not know the answer and the outcome is that identified by the Law Commission in its publication: Expert Evidence in Criminal Proceedings in England and Wales (Law Com No. 325, 2011) at para 1.21:
‘… cross-examining advocates tend not to probe, test or challenge the underlying basis of an expert’s opinion evidence but instead adopt the simpler approach of trying to undermine the expert’s credibility. Of course, an advocate may cross-examine as to credit in this way for sound tactical reasons; but it may be that advocates do not feel confident or equipped to challenge the material underpinning expert opinion evidence.’
Needless to say, a conference with one’s own expert is the time to explore in depth not only the base material but also the material underpinning the evidence of that expert and of the other side’s expert. ‘Why are you right and why are they wrong?’
In some areas of expert evidence, it becomes significantly harder for the advocate to fully understand the material underpinning the expert opinion. There is no better example than in the case of statistics. Few advocates have studied mathematics to a level that equips them with sufficient knowledge to readily understand statistics. The situation is made worse by the fact that, more often than not, an expert in a case has no background in statistics either. For example, a medical expert may cite statistical evidence gleaned from literature but have no concept of the reliability or true interpretation of that evidence. In those circumstances, the assistance that the advocate would normally wish to receive from his or her expert in understanding the material underpinning that expert’s or the opposing expert’s opinion is lacking.
It would be convenient to dismiss statistical evidence as affecting only a minority of cases mainly in the criminal field. If that were ever true, it is no longer so. Statistical evidence and probabilistic reasoning form part of expert witnesses’ testimony in an increasingly wide range of litigation spanning criminal, civil and family cases as well as more specialist areas such as tax appeals, sports law, and discrimination claims.
In recognition of this state of affairs, the ICCA and the Royal Statistical Society (RSS) have jointly published a booklet containing guidance to help advocates better understand the use of statistical evidence when presented in court. The guide is entitled: Statistics and probability for advocates: Understanding the use of statistical evidence in courts and tribunals. It provides a walk-through of the main concepts within the subject that advocates need to grasp.
Consider the following two sentences where ‘match’ refers to an agreement between a DNA profile obtained from a defendant and a DNA profile obtained from the evidence:
a) ‘The probability of a match if the semen came from another person is one in 40 million.’
b) ‘The probability that the semen came from another person is one in 40 million.’
How many people, including barristers and judges, would view sentence B as being a consequence of sentence A? Yet the two sentences are describing very different scenarios. To infer that sentence B is a consequence of sentence A, and that it reflects the chances of a defendant being innocent, is hugely erroneous.
"Few advocates have studied mathematics to a level that equips them with sufficient knowledge to readily understand statistics. The situation is made worse by the fact that, more often than not, an expert in a case has no background in statistics either"
The guide explains how such an error (‘the Prosecutor’s Fallacy’) can more easily be understood and thereby avoided. The guide also takes the reader through statistics and probability as they affect all areas of litigation. In doing so, it illustrates these topics with examples taken from miscarriages of justice in real life cases and/or from papers and textbooks on statistics. It looks at concepts such as:
It would be foolish to claim that the guide will arm its reader with an in-depth knowledge of what are, on any view, complex topics. It will, however, alert the advocate to pitfalls in relation to statistical (and epidemiological) evidence and, at the very least, to know when it would be appropriate to obtain the advice of a statistician.
To encourage practitioners to start testing their understanding and putting the guidance into practice, the booklet also provides four case studies in different areas of law. These are designed to incite the reader to plan how to question an expert in a variety of different situations and to scrutinise the methods used in presenting different kinds of conclusions based or allegedly based on statistics.
The guide also considers whether further activity may be required to improve the quality of scientific evidence to the degree identified as necessary by the Law Commission in its 2011 report. It signposts advanced thinking around current and controversial concepts (the likelihood ratio, Bayesian reasoning, the doubling of risk test and life expectancy estimates) for the purpose of warning advocates that this is a rapidly advancing subject whereby concepts currently accepted by the courts are likely to be challenged by statisticians.
The final chapter directs readers to discover more information on particular problems, and complements the reading lists included within each section of the guide. The resources offered reflect ICCA’s and RSS’ joint aim to encourage practitioners to develop their understanding of this complex but important topic and increase their confidence when they are conferring with experts, or challenging scientific evidence and expert opinion in court.
As science progresses, the role of expert evidence in litigation will only augment. As such, there will be an even greater weight of responsibility on the advocate to ensure that such evidence is properly tested and that the prospect of miscarriages of justice is minimised.
Contributor Colin McCaul QC, The Inns of Court College of Advocacy
From lab bench to judicial bench, the UK’s science academy is collaborating with the judiciary to ensure that the best scientific guidance is available to the courts. By Dr Julie Maxton
In a unique collaboration with the judiciary and the Royal Society of Edinburgh, the Royal Society launched the first of a series of succinct, jargon-free and easy to understand scientific guides, or primers, for judges. As the UK’s science academy we believe that the very best scientific advice should be available in all spheres of society, and that includes the courts.
The primers are designed to assist the judiciary when handling scientific evidence in the courtroom, explaining how far the science goes at any given time and what the limitations of the science are. They have been written by leading scientists and working judges and peer reviewed by legal practitioners, all of whom have generously volunteered their time to the project. It was Lord Chief Justice Thomas who first envisaged their development, and they have been steered to publication by Lord Anthony Hughes from the Supreme Court.
The first two primers cover DNA fingerprinting, widely used in court, and gait analysis, a relatively new form of evidence identifying people from the way they walk from CCTV. Future primers on statistics and the physics of vehicle collisions are planned. The project’s steering committee identifies topics by speaking to practising judges. It is absolutely vital that this project is demand-led and addresses practical trial-related questions which are relevant and important to judges themselves.
Dame Sue Black, a world expert in forensic anthropology, and criminal trial judge, His Honour Mark Wall QC, led the primer on forensic gait analysis, which was first admitted as evidence in a UK court in a case of armed robbery in 2000. It goes on to discuss the discipline’s evidence base, describing this as ‘sparse’, and making clear it is largely translated from more developed fields, such as clinical gait analysis, which provides information to health professionals. It recommends, ‘care is required in assuming that techniques developed in one field can be applied in another with quite different objectives’.
It also highlights that there is no evidence to support the assertion that the way people walk is unique, and explores the related issue of the variability shown in an individual’s gait pattern when factors such as fatigue, walking surface, footwear, injury or intoxication are taken into account. Finally, particularly as the use of forensic gait analysis comes more to the public’s attention, it concludes it is also possible that individuals may choose to walk differently to evade detection. The guide ends with a discussion of the different type of professionals that may present as gait analysts, and the diverse qualifications that they hold.
Professor Niamh Nic Daéid, a professor of forensic science at the University of Dundee and the Court of Appeal’s Lady Justice Anne Rafferty, led the DNA-themed primer. Its editorial board also included the expertise of Sir Alec Jeffreys, the inventor of genetic fingerprinting who in 1984 discovered a method of showing the variation in the DNA of individuals, and Nobel Prize-winning scientist Sir Paul Nurse.
The primer discusses the history of DNA profiling, the science of DNA analysis, factors to be considered in the evaluation of DNA evidence, including current limitations, and the future development of new DNA methods. Specific questions such as how DNA profiles are compared and interpreted are explored, with the authors discussing statistical approaches such as match probability and the likelihood ratio, and how they can very easily be misunderstood, known as the defence and prosecutor’s fallacy.
Hard copies of the primers have been distributed to courts in England and Wales, Scotland and Northern Ireland through the Judicial College, the Judicial Institute, and the Judicial Studies Board for Northern Ireland. The primers are also available to download from the Royal Society website.
The primers project is just one part of the collaboration between the scientific and judicial communities. With the Judicial College, the Royal Society has also hosted a series of seminars for senior judges on scientific and new technology topics relevant to court proceedings. Topics have included memory in testimony, uncertainty and probability, mental capacity, pain, substance addiction, and AI and machine learning. Future seminars will consider human gene editing, robotics, and causation. Further primer topics may well come out of some of these seminars.
Science and the law are of fundamental importance to society. As society changes and responds to fast-moving and far-reaching developments such as neuroscience and AI, both communities need to continue to work together to accommodate and understand the challenges of these new technologies.
Contributor Dr Julie Maxton CBE, Executive Director of the Royal Society, and a member of the primers steering group
With the power which an expert has to influence the decision of a fact-finding tribunal, whether judge or jury, goes responsibility. As some controversial cases have shown, the abuse by an expert of the power which he or she is given can cause serious harm and injustice.’
Lord Hodge, Justice of The Supreme Court, October 2017
Such have been the advances in advocacy training that most advocates nowadays will do a decent job of examining a witness of fact either in chief or by way of cross-examination. A thorough knowledge of the facts of the case, combined with careful preparation, will give the advocate the confidence and ability to bring out truth and to expose untruths.
When it comes to expert evidence, the landscape changes. Not only does the subject matter comprise material that often constitutes difficult ground for the advocate, but the experts’ opinions are themselves underpinned by what often appears to be impenetrable material. Many such experts have considerable presence and speak with great authority. A combination of all these factors provides the potential for the perfect storm whereby flawed opinion evidence is delivered to the court, it is not tested in a way that would unearth those flaws and a judgment is given based on that flawed evidence.
The Inns of Court College of Advocacy (ICCA) has drawn upon the expertise of seasoned, senior practitioners to create generic guidance for the assistance of all advocates involved in the preparation, admission and examination of expert evidence in any court or tribunal. This advice is compiled in the paper: Guidance on the preparation admission and examination of expert evidence. It has been distilled from principles and practices which are now well established across all areas of legal practice – civil, family, and criminal law, and covers all types of tribunal.
The guidance reminds advocates that, in addition to the relevant rules of court by which experts are bound, their expert might also be constrained by professional rules of practice or case-specific literature relevant to their own discipline. It is the responsibility of advocates to satisfy themselves that their expert is up-to-date and fully conversant with current regulations, research and good practice. Beware the expert who has retired from active practice.
To identify the central issues in dispute and expose the strengths and weaknesses of an opposing expert, early consultation with one’s own expert must be well-prepared and organised. The guidance contains much advice on this topic. Advocates must ensure that:
The importance of fully understanding every part of the evidence cannot be overstated. Without such an understanding, the advocate does not have the confidence or armoury with which to confront an expert witness. Add to this the fact that it is (rightly) drummed into the advocate from an early stage that no question should be asked to which he or she does not know the answer and the outcome is that identified by the Law Commission in its publication: Expert Evidence in Criminal Proceedings in England and Wales (Law Com No. 325, 2011) at para 1.21:
‘… cross-examining advocates tend not to probe, test or challenge the underlying basis of an expert’s opinion evidence but instead adopt the simpler approach of trying to undermine the expert’s credibility. Of course, an advocate may cross-examine as to credit in this way for sound tactical reasons; but it may be that advocates do not feel confident or equipped to challenge the material underpinning expert opinion evidence.’
Needless to say, a conference with one’s own expert is the time to explore in depth not only the base material but also the material underpinning the evidence of that expert and of the other side’s expert. ‘Why are you right and why are they wrong?’
In some areas of expert evidence, it becomes significantly harder for the advocate to fully understand the material underpinning the expert opinion. There is no better example than in the case of statistics. Few advocates have studied mathematics to a level that equips them with sufficient knowledge to readily understand statistics. The situation is made worse by the fact that, more often than not, an expert in a case has no background in statistics either. For example, a medical expert may cite statistical evidence gleaned from literature but have no concept of the reliability or true interpretation of that evidence. In those circumstances, the assistance that the advocate would normally wish to receive from his or her expert in understanding the material underpinning that expert’s or the opposing expert’s opinion is lacking.
It would be convenient to dismiss statistical evidence as affecting only a minority of cases mainly in the criminal field. If that were ever true, it is no longer so. Statistical evidence and probabilistic reasoning form part of expert witnesses’ testimony in an increasingly wide range of litigation spanning criminal, civil and family cases as well as more specialist areas such as tax appeals, sports law, and discrimination claims.
In recognition of this state of affairs, the ICCA and the Royal Statistical Society (RSS) have jointly published a booklet containing guidance to help advocates better understand the use of statistical evidence when presented in court. The guide is entitled: Statistics and probability for advocates: Understanding the use of statistical evidence in courts and tribunals. It provides a walk-through of the main concepts within the subject that advocates need to grasp.
Consider the following two sentences where ‘match’ refers to an agreement between a DNA profile obtained from a defendant and a DNA profile obtained from the evidence:
a) ‘The probability of a match if the semen came from another person is one in 40 million.’
b) ‘The probability that the semen came from another person is one in 40 million.’
How many people, including barristers and judges, would view sentence B as being a consequence of sentence A? Yet the two sentences are describing very different scenarios. To infer that sentence B is a consequence of sentence A, and that it reflects the chances of a defendant being innocent, is hugely erroneous.
"Few advocates have studied mathematics to a level that equips them with sufficient knowledge to readily understand statistics. The situation is made worse by the fact that, more often than not, an expert in a case has no background in statistics either"
The guide explains how such an error (‘the Prosecutor’s Fallacy’) can more easily be understood and thereby avoided. The guide also takes the reader through statistics and probability as they affect all areas of litigation. In doing so, it illustrates these topics with examples taken from miscarriages of justice in real life cases and/or from papers and textbooks on statistics. It looks at concepts such as:
It would be foolish to claim that the guide will arm its reader with an in-depth knowledge of what are, on any view, complex topics. It will, however, alert the advocate to pitfalls in relation to statistical (and epidemiological) evidence and, at the very least, to know when it would be appropriate to obtain the advice of a statistician.
To encourage practitioners to start testing their understanding and putting the guidance into practice, the booklet also provides four case studies in different areas of law. These are designed to incite the reader to plan how to question an expert in a variety of different situations and to scrutinise the methods used in presenting different kinds of conclusions based or allegedly based on statistics.
The guide also considers whether further activity may be required to improve the quality of scientific evidence to the degree identified as necessary by the Law Commission in its 2011 report. It signposts advanced thinking around current and controversial concepts (the likelihood ratio, Bayesian reasoning, the doubling of risk test and life expectancy estimates) for the purpose of warning advocates that this is a rapidly advancing subject whereby concepts currently accepted by the courts are likely to be challenged by statisticians.
The final chapter directs readers to discover more information on particular problems, and complements the reading lists included within each section of the guide. The resources offered reflect ICCA’s and RSS’ joint aim to encourage practitioners to develop their understanding of this complex but important topic and increase their confidence when they are conferring with experts, or challenging scientific evidence and expert opinion in court.
As science progresses, the role of expert evidence in litigation will only augment. As such, there will be an even greater weight of responsibility on the advocate to ensure that such evidence is properly tested and that the prospect of miscarriages of justice is minimised.
Contributor Colin McCaul QC, The Inns of Court College of Advocacy
From lab bench to judicial bench, the UK’s science academy is collaborating with the judiciary to ensure that the best scientific guidance is available to the courts. By Dr Julie Maxton
In a unique collaboration with the judiciary and the Royal Society of Edinburgh, the Royal Society launched the first of a series of succinct, jargon-free and easy to understand scientific guides, or primers, for judges. As the UK’s science academy we believe that the very best scientific advice should be available in all spheres of society, and that includes the courts.
The primers are designed to assist the judiciary when handling scientific evidence in the courtroom, explaining how far the science goes at any given time and what the limitations of the science are. They have been written by leading scientists and working judges and peer reviewed by legal practitioners, all of whom have generously volunteered their time to the project. It was Lord Chief Justice Thomas who first envisaged their development, and they have been steered to publication by Lord Anthony Hughes from the Supreme Court.
The first two primers cover DNA fingerprinting, widely used in court, and gait analysis, a relatively new form of evidence identifying people from the way they walk from CCTV. Future primers on statistics and the physics of vehicle collisions are planned. The project’s steering committee identifies topics by speaking to practising judges. It is absolutely vital that this project is demand-led and addresses practical trial-related questions which are relevant and important to judges themselves.
Dame Sue Black, a world expert in forensic anthropology, and criminal trial judge, His Honour Mark Wall QC, led the primer on forensic gait analysis, which was first admitted as evidence in a UK court in a case of armed robbery in 2000. It goes on to discuss the discipline’s evidence base, describing this as ‘sparse’, and making clear it is largely translated from more developed fields, such as clinical gait analysis, which provides information to health professionals. It recommends, ‘care is required in assuming that techniques developed in one field can be applied in another with quite different objectives’.
It also highlights that there is no evidence to support the assertion that the way people walk is unique, and explores the related issue of the variability shown in an individual’s gait pattern when factors such as fatigue, walking surface, footwear, injury or intoxication are taken into account. Finally, particularly as the use of forensic gait analysis comes more to the public’s attention, it concludes it is also possible that individuals may choose to walk differently to evade detection. The guide ends with a discussion of the different type of professionals that may present as gait analysts, and the diverse qualifications that they hold.
Professor Niamh Nic Daéid, a professor of forensic science at the University of Dundee and the Court of Appeal’s Lady Justice Anne Rafferty, led the DNA-themed primer. Its editorial board also included the expertise of Sir Alec Jeffreys, the inventor of genetic fingerprinting who in 1984 discovered a method of showing the variation in the DNA of individuals, and Nobel Prize-winning scientist Sir Paul Nurse.
The primer discusses the history of DNA profiling, the science of DNA analysis, factors to be considered in the evaluation of DNA evidence, including current limitations, and the future development of new DNA methods. Specific questions such as how DNA profiles are compared and interpreted are explored, with the authors discussing statistical approaches such as match probability and the likelihood ratio, and how they can very easily be misunderstood, known as the defence and prosecutor’s fallacy.
Hard copies of the primers have been distributed to courts in England and Wales, Scotland and Northern Ireland through the Judicial College, the Judicial Institute, and the Judicial Studies Board for Northern Ireland. The primers are also available to download from the Royal Society website.
The primers project is just one part of the collaboration between the scientific and judicial communities. With the Judicial College, the Royal Society has also hosted a series of seminars for senior judges on scientific and new technology topics relevant to court proceedings. Topics have included memory in testimony, uncertainty and probability, mental capacity, pain, substance addiction, and AI and machine learning. Future seminars will consider human gene editing, robotics, and causation. Further primer topics may well come out of some of these seminars.
Science and the law are of fundamental importance to society. As society changes and responds to fast-moving and far-reaching developments such as neuroscience and AI, both communities need to continue to work together to accommodate and understand the challenges of these new technologies.
Contributor Dr Julie Maxton CBE, Executive Director of the Royal Society, and a member of the primers steering group
Avoid the pitfalls of probabilistic reasoning and examine expert evidence with more confidence: Colin McCaul QC introduces new guidance from the Inns of Court College of Advocacy and the Royal Statistical Society
‘No practising lawyer should underestimate the difficulty involved in preparing and mounting an effective challenge to a well-prepared expert’s evidence by cross-examination, even when assisted by his or her client’s own expert.
The beginning of the legal year offers the opportunity for a renewed commitment to justice and the rule of law both at home and abroad
By Louise Crush of Westgate Wealth Management sets out the key steps to your dream property
A centre of excellence for youth justice, the Youth Justice Legal Centre provides specialist training, an advice line and a membership programme
By Kem Kemal of Henry Dannell
By Ashley Friday of AlphaBiolabs
Providing bespoke mortgage and protection solutions for barristers
Joanna Hardy-Susskind speaks to those walking away from the criminal Bar
From a traumatic formative education to exceptional criminal silk – Laurie-Anne Power KC talks about her path to the Bar, pursuit of equality and speaking out against discrimination (not just during Black History Month)
Yasmin Ilhan explains the Law Commission’s proposals for a quicker, easier and more effective contempt of court regime
Irresponsible use of AI can lead to serious and embarrassing consequences. Sam Thomas briefs barristers on the five key risks and how to avoid them
James Onalaja concludes his two-part opinion series
