Like many Americans, we were riveted by “Making a Murderer,” Netflix’s true-life series about Steven Avery, a Wisconsin man who served 18 years in prison for sexual assault, was exonerated of that crime and released, then arrested and convicted for the 2005 murder of photographer Teresa Halbach. Avery’s nephew, Brendan Dassey, also was convicted of the murder.

The series has raised questions about the actions and motives of law enforcement. Avery was exonerated of the sexual assault because of DNA evidence and the help of the Wisconsin Innocence Project. Now, DNA is an issue in efforts to overturn his murder conviction.

We have a professional interest in forensic science. We are academics who have been participating in a year-long evaluation of forensics sponsored by the Statistical and Applied Mathematical Sciences Institute with funding from the National Science Foundation. Among other issues, participants are studying fingerprint and shoe print quality and examiner bias. 

In the evidence against Steven Avery, we saw striking examples of problems we have been studying.

• • • 

According to a 2009 report of the National Academy of Sciences, many forensic science methods are inadequately validated, which means they have not been sufficiently tested to establish how well they work and how often and under what conditions they fail. Avery’s case provides a dramatic example of the uncertainty and confusion that can arise when experts rely on such methods, especially when the court allows the findings to be introduced as scientific evidence at trial.

Among the prime scientific issues in the Avery case was whether blood stains found in Halbach’s car contain a chemical preservative known as EDTA, which is placed in vials of preserved blood.

DNA tests indicated the blood in Halbach’s car was Avery’s. The presence of EDTA would support Avery’s claim that the blood came from a vial that the police had collected from him in the sexual assault case. The absence of EDTA would support the prosecution’s claim that the blood came directly from Avery at the time of the murder.

At the request of the prosecution, the FBI quickly developed a test to detect EDTA in dried blood. FBI chemist Marc LeBeau reported that this test had failed to detect EDTA in three of the six stains. Based on this, he concluded “within a reasonable degree of scientific certainty” that EDTA was absent from all six stains even though three of them were never tested.

But how often and under what conditions would the FBI’s test have detected (or failed to detect) EDTA in dried blood taken from old vials under conditions like those in question?

Without extensive research and analysis of this question — research the FBI could not have conducted in the limited time available — it is difficult to assess the validity of Mr. LeBeau’s conclusion.

There is a saying among scientists that absence of evidence isn’t necessarily evidence of absence, and that appears to be the case here. If the quantity of EDTA present in the stains was near the threshold of detection, it might have been detected in some of the stains but not others. In fact, if only one of the samples had EDTA above the threshold of detection, there was only a 50 percent chance that it would have been detected in the three analyzed samples.

Hence it was problematic for Mr. LeBeau to draw conclusions with any scientific certainty about all six of the stains after testing only three of them. A defense expert, Janine Arvizu, pointed out the problem, but we wonder whether this “battle of experts” left the jury with an inadequate appreciation of the underlying scientific and statistical uncertainties.

• • • 

Another common problem in forensic science is contextual bias — the tendency for forensic scientists to be influenced inappropriately by nonscientific aspects of a case.

To avoid bias, forensic scientists sometimes “blind” themselves to information that is not needed for their tests, such as information about what the police expect and hope they will find. This did not happen in the Avery case. DNA analyst Sherry Culhane had written in her laboratory notes that she should “try to put her [Halbach] in his [Avery’s] house or garage.” Ms. Culhane claimed that this information “had no bearing on my analysis at all.”

But this claim is difficult for us to accept. There is a large body of research showing that human judgments, including judgments of scientific experts, can unknowingly be influenced by contextual information.

Indeed, bias may explain Ms. Culhane’s decision to disregard the failure of a critical scientific control when testing a bullet that was allegedly found in Avery’s garage.

She reported finding traces of Halbach’s DNA on the bullet, but she also found traces of  DNA (her own!) in control samples that were supposed to contain no DNA. Because this indicates a serious problem, Ms. Culhane’s laboratory protocols required her to disregard the findings and run the test again, which was impossible because she used up the sample in her first test. We wonder whether she would have decided to ignore her own protocol and report the highly problematic results of this test had she not known of the pressing desire of the police to “put” Halbach in Avery’s garage.

• • • 

Like Ms. Culhane, Mr. LeBeau was well aware of what the police and prosecutors wanted him to find when he devised and conducted his analysis of the EDTA. The letter requesting the analysis stated that its purpose was “eliminating the allegation that his vial was used to plant evidence.”

His conclusions would have been more convincing had he reached them without knowing the consequences for Avery’s case in advance. Someone at the FBI should have intervened between the police and the scientific staff, so that the questions posed for scientists could have been stated in a neutral manner that did not indicate the preferred outcome.

These problems are not unique to the Avery case.

According to the Innocence Project, about half of DNA-related exonerations resulted from the authorities’ misapplication of forensic science; one analysis found that forensic scientists overstated their findings in more than 60 percent of cases.

We see inadequate validation and potential issues of bias frequently in the forensic science evidence we examine. But the Avery case illustrates dramatically why it is important that these problems be addressed. When the liberty of a human being is on the line, the scientific evidence on which we rely must be as valid and unbiased as possible.

Lucas Mentch (lmentch@samsi.info) is an assistant professor of statistics at the University of Pittsburgh, currently on leave to participate in the 2015-2016 SAMSI program dealing with statistics in forensic science. Maria Cuellar (mcuellar@cmu.edu) is a doctoral student in statistics and public policy at Carnegie Mellon University. William C. Thompson (william.thompson@uci.edu) is professor of criminology, law and society at the University of California, Irvine. Clifford Spiegelman (cliff@stat.tamu.edu) is distinguished professor of statistics at Texas A&M University.

Editor’s note, March 14: An earlier version of the story misrepresented the number of DNA-related  exonerations that resulted from the authorities’ misapplication of forensic science.

First Published: March 13, 2016, 5:00 a.m.