How the flawed ‘science’ of bite mark analysis has sent innocent people to prison

This is an excerpt from an excellent four part series on bitemark evidence. You can access the full story at this link.

Excerpt from Part 3 of 4:

In 2007 Mary and Peter Bush, a married couple who head up a team of researchers at the State University of New York at Buffalo, began a project to do what no one had done in the three decades — conduct tests to see whether there’s any scientific validity to the bite mark evidence presented in courts across the United States.

The Bushes sought to test the two main underlying premises of bite mark matching — that human dentition is unique and that human skin can record and preserve bite marks in a way that allows for analysts to reliably match the marks to a suspect’s teeth. The Bush team was the first to apply sophisticated statistical modeling to both questions. It was also the first to perform such tests using dental molds with human cadavers. Previous tests had used animal skins.

When they first set out on the project, the Bushes received preliminary support from some people in the bite mark analyst community. “Franklin Wright was the ABFO president at the time,” says Mary Bush. “He visited our lab, and then put up a message praising our work on the ABFO website.” They also received a small grant from the ASFO, the discipline’s non-accrediting advocacy and research organization.

“There was a lot enthusiasm at the outset,” says Fabricant. “I think some analysts were excited about the possibility of getting some scientific validation for their field.”

But when the Bushes began to come back with results that called the entire discipline into question, that support quickly dried up.

The Bushes’ research found no scientific basis for the premise that human dentition is unique. They also found no support for the premise that human skin is capable of recording and preserving bite marks in a useful way. The evidence all pointed to what critics such as Bowers had always suspected: Bite mark matching is entirely subjective. The Bushes’ first article appeared in the January 2009 issue of the Journal of Forensic Sciences. The couple have since published a dozen more, all in peer-reviewed journals.

Outside of ABFO and their supporters, the Bushes’ research has been lauded. “I think there’s a chance that because of the Bushes’ research, five years from now we aren’t going to be talking about bite mark evidence anymore,” says Risinger. “It’s that good. Their data is solid. Their methodology is solid. And it’s conclusive.”

Other legal scholars and experts on law and scientific evidence interviewed for this article shared Risinger’s praise for the Bushes’ research but were less optimistic about its implications, in part because the criminal justice system so far hasn’t recognized the significance of their work.

But from a scientific standpoint, the Bushes’ research was a direct and severe blow to the credibility of bite mark analysis. At least initially, it threatened to send the entire field the way of voice print matching and bullet lead analysis, both of which have now been discredited. And so when defense attorneys began asking the couple to testify in court, the bite mark analysts fought back with a nasty campaign to undermine the Bushes’ credibility. In a letter to the editor of the Journal of Forensic Sciences, seven bite mark specialists joined up to attack the Bushes in unusually harsh terms for a professional journal. When that letter was rejected for publication, five of the same analysts wrote another. That, too, was rejected. A toned-down but still cutting third letter was finally published.

In the unpublished letter dated November 2012, the authors — all bite mark analysts who hold or have held positions within ABFO — declared it “outrageous that any of these authors would go into courts of law and give sworn testimony citing this research as the basis for conclusions or opinions related to actual bite mark casework, especially considering that no independent research has validated or confirmed their methods or findings.”

Of course, critics would say this was a bit of rhetorical jujitsu — that the last clause could describe exactly what bite mark analysts have been doing for 35 years. For emphasis they added, “This violates important principles of both science and justice.” In the other letter, the authors referred to the Bushes’ testimony in an Ohio case, which was based upon their research, as “influenced by bias” and “reprehensible and inexcusable.”

The primary criticism of the Bushes’ research is that they used vice clamps to make direct bites into cadavers that were stationary through the entire process. This is quite a different scenario than the way a bite would be administered during an attack. During an assault, the victim would probably be pulling away, causing the teeth to drag across the skin. For the Bush tests, the clamp they used to make the bites moved only up and down. A human jaw also moves side to side. A biter might also twist his head or grind his teeth. A live body will also fight the bite at the source to prevent infection, causing bruising, clotting and various other defenses that would alter the appearance of the bite.

“We acknowledge that our lab tests are different from how bites are made in the real world,” says Mary Bush. “But to the extent that our tests differed, they should have made for better preserved samples.”

In other words, the tests that the Bushes conducted made for cleaner, clearer bites that could be easily analyzed. If they were in error, they were in error to the benefit of the claims of bite mark analysts. And they still found no evidence to support the field’s two basic principles.

“That’s exactly right,” says Risinger. “If there was any validity to bite mark analysis at all, these tests would have found it. They gave the field the benefit of the doubt. The evidence just wasn’t there. Their data is very, very strong.”

To argue that the Bushes’ experiments should be disregarded because they weren’t able to replicate real-world bites is also an implicit acknowledgment that real-world bites aren’t replicable in a lab, and therefore aren’t testable. You won’t find many people volunteering to allow someone else to violently bite them for the purposes of lab research. Even if you could, a volunteer won’t react the same way to a bite that an unwitting recipient might.

The Bushes’ research not only failed to find any scientific support for bite mark matching, but it also exposed the fact that for four decades the bite mark community neglected to conduct or pursue any testing of its own. It put the ABFO and its members on the defensive. The bite mark analysts responded by intensifying their attacks on the couple and making the attacks more personal.

At the February 2014 AAFS conference in Seattle, the ABFO hosted a dinner for its members. The keynote speaker was Melissa Mourges, an assistant district attorney in Manhattan, one of the most outspoken defenders of bite mark matching in law enforcement.

Mourges already had a high profile. The combative, media-savvy prosecutor was part of the prosecution team featured in the HBO documentary “Sex Crimes Unit,” which followed the similarly named section of the Manhattan DA’s office, the oldest of its kind in the country. Mourges herself founded a cold-case team within that unit. At the 2012 AAFS conference she spoke on a panel called “How to Write Bestselling Novels and Screenplays in Your Spare Time: Tips From the Pros.” At this year’s conference, she’ll be on a panel that’s titled “Bitemarks From the Emergency Room to the Courtroom: The Importance of the Expert in Forensic Odontology.” She’ll be co-presenting with Franklin Wright, the former ABFO president who initially supported the Bushes’ research.

Mourges was also the lead prosecutor in State v. Dean, a New York City murder case in which the defense challenged the validity of the state’s bite mark testimony. In 2013, Manhattan state Supreme Court Judge Maxwell Wiley held a hearing on the scientific validity of bite mark evidence. Mary Bush testified about the couple’s research for the defense. It was the first (and so far the only) such hearing since the NAS report was released, and both sides of the bite mark debate watched with anticipation. In September 2014, Wiley ruled for the prosecution, once again allowing bite mark evidence to be used at trial. (I’ll have more on the Dean case in part four of the series.) Mourges’s talk at the ABFO dinner was basically a victory lap.

There’s no transcript of Mourges’s speech, but those in attendance say it was basically a no-holds-barred attack on Mary Bush. Cynthia Brzozowski has been practicing dentistry in Long Island for 28 years and sits on the ABFO Board of Directors. She practices the widely accepted form of forensic dentistry that uses dental records to identify human remains, but she doesn’t do bite mark matching, and she won’t testify in bite mark cases. Brzozowski was at the dinner in Seattle and says she still can’t believe what she heard from Mourges.

“Her tone was demeaning,” Brzozowski says. “It would be one thing if she had just come out and presented the facts of the case, but this was personal vitriol against the Bushes because of their research.”

According to Brzozowski, Mourges even went after Mary Bush’s physical appearance. “At one point, she put up an unflattering photo of Mary Bush on the overhead. I don’t know where she got it, or if it had been altered. Mary Bush is not an unattractive person. But it was unnecessary. You could hear gasps in the audience. It was clear that she had chosen the least flattering image she could find. Then she said, ‘And she looks better here than she does in person.’ It was mean. I had to turn my back. I was mortified.”

Other ABFO members — including two other members of the board of directors — also complained, to both the ABFO and the AAFS. The complainants described Mourges’s attack on Bush as “malicious,” “bullying” and “degrading.” According to accounts of those in attendance, other members were also upset by Mourges’s remarks but didn’t file formal complaints for fear of professional retaliation.

A few weeks later, Loomis sent an e-mail to the ABFO Board of Directors to address the complaints. Loomis defended Mourges and her presentation. He described the dinner as a “convivial affair” where members can socialize, have a libation and “be entertained” by the invited speaker. He argued that “anyone who understands litigation” should not have been unsettled by the talk and described the presentation as “sarcastic, serious, and even light-hearted.” He stood by the decision of his predecessor, Greg Golden, to invite Mourges, calling it “a good decision,” adding, “I apologize to those who were offended. However, I do not apologize for the message.”

“‘Bullying’ is exactly what it is,” says Peter Bush. “We’re scientists. We’re used to collegial disagreement. But we had no idea our research would inspire this kind of anger.”

. . .

Addendum: After this post was published, the office of Manhattan District Attorney Cyrus Vance sent the following statement:

Melissa Mourges is a veteran prosecutor and a nationally recognized leader in her field. As Chief of the Manhattan District Attorney’s Forensic Science/Cold Case Unit, she has solved dozens of cold case homicides, including two recently attributed to “Dating Game” serial killer Rodney Alcala. In addition to being a Fellow at the American Academy of Forensic Sciences, ADA Mourges has also served as co-chief of the DNA Cold Case Project, which uses DNA technology to investigate and prosecute unsolved sexual assaults. As part of that work, she pioneered the use of John Doe indictments to stop the clock on statutes of limitation and bring decades-old sexual assaults to trial. Her work and reputation are impeccable, and her record speaks for itself.

Excerpt from Part 4 of 4:

The most significant challenge to bite mark evidence since the NAS report was released came in State v. Dean, the New York case mentioned in part three of this series. In 2013, attorneys for defendant Clarence Dean challenged the prosecution’s plan to use bite mark evidence against their client. Manhattan state Supreme Court Judge Maxwell Wiley granted a hearing to assess the validity of bite mark matching. It was the first such hearing since the NAS report was published, and both sides of the bite mark debate watched closely. Mary Bush testified for the defense, as did Karen Kafadar, chair of the statistics department at the University of Virginia and a member of the National Institute of Standards and Technology’s Forensic Science Standards Board.

The prosecutor in that case was Manhattan assistant district attorney Melissa Mourges, an aggressive 30-year prosecutor with a high profile. Mourges was featured in a 2011 HBO documentary and holds the title of chief of the District Attorney’s Forensic Science/Cold Case Unit in what is arguably the most influential DA’s office in the country. So her advocacy for bite mark matching is significant.

As reported in part three, Mourges has not only defended bite mark evidence but also seems to be on a campaign to denigrate its critics, going so far as to heckle scientific researchers Mary and Peter Bush at a panel, and then to personally attack Mary Bush during a dinner talk at a forensics conference. Her bite mark brief in the Dean case compared bite mark evidence critic Michael Bowers to the notorious bite mark charlatan Michael West. It was a particularly egregious comparison because Bowers had helped expose West back when he was still embraced by the ABFO.

In her brief, Mourges first encouraged Wiley to embrace the “soft science” approach to bite mark analysis used by the Texas court in Coronado. Conveniently, doing so would allow bite mark specialists to testify to jurors as experts with almost no scrutiny of their claims at all.

Mourges next argued that if the court must do an analysis of the validity of bite mark testimony, it do so on the narrowest grounds possible. When it comes to assessing the validity of scientific evidence, New York still goes by the older Frye standard, which states that evidence must be “generally accepted” by the relevant scientific community. The question then becomes: What is the relevant scientific community?

In her brief, Mourges urged Wiley to limit that community to analysts who “have actually done real-world cases.” In other words, when assessing whether bite mark matching is generally accepted within the scientific community, Mourges says the only relevant “community” is other bite mark analysts.

Saks offers a metaphor to illustrate what Mourges is asking. “Imagine if the court were trying to assess the scientific validity of astrology. She’s saying that in doing so, the court should only consult with other astrologers,” he says. ”She’s saying the court shouldn’t consult with astronomers or cosmologists or astrophysicists. Only astrologers. It’s preposterous.”

Saks, who submitted a brief in the case on behalf of Dean, also offers a real-world example: the now-discredited forensic field of voiceprint identification. The FBI had used voiceprinting in criminal cases in the 1970s but discontinued the practice after an NAS report found no scientific support for the idea that an expert could definitively match a recording of a human voice to the person who said it.

“If you look at the Frye hearings on voiceprint identification, when judges limited the relevant scientific community to other voiceprint analysts, they upheld the testimony every time,” Saks said. “When they defined the relevant scientific community more broadly, they rejected it every time. It really is all about how you define it.”

In urging Wiley to only consider other bite mark analysts, Mourges also casts aspersions on the scientists, academics and legal advocates urging forensics reform. She writes:

The make-up of the relevant scientific community is and should be those who have the knowledge, training and experience in bitemark analysis and who have actually done real world cases. We enter a looking-glass world when the defense urges that the Court ignore the opinions of working men and women who make up the ranks of board-certified forensic odontologists, who respond to emergency rooms and morgues, who retrieve, preserve, analyze and compare evidence, who make the reports and who stand by their reasoned opinions under oath. The defense would instead have this Court rely on the opinions of statisticians, law professors and other academics who do not and could not do the work in question.

Of course, one needn’t practice astrology or palm reading to know that they aren’t grounded in science. And if police and prosecutors were to consult with either in a case, we wouldn’t dismiss critics of either practice by pointing out that the critics themselves have never read a palm or charted a horoscope.

Mourges also attempts to both discredit the NAS report and claim that it isn’t actually all that critical of bite mark analysis. For example, she laments that the report was written by scientists and academics, not bite mark analysts themselves. This, again, was entirely the point. The purpose of the NAS report was to research the scientific validity of entire fields. If it were written by active practitioners within those fields, every field of forensics would have been deemed valid, authoritative and scientifically sound.

Mourges also misstates and mischaracterizes what the report actually says. She writes in one part of her brief that “the NAS report does not state that forensic odontology as a field should be discredited.” That’s true. But bite mark matching is only one part of forensic odontology. The other part, the use of dental records to identify human remains, is widely accepted. What the report makes abundantly clear is that there is zero scientific research to support bite mark analysis in the manner it is widely practiced and used in courtrooms.

In another portion of the brief, Mourges selectively quotes part of the the report, cutting out some critical language. She writes:

When Dr. Kafadar and her NAS committee created the NAS report, they wrote a summary assessment of forensic odontology. In it they said that “the majority of forensic odontologists are satisfied that bite marks can demonstrate sufficient detail or positive identification …

That ellipsis is important, as is the word that comes before the quote. Here’s the passage quoted in full:

Although the majority of forensic odontologists are satisfied that bite marks can demonstrate sufficient detail for positive identification, no scientific studies support this assessment, and no large population studies have been conducted. In numerous instances, experts diverge widely in their evaluations of the same bite mark evidence, which has led to questioning of the value and scientific objectivity of such evidence.

Bite mark testimony has been criticized basically on the same grounds as testimony by questioned document examiners and microscopic hair examiners. The committee received no evidence of an existing scientific basis for identifying an individual to the exclusion of all others.

The report only acknowledges the near consensus within the community of bite mark analysts for the purpose of criticizing them. Mourges’s selective quotation implies that the report says the relevant scientific community accepts bite mark matching. The full passage reveals that the report is essentially pointing out just the opposite: The insular community of bite mark analysts may believe in what they do, but the larger scientific community is far more skeptical.

One common tactic that shows up in Mourges’s brief and has also shown up in defenses of bite mark analysis across multiple forums — court opinions, forensic odontology journals and public debates — is a sort of meticulous recounting of the care and precision into which bite mark analysts collect and preserve evidence as well as the scientific-sounding nomenclature used by the field’s practitioners. Mourges devotes more than 10 pages to laying out the procedures, methods and jargon of bite mark matching.

In any field of forensics it’s of course important that evidence be carefully handled, properly preserved and guarded against contamination. But to go back to the astrology metaphor, even the most careful, conscientious, detail-oriented astrologer . . . is still practicing astrology. If the field of bite mark analysis cannot guarantee reliable and predictable conclusions from multiple practitioners looking at the same piece of evidence, if it cannot produce a margin for error, if its central premises cannot be proved with testing, then it doesn’t matter how pristine the bite mark specimens are when they’re analyzed or what the mean number of syllables may be in each word of a bite mark analyst’s report.

But ultimately, Mourges was effective. In September 2013, Wiley rejected the defense challenge to bite mark evidence in the Dean case. He never provided a written explanation for his ruling. In an e-mail, Joan Vollero, director of communications for the Manhattan District Attorney’s Office, wrote of the ruling: “Following the months-long Frye hearing, Judge Wiley denied the defendant’s motion to preclude the bite mark evidence, finding that the field of bite mark analysis and comparison comports with New York State law.”