Questioning the Reliability of DNA Testing

The LA Times runs what it wants to think of as a serious investigative piece on the reliability of current DNA testing practices. And while there’s plenty of investigation, it’s also plenty alarming:

State crime lab analyst Kathryn Troyer was running tests on Arizona’s

DNA database when she stumbled across [two felons with remarkably

similar genetic profiles]2.

The men matched at nine of the 13 locations on chromosomes, or loci, commonly used to distinguish people.

The FBI estimated the odds of unrelated people sharing those genetic

markers to be as remote as 1 in 113 billion. But the mug shots of the

two felons suggested that they were not related: One was black, the

other white.

In the years after her 2001 discovery, Troyer found dozens of similar matches — each seeming to defy impossible odds.

As word spread, these findings by a little-known lab worker raised

questions about the accuracy of the FBI’s DNA statistics and ignited a

legal fight over whether the nation’s genetic databases ought to be

opened to wider scrutiny.

But here’s the thing: A nine-loci “match” (because forensic scientists don’t use those terms in court and are cautioned against ever using those terms) is not and can never be as statistically significant as a 13-loci match. And the way Troyer ran her searches created results that don’t tell the whole story:

Bureau officials say critics have exaggerated or misunderstood the implications of Troyer’s discoveries.

Indeed, experts generally agree that most — but not all — of the

Arizona matches were to be expected statistically because of the

unusual way Troyer searched for them.

In a typical criminal case, investigators look for matches to a

specific profile. But the Arizona search looked for any matches among

all the thousands of profiles in the database, greatly increasing the

odds of finding them.

As a result, Thomas Callaghan, head of the FBI’s CODIS unit, has dismissed Troyer’s findings as “misleading” and “meaningless.”

In other words, Troyer wasn’t looking for one needle in a haystack, but searched all haystacks for all possible needles and then was “shocked” to discover the presence of more than one needle in a haystack. Which isn’t to say that DNA testing standards should remain at the status quo, especially as techniques improve and detection levels grow ever smaller, and the FBI (and other state and local crime labs) figure out how to improve interpretation and minimize what are often catastrophic errors with forensic evidence. But there is an easy, albeit time-consuming from a validation standpoint, way to get around this: bump up the number of loci required for statistical significance.