Expert Systems Help Labs Process DNA Samples
The products, manufacturers and organizations discussed in this article are presented for informational purposes only, and
their discussion does not constitute product approval or endorsement by the U.S. Department of Justice.
Criminal justice professionals who work in the field of DNA analysis know that a backlog of convicted offender samples exists
in our nation's laboratories. It takes a long time to analyze a DNA sample of a convicted offender. Two forensic analysts
must visually review the sample and apply a set of standard operating procedures that can have many sets of rules. The procedures
can be difficult to apply consistently. After the review is complete, the data must be entered into the national database.
Completing all the steps quickly is a formidable challenge.
New software programs called "expert systems" are helping increase the speed of the review process. Expert systems capture
all possible circumstances that experts encounter when they do their jobs and dictate what the appropriate responses should
be. For forensic analysts, expert systems not only allow them to get consistent, accurate results more quickly, they also
help them review and upload many DNA profiles into the national database faster.
But can these expert systems help reduce the backlog of convicted offender samples that exists in our nation's laboratories?
According to the results of an evaluation by the National Institute of Justice (NIJ), they can. The NIJ Expert Systems Testbed
(NEST) project, hosted by the Marshall University Forensic Science Center, evaluated the ability of three commercially available
expert systems, designed specifically for forensic DNA laboratories, to rapidly and accurately review convicted offender single-source
DNA samples for eventual upload into the national DNA database. Project researchers — including this author — found that the
three programs will help reduce the backlog and ensure timely submission of data into the national database so ultimately
more crimes can be solved.,
What Are Expert Systems?
Expert systems are a subset of artificial intelligence. They store knowledge on how to respond to a particular result and,
when a challenge is presented, use the stored knowledge in the program to respond with an explanation. The system, however,
cannot completely replace a human expert because it contains only rules to solve most commonly encountered problems. The system
must be able to recognize cases that are outside its rules and scope of knowledge or those in which there is a possible alternate
judgment and alert a human expert reviewer.
Expert systems are developed with the close collaboration of the software development team and forensic science experts. Developers
create the system by first conducting extensive interviews with experts in the field where the software will be used, writing
algorithms to apply to the data and testing the programs with data appropriate for the software. The knowledge from these
experts is then taken and applied consistently to the data review process by using the algorithms, automating the analysis
and freeing analysts to focus on more complex challenges.
Use of this technology continues to increase in several industries as they become more aware of its benefits — for instance,
expert systems are assisting physicians in making medical diagnoses, supporting NASA's space program and managing inventories
for large factories. They serve as scheduling and planning tools when the everyday consumer makes an online airline reservation.
A search engine finds all possible airline companies, determines the routes based on particular requests and attempts to find
the requested time frame. Once the consumer chooses a particular flight plan, the system can determine the class of travel,
price and seat assignments.
For forensic DNA analysis, expert systems could easily be one of the most important advances in analyzing convicted offender
samples. By reviewing routine data rapidly and accurately, the system will allow analysts to focus on those samples with problems
or "flags." Also, because DNA analysis requires considerable training and experience, software that interprets information
with little or no work by the forensic analyst is a noteworthy advancement for the DNA community.
Rapid, Accurate Analysis
Our evaluation focused on the technical review of software programs that:
- Are publicly available for purchase.
- Are configurable, off-the-shelf software.
- Are housed in a laboratory.
- Can be used by people who are unfamiliar with computer code.
- Meet technical specifications outlined in the forensic standards.
We evaluated three software packages that met these criteria:
- GeneMapper® ID v. 3.2 (Applied Biosystems, Foster City, Calif.).
- FSS-i3™ (Promega Corporation, Madison, Wis.), used with GeneMapper® ID v. 3.2 for peak detection and sizing.
- TrueAllele® System 2 (Cybergenetics, Pittsburgh).
We evaluated every step involved from buying the programs to running them. Our evaluation examined the time it takes to import
data, run the application and complete the analysis. We also critically checked the flags, rules and features available with
each program. We evaluated their ability to run without assistance by the analyst and to flag the samples needing human expert
review. We further looked at the customer service and training provided by the vendors' technical staff.
Our team found that the three expert systems evaluated for single-source samples will help reduce the DNA convicted offender
backlog. We determined that each is able to rapidly and accurately conduct routine reviews (reviews of DNA samples without flags)
and can significantly reduce the time spent in the human review of DNA profiles. When implementing one of these expert systems,
only one analyst is needed to accept the good quality samples (with no expert system in place, two analysts are needed during
a review), thereby reducing labor hours by half.
We determined, however, that the systems cannot fully replace a human expert or DNA analyst. The systems can be used as tools
to evaluate single-source DNA samples and to alert the DNA analyst when results do not meet defined rules or there is a possible
alternate judgment. But the final decision-making process in cases when results raise flags must remain in the hands of the
trained DNA analyst.
As part of our evaluation, we also presented the features and limits of each system to help forensic analysts determine which
program will most benefit their laboratories and to aid managers when deciding whether to purchase or subscribe to expensive
software. Here is a brief summary of each expert system that we evaluated.
GeneMapper® ID Software v. 3.2
We found the purchase of GeneMapper ID straightforward. The laboratory in which we evaluated the product purchases only software from the vendor; hardware is purchased
separately. The consumer determines the number of licenses required by the laboratory and can self-install the software from
a CD. The vendor provides a one-day on-site training on setup and basic software tools with purchases as well as technical
support for telephone and electronic questions and free periodic webinars. More extensive training can be purchased. Our evaluation
found many features in GeneMapper ID to be intuitive and the sizing and genotyping analysis to be simple and effective.
FSS-i3™ Expert Systems Software
With FSS-i3, hardware is purchased separately; the laboratory purchases only software from the vendor. The vendor offers on-site or company-site
training at a separate cost. The five-day sessions include installation, optimization and training. The vendor also offers
technical support for telephone and electronic questions. Overall, we found the operation of the software straightforward.
TrueAllele® System 2
With TrueAllele, hardware is included with the software. The laboratory can renew its support and updates annually. The vendor
provides two company-site training sessions: a two-day executive training session for laboratory managers and administrators
and a four-day training session for scientists using the software. The vendor also provides technical support for telephone
and electronic questions. During our evaluation, we found that we had a steeper learning curve for TrueAllele, but once we
understood the setup, it was fairly straightforward and the analysis of the size standard and samples was simple and effective.
It is not the objective of the NEST project to identify the single best expert system for a laboratory. We recognize that
crime laboratories are different, and each laboratory should consider its specific needs when choosing an expert system. Instead,
the goal of the NEST project is to continue to give managers and analysts the tools they need to determine which expert system
would best fit their laboratory and ultimately help reduce the backlog of convicted offender samples across the country so
more crimes can be solved.
NIJ Journal No. 260, July 2008
About the Author
Rhonda Roby has been the NEST project technical director for NIJ since the project's inception in May 2005. For the past eight years,
she has worked with software development teams and evaluated expert systems for forensic DNA analysis. Roby also conducts
research for the Missing Persons Program at the University of North Texas Center for Human Identification, where she focuses
on automation, development of assays and procedures, and mitochondrial DNA research. Back to the top.
[note 1] This article discusses the results from phase 1 of the project, which evaluated software for use with single-source DNA samples.
We are now assessing software that can handle degraded DNA and mixtures that include DNA samples from more than one person.
Results of phase 2 of the project are expected in 2009.
[note 2] The goal of the NEST project is to evaluate, not validate, expert systems software. Validation refers to formal testing that
must be completed before laboratories can upload genetic profile information to the national database. Some of these software
programs have been validated and approved by the National DNA Index System (NDIS). They are now in use at the Florida Department
of Law Enforcement and New York State Police. According to the standards, the specific combination of instrument, chemistry
and software must be developmentally validated. Once NDIS validates and approves a particular combination, other laboratories
are required to perform internal validation only if they adopt the same combination.
[note 3] The forensic standards for an expert system call for specific criteria that address different functions.
[note 4] Other software packages are currently available that were not on the market when we began the evaluation. The new programs
are worthy of serious consideration.
[note 5] Before a laboratory can adopt any expert system into its processes for submission of data into NDIS or the Combined DNA Index
System applications, the system must first receive the approval of the NDIS board.
Date Created: July 16, 2008