Social Science Research on Forensic Science Topical Working Group Meeting

Washington, DC
January 23–24, 2013

Meeting Overview

On January 23–24, 2013, the National Institute of Justice (NIJ) held a working-group meeting to discuss the role of forensic evidence from a social science perspective.

Greg Ridgeway, NIJ’s Acting Director, welcomed participants to the meeting. He briefly discussed the 2009 National Research Council report, Strengthening Forensic Science in the US:  A Path Forward, and noted that as NIJ continues its work in the role of social sciences in forensics, innovation will be key.

Katharine Browning, Ph.D., a senior social scientist at NIJ who chaired the meeting, said the mission of the meeting was to get input from the field regarding current trends and research questions that could be studied to help the criminal justice system use its resources — including forensic resources — as effectively and efficiently as possible.

The list of attendees appears at the end of this document.

Historical Overview of NIJ’s Forensics in Social Science Portfolio

Dr. Browning provided a brief historical overview of NIJ’s social science and forensics portfolio, reporting that NIJ launched a forensics in social sciences program in 2004 to examine research questions such as:

  • Are advances in forensics furthering justice?
  • How can forensic evidence be used as effectively and efficiently as possible to increase the reliability of criminal justice outcomes?
  • What are the potential ramifications of new policies and procedures?
  • What impact do forensic advances have on law enforcement, the courts, and corrections?

One of NIJ’s first major studies was a field experiment on the use of DNA in property crimes. For more on the findings from the study, see DNA Solves Property Crimes: But Are We Ready for That? an article in the NIJ Journal, and the Urban Institute’s final report (pdf, 164 pages) on the project.

In 2006, NIJ funded two studies that examined the role and impact of forensic evidence in the criminal justice process. The first was conducted by Joseph Peterson, Ph.D., and colleagues at California State University, Los Angeles; the final report, The Role and Impact of Forensic Evidence in the Criminal Justice Process (pdf, 151 pages). The second study was conducted by Tom McEwen, Ph.D.; the final report, also titled The Role and Impact of Forensic Evidence in the Criminal Justice Syste (pdf, 130 pages).

In 2009, NIJ published a survey of law enforcement agencies that looked at the processing of forensic evidence and how it was used to improve investigations. That study was performed by Kevin J. Strom and his colleagues at RTI International; learn more about The 2007 Survey of Law Enforcement Forensic Evidence Processing.

NIJ faced a number of challenges in developing this research portfolio. Few researchers examined forensics from a social science perspective, generally lacking expertise in both the social sciences and the forensic sciences. In general, there was a lack of baseline data upon which to build a scientific research portfolio. And, too often, advocacy groups framed the issues surrounding the collection and use of forensic data, resulting in narrowly focused agendas. Mindful of its role in the administration of justice and protecting the integrity of science, NIJ has widened that lens on the issues, looking more broadly at forensics and its relationship to social science concerns.

See the slides from Dr. Browning’s presentation (pdf, 18 pages).

Forensic Blueprint for Law Enforcement Overview

Presenters: Kelly Walsh and John Roman
Urban Institute, Justice Policy Center

John Roman and Kelly Walsh of the Urban Institute’s Justice Policy Center (Urban) gave an overview of the preliminary results of an ongoing NIJ-funded project to develop a “Forensic Blueprint” for law enforcement. The study involved an online survey of investigators, police chiefs, lab analysts, lab directors, and prosecutors, focusing on practices, cases, opinions, and perceptions regarding the processing and use of forensic evidence. Although the results are preliminary, Urban has identified 40–50 “roadblocks,” or barriers, to using DNA more effectively in the criminal justice system.

The Urban study aimed to establish baseline information about forensic labs, their caseloads, practices, and staff size and capabilities. Most survey respondents said that: (1) the DNA database in their jurisdiction should be expanded, (2) biological materials are packaged correctly, (3) DNA evidence is effective, and (4) lab analysts receive sufficient training.

In general, respondents said that problems arise in the lab because of a casework backlog and the lab’s capacity to analyze submitted evidence. Urban’s preliminary findings indicate that crime lab professionals believe they provide law enforcement with excellent service, answering their questions and generally recognizing the forensic needs of prosecutors and investigators. Respondents described the existing system as fine, despite the small number of unknown offenders actually identified through DNA testing.

The ongoing study addresses evidence collection, including who collects it. For example, although there generally are written policies regarding evidence collection in a homicide, there is substantial discretion regarding evidence collection in a burglary.

The researchers also looked at practices in the United Kingdom, where the average turnaround time for DNA analysis at that time was 3–5 days. They noted that when the system was privatized, a large monetary investment by the U.K. government increased lab capacity.

Many questions remain regarding best forensic practices, which, as Walsh suggests, means that “every assumption should be questioned,” including assumptions that: (1) DNA is used frequently to identify unknown offenders, (2) all backlogged cases are “open” and should be tested, (3) bigger is necessarily better, and (4) having an adversarial system and accreditation ensure quality.

See Walsh’s slides from this presentation (pdf, 12 pages).

During the discussion that followed, participants discussed the role of forensics in identifying a suspect versus improving a case outcome. Participants noted that a DNA hit may eliminate a suspect, but is not always recorded in police department files in a way that researchers can use later to explore the value of DNA testing and other issues. Another participant observed that criminal justice practitioners can overestimate the importance of DNA evidence and wondered whether this merits further study.

Analysis for Investigations and Confirmations

Presenter: Tom McEwen
McEwen & Associates, LLC

Tom McEwen, Ph.D., offered an overview of his findings from an NIJ-funded study, The Role and Impact of Forensic Evidence in the Criminal Justice System (pdf, 130 pages).

See Dr. McEwen's slides from this presentation (pdf, 10 pages).

The discussion that followed touched on three primary issues:

  • collecting evidence and what should be sent to the lab
  • prioritizing testing
  • system issues

Collecting evidence and what should be sent to the lab
Some investigators regard DNA evidence as a last-ditch effort. Participants noted, however, the importance of law enforcement understanding the difference between using DNA as an investigatory, rather than just a confirmatory, tool.

An investigator responding to a crime scene may collect DNA evidence but hold off on submitting it to the lab for analysis. The question becomes: What do we know about the role of the investigator in making strategic decisions on what is sent to the lab?

Investigators and prosecutors triage evidence to separate the potentially probative from unrelated facts or “noise.” But communications can be a problem, particularly when investigators do not ask the right questions, such as, what the assailant might have touched.

DNA is used primarily in homicide and sexual assault cases, although its use for burglary and robbery is increasing.

Although DNA is important, its importance is often overemphasized and this overemphasis can influence public policies.

Often, too much evidence is sent to the lab and the lab is expected to figure out what is relevant. This can exacerbate a backlog. Research might look at whether there is, indeed, an issue with investigators and prosecutors clogging the system with material that does not need to be analyzed — or not informing the lab of important information, which results in the lab having to analyze everything.

How can we ensure that the “right” evidence is sent to the lab? Part of the solution is better training of forensic examiners, detectives and investigators. As a way to help labs determine and set priorities, we could study jurisdictions in which we think it is being done well and compare processes and protocols with other jurisdictions.

Do we want DNA analysis to be used more frequently? Again, participants noted that, although the lab’s role is to analyze investigative information, they currently play a more confirmatory role. Time is an issue, especially in homicide cases, and investigators take on other cases as time passes —the sooner investigators get lab results, the more likely they can be used in an investigation. However, investigators often cannot wait for lab analysis, even if DNA is the “gold standard.”

How much  evidence should be DNA-tested is a question that comes up, especially with known offenders. We might identify a serial rapist if all DNA is tested, but some investigators think, Why test if we know who it is? There are cost-benefit issues to consider with respect to what is needed in a particular case versus what might be valuable in another case.

Prioritizing Testing
Language is important: the words “triage” and “prioritization” suggest that some cases are more important than others. “Tracking” describes the ability to know where cases are in the system.

Some crime labs prioritize evidence based on the type of analysis to be done and when it has to be done: a homicide case, for example, may need only a few blood spots analyzed, not a full-blown work-up of clothing, bedding, etc. Calls for efficiencies in labs should address such issues and for that to happen, decision makers must understand the particulars of a case to determine which approach is most appropriate. This, however, raises issues regarding context bias. One discussant suggested that research on lab efficiency could focus on the 1–2 percent of cases expected to go to trial. For this to happen, research requires increased coordination among criminal justice agencies.

In some jurisdictions, testing controlled substances creates a huge backlog in the crime lab; however, only a small percentage of these cases ever goes to trial. In one county, for example, the lab analyzes evidence only when requested by a prosecutor, resulting in a county with no backlog and a 3- to 4-day turnaround. On the other hand, other jurisdictions do not have such a triage system; rather, they want everything tested, which puts a tremendous burden on labs.

Could there be a different procedure/practice in the lab for suspects who may be ruled out? One discussant mentioned a jurisdiction that requires ruling people out before a case can proceed. A good topic for research would be exploring a reasonable way to triage DNA testing so it does not appear to be subjective, which would help assure defense counsel that the lab has not pushed evidence to the side while a suspect/client is sitting in jail.

One discussant noted that the prosecutor determines what is important in a case. There are those who think a jury will not convict without DNA evidence, despite the fact that the DNA evidence may be superfluous. Over-collecting is good, but over-testing is not, nor is over-presenting the reliability of the evidence to the jury. Therefore, prosecutors must learn how to present a good explanation about why a piece of evidence was or was not tested. This will help mitigate the current attitude held by many, ”Test it all and let the jury sort it out.”

System Issues
Discussion followed regarding whether it would be important for NIJ to fund improvements in laboratory information systems. Discussants observed that if labs collect the right information, it becomes easier to perform social science research that answers questions regarding improved judicial outcomes.

We often talk about the criminal justice system as it exists currently, but we should be mindful of new trends and developments. With respect to the development of rapid DNA testing, there is a perception that DNA could be used (in terms of collecting and testing it immediately) more as a biometric than a forensic. Rapid DNA could lead to collection, analysis, and development of a CODIS profile within two hours, which could be useful in cases involving violent offenders, including tying into cold cases. This, however, could affect lab resources and management in unforeseen ways. Would rapid DNA prevent crimes? One of the arguments for rapid DNA is to make evidence available at booking, which would mean that using DNA evidence in this way would be similar to the way fingerprints or mug shots are used currently.

To enhance public safety and criminal justice, social science, communications and operations research should come together more creatively. Social science cannot work in isolation from physical science and vice versa. In the same way, practitioners should be an integral part of the work vis-à-vis translational criminology. That said, one participant maintained that looking for ways to increase the use of DNA is not a social science issue; rather, it is the value of DNA that is the social science issue, just as with non-DNA forensic evidence. We need to describe forensics better throughout the entire system because, currently, each person sees only his or her own piece of the puzzle. For example, we need to know more broadly what evidence is out there and how it is being used.

Discussion about the distinction between operations research and social science research followed. How to make a lab more efficient is an example of operations research; how many crimes someone would have committed without rapid DNA is an example of social science research. Yet, capacity-building versus knowledge-building cannot be separated from justice outcomes. For example, research on the deterrent effect of having a DNA profile in CODIS shows little effect. Therefore, would rapid DNA offer a close-up deterrent effect?

Social science projects need practitioners during the development phase of the research to define the meaning of research in practical terms and determine the methods of disseminating results to the appropriate users. NIJ can sponsor research that asks people in the field to do something (e.g., implement new procedures) and then study it, or NIJ can examine new practices in the field and compare them with current practices: a “naturally occurring” experiment.

Paying for Forensic Services

Presenter: John Roman
Urban Institute, Justice Policy Center

John Roman, Ph.D., discussed new financing techniques and privatization issues that affect forensic services funding. The U.K., for example, explored a fee-for-service model, focusing on potential security concerns. In that model, the accreditation process can help alleviate concerns about possible security breaches that could arise as a result of broader access to data. In the U.S., market forces help prioritize funding, but leave unanswered the question of where and how to start.

Participants noted that with respect to lab privatization, a private company has more freedom to narrow its focus and do what is profitable. On the negative side, however, innovation is less likely to occur. Furthermore, law enforcement agencies may shop around for the result they want. In addition, there is a question as to whether government would be able to compete with a fee-for-service model. In considering privatization, it becomes important to define what it means to “compete” and not think only in terms of dollars.

Discussants noted that attitudes regarding privacy are different in the U.K. from those in the U.S. In a fee-for-service lab, you give up some rights, including privacy (that is, a private entity having access to DNA information). The American criminal justice system may not be ready to do that.

Another issue to consider is that labs are responsible for more than simply testing evidence. They must maintain quality and be available to testify and consult with prosecutors, etc.  The cost of these tasks must be considered.

Social science research could help inform funding and other issues with respect to privatization. What, for example, would be done if a small law enforcement agency must investigate a sexual assault, but cannot afford lab testing? Where will they get the money? Number of cases vary from year-to-year. An agency may have one case a year or five. However, if the lab is funded for three, for example, how would the additional two cases be handled? Would they have to wait to get extra funds? And, too, who do we want deciding where to save money? Cost/benefit issues could be explored in much the same way as was done with the privatization of correctional institutions. Issues to be explored in considering privatization include where the money would come from and where cuts would be made?

One discussant observed that it is not simply a public versus private issue. In Houston, for example, the Local Government Corporation allows the Houston Police Department (HPD) crime lab to break even because HPD receives money — via direct billing and contracts — from the county government paying for services. Texas law allows the collection of restitution fees, but participants observed that it is not a good source of funding. HPD has found that reducing lab turn-around time reduces jail time and associated expenses (18 days versus 18 months).

At the Minnesota Bureau of Criminal Apprehension (an independent agency), the crime labs report to the prosecutor. Is there an impact on the criminal justice system based on who is in charge? And how do defense attorneys feel about accountability? A private lab may be just as likely to violate someone’s rights as a public lab.

Are there natural experiments available to look at these issues? We could find the records from disbanded private labs and perhaps compare data to jurisdictions that currently use different pay models.

Participants noted that fragmentation of crime labs could be an issue. Every citizen should be able to get the same level of service anywhere in the country. To get the benefits of privatization, would the requestor have to pay?

Designer drugs are getting harder to identify, which puts more pressure on labs. As less funding for labs is available, it may become necessary to remove some services.

We have little foundational information about how labs interact with other players in the criminal justice system. This can vary from state-to-state. In West Virginia, for example, state law dictates the way in which Marshall University (which performs forensic testing) and the police department work together.

In addition to studying private, public, quasi-private, private/public labs, we should look to other fields for examples of success, such as the Ben & Jerry business model. Manufacturing, for example, has offered innovations such as using bar codes — social science research can help us understand the value of such innovation. Also, it was noted, for any kind of scientific innovation, we need sound business models before we decide how to implement it.

One participant noted that for privatization to be successful we would need a regulating body akin to the Food and Drug Administration.

A question arises as to the issue of evidence storage. In the past, there has been an issue with private labs destroying evidence when the lab closed.

Issues in the Laboratory

Presenter: Kelly Walsh
Urban Institute, Justice Policy Center

Kelly Walsh discussed the Urban Institute’s evaluation of NIJ’s DNA Efficiency Project in which five labs were tasked with increasing their efficiency (as opposed to capacity). She noted that, in 2001, 75 percent of laboratories said they had a Laboratory Information Management System (LIMS). By 2009, this had increased to 84 percent overall (97 percent of state labs and 56 percent of municipal labs). A LIMS generates reports, tracks evidence and case numbers, documents chain of custody, monitors backlogs, calculates turnaround time, and tracks case status.

Referring to Urban’s Forensics Blueprint project survey (see above discussion), Walsh noted that more than 53 percent of lab analysts do not obtain information on whether a case has been closed or resolved prior to DNA-testing the evidence, and none do this automatically through LIMS. This, she noted, may not be a problem if there is no backlog.

In the Blueprints survey, respondents reported that testing prioritization was done based on: the nearest trial date (20 respondents), seriousness of the crime (13), probative value (4), and whether the suspect had been identified (7). The Urban evaluation is finding potential issues with documentation of evidence, the lack of lab discretion, and use of a written policy.

The role of “context” is another issue. With respect to technology implementation, for example, several issues must be kept in mind, including purchasing, installation, integration, training, validation, maintenance, and performance. With any new technology, there will be some systems change that requires multi-level buy-in to be successful. But often, Walsh said, a purchase is made before a plan has been established, which can lead to failure. It would be interesting to know, for example, how often a plan was in place before the purchase decision was made.

Other potential research issues include: (1) How will the current LIMS configuration impact the ability to do social science research? (2) Can we evaluate the innovators? and (3) How does implementation timing impact the effectiveness of social science research?

See Walsh's slides (pdf, 21 pages).  

During the ensuing discussion, participants mentioned that some departments have a UPS–type tracking system, which makes it easy to keep track of evidence. Some discussants wondered if victims should have access to such a system. For example, California has a system that gives people access to blood alcohol testing results in drunk-driving cases. Participants also discussed bar-coding items in LIMS, which could provide information on what staff is working on at any point in time.

One participant noted that the term “backlog” can be a misnomer. Labs may have evidence that does not need to be tested — when, for example, cases have been closed already or when testing would add no value to the case. Testing protocols can vary among jurisdictions. In some jurisdictions, evidence is discussed at the prosecutor’s office before testing is requested. This means that education of the prosecutor is important in terms of evaluating the need to test. In some jurisdictions, if the statute of limitations has expired, they test the evidence, but de-prioritize it.

There was discussion regarding Los Angeles County’s testing, a number of years ago, of sexual assault kits that were in the police and sheriff departments’ property rooms. After a policy decision was made to test all kits, a questionnaire was created to help prioritize testing: suspect unknown; not able to identify a suspect without DNA testing; suspect already identified, but confirmation needed, etc. To learn more about this project, read Solving Sexual Assaults: Finding Answers Through Research. The full evaluation report (pdf, 151 pages) also is available.

Participants discussed the issue of follow-up investigation, noting that if all financial resources go to testing, nothing is left for police follow-up, which could mean testing becomes a waste of money. One discussant noted that a jurisdiction that is testing evidence in property crimes lacks sufficient investigators to follow-up or jail space for those found guilty of low-level burglaries.

Crime labs generally do not have the tools to measure and evaluate their efficiency in meeting client needs — and, in fact, some clients cannot define their needs. Research would be helpful in determining the needs of a forensic delivery system and how effectiveness should be measured. Research also is needed in the area of cognitive bias issues, expert systems impact lab protocols and efficiency, and what the impact of DNA field-testing would be.

Finally, one discussant queried the implications of disruptions to the work flow, especially in light of the Crawford decision, which essentially grants the defense the right to confront the lab analyst’s findings.

New Research Possibilities on Criminal Behavior

Avinash Bhati (Maxarth, LLC)
Greg Ridgeway (National Institute of Justice)

Avinash Bhati, Ph.D., gave a brief overview of his research study, Quantifying the Specific Deterrent Effects of DNA Databases (pdf, 98 pages) Exit Notice . Following this, Greg Ridgeway, NIJ’s Acting Director, facilitated a discussion on future research, posing the question, ”Does adding profiles to the CODIS database deter crime?“ States increasingly invest in collecting evidence from crime scenes, but what do we know about the impact on deterrence of having one’s DNA in a criminal justice database?

Deterrence, Ridgeway observed, relies on the probability of the suspect being apprehended quickly and sanctioned severely. We know that DNA testing can be a powerful tool, but does it — or can it — have a deterrent effect? How does the use of DNA evidence affect certainty and severity? If we wanted to enhance the benefits of DNA from just a probative tool to a deterrence tool, we would need to educate the people who would be affected — that is, those whose profiles are in CODIS and other criminal justice databases.

Discussants said that it would be interesting to know what offenders know about DNA and how it is used. Do they know what DNA is? Would there be a deterrent value in educating offenders about what it means that their DNA profiles are in a database? With respect to educating offenders about the ramifications of having a DNA profile in CODIS or other database system, an analogy was offered regarding seatbelts: seatbelt use increased after laws were passed that resulted from educating consumers. Because reentry programs are growing, this may be a good place to include information/education on what it means to have one’s DNA profile in a database such as CODIS. With respect to “learning,” however, participants observed that people process information differently. For example, despite knowing they have the right to remain silent, some suspects confess. Another possibly fruitful area for research would be to explore how the existence of DNA evidence is used in interrogation.

With respect to deterrence regarding the presence of DNA in a database, a discussant noted that many crimes are crimes of passion, and most perpetrators do not think in terms of consequences. A number of participants maintained that to understand the “context” of the data, researchers should know that criminals do not care about deterrent effects. A robber is interested in the immediate outcome, not the odds of getting caught in the future. Participants observed that deterrence has been effective in decreasing driving under the influence of alcohol, for example, but not in decreasing street crime.

When considering the possible deterrent value, there could be a difference between the probative versus perceived consequence of having a person’s DNA on record. In this regard, one discussant noted that the probative value is not the issue; it is the perceived value — the perception of the offender — that could affect deterrence.

Another discussant maintained that, with respect to any deterrence issue, getting caught is not the issue; rather, what happens after you are caught. This prompted a discussion about how some criminals modify their behavior to accommodate new obstacles; that is, their behavior does not change, but their tactics do. One participant said that teaching criminals more about the effect of having one’s DNA in a database would not change their criminality — it would only help them to be smarter criminals. For instance, a rapist returning to the community who understands that his DNA profile is in CODIS may still rape in the same neighborhood, but he may choose to use a condom and gloves.

Since the 1980s, forensic evidence has increased the certainty and severity of sentencing, which is an issue worthy of further exploration. But whether certainty and severity affects the propensity to commit a crime is another question. One factor to keep in mind as these issues (and their effect on deterrence) are studied further is income group. For example, a DUI charge may be considered very serious to someone in the middle-class.

Finally, a participant offered the example of fingerprints, noting that when the Federal Bureau of Investigation piloted a new system of collecting fingerprints that enabled a match to be made within 15 minutes, people stopped trying to falsify their identity. The hope is that rapid DNA (24 hours/day, 7 days/week) could yield a similar effect.

January 24, 2013

Recap of Day One

Dr. Angela Moore, of the National Institute of Justice, recapped the previous day’s discussions, noting some overarching take-aways, including:

  • NIJ uses “base funds” to support a broad range of research. Two of the primary challenges regarding future research are a lack of expertise in forensics by social science researchers and a lack of baseline data.
  • There is great need to question assumptions and the handling of backlogs of evidence.
  • DNA evidence can be used as an investigative versus confirmatory or exoneration tool.
  • Research questions for the future include: (1) What happens to evidence when private labs go out of business? (2) Would a “mixed model” work best? (3) Should crime labs be part of law enforcement? and (4) Who determines performance measures?
  • Data in LIMS cannot be used easily for research purposes. How will the current LIMS configuration impact the ability to do research?

Hey Wait, It’s Not Just About DNA

Presenter: William King
Sam Houston State University, College of Criminal Justice

Run by the Bureau of Alcohol, Tobacco, Firearms, and Explosives (ATF), the National Integrated Ballistic Information Network (NIBIN) program uses ballistics imaging as a source of investigatory intelligence. NIBIN digitizes spent brass (such as cartridge cases) found at a crime scene and compares data from this process with data in the database. Three types of correlations can be made: matching brass to guns, weapons to weapons, and crimes to crimes.

The NIBIN report (including hits) may go to the supervisor of the case, an investigator or supervisor, or the command staff. Labs communicate results via telephone (45.5 percent), email directly from LIMS (30.4 percent), email (17.9 percent) and fax (6.2 percent). The amount of information communicated differs from jurisdiction to jurisdiction — and issues that can affect this include agencies not using the same case number and variation among victims’ names and suspects’ names.

eTrace (Electronic Tracing System) is an Internet-based system that allows law enforcement agencies to submit firearm traces to the ATF National Tracing Center (NTC). Authorized users can receive firearm trace results via the Web site, search a database of all firearm traces submitted by their agency, and perform analytical functions. Although eTrace is a national database networking 12 regions together, it must be run by each individual region because jurisdictional boundaries affect levels of information-sharing. Budget cuts to NIBIN have forced the closure of more than 100 sites, creating a potential “natural experiment.”

One important question is whether to put everything in the database or selectively input data. Survey data from NIBIN sites reveal delays in processing ballistics evidence: the mean is 57 days; the median is 27 days. A “hit” gives a list of high-confidence candidates. Then the brass can be pulled from the collection and examined under a comparison microscope; both steps must be done to qualify as a hit.

See Dr. King's slides (pdf, 18 pages).

Here are some of the points made during the ensuing discussion:

Current challenges include difficulties in contacting other agencies, funding, and delays in getting brass, which slows down analysis by regional crime labs.

Many NIBIN hits come from drive-by shootings that produce brass.

Discussion followed regarding the Law Enforcement National Data Exchange (N-DEx), an information-sharing tool that allows officers around the country to pull together information. However, fusion centers that use N-DEx often have difficulty securing information because guns are moved around purposefully. In cities, the guns generally used are semiautomatics, which means that many sites do not enter data into NIBIN regarding revolvers and shotguns.

Investigators use NIBIN to help gather information from convicted felons who may turn in their colleagues. Half of homicide cases are cleared in 7–10 days. NIBIN hits can be used to gather intelligence on criminal organizations (including gangs), but this requires collaboration. With a network analysis, investigators can piece together information about suspects or organizations, which can be useful in Racketeer Influenced and Corrupt Organizations Act (RICO) prosecutions.

Although the system cannot be used to compare gang cultures, it can classify cities based on the density of gun use by criminals. Because 95 percent of weapons are used only once, they produce no hit in NIBIN. Furthermore, there is no information on whether the gun is legal and there is variability among sites regarding staff’s ability to use eTrace. It is possible to use eTrace to establish the time a crime was committed as long as a firearm is recovered. In fact, 60–70 percent of traces involve brass-to-brass comparisons, but no test fire.

One participant advocated putting everything into NIBIN to aid in going after criminal organizations, including how many times the gun was fired, location, etc. Other participants noted that, as has already been done with DNA and fingerprints, processing impediments must be removed to identify suspects more quickly. This includes prioritizing cases for fast-track analysis based on solvability and triaging cases.

NIBIN sites test-fire nearly everything. One research question might be whether test-firing an individual’s weapon when (s)he is still in jail — allowing the investigator to receive information within 24–48 hours — would help solve crimes. From the social-science perspective, one participant queried, does it matter if the process is long if it results in jail time? The question was posed regarding prosecutions: No matter how much the lab does, if the results are not processed downstream, is there a need to spend money on it?

NIJ’s Office of Investigative and Forensic Sciences focuses on the core sciences; however, social science focuses on issues such as the speed of processing and what difference this can make. Currently, much social science research is focused on DNA, which may be over-valued — and NIJ, therefore, would like to consider broadening topics to include, for example: What do jurors really understand? How do lawyers fail to understand the significance of forensics? What is the judge hearing and what does that mean to her? With respect to such questions, however, a participant noted that only a small proportion of cases go to a jury.

One participant observed that there are more gun deaths from suicide than homicide. Prosecutors make decisions on whether a case goes to trial and how they think jurors will view the evidence. Do many district attorney offices have an expert in forensics? Do we need to limit which prosecutor should be able to try a case with complicated forensics?

Improving Processes

Katharine Browning led a discussion about evidence collection. Noting that evidence may be collected to generate a lead or it may be valuable for use in court, she pointed to studies done by Tom McEwen and Joe Peterson regarding the progression of evidence through the system, including the processing, adjudication and sentencing stages. It is important to understand what forensic evidence pays off at what stage and in which circumstances.

In the ensuing discussion, participants made the following points:

The criminal justice system seems to be pushing to collect more and more evidence; but is this necessary? Are we collecting the right material? Some jurisdictions are adding training on DNA evidence collection to their academy classes. However, graduates may complete training without learning much about the DNA evidence they are collecting.

What does a crime scene investigator (CSI) see as evidence versus artifact? Many operate without a sense of “context,” which is important in determining what to collect. Most potential evidence collected is later determined to be not related to the crime, but investigators tend to believe that any relevance will be figured out later. Often, however, that does not happen and such determinations end up being pushed down the line to the lab.

CSIs must do a better job of information-sharing, including what evidence must be gathered before the scene is released. From the police perspective, there are three decision-makers: the patrol officer who decides whether a scene is contaminated or could contain probative evidence; a crime scene investigator who decides what to collect; and an investigator who decides whether to have collected evidence sent to the lab. The crime lab makes further decisions about what to test. Often, for example, the lead investigator tells the crime scene people what to collect. Does this need to change? There are many different models of evidence collection and handling — and social science research can explore appropriate models. We should define best practices, particularly in this era where “touch DNA” — in which a DNA profile is established from an object that has been touched by a suspect — is being used more frequently.

Other possible social science questions include: Should the starting point for training be state-to-state or jurisdiction-to-jurisdiction? Should crime scene investigators be sworn personnel or civilians? Civilians can be used at much lower cost, and it generally has been found that it does not matter who collects evidence. Could the mix of uniformed officers and civilians for field testing be changed?

It is important to get data not only on what is collected, but also what the scene is like — whether it is contaminated, for example. Because input can come from detectives, forensic analysts, and prosecutors — all of whom have different ways of determining how probative evidence may be — we need a more holistic look at the process and best practices.

Digital evidence is a growing area in which social science research could play an important role, including how to collect cell phones and computers and retrieving information from them. Analysis of video tapes is also a needed skill-set because video recordings of crime scenes are becoming more common. One participant observed that the increasing use of digital forensics might reduce DNA use.

Training in digital forensic evidence examination is needed. Moreover, technology is changing fast. People do not want to spend time in training and testing, but without training, they cannot stay ahead of new technologies whose developers may market unproven technologies. Also, with respect to knowing what to test, it is important to have a baseline of promising practices, including what people are trying already. This applies to what we are training a CSI to collect, for example. We need a best-practices benchmark regarding investigations.

A Fresh Look at Hot Topics: Sexual Assault Kit Action-Research Project in Wayne County, MI

Presenter: Becki Campbell, Ph.D.
Michigan State University, Department of Psychology

Dr. Campbell discussed her work as the lead researcher on an NIJ-funded “action-research” project in Detroit (Wayne County), Michigan. This research aims to develop and evaluate solutions for testing SAKs, victim notification, and investigation and prosecution. In 2009, approximately 10,000 sexual assault kits (SAKs) were discovered in police storage in Detroit; it was unknown whether they had been tested or not. Ultimately, 11,304 SAKs were audited: 2,399 had a lab number; 8,505 had never been submitted to the lab.

The NIJ-funded team developed four studies — or “waves” — to test 1,600 of these SAKs in an effort to determine the utility of DNA testing:

  • 450 “stranger” (unknown perpetrator) cases
  • 450 non-stranger cases
  • 350 cases selected for comparison of DNA testing methods
  • 350 older cases in which the statute of limitations had expired or was about to expire.

In the first wave of testing (stranger rapes), 248 kits were submitted for testing and 196 were able to be DNA-tested. These yielded 100 profiles that could be entered into CODIS. There were 64 CODIS hits: 42 offender hits, 4 forensic hits, 18 offender and forensic hits.

With respect to the controversial issue of victim notification, the tentative plan is to develop a pilot program for the first two waves of testing, notifying the victim only if there is a CODIS hit and a possible prosecution. Notification will be done by an investigator/advocate team, but it will vary by case — and they will be evaluating as they go along.

Campbell said that there is a strong commitment to follow-up investigations and prosecutions, but there are funding concerns. The team currently is working on the first 12 hits, of which 7 are eligible for victim notification.

See Dr. Campbell's presentation slides (pdf, 68 pages).

Here are some of the points made during the ensuing discussion:

Historically, there has been a new chief of police in Detroit every two years; the city is currently operating under a federal consent decree.

Record-keeping is a problem in trying to determine why cases were not tested; this also affects efforts to track the rate of sexual and other assaults. The data may offer red flags that could help other jurisdictions identify organizational “risk factors” for developing a large number of SAKs that are maintained in police property rooms but not sent to the lab for testing.

Explanations for why there are untested SAKs in a police evidence room vary across jurisdictions. To learn more about a project in Los Angeles in which a sample of SAKs in police and sheriff property rooms were tracked and evaluated in an NIJ-funded project, read Solving Sexual Assaults: Finding Answers Through Research, an NIJ Journal article or the full report, The Role and Impact of Forensic Evidence on the Criminal Justice Process (pdf, 151 pages).

One participant suggested that a best practice would be for all cases to have a multi-disciplinary team analyze the evidence. Participants also noted that untested SAKs is not a regional problem, but a national problem. Although sexual assault investigations are no different from any other crime category, it is important to remember that they require an invasive evidence collection and are susceptible to false promises; these are factors that could be studied. Finally, it was noted that the problem does not rest simply within the criminal justice system. There may be 4,000 hospitals with 60,000 staff nationwide who have never received any sexual assault training or education.

Next Steps Discussion

The following topics/areas were raised as potentials for possible social science research in the future:

  • In the victim area, we need to know more about many issues, including the impact of trauma.
  • Witness intimidation and witness protection.
  • Focusing on outcomes of forensic analysis rather than outputs.
  • The impact of the current trend of early releases from prison.
  • Business practices and collaboration.
  • The issue of non-reporting in sexual assaults. In West Virginia, for example, victims must decide within 18 months whether they want to pursue prosecution. At that point, the state is authorized to use a SAK for research or destroy it. But what happens with respect to destroying potential evidence and issues of storing, etc.? What is the victim point of view with respect to storage of SAKs?
  • What are the ramifications of wrongful convictions?
  • When cases are old, we apply new technology to samples collected with no concern about how those samples were collected. For example, swabs may not have been taken with the same level of cleanliness/sterility that we now know is important.
  • Should procedures change with respect to testing for cannabis now that it is becoming legal in some jurisdictions?
  • Using DNA evidence to interrupt the progress of a career criminal; if a criminal can be stopped at the stage of doing property crimes, it may stop more serious crimes in the future.
  • Bringing the standards for death investigation up to par should include a discussion of whether coroners are trained adequately to perform investigations.
  • The FBI is reviewing cases with microscopic hair evidence, which could result in a push for that kind of retrospective exam across the country.
  • As practices and procedures change in criminal justice agencies, there are likely to be economic/cost-benefit issues.
  • Incompetence in crime lab staff; individuals are not being held accountable.

Dr. Browning concluded the meeting by noting that NIJ is doing work in the area of system error. NIJ plans to issue two solicitations: one for social science research (she noted that participation in this working group is not a conflict of interest) and one regarding firearms. Other forthcoming solicitations include building researcher/practitioner relationships; offender issues; sexual violence; and research and evaluation on violent victimization.


Bethany Backes, National Institute of Justice
Avinash Bhati, Maxarth, LLC
Katharine Browning, National Institute of Justice
Eric Buel, Champlain Forensics
Marc Buslik, Chicago Police Department
Rebecca Campbell, Michigan State University
Jamie Downs, Georgia Bureau of Investigations
James Doyle, Visiting Fellow, National Institute of Justice
Matt Durose, Bureau of Justice Statistics
Valerie Evanoff, CrossResolve
Terry Finger, Marshall University Forensic Science Center
David Foran, Michigan State University
Christine Funk, Christine Funk Law Firm
Dean Gialamas, Los Angeles County Sheriff Department
Ted Hunt, Jackson County, Missouri Prosecutor’s Office
William King, Sam Houston State University
Gerald LaPorte, National Institute of Justice
Eric Martin, National Institute of Justice
Tom McEwen, McEwen & Associates, LLC
Danielle McLeod-Henning, National Institute of Justice
Angela Moore, National Institute of Justice
Mark Nelson, National Institute of Justice
Phyllis Newton, National Institute of Justice
Joseph Peterson, California State University, Los Angeles
Ron Reinstein, Judge and Judicial Consultant, Arizona Supreme Court (ret.)
Greg Ridgeway, National Institute of Justice
Nancy Ritter, National Institute of Justice
John Roman, Urban Institute, Justice Policy Center
Jeri Ropero-Miller, RTI International
Steven Siegel, Denver District Attorney’s Office
Patricia Speck, University of Tennessee, Health Science Center, College of Nursing
Stephanie Stoiloff, Miami-Dade Police Department, Forensic Services Bureau
Kelly Walsh, Urban Institute, Justice Policy Center
Heather Waltke, National Institute of Justice
Vincent Webb, Sam Houston State University
Victor Weedn, George Washington University, Department of Forensic Sciences
Tate Yeatman, Palm Beach County Sheriff’s Office Crime Laboratory

Date Created: November 7, 2013