To Err is Human: Using Science to Reduce Mistaken Eyewitness Identifications Through Police Lineups

by Maureen McGough

Researchers take police lineup studies from the laboratory to the field.


© (see reuse policy). Image is used for illustrative purposes only and any person depicted is a model.

In 1984, a Cook County, Ill., jury found 27-year-old Ronnie Bullock guilty of raping a 9-year-old girl in Chicago's south side. He was sentenced to 60 years in prison. Crucial to the prosecution's case was the victim's identification of Bullock in a police lineup. A second rape victim — a 12-year-old girl — also viewed a police lineup and identified Bullock as her attacker.[1]

Maintaining his innocence, Bullock sought relief from the courts. His conviction was upheld on appeal in 1987, and two state postconviction petitions were unsuccessful. Bullock's federal habeas petition was denied in 1991.[2]

In June of 1993, he was granted a motion to have impounded evidence released for DNA testing. Tests revealed that Bullock was not the source of the semen found on the victim's clothing, and a judge dismissed the charges against him in 1994.[3]

Bullock spent 10 and a half years in prison for a crime he did not commit.[4]

Nationwide, mistaken eyewitness identifications have played a role in 75 percent of convictions later overturned because of DNA evidence,[5] and criminal justice practitioners and researchers have a pervasive interest in finding ways to improve the methods used for eyewitness identifications. A good deal of research has focused on the police lineup, in which victims and witnesses attempt to distinguish a suspect from other individuals presented (known as "fillers").

A recent study from the American Judicature Society (AJS) is adding to the body of research by investigating which lineup method results in fewer mistaken identifications:[6]

  • Sequential, in which the witness views lineup members one at a time and makes a decision on each individual member, or
  • Simultaneous, in which the witness views the entire lineup at once

Past research using controlled laboratory experiments consistently showed that sequential methods yielded fewer mistaken identifications. But in 2006, a field study in Illinois called into question the superiority of the sequential method (and with it, the use of controlled laboratory experiments as approximations for actual eyewitnesses to crimes).

Scientists, however, identified flaws in the Illinois study's design and implementation. As a result, some experts have deemed the results "difficult or impossible to interpret."[7]

To produce more rigorous data using field techniques rather than laboratory techniques, the AJS research team developed an improved research design for its study.

The initial report on the AJS study indicates that sequential lineups significantly reduce the number of filler identifications without significantly reducing the number of accurate positive identifications. Thus, the AJS findings support results from past laboratory experiments.

Research From Laboratories to the Field

Given the vital role of eyewitness testimony in the administration of justice and the inherent risks therein, extensive research has been dedicated to developing lineups that minimize identification of fillers without significantly reducing accurate, positive identifications.

However, many of the variables that may affect the accuracy of eyewitness identification are out of the control of the criminal justice system.[8] These include lighting of the crime scene; length of time a witness was exposed to the perpetrator; severity of the crime; and characteristics of the witness and perpetrator, such as race, age and sex. These variables are helpful in estimating the likely accuracy of eyewitness identification, but they cannot be controlled in actual criminal cases.[9]

Several renowned eyewitness researchers have focused studies on variables that the criminal justice system could control, such as who administers the lineup, how the lineup is administered, lineup compositions and instructions given to witnesses.[10]

By focusing on these controllable variables, researchers have produced findings from laboratory experiments that shape investigative practices and procedures.[11] These science-based practices include:

  • Using fillers in lineups that match the verbal description of the perpetrator
  • Informing the witness that the perpetrator may or may not be present in the lineup
  • Using a double-blind administration in which the lineup administrator does not know who the suspect is and therefore is unable to transmit inadvertent cues or feedback to the identifying witness[12]

Laboratory tests also show that sequential lineups offer a better ratio of accurate to mistaken identifications than simultaneous lineups. Sequential lineups require witnesses to compare each individual they see to their recollection of the suspect.[13] This increases accuracy and reduces the risk that witnesses will make a judgment based on a relative comparison of who among the group looks most like the perpetrator relative to the other lineup members. In fact, when a double-blind lineup was administered using the sequential technique in laboratory testing, identifications were twice as reliable as those from traditional lineups.[14]

The 2006 Illinois Report

Although laboratory results were promising, proposed changes in investigative practices needed to be field tested before they could be used to support widespread procedural overhauls. In 2003, the Illinois state legislature charged the Illinois State Police with conducting a yearlong field test of the effectiveness of the sequential, double-blind lineup compared with the traditional (non-blind, simultaneous) lineup.[15]

The results were surprising. In 2006, the Illinois State Police released a report (often referred to as The Mecklenberg Report after its author, Sheri Mecklenberg) showing that, in two of the three jurisdictions participating in the study, double-blind, sequential lineups produced a higher rate of identification of innocent fillers and a lower rate of identification of suspects.[16] In other words, this report contradicted what laboratory experiments had shown for years, and it recommended against instituting changes based solely on laboratory science.[17] The report, which was widely publicized, drew resolute support and severe criticism, particularly regarding the design and implementation of the study.[18]

First and foremost, critics stressed that the study confounded the simultaneous/sequential and non-blind/double-blind variables, rendering results largely uninterpretable. It was impossible to determine whether the better outcome using the simultaneous lineups was partly or entirely attributable to the influence of the non-blind administrator.[19] Notably, some proponents of the study felt that confounding these variables did not color the results, citing research that compared double-blind conditions to non-blind conditions and finding no effects.[20]

Additionally, critics cited the fact that cases were not randomly assigned to either group, and cases thought to be "tougher," such as cross-race identifications or those in which the lineup took place after a delay, were more likely to be assigned to the sequential group, thus negatively skewing the sequential results.[21] Critics also noted that some filler identifications were not recorded in simultaneous lineups, thus positively skewing the simultaneous results.[22]

The Greensboro Protocols

To address critiques of The Mecklenberg Report, the AJS convened scientists, lawyers, prosecutors and police in Greensboro, N.C., to develop a set of guidelines for conducting field experiments testing the simultaneous/sequential variable. The group was committed to conducting field research that would gather reliable data on the administration of the lineup and witness and event variables. Data determined to be essential for a scientifically sound field experiment included time between crime and lineup, type of crime, whether a weapon was present, viewing conditions, sobriety of the witness, certainty of the witness, and whether it was a cross-race identification.[23]

The "Greensboro Protocols" emphasized the importance of true random assignment of lineups into the sequential or simultaneous groups, and the consistent use of double-blind lineups in both groups was also deemed essential for conducting a scientifically sound field experiment.

The protocols also highlighted the importance of using computers — both for administering the lineups and for recording witness responses — to ensure that procedures were fairly conducted in accordance with best practices. Computers were deemed especially important because they could ensure uniform administration of lineups according to protocol, randomly assign lineups as either sequential or simultaneous, and randomly order the photos within a lineup. Computers would also allow for uniform, reliable and complete recordings of witness responses, including the time it took for witnesses to make a determination.

The AJS Field Study

Relying on the Greensboro Protocols, the AJS developed a field experiment that compensated for the deficiencies of the 2006 Illinois study. The field experiment was conducted at four sites: the Austin Police Department (Texas), the Charlotte-Mecklenburg Police Department (N.C.), the San Diego Police Department (Calif.) and the Tucson Police Department (Ariz.).

The AJS research team excluded lineups that were not conducted using a double-blind procedure, as well as lineups in which the eyewitness had prior knowledge of the suspect through a previous acquaintance. This left researchers with 497 protocol-consistent lineups for crimes, ranging from simple assault to murder.


There were no significant differences in a witness's ability to identify the suspect between the simultaneous and sequential techniques. Witnesses identified the suspect 25.5 percent of the time in simultaneous lineups and 27.3 percent of the time in sequential lineups. This small difference in identification rates falls within the margin of error and should not be considered a meaningful difference.

However, simultaneous lineups resulted in 18.1 percent identification of fillers, whereas sequential lineups resulted in 12.2 percent identification of fillers. This 5.9 percent difference in filler identifications was found to be statistically significant.

The AJS study results are consistent with the results of decades of laboratory tests showing that sequential lineups reduce mistaken identifications without significantly reducing accurate identifications.

Next Steps

Researchers plan to conduct additional analyses of the data to determine:

  • Whether witnesses are more certain about their mistaken identifications in sequential or simultaneous lineups
  • Whether accuracy changes with the witness's status as a victim or bystander
  • Whether the identifications were same-race or cross-race

Additionally, although identifications of fillers are clearly erroneous, identifications of the suspect are not necessarily accurate because the suspect is not always the perpetrator. The Police Foundation is leading a second phase of research to follow up on this area.

The police lineup is an inherently human process, and therefore inherently flawed. Although no single study can lead to the development of a procedure guaranteeing consistently accurate identifications, a well-designed field study can be an important step in developing best practices for lineups and other identification practices.

NIJ Journal No. 270, June 2012
NCJ 238486

About the Author

Maureen McGough is an attorney and NIJ's outreach coordinator.

Back to the top.

For More Information


[note 1] Conners, Edward, Thomas Lundregan, Neal Miller and Tom McEwen, "Convicted by Juries, Exonerated by Science: Case Studies in the Use of DNA Evidence to Establish Innocence After Trial" (pdf, 117 pages), Final report to the National Institute of Justice, grant number OJP-95-215, June 1996, NCJ 161258.

[note 2] Garrett, Brandon L., "Judging Innocence", Columbia Law Review 108 (2008): 65.

[note 3] Conners, "Convicted by Juries, Exonerated by Science," 40.

[note 4] Ibid.

[note 5] The Innocence Project, "Eyewitness Misidentification".

[note 6] One of the four study sites, Tucson, Ariz., was initially funded as a standalone field study by NIJ. The results of that site analysis were included in the AJS Field Study after the NIJ study received funding to use laptop computers.

[note 7] Schacter, Daniel L., Robin Dawes, Larry L. Jacoby, Daniel Kahneman, Richard Lempert, Henry L. Roediger and Robert Rosenthal, "Policy Forum: Studying Eyewitness Investigations in the Field," Law and Human Behavior 32 (2008): 3–5; see generally Conners, "Convicted by Juries, Exonerated by Science."

[note 8] Wells, Gary L., "Applied Eyewitness-Testimony Research: System Variables and Estimator Variables", Journal of Personality and Social Psychology 36 (1978): 1546–1557.

[note 9] Ibid., 1548.

[note 10] See, for example, Doyle, James M., "Learning From Error in American Criminal Justice" (pdf, 40 pages), The Journal of Criminal Law & Criminology 100 (2010): 109–148.

[note 11] Ibid., 116.

[note 12] Ibid., 17.

[note 13] Schacter, "Policy Forum."

[note 14] Steblay, Nancy, Jennifer Dysart, Solomon Fulero and R.C.L. Lindsay, "Eyewitness Accuracy Rates in Sequential and Simultaneous Lineup Presentations: A Meta-Analytic Comparison,", ​aw and Human Behavior 25 (2001): 459–473.

[note 15] Mecklenberg, Sheri H., Report to the Legislature of the State of Illinois: The Illinois Pilot Program on Sequential Double-Blind Identification Procedures, Springfield, IL: Illinois State Police, 2006.

[note 16] Ibid.

[note 17] Schacter, "Policy Forum."

[note 18] Ibid., 2.

[note 19] Ibid.

[note 20] Malpass, Roy S., "Notes on the Illinois Pilot Program on Sequential Double-Blind Lineup Procedures," Public Interest Law Reporter 2006: 5–8, 39–47; See also Ebbesen, Ebbe B., "Comments on IL Simultaneous v. Sequential Lineup Field Test," 2006.

[note 21] Wells, Gary L., Nancy K. Steblay and Jennifer E. Dysart, A Test of the Simultaneous vs. Sequential Lineup Methods: An Initial Report of the AJS National Eyewitness Identification Field Studies (pdf, 28 pages), Des Moines, IA: American Judicature Society, 2011.

[note 22] Ibid.

[note 23] Ibid., ix.

Date Created: June 15, 2012