Test and Survey Results from the “Black Box” Study

The public data supporting the “Black Box” study¹ is provided as two tab-delimited text tables described below.

Examiner responses to the test questions

The following is a description of the “Test Responses” data table containing the responses to the test questions. There is one row for each presentation of an image pair to an examiner: 169 examiners responded to approximately 100 presentations each of image pairs yielding 17,121 sets of responses. The meaning of these data values is further explained in the paper (1); see especially the instructions to participants, section 1.5 of the Supporting Information.

Column header	Data content	Data format
Examiner_ID	Unique, anonymized identifier for the examiner.  169 distinct values	Annn (the letter “A”  followed by 3 digits)
Pair_ID	Unique identifier for the image pair. 744 distinct values	Annnnn (letter indicating mating, followed by 6 digits) The first 3 digits uniquely identify the latent, and the last 3 digits uniquely identify the exemplar.
Mating	Whether the latent and exemplar originate from the same source	{“Mates”,”Non-mates”}
Latent_Value	Examiner’s latent value decision	{“NV”,”VEO”,”VID”}
Compare_Value	Examiner’s comparison decision	{“Exclusion”, ”Individualization, ”Inconclusive”, “NA”}
Inconclusive_Reason	Examiner’s reason for inconclusive decision	{ “Close”, “Insufficient”, “Overlap”, “NA”}
Exclusion_Reason	Examiner’s reason for exclusion decision	{“Minutiae”, “Pattern”, “NA”}
Difficulty	Examiner’s rating of difficulty to make comparison decision	{“A_Obvious”, “B_Easy”, “C_Medium”, “D_Difficult”, “E_VeryDifficult”, “NA”}

Examiner survey responses

The “Survey Responses” has one row per survey respondent (159 participants completed the survey). The full text of each question is published in the paper (1), section 1.4 of the Supporting Information.

There is one column for each possible response to the multiple choice questions; the header of each column is of the format “QuestionNumber.Response” where Response indicates the response category; the responses selected by participants are indicated by a “Y”. For example, there are two columns for the first question, with column headers “1.Male” and “1.Female”. Some questions were of the form “check all that apply,” so multiple columns may be selected.

There is a single column for each of the free-format text responses (questions 2, 5, 7, 9, 19). No response was permitted on the free-format and “check all that apply” questions. Survey responses are not associated with Examiner_IDs to protect participant anonymity, which was part of the Institutional Review Board for Human Subject Research approval for this study (described in the paper (1), section 1.2 of the Supporting Information).

As stated in the paper (section 1.4 of the Supporting Information, #13) “On question 13, responses were available for 161 of the participants (as opposed to 159 for all of the other questions), and therefore percentages are based on a total of 161.” Two additional participants did not complete the survey but indicated that they were certified by an accredited employer.

---

¹B.T. Ulery, R.A. Hicklin, J. Buscaglia, M.A. Roberts (2011) “Accuracy and reliability of forensic latent fingerprint decisions,” Proc. Natl. Acad. Sci. U. S. A., May 10, 2011, Vol. 108, no. 19, 7733-7738, published online before print April 25, 2011, doi:10.1073/pnas.1018707108.