Nervous System: Taking Biometrics at Face Value

David Kalat writes about the history of identifying criminals through photographs, the Bertillon method of pairing photography with specific measurements presumed to be immutable attributes of a person, and how the computer age affected identification.

Read the article.

In 1887, Warden R.W. McClaughry of the Illinois State Penitentiary at Joliet (now Joliet Correctional Center) introduced a radical new idea. To keep track of recidivist offenders and identify prisoners of extra-special concern due to past crimes, he implemented a “database” of criminals. The database consisted of a collection of paper-based records containing pictures of prisoners’ faces, along with a set of facial and bodily measurements. McClaughry got the idea from a French policeman and pioneer of biometrics, Alphonse Bertillon.

As described in Signaletic Instructions Including the Theory and Practice of Anthropometrical Identification, his landmark book of 1885, Bertillon sought to apply a rigorous scientific framework to the then-haphazard practice of tracking and cataloging criminals. The use of photography to document arrests was nothing new—in Victorian England, the camera was ironically called the “angel copier,” and was seen as a humane alternative for keeping track of former offenders, as opposed to physically branding them. Bertillon’s innovation was to pair photography with a set of specific measurements he presumed to be immutable attributes of the person: head length, head breadth, length of the middle finger, length of the left foot, and length of the cubit (the forearm from the wrist to the elbow). Bertillon posited that the combination of these measurements together with his “mugshot” of face-on and sideways views of the subject would provide a definitive identification of a unique individual.

The photos and biometric measurements would then be printed onto a “Bertillon Card” to be stored with other Bertillon Cards as a virtual library of criminals. This manually searchable database was an enormous boon to the field of criminology, and the “Bertillon System” quickly took hold across Europe and the United States. It had a couple of notable drawbacks, however. For one, even Bertillon himself agreed that it was a grueling and time-consuming task to find a single face “if you have no other means but your eyes to search for the photograph among the thousands in an ordinary collection.” Second, the system was designed to be reactive, to help identify suspects and monitor repeat offenders, but did little to proactively prevent crime.

The computer age offered solutions to both problems. If a machine could become as adept as a person at recognizing faces, that machine could sort and filter its way through a Bertillon-style database faster than any person could—and if such a machine were paired with surveillance cameras, it might be possible to identify known troublemakers in a crowd before they even did anything. Setting aside the many privacy rights concerns about whether such a Big Brother-style surveillance state would be socially desirable, the technical question was: would it even be doable?

Woodrow Wilson Bledsoe was a gifted computer scientist with a particular knack for pattern recognition. He had previously worked on Defense Department research funded by the Atomic Energy Commission to teach computers to recognize letters, both printed and handwritten (this was early research into what became optical character recognition). In the mid-1960s, Bledsoe led a team of researchers at Panoramic Research to teach computers to try to recognize faces. It was only years later that curious researchers discovered Bledsoe’s facial recognition work was in fact financed by the Central Intelligence Agency. Much of his work remains classified today.

Following the lead of Bertillon, Bledsoe started by dividing up the human face into subsections to be classified. By measuring details like the distance from the hairline to the eyebrows, the distance between the eyes, or the width of the mouth, he could create a facial map from which to draw statistical conclusions about how closely another photograph matched that dimensional model of a given face. Bledsoe’s researchers painstaking coded photographs by hand using a grid and a handheld stylus that emitted electromagnetic pulses to mark the coordinates of twenty critical biometric distances.

The software was designed to proceed down a decision tree from larger categories to smaller—for example, deciding the gender of the subject first, and then making broad racial categorizations, before proceeding to analyze more specific details.

Bledsoe published his first study, A Proposal for a Study to Determine the Feasibility of a Simplified Face Recognition Machine, on January 30, 1963—the first known work in facial recognition software. In this project, Bledsoe and his team encoded high-quality, high-resolution photographs of various faces taken from the same angle of view. They then fed the new photos of the same faces into the system to evaluate how the program matched them to the known faces.

On March 6, 1964, Bledsoe’s second paper, A Facial Recognition Project Report, reported the results using photographs taken from a variety of angles and lighting conditions. Glumly, Bledsoe declared “picture recognition by machines for a large sample of people is beyond the state of the art of the present pattern recognition and computer technology at this time.” The problem was rooted in the variations introduced by different camera angles, lighting conditions, tilt and rotation of the subject’s head, and other differences that would exist between pictures irrespective of the differences of the subject face itself. Computing power in 1964 was stretched to the limit to try to account for those differences.

A few years after Bledsoe left the project, Peter Hart continued the research at the Stanford Research Institute in 1966. In the spirit of John Henry pitting his mettle against the steam drill, Hart tested the software against humans in face-recognition trials using a database of 2,000 photographs. Hart found his computer consistently outperformed the humans, exclaiming, “It really worked!” It should be emphasized, however, that this “success” rested on the surprisingly poor performance of the human testers, which left a fairly low bar for the computer to clear to do better by comparison.

The inherent challenges in using biometrics and facial recognition for identification is perhaps illustrated by the incident that brought the Bertillon System to an end in 1903. Officials at Leavenworth Penitentiary in Kansas were processing a new inmate, named Will West, and performed the usual Bertillon procedure of capturing mugshots and biometric measurements. Vexingly, his measurements matched exactly an existing Bertillon Card for someone named “William West.” This was especially puzzling because William West was already incarcerated at Leavenworth and had been for two years serving a conviction for murder. Here were two men, with basically the same name, strikingly similar appearances, and identical Bertillon measurements—yet they were indeed two different people, unrelated and unknown to one another. The prison officers fingerprinted both men to show their fingerprints differed, thereby ushering in a new era of identification—but that’s another story.

The views and opinions expressed in this article are those of the author(s) and do not necessarily reflect the opinions, position, or policy of Berkeley Research Group, LLC or its other employees and affiliates.