Face Matching Methods Questioned

Rebecca of the Yale University School of Art writes:

It is something cartoonists do but can never explain. Setting aside for the moment props like FDR’s cigarette holder or Charley Chaplin’s bowler and cane, cartoonists with only facial attributes to go on simultaneously minimize and exaggerate what they see to make people instantly recognizable. We can see it has nothing to do with Vignony like markings or points of congruence and everything to do with artistic genius. Ask yourself how it is we recognize these four presidents.

I don’t know and neither do you.


Ask yourself how it is it that we know the sculpture in a dried up apple looks like Jack Nicholson as the Joker. Is it the great lips, the eyes, points of congruence or some Vignony like markings? It is something no one can explain. 


The artists of early Christianity portrayed Jesus based on something. Maybe it was the Shroud which is what I think. I can’t explain why, however. One thing I am sure of though, you can’t claim anything based on a fake scientific method like points of congruence. There is no such method for matching faces.

Note: The examples were attached to an email. I resized them and inserted them into the email where it seemed appropriate.

8 thoughts on “Face Matching Methods Questioned”

  1. Rebecca has a point, but is unnecessarily defeatist.
    “I don’t know, and nether do you.”
    “It is something no one can explain.”
    “One thing I am sure of though, you can’t claim anything based on a fake scientific method like points of congruence. There is no such method for matching faces.”

    The second two quotations are simply untrue. I don’t know about cartoons, but across the world people are allowed into and out of countries on the basis of a computer comparison of their appearance in a camera and their photo on a passport. Algorithms for identifying faces have been devised, are being continually refined, and are sufficiently good at their job to be used at immigration control. They are no doubt biased to avoid false positives, which is why they throw up so many false negatives, but they are there and on the whole they work.

  2. Rebecca has a good point questionning face matching between the shroud “photograph like” and frescoes/icons “cartoon like” using facial recognition algorithms which are known to fail at cross-matching between cartoons based on facial attributes and photographs.

    Why do such methods fail ? because it is not how our brain works.

    1. The special knack of the cartoonist to conjure up an instantly recognizable face with a few strokes of a pencil is highly intriguing. I suspect it taps into something fairly profound as to the manner in which the human visual cortex operates, far behind the eyes, connected only by the optic nerves, like telecommunication cables.

      It’s known already that we acquire and store visual templates in early life that play an initial filtration and storing role. Francis Crick of DNA double helix fame switched to this aspect of brain function late in his career and wrote a fascinating book on the subject.

      My view, no doubt fanciful, is that we don’t store images like multi-pixel photographs – that would place too great a demand on our neurons and synapses – despite our having billions of them.

      What we do is scan an image, received through the eye, projected first onto the retina as a real inverted image, then transmitted to the cortex, to become an entirely ‘virtual’ image (human computer code – but non binary!). The latter has then to be quickly captured and stored as a neuronal ‘snapshot’ on the basis of its peculiarities, i.e. solely those aspects that make it DIFFERENT from our stored archive of standard types. We match to a template, but then store the little bits that don’t fit.

      The genius of the cartoonist is to focus mainly on those tiny give-aways, the ones that make a face instantly recognizable, and then exaggerate them slightly, making an immediate impact on our visual apparatus, thereby hitting all the right notes, if you’ll pardon the musical analogy.

      It’s perhaps not surprising that computer algorithms for image recognition cannot cope with cartoons. The human brain does not operate on binary code.

      1. Perhaps this is a way to test the limits of artificial intelligence. Going sci-fi here, but maybe the way we can identify a Terminator (in the future) is by showing him a cartoon of Arnold Schwarzenegger and asking, “Who is this?” If he doesn’t guess Arnie, start blasting.

      2. “It’s perhaps not surprising that computer algorithms for image recognition cannot cope with cartoons.” Not yet, perhaps, but it’s being worked on quite seriously. In a year or so not even Donald Duck will escape recognition..

  3. Despite our constitutional monarchy preoccupations in NZ, like O.K. I can still recognise at least three of the U.S. presidents in the cartoons. But it is I think significant that facial recognition computer software cannot. Hugh notes that such software is used for pass-port entries at immigration controls, and most of the time is successful, but that there are also false positives and false negatives. Colin has made some shrewd observations of distinctions between such software and the way our brains work. The software provides objective measurement but is imperfect. Our brain works on some kind of subjective measurement, but making it difficult to express communication of the comparisons between different observers. This makes discussion of whether perceived images of subjects A and B are the of same person or not controversial and even contentious.

    We see this all the time in our discussions of comparisons between Christ-icons and the Shroud image. We have come a long way since the Vignon markings; they might be seen as an analogue of the computer software. But Vignon markings are often considered suspect when it comes to asserting a Shroud-like facial image on an icon.

    Nevertheless many of us see the likeness of icons to the Shroud facial image. Others may deny such likeness. How much of it is wishful thinking; How much of it is merely personal denial? We cannot say. It seems to be merely in the perceptions of the beholder! Presumably one might draw a cartoon of the Shroud facial image, instantly recognisable by many observers, but which would be rejected by all facial recognition software, a significant point, I think!

Comments are closed.