Leslie Zebrowitz’s talk brought her work—which deals with how deep-seated tendencies to overgeneralize distort our assessment of people,–to bear on the issue of justice in the courtroom.

The premise of Zebrowitz’s work on overgeneralization is that the adaptive, useful reactions we have to features such as a baby’s cute face, the disfiguration caused by disease or genetic malady, or the familiarity of our family members, also distort our impression of people whose faces happen to have those characteristics.

It is essential for the survival of the species that we should perceive babies as attractive, innocent, needing and deserving our care and protection. Babies’ faces are round, with little noses and chins, big eyes and high foreheads. Some adults, through no action of their own but simply due to the genetics of their bone structure, happen to have faces with baby-like features. Adults with “baby faces” are perceived as having the qualities of a baby: naive, guileless, etc. Adults with mature faces – strong square chin, heavy brow, and lower forehead — are perceived to be more aggressive, but also more responsible and intelligent. Although these perceived qualities do not have any basis in the actions or character of the subject, arising from how their bones, hair and skin developed, they may strongly affect the observer’s impressions. These misapprehensions, which can result in costly errors for the perceiver, subject, or both, persist because the evolutionary value of reacting with nurturance, acceptance, and patience to an actual baby outweighs the disadvantage of sometimes misperceiving adult characteristics due to the look of a face.

There are times, however, when this misapprehension can be extremely costly. One of them is in the court system. Dr. Zebrowitz, with her colleagues and students, has looked at “baby-face” perceptions and other overgeneralizations and how they affect justice in court. They found that these effects do influence court outcomes; for example, babyfaced defendants are more likely to be found innocent in cases involving intentional actions, and more likely to be found guilty in cases involving negligence. Justice is not, in fact, blind. But these cases highlight why the ideal of justice is portrayed as blind or unbiased. Seeing the participants in the case introduces errors in person perception, errors which can be serious yet subtle and hard to address.

But can we make justice blind? What, ideally, should juries and judges be able to see of the defendant, plaintiff or witnesses? Many feel that it they need to see the accused in order to assess their words and gauge how believable they are. Yet if that judgment is based on distorted character perception, then perhaps justice is better served by being blind, by not seeing the participant in a case, no matter how important it feels.

Would we be better off with a purely audio court? This is an intriguing idea. There are studies that show that people are better at distinguishing truth from deception when only listening to audio recordings, rather than seeing a video (with audio). Yet, for other assessments, such as understanding the rapport and relationship between people, visual cues are more important. The issue is further complicated because the existence of cues to deception in a particular modality does not necessarily match the observer’s use of those cues – indeed, the problem Dr Zebrowitz’s work addresses is people’s erroneous assessments.

These are not just hypothetical questions. Virtual courts are being developed. As we design the technology for these computer-mediated justice systems, we must think about the impact of different ways of representing people, and how those representations might impact our ability to assess honesty, character, and relationships. Although the first impulse might be to use the most richly detailed media available; however, if one channel tends to make us less accurate in our perceptions, is its absence better?

Intriguingly, we can also investigate the possibility of excising only the problematic aspects of visual appearance. It can be very useful to see a person’s expressions and their gestures. A subject who sits calmly and attentively gives off a different impression than one who fidgets constantly and seems uninterested, distracted or annoyed, which is different again from the one who glares menacingly at the jury. These gestures and expressions, unlike the features that lead to overgeneralization problems, are not an artifact of genetic inheritance, but actions generated by the person, and are useful and relevant in assessing him or her. Could we create a court technology where all the key participants were given the same neutral appearance, but where gestures and expressions were still visible?

It is possible to use the actions of a person to drive the animation of an avatar. Would this be desirable in a future virtual court? It eliminates much of the facial overgeneralization issues. It retains the voice, which is necessary for detecting deception. The voice would usually reveal gender, and sometimes race and socioeconomic position. Such an experiment raises very interesting questions about what aspects of a person’s physique are relevant to the court. For instance, we could make all these avatars equal in size. But in a case where much of the case depends on knowing if the plaintiff had reason to be scared of the defendant, seeing that the plaintiff is 5 feet and frail and the defendant is a hulking 6 foot plus, makes a difference. So here basic physical shape would need to be conveyed.

This opens other questions. The underlying thought experiment is as follows: in the context of a trial, if we can control what aspects of a person’s physical appearance are introduced into the proceedings, what elements should we include, and what should we omit? If a defendant is very baby-faced and innocent looking, and is charged with say, fraud or with deceptive sales practices, is it not relevant to see what he or she looks like, in order to have insight into how the plaintiffs could have been taken in?

Another issue is familiarity. If someone looks like you, or like many people you know, you’ll believe them to be more trustworthy. This is an artifact of your perception, not a fact based on any trait of theirs. Similarly, if someone resembles an individual about whom you have strong feeling, you are likely to transfer those feelings to that person. Thus, the defendant who has the luck of looking like a juror’s beloved uncle may be warmly defended by that juror. (see also Bailenson’s work showing that making another look more like you, by blending photographs, makes them seem more trustworthy).

Zebrowitz notes the effect of facial features is different in determining guilt and in sentencing: attractive individuals have been found to receive lighter sentences, though attractiveness did not affect whether they were found guilty. One could address this problem without high tech solutions, by maintaining face to face court proceedings, but having a separate judge determine sentencing. The sentencing judge would be given the written summary of the case, with all the relevant information, but would not see or hear the participants, thus working from all – and only – the information deemed legally relevant for that decision.

What, ideally, would we see of others in a given situation in order to have all the information we need about them, but without having the information that leads us to make erroneous assumptions? The court scenario shows vividly and with serious consequences the effects of our overgeneralizations and other error-prone impression formation processes – but this question has relevance far beyond the legal system. It has implications in how we perceive politicians, in how we determine who to trust. It makes us think about what aspects of a person’s appearance are germane to a discussion.