Journal of Undergraduate Research
Volume 1, Issue 8 - May 2000

Digitizing the Moving Face: Asymmetries of Emotion and Gender

Ashish Desai

ABSTRACT

In a previous study with dextral males, Richardson and colleagues [4] digitized real time video signals and found movement asymmetries over the left lower face for emotional, but not non-emotional expressions. These findings correspond to observations, based on subjective ratings of static pictures, that the left side of the face is more intensely expressive than the right [6]. From a neuropsychological perspective, one possible interpretation of these findings is that emotional priming of the right hemisphere of the brain results in more muscular activity over the contralateral left than ipsilateral right side of the lower face.

The purpose of the present study was to use computer-imaging methodology to determine whether there were gender differences in movement asymmetries across the face. We hypothesized that females would show less evidence of facial movement asymmetries during the expression of emotion. This hypothesis was based on findings of gender differences in the degree to which specific cognitive functions may be lateralized in the brain (i.e., females less lateralized than males).

Forty-eight normal dextral college students (25 females, 23 males) were videotaped while they displayed voluntary emotional expressions. A quantitative measure of movement change (called entropy) was computed by subtracting the values of corresponding pixel intensities between adjacent frames and summing their differences. The upper and lower hemiface regions were examined separately due to differences in the cortical enervation of facial muscles in the upper (bilateral) versus lower face (contralateral). Repeated measures ANOVA's were used to analyze for the amount of overall facial movement and for facial asymmetries.

Certain emotions were associated with significantly greater overall facial movement than others (p<.0001), beginning with surprise and followed by happy> fear> (angry = sad) > neutral. Both males and females showed this same pattern, with no gender differences in the total amount of facial movement under voluntary conditions. In males, movement asymmetries favoring the lower left side of the face occurred for most emotional expressions. For females, all emotions were symmetric over the lower face.

Our findings with computer digitizing techniques support the hypothesis that there are gender differences in facial movement asymmetries during the expression of emotion. They further underscore the view that emotional processing may represent a more widely distributed system throughout the brain in women than in men, corresponding to previous reports that language processes are also less lateralized in women.

INTRODUCTION

Facial expressions are rapid signals caused by changes in facial muscles that are brief and last only a few seconds. Rarely do they endure more than five seconds or fewer than 250 ms. Many studies have shown that the left side of the face is more emotionally expressive than the right [1,6]. Even though the basis for these asymmetries is unclear, one popular interpretation is that they reflect greater contribution of right hemisphere systems to emotional processing by contralateral enervation of the left side of the face. This is known as the right hemisphere emotional priming hypothesis.

The presumed mechanism for this hypothesis is that 1) The right hemisphere is important for emotional processing, and 2) The motor control of the lower two-thirds of each face is by the contralateral frontal area. Thus, the left hemisphere controls the movement of the right lower face, and the opposite is true for the right hemisphere [5].

Previous research of facial expression asymmetries among normal individuals has largely relied on subjective ratings by human judges of still photographs or video frames. Nonetheless, human observers typically do not observe facial signals as static stimuli. Facial expressions are dynamic interactions in which the face moves and changes from one expression to another. A recent study [2] was able to evaluate facial expressions dynamically.

In the Leonard study, they digitized video images and computed a quantitative measure of movement change (entropy) by subtracting the corresponding pixel intensities between adjacent frames and summing their differences. Therefore, these changes in signal value on a frame-by-frame basis represented the "signal" that corresponded to movement over the face.

Richardson and colleagues [4] have implemented techniques for analyzing video images that originated from [1]. In this study, video signals were digitized and analyzed for changes in pixel intensity on a frame-by-frame basis. The results revealed movement asymmetries over the lower face, consistent with the emotion-priming hypothesis. The emotional processing "primes" right frontal lobe motor systems, resulting in more movement over the left hemiface. We have modeled our study after Richardson and colleagues work.

In the study, forty right-handed males were used as subjects. Therefore the results can be generalized for males, but not for females. For our study, the purposes are 1) To replicate the Richardson and colleagues' findings, and 2) To determine whether females also show movement asymmetries across the lower face during voluntary emotional expressions.

When examining gender differences in facial asymmetry, a similar methodological problem is faced with most of the facial research: a strong basis on subjective ratings. Moreover, lesion and neuroimaging studies have suggested that cognitive functions are more bilaterally represented across both hemispheres of the brain in females, and males appear to have more of a lateralized representation of brain functioning [3]. With the previous research findings and the ability to have an objective rating of facial expressions, we hypothesize that females will display less robust facial movement asymmetries than males. Therefore, in females, emotional priming will occur from both hemispheres, resulting in similar movement across each side of the face.

Our lab adopted a computer imaging methodology to have the ability to systematically quantify movement changes over the face during the course of an expression. Two aspects are important to note for using this new methodology:

  1. We are observing dynamic, moving facial expressions
  2. We are able to obtain objective, quantifiable data

These factors are what previous research has lacked, and will enable researchers to have more powerful empirical studies on facial expressivity.

METHODS

Subjects

Forty-eight normal dextral subjects (25 female, 23 male) were recruited from the University of Florida student population. The reason only right handed subjects were used is because it is known that language is represented by the left hemisphere by the majority of right handed people, whereas left handed individuals have greater variability for language representation. Therefore we used right-handed people because we wanted to use a homogenous group of subjects.

Videotaping Facial Expressions

Subjects were told that they were participating in a study of facial expressions and would be videotaped by the experimenter throughout the session. The subjects were instructed to make voluntary facial expressions (e.g. happy, sad, anger, frightened, surprise, and disgust) in a randomized order. Each trial began with the presentation of a card, to the camera, denoting the target expression. After, there was a one-second delay, and an auditory tone was used to signal the subject to begin the target emotion. The tone cued was also synchronized with the onset of light diodes that provided a visual marker of trial onset on the videotape. During the videotaping portion we controlled for movement and lighting biases by securing the head in a comfortable head restraint while providing indirect lighting. If we did not control for these factors, our results would be biased.

Capturing and Digitizing Video Frames

After the videotaping portion, the tapes were analyzed. Trained research assistants who were blinded to the experimental hypotheses edited the videotapes. Once the target expression was chosen, the initial light of the diodes identified the onset frame of the expression. 30 consecutive still frames from a 900 ms portion of the expression were selected for digitizing using the Eyeview software. Each frame was 30.75 ms in duration and consisted of a 640 X 480 ms pixel array depicted at 256 levels of grey scale.

Landmarking and Extracting Regions of Interest

Once all the images were collected, we needed to segment the face into different regions. Our interest was in right versus the left side of the face, and upper versus the lower regions of the face. We were interested in the different regions because they are differently innervated by contralateral and bilateral motor systems. The different muscle groups of the face determined the regions. Didem Gokcay developed a semi-automated method for automatically segmenting the face into different regions. There were eighteen anatomic landmarks were located on the face using a mouse. Custom software (in PV-Wave) used these landmarks to compute geographic boundaries for different regions of the face (upper, lower, right, left). The boundaries were automatically applied to all face images in a particular expression. Even though we were able to divide the face into different hemiface regions, we combined the middle and lower regions. Our focus was on the differences between upper, lower, left, and right hemiface.

Quantifying Expression Change

Two inherent assumptions for digitizing facial expressions were that a) changes in the surface lighting of the face reflect movement, and b) The signal change was direct quantitative index of facial mobility. Each digitized image represented a 640 X 480 pixel array at 256 levels of grey scale. For each expression, we computed differences in pixel intensity, point by point, over consecutive frames for specified regions of the face, divided by the number of pixels in that region. This computation was repeated for each pair of adjacent frames. The resulting value is referred to as "ENTROPY". The formula for computing Entropy:


Ei(t) =Snj(t)/Ni*log(nj(t)/Ni)

Thus, entropy is a measure of pixel intensity change that occurred over the face as it moved during the course of expression. Again, this was done automatically, in a software program in PV wave.

Figure 1 shows graphs of entropy over consecutive frames for the right and left hemiface. The person was making a happy facial expression. The maximum point is the maximum amount of entropy or movement change. If you were to show an individual video frame, this is the point where someone is most likely to identify the facial expression. Entropy was used as the dependent variable in various statistical analyses of variance that were completed. It is important to remember that this approach is highly quantitative with no subjective ratings.

Figure 1. Graphs of entropy over consecutive frames for the right and left hemiface.
Figure 1. Graphs of entropy over consecutive frames for the right and left hemiface.

RESULTS

Whole Face Entropy

We initially analyzed which facial expressions resulted in the greatest overall amount of change or movement by examining entropy scores from the entire face by gender. Entropy was the dependent measure used. The emotional expressions (surprise, happy, fear, anger, disgust, and sad) were analyzed by analyses of variance (ANOVA). From Figure 2, certain emotions were associated with more overall facial movement than others were, and this pattern was similar for both males and females. Surprise, happy, and fear were significantly different, whereas anger, disgust, and sad were not. There was no sex difference in total facial movement. Thus, males have the capacity to be as expressive as females.

Figure 2. Whole Face Entropy.
Figure 2. Whole Face Entropy.

Hemiregional Asymmetries

A second set of analyses studied whether there were hemiregional differences in signal change across the face during emotional expressions by gender (Figure 3). For the lower face, males displayed movement asymmetries favoring the left hemiface during voluntary facial emotions. This finding corresponds to observations of [4] and is consistent with the "emotion priming" hypothesis. However, females showed no hemiface movement asymmetries across the lower face. This supports our hypothesis. For the upper face, neither male nor females showed consistent, significant asymmetries. This evidence is in line with bilateral ennervation of the upper 1/3 of the face.

Figure 3. Lower Face: Right versus Left Hemiface.

Figure 3. Lower Face: Right versus Left Hemiface.

DISCUSSION

Previous focal lesion and neuroimaging studies have suggested that language and other cognitive functions may be less lateralized in women than in men. The current findings add to this literature by suggesting gender differences in facial movement asymmetries during the expression of voluntary facial emotions. In males, emotions are more aligned with right hemisphere systems which prime right (frontal) motor areas. This results in greater movement over left hemiface. In females, emotions are more widely distributed and prime motor areas of both hemispheres. This results in symmetrical movements over the left and right face. In women, our finding of no difference between the left and right hemiface, support the notions that emotional processing may represent a more widely distributed and less "lateralized" system than in does for men.

REFERENCES

  1. Borod, J.C., Haywood, C.S., and Koff, E. (1997). Neuropsychological aspects of facial asymmetry during emotional expression: a review of the normal adult literature. Neuropsychological Review 7, 41-60.
  2. Leonard, C., Voeller, K.K.S., and Kuldau, J.M. (1991). When's a smile a smile? Or how to detect a message by digitizing the signal. Psychological Science 2, 166-172.
  3. McGlone, J. (1980). Sex differences in human brain asymmetry: A critical survey. The Behavioral and Brain Sciences 3, 215-262.
  4. Richardson, C., & Bowers, D. (1999). Digitizing the Moving Face During Dynamic Facial Expressions (in press).
  5. Rinn, W.B. (1984). The neuropsychology of facial expression: A review of the neurological and psychological mechanisms for producing facial expression. Psychological Bulletin 95, 52-77.
  6. Sackeim, H.A., Gur, R.C., and Saucy, M.C. (1978). Emotions are expressed more intensely on the left side of the face. Science 202, 834-836.

--top--

Back to the Journal of Undergraduate Research