The emotion predictors use the observed facial expressions as input to calculate the likelihood of an emotion. Our Facial expression to emotion mapping builds on EMFACS mappings developed by Friesen & Ekman. A facial expression can have either a positive or a negative effect on the likelihood of an emotion. The following table shows the relationship between the facial expressions and the emotions predictors.

Emotion Increase Likelihood Decrease Likelihood
Joy Smile Brow Raise
Brow Furrow
Anger Brow furrow
Lid Tighten
Eye Widen
Chin Raise
Mouth Open
Lip Suck
Inner Brow Raise
Brow Raise
Smile
Disgust Nose Wrinkle
Upper Lip Raise
Lip Suck
Smile
Surprise Inner Brow Raise
Brow Raise
Eye Widen
Jaw Drop
Brow Furrow
Fear Inner Brow Raise
Brow Furrow
Eye Widen
Lip Stretch
Brow Raise
Lip Corner Depressor
Jaw Drop
Smile
Sadness Inner Brow Raise
Brow Furrow
Lip Corner Depressor
Brow Raise
Eye Widen
Lip Press
Mouth Open
Lip Suck
Smile
Contempt Brow Furrow
Smirk
Smile

How do we calculate valence?

The Valence metric likelihood is calculated based on a set of observed facial expressions:

Increase Positive Likelihood Increase Negative Likelihood
Smile
Cheek Raise
Inner Brow Raise
Brow Furrow
Nose Wrinkle
Upper Lip Raise
Lip Corner Depressor
Chin Raise
Lip Press
Lip Suck

How do we calculate engagement?

Engagement or expressiveness is a weighted sum of the following facial expressions:

  • Brow raise
  • Brow furrow
  • Nose wrinkle
  • Lip corner depressor
  • Chin raise
  • Lip pucker
  • Lip press
  • Mouth open
  • Lip suck
  • Smile