Smile to Me: Investigating Emotions and their Representation in Text-based Messaging in the Wild

Abstract

Emotions are part of human communication shaping mimics and representing

feelings. For this, conveying emotions has been integrated

in text-based messaging applications using emojis. While visualizing

emotions in text messages has been investigated in previous

work, we studied the effects of emotion sharing by augmented the

WhatsApp Web user interface – a text messenger people already use

on daily basis. For this, we designed and developed four different

visualizations to represent emotions detected through facial expression

recognition of chat partners using a web cam. Investigating

emotion representation and its effects, we conducted a four weeks

longitudinal study with 28 participants being inquired via 48 semistructured

interviews and 64 questionnaires. Our findings revealed

that users want to maintain control over their emotions, particularly

regarding sharing, and that they preferably view positive emotions

avoiding unpleasant social situations. Based on these insights, we

phrased four design recommendations stimulating novel approaches

for augmenting chats.