Mixed emotion detection in chat messages - it feels so good and so bad

Cloud created by:

Rebecca Ferguson
15 May 2017

Garron Hillaire, Jenna Mittelmeier, Bart Rienties, Mark Fenton-O'Creevey, Zdenek Zdrahal and Dirk Tempelaar

In this presentation a proposed measure for mixed emotions in text will be described. A brief outline of why mixed emotions are important to measure in the context of learning will precede the measure validation results. The data for this study will be from a pilot study with N=982 freshman students in the Netherlands in a statistics course. The study examines their activity working in a computer supported collaborative working environment during a 72-minute course lab where they were using a chat feature in order to create a group response to a case study. The standard for identification of mixed emotional expression used in this analysis are messages identified as mixed emotion expression by students when examining their own group work conversations in a post activity. The lab ran during the span of a school week with participants attending one lab session during the week. The post activity was completed by the following Sunday after the week of labs. A brief interpretation of what messages from group chat were categorized as mixed emotion will be provided to help gain insights into what people are identifying as mixed emotional expression. While the focus of the proposed measure is on detecting mixed emotional expression in written communication the categories of positive, negative, neutral, and mixed will be benchmarked against a state-of-the-art sentiment analysis technology. While valence is not precisely emotion detection a second benchmark will be a machine learning classifier of mixed emotion expression. When establishing the proposed measure, a machine learning bag-of-words approach towards classifying mixed emotion expression was created using the examples of mixed emotion messages provided by the students. By comparing the new measure to both a state of the art sentiment analysis method as well as a machine learning classifier, the results show how the proposed measure fits between valence measurement and bag-of-words classification of written expression for the purpose of mixed emotion detection.

Extra content

Embedded Content