Prediction and Localization of Student Engagement in the Wild

Aamir Mustafa, Amanjot Kaur, Love Mehta, Abhinav Dhall

Published 2018-04-03, updated 2018-04-16Version 3

Student engagement localization can play a key role in designing successful e-learning systems. Facial expressions of an individual in response to stimuli videos provide important cues in estimating variations in engagement level. In this paper we study the association of a \textbf{subject}'s facial expressions with his/her engagement level, as annotated by labelers. We then localize engaging/non-engaging parts in the stimuli videos using a deep multiple instance learning based framework, which can give useful insight into designing Massive Open Online Courses (MOOCs) video material. Recognizing the lack of any publicly available dataset in the domain of user engagement, a new `in the wild' database is created to study subject engagement. The dataset contains 725,000 frames (about 9 hours of recording), 102 videos captured from 75 subjects. The problem of engagement prediction is modeled as a weak label learning problem. The dataset is manually annotated by labelers for four levels of engagement and the predicted labels of videos are correlated with the manual labels. This dataset creation is an effort to facilitate research in various e-learning environments such as intelligent tutoring systems , MOOCs and others.