Project 65: Multimodal Emotion Recognition in Response to Oil Paintings
Contact Information
Prof. Bao-Liang Lu
Email: bllu@sjtu.edu.cn
Project Description and Objectives
Most artworks are created to raise strong emotional responses and emotions in aesthetics are contained within the narrative. Artworks could be another potential approach to study how our brain perceives and processes affective information. Could we develop computational models to recognize human emotions in response to oil paintings? Based on our previous work in the lab, we have collected multimodal data including EEG and eye tracking while watching the oil paintings. We use oil paintings as stimuli to evoke three types of emotions, namely, negative, neutral, and positive, in affective brain-computer interfaces.
This project mainly analyzes the multimodal data to classify different emotions using multimodal deep neural networks and explores critical EEG and eye movement features of different emotions.
Eligibility Requirements
Interested students should have basic knowledge of machine learning and programming skills in Python.
Experience with TensorFlow or PyTorch is preferred.
Main Tasks
Finish a research report.
Develop multimodal deep learning methods to analyze multimodal data.
Website