Result description
Human emotions entail a complex set of behavioral, physiological and cognitive changes. Current state-of-the-art models fuse the behavioral and physiological components using classic machine learning, rather than recent deep learning techniques. We propose to fill this gap, designing the Multimodal for Video and Physio (MVP) architecture, streamlined to fuse video and physiological signals.
MVP exploits the benefits of attention to enable the use of long input sequences (1-2 minutes).We have studied video and physiological backbones for inputting long sequences and evaluated our method with respect to the state-of-the-art. Our results show that MVP outperforms former methods for emotion recognition based on facial videos, EDA, and ECG/PPG.
Addressing target audiences and expressing needs
- Business partners – SMEs, Entrepreneurs, Large Corporations
- Technology Transfer Expertise
- Collaboration
Partners
- Other Actors who can help us fulfil our market potential
- Research and Technology Organisations
- Academia/ Universities
R&D, Technology and Innovation aspects
Research publication
Result submitted to Horizon Results Platform by DEUTSCHES FORSCHUNGSZENTRUM FUR KUNSTLICHE INTELLIGENZ GMBH
