Abstract
Continuous affect recognition from facial images aims to estimate the values of multiple affective dimensions from a facial image sequence. To leverage relevant information between multiple affective dimensions, multitask learning has been used in the estimation of continuous affective states. Most of the existing multitask continuous affect recognition methods focus on designing elaborate multitask networks. Meanwhile, a few research works consider using multitask training strategies for continuous affect recognition. In general, existing multitask continuous affect recognition methods face the problem of unstable training effects. In this work, to improve the stability of multitask learning, we propose an ensemble learning-enhanced multitask network architecture for continuous affect recognition. In addition, we introduce a novel adaptive weighted loss-based multitask learning strategy to effectively train the proposed multitask continuous affect recognition model. Experimental results, on the RECOLA, SEMAINE and AFEW-VA datasets for continuous affect recognition, demonstrate the potential of the proposed method compared to state-of-the-art methods.
Original language | English |
---|---|
Article number | 121290 |
Pages (from-to) | 1-12 |
Number of pages | 12 |
Journal | Expert Systems with Applications |
Volume | 236 |
Issue number | February 2024 |
Early online date | Aug 2023 |
DOIs | |
Publication status | Published - Feb 2024 |
Bibliographical note
Funding Information:This work is supported by the Natural Science Basic Research Program of Shaanxi Province (Program No. 2022JQ-662 ), the National Natural Science Foundation of China (Program No. 62206219 ), the Special Construction Fund for Key Disciplines of Shaanxi Provincial Higher Education , and the National Natural Science Foundation of China (Program No. 62376215 ).
Publisher Copyright:
© 2023 Elsevier Ltd