Enhancing Sentiment Classification in Video Content Via CrossModel Attention on Text, Audio and Visual Modalities

Abstract

Multi-modal sentiment analysis aims to understand human emotions by combining information from different data sources such as text, audio, and visual signals. However, many existing methods struggle to effectively capture and align these modalities, especially in video-based content where emotional cues are complex and interdependent. This paper proposes a unified framework for sentiment classification that integrates textual, acoustic, and visual features. The model uses BERT for text representation, ResNet for visual feature extraction, and Wav2Vec for audio processing. A crossmodal attention mechanism is applied to dynamically fuse information from all modalities, enabling better interaction and understanding between them. Experiments are conducted on the CMU-MOSEI dataset, A widely used benchmark for multi-modal sentiment analysis. The proposed approach is expected to improve classification accuracy compared to traditional uni-modal and simple fusion methods. The results demonstrate the importance of cross-modal interaction in capturing rich emotional context in real-world video data.