Multimodel Intension Model

Talk overview

Nova presents a Multimodal Intention Model (MMIM) for On-Device Deployment. Conventional intent recognition often relies on a single modality, such as speech or text, and thus struggles to capture users' implicit emotional and social intentions. To address this limitation, MMIM integrates multimodal signals—including facial expressions, voice, language, posture, and physiological cues—into a unified intention space. It employs cross-modal alignment, temporal fusion, and context-aware reasoning to jointly infer explicit requests and implicit affective intentions. The model is designed to support empathetic, personalized, and trustworthy human–robot interaction in scenarios such as elderly care, child companionship, and mental health support. Experimental results demonstrate that MMIM improves intention recognition accuracy, robustness under missing modalities, and user-perceived naturalness compared with unimodal and conventional multimodal baselines. The proposed model offers a step toward more emotionally intelligent companion robots.