Harnessing Haptic Technology for Real-Time Emotion Detection

Lital Levy; Yuval Blum; Asmare Ambaw; Roi Yozevitch; Eldad Holdengreber

doi:10.1109/LSENS.2025.3538804

Harnessing Haptic Technology for Real-Time Emotion Detection

Lital Levy, Yuval Blum, Asmare Ambaw, Roi Yozevitch, Eldad Holdengreber

Research output: Contribution to journal › Article › peer-review

Abstract

This letter introduces a novel multi-modal environmental translator for real-time emotion recognition. The system integrates facial expression recognition (FER) and speech emotion recognition (SER) to analyze visual and vocal cues while conveying emotional feedback through vibrotactile signals. Emotions are mapped to distinct vibration frequencies - ranging from 0.4 Hz for neutral to 35 Hz for anger - enabling users to identify seven core emotions through tactile sensations intuitively. A user study involving ten participants demonstrated an average adaptation time of fewer than 7 min, indicating the system's effectiveness in quickly familiarizing users with the vibration signals. Overall, this innovative solution provides a robust approach to enhancing real-time emotion recognition through haptic feedback, making it suitable for everyday social interactions.

Original language	English
Article number	5500804
Journal	IEEE Sensors Letters
Volume	9
Issue number	3
DOIs	https://doi.org/10.1109/LSENS.2025.3538804
State	Published - 2025

Keywords

Sensor systems
emotion recognition
facial expression recognition (FER)
haptic feedback
machine learning (ML)
sensory substitution
speech emotion recognition (SER)

Access to Document

10.1109/LSENS.2025.3538804

Cite this

@article{d99b8f637373414c9edfbbb9e7d74cd8,

title = "Harnessing Haptic Technology for Real-Time Emotion Detection",

abstract = "This letter introduces a novel multi-modal environmental translator for real-time emotion recognition. The system integrates facial expression recognition (FER) and speech emotion recognition (SER) to analyze visual and vocal cues while conveying emotional feedback through vibrotactile signals. Emotions are mapped to distinct vibration frequencies - ranging from 0.4 Hz for neutral to 35 Hz for anger - enabling users to identify seven core emotions through tactile sensations intuitively. A user study involving ten participants demonstrated an average adaptation time of fewer than 7 min, indicating the system's effectiveness in quickly familiarizing users with the vibration signals. Overall, this innovative solution provides a robust approach to enhancing real-time emotion recognition through haptic feedback, making it suitable for everyday social interactions.",

keywords = "Sensor systems, emotion recognition, facial expression recognition (FER), haptic feedback, machine learning (ML), sensory substitution, speech emotion recognition (SER)",

author = "Lital Levy and Yuval Blum and Asmare Ambaw and Roi Yozevitch and Eldad Holdengreber",

note = "Publisher Copyright: {\textcopyright} 2025 IEEE.",

year = "2025",

doi = "10.1109/LSENS.2025.3538804",

language = "אנגלית",

volume = "9",

journal = "IEEE Sensors Letters",

issn = "2475-1472",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "3",

}

TY - JOUR

T1 - Harnessing Haptic Technology for Real-Time Emotion Detection

AU - Levy, Lital

AU - Blum, Yuval

AU - Ambaw, Asmare

AU - Yozevitch, Roi

AU - Holdengreber, Eldad

PY - 2025

Y1 - 2025

N2 - This letter introduces a novel multi-modal environmental translator for real-time emotion recognition. The system integrates facial expression recognition (FER) and speech emotion recognition (SER) to analyze visual and vocal cues while conveying emotional feedback through vibrotactile signals. Emotions are mapped to distinct vibration frequencies - ranging from 0.4 Hz for neutral to 35 Hz for anger - enabling users to identify seven core emotions through tactile sensations intuitively. A user study involving ten participants demonstrated an average adaptation time of fewer than 7 min, indicating the system's effectiveness in quickly familiarizing users with the vibration signals. Overall, this innovative solution provides a robust approach to enhancing real-time emotion recognition through haptic feedback, making it suitable for everyday social interactions.

AB - This letter introduces a novel multi-modal environmental translator for real-time emotion recognition. The system integrates facial expression recognition (FER) and speech emotion recognition (SER) to analyze visual and vocal cues while conveying emotional feedback through vibrotactile signals. Emotions are mapped to distinct vibration frequencies - ranging from 0.4 Hz for neutral to 35 Hz for anger - enabling users to identify seven core emotions through tactile sensations intuitively. A user study involving ten participants demonstrated an average adaptation time of fewer than 7 min, indicating the system's effectiveness in quickly familiarizing users with the vibration signals. Overall, this innovative solution provides a robust approach to enhancing real-time emotion recognition through haptic feedback, making it suitable for everyday social interactions.

KW - Sensor systems

KW - emotion recognition

KW - facial expression recognition (FER)

KW - haptic feedback

KW - machine learning (ML)

KW - sensory substitution

KW - speech emotion recognition (SER)

UR - http://www.scopus.com/inward/record.url?scp=85217528524&partnerID=8YFLogxK

U2 - 10.1109/LSENS.2025.3538804

DO - 10.1109/LSENS.2025.3538804

M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???

AN - SCOPUS:85217528524

SN - 2475-1472

VL - 9

JO - IEEE Sensors Letters

JF - IEEE Sensors Letters

IS - 3

M1 - 5500804

ER -

Harnessing Haptic Technology for Real-Time Emotion Detection

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this