A real-time environmental translator for emotion recognition in autism spectrum disorder

Autism spectrum disorder (ASD) involves challenges in communication and social interaction, including challenges in recognizing emotions. Existing technological solutions aim to improve social behaviors in individuals with ASD by providing learning aids. This paper presents a real-time environmental translator designed to enhance social behaviors in individuals with ASD using sensory substitution. Our system utilizes vibrotactile and visual feedback to interpret and convey emotional states through vibration patterns emitted from small vibration motors on the user’s temple, complemented by color-coded displays of emotional intensity. It can detect seven emotions: neutral, sad, happy, angry, disgust, surprise, and fear. Testing with adults with ASD showed they could adapt to the system in about 19 min, enabling them to intuitively and immediately recognize others’ emotions. This innovative approach presents a promising advancement in emotion recognition technology for individuals with ASD, offering potential benefits in enhancing their social interactions and communication skills.