Detecting sentences that may be harmful to children with special needs

Children and adults with special needs may find it difficult to recognize danger and threats as well as socially complex situations. Thus, they are under the risk of being victims of exploitation, violence and attacks. In addition, they may find themselves unintendedly insulting their friends, relatives or caregivers. In this paper, we propose an autonomous agent to assist the special needs person (child or adult) in the goal of recognizing risky or insulting situations. The autonomous agent will detect these situations and will signal them to the user (by text, speech, or other signaling forms). We composed a dataset containing 13,490 sentences, categorized into one of four classes: A 'normal' sentence, an insulting sentence, a negative sentence about a different person, or a risky sentence that may indicate a dangerous situation for the special needs person, which requires immediate intervention. We used several machine learning methods, and we found that the most accurate methods were the random forest method with 100 estimators, a voting method using several classifiers, and a convolutional neural network (CNN) with embedding. All of these mechanisms reached an accuracy close to 70% in classifying the sentences in the test set. Finally, using an ensemble method comprising a panel of the 5 best CNN based methods, improves the accuracy of the results and the F1-score. Our results demonstrate the feasibility of building an assisting agent that will accompany the special needs children and adults, and assist them in their daily social interactions.