TY - JOUR
T1 - Social Media Images Can Predict Suicide Risk Using Interpretable Large Language-Vision Models
AU - Badian, Yael
AU - Ophir, Yaakov
AU - Tikochinski, Refael
AU - Calderon, Nitay
AU - Klomek, Anat Brunstein
AU - Fruchter, Eyal
AU - Reichart, Roi
N1 - Publisher Copyright:
© 2023 Physicians Postgraduate Press, Inc.
PY - 2024/1
Y1 - 2024/1
N2 - Background: Suicide, a leading cause of death and a major public health concern, became an even more pressing matter since the emergence of social media two decades ago and, more recently, following the hardships that characterized the COVID-19 crisis. Contemporary studies therefore aim to predict signs of suicide risk from social media using highly advanced artificial intelligence (AI) methods. Indeed, these new AI-based studies managed to break a longstanding prediction ceiling in suicidology; however, they still have principal limitations that prevent their implementation in real-life settings. These include “black box” methodologies, inadequate outcome measures, and scarce research on non-verbal inputs, such as images (despite their popularity today). Objective: This study aims to address these limitations and present an interpretable prediction model of clinically valid suicide risk from images. Methods: The data were extracted from a larger dataset from May through June 2018 that was used to predict suicide risk from textual postings. Specifically, the extracted data included a total of 177,220 images that were uploaded by 841 Facebook users who completed a gold-standard suicide scale. The images were represented with CLIP (Contrastive Language-Image Pre-training), a state-of-the-art deep-learning algorithm, which was utilized, unconventionally, to extract predefined interpretable features (eg, “photo of sad people”) that served as inputs to a simple logistic regression model. Results: The results of this hybrid model that integrated theory-driven features with bottom-up methods indicated high prediction performance that surpassed common deep learning algorithms (area under the receiver operating characteristic curve [AUC] = 0.720, Cohen d=0.82). Further analyses supported a theory-driven hypothesis that at-risk users would have images with increased negative emotions and decreased belonginess. Conclusions: This study provides a first proof that publicly available images can be leveraged to predict validated suicide risk. It also provides simple and flexible strategies that could enhance the development of real-life monitoring tools for suicide.
AB - Background: Suicide, a leading cause of death and a major public health concern, became an even more pressing matter since the emergence of social media two decades ago and, more recently, following the hardships that characterized the COVID-19 crisis. Contemporary studies therefore aim to predict signs of suicide risk from social media using highly advanced artificial intelligence (AI) methods. Indeed, these new AI-based studies managed to break a longstanding prediction ceiling in suicidology; however, they still have principal limitations that prevent their implementation in real-life settings. These include “black box” methodologies, inadequate outcome measures, and scarce research on non-verbal inputs, such as images (despite their popularity today). Objective: This study aims to address these limitations and present an interpretable prediction model of clinically valid suicide risk from images. Methods: The data were extracted from a larger dataset from May through June 2018 that was used to predict suicide risk from textual postings. Specifically, the extracted data included a total of 177,220 images that were uploaded by 841 Facebook users who completed a gold-standard suicide scale. The images were represented with CLIP (Contrastive Language-Image Pre-training), a state-of-the-art deep-learning algorithm, which was utilized, unconventionally, to extract predefined interpretable features (eg, “photo of sad people”) that served as inputs to a simple logistic regression model. Results: The results of this hybrid model that integrated theory-driven features with bottom-up methods indicated high prediction performance that surpassed common deep learning algorithms (area under the receiver operating characteristic curve [AUC] = 0.720, Cohen d=0.82). Further analyses supported a theory-driven hypothesis that at-risk users would have images with increased negative emotions and decreased belonginess. Conclusions: This study provides a first proof that publicly available images can be leveraged to predict validated suicide risk. It also provides simple and flexible strategies that could enhance the development of real-life monitoring tools for suicide.
UR - http://www.scopus.com/inward/record.url?scp=85178240505&partnerID=8YFLogxK
U2 - 10.4088/JCP.23M14962
DO - 10.4088/JCP.23M14962
M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???
C2 - 38019588
AN - SCOPUS:85178240505
SN - 0160-6689
VL - 85
JO - Journal of Clinical Psychiatry
JF - Journal of Clinical Psychiatry
IS - 1
M1 - 23M14962
ER -