Overview of the CLPsych 2024 Shared Task: Leveraging Large Language Models to Identify Evidence of Suicidality Risk in Online Posts

Jenny Chim, Adam Tsakalidis, Dimitris Gkoumas, Dana Atzil-Slonim, Yaakov Ophir, Ayah Zirikly, Philip Resnik, Maria Liakata

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

14 Scopus citations

Abstract

We present the overview of the CLPsych 2024 Shared Task, focusing on leveraging open source Large Language Models (LLMs) for identifying textual evidence that supports the suicidal risk level of individuals on Reddit. In particular, given a Reddit user, their predetermined suicide risk level (‘Low’, ‘Moderate’ or ‘High’) and all of their posts in the r/SuicideWatch subreddit, we frame the task of identifying relevant pieces of text in their posts supporting their suicidal classification in two ways: (a) on the basis of evidence highlighting (extracting sub-phrases of the posts) and (b) on the basis of generating a summary of such evidence. We annotate a sample of 125 users and introduce evaluation metrics based on (a) BERTScore and (b) natural language inference for the two sub-tasks, respectively. Finally, we provide an overview of the system submissions and summarise the key findings.

Original languageEnglish
Title of host publicationCLPsych 2024 - 9th Workshop on Computational Linguistics and Clinical Psychology, Proceedings of the Workshop
EditorsAndrew Yates, Bart Desmet, Emily Prud�hommeaux, Ayah Zirikly, Steven Bedrick, Sean MacAvaney, Kfir Bar, Molly Ireland, Yaakov Ophir, Yaakov Ophir
PublisherAssociation for Computational Linguistics (ACL)
Pages177-190
Number of pages14
ISBN (Electronic)9798891760806
StatePublished - 2024
Event9th Workshop on Computational Linguistics and Clinical Psychology, CLPsych 2024 - St. Julian's, Malta
Duration: 21 Mar 2024 → …

Publication series

NameCLPsych 2024 - 9th Workshop on Computational Linguistics and Clinical Psychology, Proceedings of the Workshop

Conference

Conference9th Workshop on Computational Linguistics and Clinical Psychology, CLPsych 2024
Country/TerritoryMalta
CitySt. Julian's
Period21/03/24 → …

Fingerprint

Dive into the research topics of 'Overview of the CLPsych 2024 Shared Task: Leveraging Large Language Models to Identify Evidence of Suicidality Risk in Online Posts'. Together they form a unique fingerprint.

Cite this