The Internal State of an LLM Knows When It's Lying

Amos Azaria, Tom Mitchell

פרסום מחקרי: פרק בספר / בדוח / בכנספרסום בספר כנסביקורת עמיתים

23 ציטוטים ‏(Scopus)

תקציר

While Large Language Models (LLMs) have shown exceptional performance in various tasks, one of their most prominent drawbacks is generating inaccurate or false information with a confident tone. In this paper, we provide evidence that the LLM's internal state can be used to reveal the truthfulness of statements. This includes both statements provided to the LLM, and statements that the LLM itself generates. Our approach is to train a classifier that outputs the probability that a statement is truthful, based on the hidden layer activations of the LLM as it reads or generates the statement. Experiments demonstrate that given a set of test sentences, of which half are true and half false, our trained classifier achieves an average of 71% to 83% accuracy labeling which sentences are true versus false, depending on the LLM base model. Furthermore, we explore the relationship between our classifier's performance and approaches based on the probability assigned to the sentence by the LLM. We show that while LLM-assigned sentence probability is related to sentence truthfulness, this probability is also dependent on sentence length and the frequencies of words in the sentence, resulting in our trained classifier providing a more reliable approach to detecting truthfulness, highlighting its potential to enhance the reliability of LLM-generated content and its practical applicability in real-world scenarios.

שפה מקוריתאנגלית
כותר פרסום המארחFindings of the Association for Computational Linguistics
כותר משנה של פרסום המארחEMNLP 2023
מוציא לאורAssociation for Computational Linguistics (ACL)
עמודים967-976
מספר עמודים10
מסת"ב (אלקטרוני)9798891760615
מזהי עצם דיגיטלי (DOIs)
סטטוס פרסוםפורסם - 2023
אירוע2023 Findings of the Association for Computational Linguistics: EMNLP 2023 - Singapore, סינגפור
משך הזמן: 6 דצמ׳ 202310 דצמ׳ 2023

סדרות פרסומים

שםFindings of the Association for Computational Linguistics: EMNLP 2023

כנס

כנס2023 Findings of the Association for Computational Linguistics: EMNLP 2023
מדינה/אזורסינגפור
עירSingapore
תקופה6/12/2310/12/23

טביעת אצבע

להלן מוצגים תחומי המחקר של הפרסום 'The Internal State of an LLM Knows When It's Lying'. יחד הם יוצרים טביעת אצבע ייחודית.

פורמט ציטוט ביבליוגרפי