Saif: A correction-detection deep-learning architecture for personal assistants

Amos Azaria, Keren Nivasch

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


Intelligent agents that can interact with users using natural language are becoming increasingly common. Sometimes an intelligent agent may not correctly understand a user command or may not perform it properly. In such cases, the user might try a second time by giving the agent another, slightly different command. Giving an agent the ability to detect such user corrections might help it fix its own mistakes and avoid making them in the future. In this work, we consider the problem of automatically detecting user corrections using deep learning. We develop a multimodal architecture called SAIF, which detects such user corrections, taking as inputs the user’s voice commands as well as their transcripts. Voice inputs allow SAIF to take advantage of sound cues, such as tone, speed, and word emphasis. In addition to sound cues, our model uses transcripts to determine whether a command is a correction to the previous command. Our model also obtains internal input from the agent, indicating whether the previous command was executed successfully or not. Finally, we release a unique dataset in which users interacted with an intelligent agent assistant, by giving it commands. This dataset includes labels on pairs of consecutive commands, which indicate whether the latter command is in fact a correction of the former command. We show that SAIF outperforms current state-of-the-art methods on this dataset.

Original languageEnglish
Article number5577
Pages (from-to)1-10
Number of pages10
Issue number19
StatePublished - 1 Oct 2020


  • Correction detection
  • Deep learning
  • Human–agent interaction
  • Implicit feedback
  • Multimodal architecture


Dive into the research topics of 'Saif: A correction-detection deep-learning architecture for personal assistants'. Together they form a unique fingerprint.

Cite this