Evaluating large-language-model chatbots to engage communities in large-scale design projects

Jonathan Dortheimer, Nik Martelaro, Aaron Sprecher, Gerhard Schubert

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


Recent advances in machine learning have enabled computers to converse with humans meaningfully. In this study, we propose using this technology to facilitate design conversations in large-scale urban development projects by creating chatbot systems that can automate and streamline information exchange between stakeholders and designers. To this end, we developed and evaluated a proof-of-concept chatbot system that can perform design conversations on a specific construction project and convert those conversations into a list of requirements. Next, in an experiment with 56 participants, we compared the chatbot system to a regular online survey, focusing on user satisfaction and the quality and quantity of collected information. The results revealed that, with regard to user satisfaction, the participants preferred the chatbot experience to a regular survey. However, we found that chatbot conversations produced more data than the survey, with a similar rate of novel ideas but fewer themes. Our findings provide robust evidence that chatbots can be effectively used for design discussions in large-scale design projects and offer a user-friendly experience that can help to engage people in the design process. Based on this evidence, by providing a space for meaningful conversations between stakeholders and expanding the reach of design projects, the use of chatbot systems in interactive design systems can potentially improve design processes and their outcomes.

Original languageEnglish
Article numbere4
JournalArtificial Intelligence for Engineering Design, Analysis and Manufacturing: AIEDAM
StatePublished - 18 Mar 2024


  • chatbot
  • crowdsourcing
  • large language model
  • participatory design
  • urban design


Dive into the research topics of 'Evaluating large-language-model chatbots to engage communities in large-scale design projects'. Together they form a unique fingerprint.

Cite this