Multimodal Generative AI for Construction-Site Management and Monitoring: A Field-Based Evaluation

Modern construction sites generate large volumes of visual, spatial, and operational data that can support data-driven project delivery, improved monitoring, and reliable decision-making within the smart-city built environment. However, construction management still relies heavily on human observation and manual interpretation, limiting the transformation of field data into structured information for sustainable urban infrastructure delivery. Multimodal generative artificial intelligence (GenAI) offers a promising approach for interpreting construction-site data, yet its performance under real site conditions remains insufficiently examined, particularly across tasks requiring different levels of visual recognition, contextual reasoning, and professional judgment. This paper presents a field-based evaluation of multimodal GenAI models using 1186 images collected from 17 active construction sites. The evaluation considered three widely available general-purpose multimodal GenAI assistants: Gemini, ChatGPT, and Microsoft Copilot. Four major construction management tasks were assessed: construction activity identification, progress tracking, execution defect detection, and safety hazard identification. The GenAI outputs were compared against ground-truth evaluations established by human experts. The results suggest that GenAI performs more reliably in descriptive and visually explicit tasks than in judgment-intensive tasks requiring engineering interpretation. Activity identification achieved the strongest performance, whereas execution defect detection was the most challenging. The findings indicate that GenAI can support visual site interpretation and improve construction management efficiency, while highlighting the need for human oversight and verification in smart-city infrastructure delivery.