【文献分享】Evaluation of large language models (LLMs) on the mastery of knowledge and skills in the heating, ventilation and air conditioning (HVAC) industry-Peng XU Research Group

【文献分享】Evaluation of large language models (LLMs) on the mastery of knowledge and skills in the heating, ventilation and air conditioning (HVAC) industry

2026-04-13 学术动态朱奕

文献来源：, , , Yang Zhao a f, , , , , ,

a: Institute of Refrigeration and Cryogenics, Zhejiang University, Hangzhou 310027, China

b: Energy Efficient Cities Initiative, Department of Engineering, University of Cambridge, Cambridge CB2 1PZ, United Kingdom

c: Architectural Design & Research Institute of Zhejiang University Co., Ltd., Hangzhou 310027, China

d: Center for Balance Architecture, Zhejiang University, Hangzhou 310027, China

e: Department of the Built Environment, Eindhoven University of Technology, Eindhoven 5612 AZ, the Netherlands

f: Key Laboratory of Clean Energy and Carbon Neutrality of Zhejiang Province, Jiaxing Research Institute, Zhejiang University, Jiaxing 314000, China

文章摘要：Abstract

Large language models (LLMs) have shown human-level capabilities in solving various complex tasks. However, it is still unknown whether state-of-the-art LLMs master sufficient knowledge related to heating, ventilation and air conditioning (HVAC) systems. It will be inspiring if LLMs can think and learn like professionals in the HVAC industry. Hence, this study investigates the performance of LLMs on mastering the knowledge and skills related to the HVAC industry by letting them take the ASHRAE Certified HVAC Designer examination, an authoritative examination in the HVAC industry. Three key knowledge capabilities are explored: recall, analysis and application. Twelve representative LLMs are tested such as GPT-3.5, GPT-4 and LLaMA. According to the results, GPT-4 passes the ASHRAE Certified HVAC Designer examination with scores from 74 to 78, which is higher than about half of human examinees. Besides, GPT-3.5 passes the examination twice out of five times. It demonstrates that some LLMs such as GPT-4 and GPT-3.5 have great potential to assist or replace humans in designing and operating HVAC systems. However, they still make some mistakes sometimes due to the lack of knowledge, poor reasoning capabilities and unsatisfactory equation calculation abilities. Accordingly, four future research directions are proposed to reveal how to utilize and improve LLMs in the HVAC industry: teaching LLMs to use design tools or software in the HVAC industry, enabling LLMs to read and analyze the operational data from HVAC systems, developing tailored corpuses for the HVAC industry, and assessing the performance of LLMs in real-world HVAC design and operation scenarios.