Demo: Towards Deploying ML-based Load Forecasting Models for Building HVAC System: an AI Evaluation Platform

Recently, machine learning (ML) models have been widely developed for building HVAC systems. Practitioners, however, have difficulties in understanding how an ML model behaves in practice, in terms of such metrics as maintainability, reliability, etc. This restricts wide adoption of AI in buildings.

Intrinsically, there is a lack of a methodology for building ML model evaluation, i.e., what to evaluate; and a platform for ML model evaluation to release the evaluation burden in recreating appropriate benchmarks, setting up experimental pipelines, etc. In this project, we propose BaiTest (Building AI Test), a new evaluation methodology for the ML models in buildings and an evaluation platform to materialize our methodology. BaiTest can be used by building operators and AI developers to compare and select appropriate ML models through the interactive visualization services. Our preliminary experiments shows the model recommended by the BaiTest platform can show 3%-10% more accuracy improvement against a model with the highest snap-shot accuracy. BaiTest can allow the effective use of a large number of ML models and accelerate ML model deployment without extra programming work.