Techno–Economic Modeling and Safe Operational Optimization of Multi-Network Constrained Integrated Community Energy Systems

The integrated community energy system (ICES) has emerged as a promising solution for enhancing the efficiency of the distribution system by effectively coordinating multiple energy sources. However, the concept and modeling of ICES still remain unclear, and operational optimization of ICES is hindered by the physical constraints of heterogeneous integrated energy networks. This paper, therefore, provides an overview of the state-of-the-art concepts for techno–economic modeling of ICES by establishing a Multi-Network Constrained ICES (MNC-ICES) model. The proposed model underscores the diverse energy devices at community and consumer levels and multiple networks for power, gas, and heat in a privacy-protection manner, providing a basis for practical network-constrained community operation tools. The corresponding operational optimization in the proposed model is formulated into a constrained Markov decision process (C-MDP) and solved by a Safe Reinforcement Learning (RL) approach. A novel Safe RL algorithm, Primal-Dual Twin Delayed Deep Deterministic Policy Gradient (PD-TD3), is developed to solve the C-MDP. By optimizing operations and maintaining network safety simultaneously, the proposed PD-TD3 method provides a solid backup for the ICESO and has great potential in real-world implementation. The non-convex modeling of MNC-ICES and the optimization performance of PD-TD3 is demonstrated in various scenarios. Compared with benchmark approaches, the proposed algorithm merits training speed, higher operational profits, and lower violations of multi-network constraints. Potential beneficiaries of this work include ICES operators and residents who could be benefited from improved ICES operation efficiency, as well as reinforcement learning researchers and practitioners who could be inspired for safe RL applications in real-world industry.