REINFORCEMENT LEARNING WITH LOAD FORECASTING FOR SMART HOME ENERGY MANAGEMENT

This paper proposes H-UPF (Hybrid Universal Policy with Forecasting), a hybrid intelligent framework for scalable sequential decision-making in heterogeneous environments under uncertainty. The architecture integrates probabilistic multi-horizon forecasting via a Temporal Fusion Transformer with continuous control via Proximal Policy Optimization, embedding predictive quantile distributions directly into the agent’s state representation. A Dynamic Adaptation Layer normalizes observations relative to instance-specific scales, enabling zero-shot policy transfer across environments with 18.5× variability in operating characteristics — without inter-agent communication or per-instance retraining. Validated on two real-world residential energy management datasets (REFIT: 20 UK households; CityLearn: 6 US buildings with real PV profiles), the framework achieves 88.4% of the theoretical optimum in zero-shot transfer, outperforming meta-learning (MAML-PPO) by 8.4 percentage points (Wilcoxon p = 0.003, Cohen’s d = 1.42). Ablation analysis identifies the adaptation layer as the dominant contributor (−16.2 p.p. upon removal), while probabilistic forecasting adds +6.8 p.p. through proactive scheduling. The learned policy is robust to reward parameter variations (≤3.2 p.p. sensitivity across 5× range) and supports practical deployment: 9.8 h one-time training, 18.4 ms inference per control step.

1. Hu, D., Ye, Z., Gao, Y., Ye, Z., Peng, Y., & Yu, N. (2022). Multi-agent deep reinforcement learning for voltage control with coordinated active and reactive power optimization. IEEE Trans. Smart Grid, 13(6), 4873-4886. https://doi.org/10.1109/TSG.2022.3185975

2. Huang, J., Zhang, H., Tian, D., Zhang, Z., Yu, C., Hancke, G. (2024). Multi-agent deep reinforcement learning with enhanced collaboration for distribution network voltage control, Engineering Applications of Artificial Intelligence, 134, 108677. https://doi.org/10.1016/j.engappai.2024.108677

3. Sang, J., Sun, H., & Kou, L. (2022). Deep Reinforcement Learning Microgrid Optimization Strategy Considering Priority Flexible Demand Side. Sensors, 22(6), 2256. https://doi.org/10.3390/s22062256

4. Xiong, L., Tang, Y., Mao, S., Liu, H., Meng, K., Dong, Z., & Qian, F. (2022). A two-level energy management strategy for multi-microgrid systems with interval prediction and reinforcement learning. IEEE Transactions on Circuits and Systems I: Regular Papers, 69(4), 1788–1799. https://doi.org/10.1109/TCSI.2022.3141229

5. Sinha, A., Vyas, R., Alasali, F., Holderbaum, W., & Vyas, O.P. (2025). A deep reinforcement learning-based approach for cyber resilient demand response optimization. Frontiers in Energy Research, 12, 1494164. https://doi.org/10.3389/fenrg.2024.1494164

6. Singh, A.R., Sujatha, M.S., Kadu, A.D., Bajaj, M., Addis, H.K., & Sarada, K. (2025). A deep learning and IoT-driven framework for real-time adaptive resource allocation and grid optimization in smart energy systems. Scientific Reports, 15(1), 2649. https://doi.org/10.1038/s41598-025-02649-w

7. Waghmare, A.V., Singh, V.P., Varshney, T., & Sanjeevikumar P. (2025). A systematic review of reinforcement learning-based control for microgrids: Trends, challenges, and emerging algorithms. Discover Applied Sciences, 7, 939. https://doi.org/10.1007/s42452-025-07529-6

8. Rehman, U.u., Faria, P., Gomes, L. & Vale, Z. (2025). Future of energy management models in smart homes: A systematic literature review of research trends, gaps, and future directions. Process Integration and Optimization for Sustainability, 9, 1169–1198. https://doi.org/10.1007/s41660-025-00506-x

9. Sajjad, A., Ullah, K., Hafeez, G., Khan, I., Albogamy, F. R., & Haider, S. I. (2022). Solving day-ahead scheduling problem with multi-objective energy optimization for demand side management in smart grid. Engineering Science and Technology, an International Journal, 36, 101135. https://doi.org/10.1016/j.jestch.2022.101135

10. Dsouza, A., Thammaiah, A., & Venkatesh, L. K. M. (2022). An intelligent management of power flow in the smart grid system using hybrid NPO-ATLA approach. Artificial Intelligence Review, 55(8), 6461–6503. https://doi.org/10.1007/s10462-022-10158-9

11. Xiong, L., Tang, Y., Liu, C. et al. (2023). A home energy management approach using decoupling value and policy in reinforcement learning. Frontiers of Information Technology & Electronic Engineering, 24(9), 1261–1272. https://doi.org/10.1631/FITEE.2200667

12. Salazar, E. J., Jurado, M., & Samper, M. E. (2023). Reinforcement learning-based pricing and incentive strategy for demand response in smart grids. Energies, 16(3), 1466. https://doi.org/10.3390/en16031466

13. Yu, Z., Zheng, W., Zeng, K., Zhao, R., Zhang, Y., & Zeng, M. (2024). Energy optimization management of microgrid using improved soft actor-critic algorithm. International Journal of Renewable Energy Development, 13(2), 329-339. https://doi.org/10.61435/ijred.2024.59988

14. Lu, W., Gao, Y., Sun, Z., & Mao, Q. (2025). An Improved Soft Actor–Critic Framework for Cooperative Energy Management in the Building Cluster. Applied Sciences, 15(16), 8966. https://doi.org/10.3390/app15168966

15. Samende, C., Fan, Z., Cao, J., Fabián, R., Baltas, G. N., & Rodriguez, P. (2023). Battery and Hydrogen Energy Storage Control in a Smart Energy Network with Flexible Energy Demand Using Deep Reinforcement Learning. Energies, 16(19), 6770. https://doi.org/10.3390/en16196770

16. Rajagopal, B. G., & Senthil Kumaran, V. N. (2025). Hybrid transformer DDPG framework for solar radiation forecasting and battery energy storage optimization in a PV-powered microgrid. International Journal of Information Technology, 1-9. https://doi.org/10.1007/s41870-025-02820-6

17. Mehdipour Pirbazari, A., Farmanbar, M., Chakravorty, A., & Rong, C. (2020). Short-term load forecasting using smart meter data: A generalization analysis. Processes, 8(4), 484. https://doi.org/10.3390/pr8040484

18. Fekri, M. N., Patel, H., Grolinger, K., & Sharma, V. (2021). Deep learning for load forecasting with smart meter data: Online Adaptive Recurrent Neural Network. Applied Energy, 282(3), 116177. https://doi.org/10.1016/j.apenergy.2020.116177

19. Semmelmann, L., Henni, S., & Weinhardt, C. (2022). Load forecasting for energy communities: a novel LSTM-XGBoost hybrid model based on smart meter data. Energy Informatics, 5(Suppl 1), 24. https://doi.org/10.1186/s42162-022-00212-9

20. Lim, B., Arık, S., Loeff, N., & Pfister, T. (2021). Temporal Fusion Transformers for interpretable multi-horizon time series forecasting. International Journal of Forecasting, 37(4), 1748-1764. https://doi.org/10.1016/j.ijforecast.2021.03.012

21. Huang, C., Zhao, T., Huang, D., Cen, B., Zhou, Q., & Chen, W. (2024). Artificial intelligence-based power market price prediction in smart renewable energy systems: Combining prophet and transformer models. Heliyon, 10(20). https://doi.org/10.1016/j.heliyon.2024.e38227

22. Khan, A., Ullah, M., Tabassum, R., and Kabir, M. (2024). A Transformer-BiLSTM based Hybrid Deep Learning Approach for Day-Ahead Electricity Price Forecasting. In IEEE Kansas Power and Energy Conference (KPEC), Manhattan, KS, USA, 1-6, https://doi.org/10.1109/KPEC61529.2024.10676111

23. Alibrahim, O., Padmanaban, S., Khan, M., Khattab, O., Alothman, B., & Joumaa, C. (2022). Deep transfer learning-enabled energy management strategy for smart home sensor networks. IEEE Transactions on Industry Applications, 59(1), 81-92. https://doi.org/10.1109/TIA.2022.3223347

24. Lin, J., Ma, J., Zhu, J., & Liang, H. (2022). Deep domain adaptation for non-intrusive load monitoring based on a knowledge transfer learning network. In IEEE Transactions on Smart Grid, 13(1), 280-292. https://doi.org/10.1109/TSG.2021.3115910

25. Dang, Y., Xu, J., Yang, F., Jiang, C., Li, D. (2025). Meta Reinforcement Learning Based Adaptive and Interpretable Energy Storage Control Meets Dynamic Scenarios. in IEEE Transactions on Sustainable Energy, vol. 16, no. 4, pp. 2560-2572. https://doi.org/10.1109/TSTE.2025.3555002

26. Dang, Y., Xu, J., Yang, F., Jiang, C., Li, D. (2025). Meta Reinforcement Learning Based Adaptive and Interpretable Energy Storage Control Meets Dynamic Scenarios. in IEEE Transactions on Sustainable Energy, vol. 16, no. 4, pp. 2560-2572. https://doi.org/10.1109/TSTE.2025.3555002

27. Huang, R., Chen, Y., Yin, T., Huang, Q., Tan, J., Yu, W., & Du, Y. (2021). Learning and fast adaptation for grid emergency control via deep meta reinforcement learning. IEEE Transactions on Power Systems, 37(6), 4168-4178. https://doi.org/10.48550/arXiv.2101.05317

28. Filiz U., Hekimoğlu M. B., Baghaee S. & Ulusoy I. (2025). Analysis of Model-Agnostic Meta-Reinforcement Learning on Automated HVAC Control. 33rd Signal Processing and Communications Applications Conference (SIU), Sile, Istanbul, Turkiye, 1-4. https://doi.org/10.1109/SIU66497.2025.11112293

29. Murray, D., Stankovic, L. & Stankovic, V. (2017). An electrical load measurements dataset of United Kingdom households from a two-year longitudinal study. Sci. Data, 4, 160122. https://doi.org/10.1038/sdata.2016.122

30. Gupta, J.K., Egorov, M., Kochenderfer, M. (2017). Cooperative Multi-agent Control Using Deep Reinforcement Learning. In: Autonomous Agents and Multiagent Systems. AAMAS 2017. Lecture Notes in Computer Science, vol 10642. Springer, Cham. https://doi.org/10.1007/978-3-319-71682-4_5

31. Nagy, Z., Vázquez-Canteli, J.R., Dey, S., & Henze, G. (2021). The citylearn challenge 2021. In Proceedings of the 8th ACM international conference on systems for energy-efficient buildings, cities, and transportation, 218-219. https://doi.org/10.1145/3486611.3492226

32. Kwon K.-B., Zhu H. (2022). Reinforcement learning-based optimal battery control under cycle-based degradation cost. IEEE Transactions on Smart Grid, 13(6), 4909-4917. https://doi.org/10.1109/TSG.2022.3180674

33. Osone Y., Kodaira D. (2025). Imbalance-Aware Scheduling for PV-Battery Storage Systems Using Deep Reinforcement Learning. IEEE Access, 13, 172245-172258. https://doi.org/10.1109/ACCESS.2025.3615960