BI for a Distributed Energy Management Platform

#Big data
#Business analytics
#Machine learning

Business Challenge

Today the energy market is moving away from centralized power plants and entering the era of the distributed grid enabling a peer-to-peer market. These changes require a completely new operating paradigm. The customer has a platform for managing distributed grids based on Artificial Intelligence that helps save energy costs by optimizing energy flow for end users, service providers, and grid operators.

The client needed analytics and prediction services for cost optimization of electricity usage. The main goal was a solar energy production forecast for his business with the improvement of the existing PV forecasting algorithms.

Solution Overview

The project is of high importance for the client, directly impacting future and current customers. Therefore, to deliver better results for our client, we provided some business intelligence services that allowed to predict energy usage production and prices, and make real-time consumption and storage decisions as well as perform peer-to-peer or peer-to-grid energy trading.

The solution enables the prediction, optimization, and trading of behind-the-meter energy. By optimizing the grid from the bottom up, the platform reduces energy costs and maximizes local energy utilization.

Project Description

The Quantum team has prototyped, developed, and deployed machine learning-based models for energy production prediction for solar panels placed on different stations that increased the complexity of the problem compared to predictions for a single station.

– Firstly we made an introductory audit of the all data. This allowed us to develop the first prototype of the solution – the XGBoost model based on historical tabular data. We have mainly used weather data and did some feature engineering based on the day of the year and timing window. The day of the year allowed us to calculate the angle of incidence of sun rays. The timing window is a useful feature to obtain information about previous observations of energy production without the usage of the classical time series approach or recurrent networks.

– After that, we integrated a model into the client’s existing platform. The original model was tested for compliance with the required efficiency. The main challenge was to integrate the model into the existing system and save the performance because all infrastructure was developed on Java that is not suitable for machine learning modeling.

– Another model based on clustering was additionally developed at this project stage. As was expected this model was more stable to seasonal changes that allowed to reduce general error. The main difference between this model and the XGBoost one is that the clustering-based model provides daily predictions (XGBoost – hourly).

– All models have been integrated into the system from scratch. As a result, we built ensembles of models based on the XGBoost model, clustering-based models, and existing models developed by the client using the Weka machine learning library. This step allowed us to reduce the error by 2%.

– One of the final stages of the project was a collection of statistics on all stations that the client had in order to check and test the models for correct operation and efficiency. We built a pipeline that is able to run all models and collect the scores on the full list of stations and generate the final report without any intervention of operators.

Let's discuss your idea!

Technological Details

Prototyping of all the models was implemented using Python programming language. Development and integration were in Java. Designed and deployed original model based on clustering algorithm for energy prediction on a daily basis. Other technologies we used were: Pandas, XGBoost, XGBoost4J, Matplotlib, Kmeans++, Weka ML, Apache Tomcat, MySQL.