Over a million developers have joined DZone.

Big Data in the Renewable Energy Sector

DZone's Guide to

Big Data in the Renewable Energy Sector

An examination of the ways in which big data processes can be applied to the need for more renewable energy sources and consumption.

· Big Data Zone ·
Free Resource

How to Simplify Apache Kafka. Get eBook.

This article is featured in the new DZone Guide to Big Data: Volume, Variety, and Velocity. Get your free copy for insightful articles, industry stats, and more!

In this article, I will look at how big data and AI can be used to improve the efficiency of renewable energy production and offer opportunities for reducing electricity consumption.


We have seen a global revolution in the use of big data to improve efficiency in manufacturing, security, and healthcare, to name just a few industries. In recent years, environmental issues, particularly climate change, have attracted concern and been widely discussed. Can the same approach be used for energy monitoring, modeling, analysis, and prediction to achieve sustainable energy objectives and to reduce the volume of carbon dioxide emissions that are causing global warming?

Clean and Efficient Electricity Generation

In the US, renewable energy sources generate 17 percent of the electricity used, and calculations suggest that solar, wind, hydroelectric, and other renewable sources are the world's fastest-growing energy source according to the Energy Information Administration of the U.S. Department of Energy

Renewable energy sources need to be scaled up in order to replace the traditional energy sources that are responsible for greenhouse gas emissions before it is too late to reverse the impacts on our ever-warming climate. In order to scale, they need to be as efficient as possible, and a combination of Big Data and artificial intelligence can help. Blending renewable energy into existing utility grids requires estimation of the power that will come from solar-, wind- and hydro-electricity sources in order for the infrastructure to function with appropriate estimation, planning, pricing, and real-time operations.

Predicting and Maximizing Solar Electricity Production

Power generation from distributed solar photovoltaic (PV) arrays has grown rapidly in recent years, with global photovoltaic capacity estimated to reach over 1 terawatt of solar capacity within the next 5 years, according to the latest global data.

Big data is used for accurate prediction of meteorological variables, pulling in disparate observational data sources and models, then using computational intelligence techniques for real-time analysis. For example, SunCast is a system from the National Center for Atmospheric Research (NCAR) that is used to provide solar energy forecasting. It is based on real-time measurements in the field and satellite data for cloud patterns. The forecast blends a number of models and tunes them according to historic observations using statistical learning and a host of artificial intelligence algorithms.

You may have driven or taken a train past a large, rural, photovoltaic array. If you live in a town, you have probably seen PV panels on rooftops. In an urban environment, where is the best place to locate PV arrays? A recent paper illustrated the use of image recognition and machine learning to determine the best sites to place rooftop-based PV arrays, allowing local decision-makers to assess the potential solar-power capacity within their jurisdiction. The approach does not require the use of 3D city models and instead uses public geographical building data and aerial images. The AItakes the geodata and outputs irradiance simulation and power generation potentials, which can be used to determine the best sites for PV panels.

Since solar panels may be placed in inaccessible areas, their owners need to be aware of environmental factors that can have negative effects on their efficiency and cause a loss of power generation, such as shading, fallen leaves, dust, snow, and bird damage, among others. Machine learning can be used to monitor the output from individual panels as a set of time series data, with the model trained to detect anomalous outputs and classify them. The AI can then indicate a problem on a particular panel's surface, which can then be scheduled for inspection and repair.

Predicting Wind-Turbine Production

Wind power provides a significant opportunity for future power generation and is growing substantially each year. One report suggests that wind power could reach nearly 2,000 GW by 2030, supplying between 16.7-18.8 percent of global electricity and help save over 3 billion tons of CO2 emissions, although this is an ambitious prediction, and I'd recommend that you consult the full report if you are interested in the nuances involved.

Wind power predictions are needed for turbine control, load tracking, power system management, and energy trading. Many different wind power prediction models have been used in combination with data mining. There are a number of approaches, such as a physical (deterministic) approach, based on lower atmosphere or numerical weather predictions using weather forecast data like temperature, pressure, surface roughness, and obstacles. An alternative statistical approach uses vast amounts of historical data without considering meteorological conditions and relies upon artificial intelligence (neural networks, neuro-fuzzy networks) and time series analysis approaches. A final approach is a hybrid model that combines both physical and statistical methods.

Reducing Electricity Consumption

A number of households are now familiar with the concept of home energy monitors, which consist of a sensor, a transmitter, and a handheld display. The sensor clips onto a power cable connected to your electricity meter box and monitors the magnetic field around the power cable to measure the electrical current passing through it. The transmitter takes data from the sensor and sends it to the handheld display unit, which calculates your power usage, the costs, and the greenhouse gas emissions (tons of CO2), assuming the electricity is from a non-renewable source. By collecting and analyzing the data from a sufficiently large number of homes, it is possible to determine where energy savings can be made or where there is flexibility in usage outside of peak hours. Consumers can then be advised on how to reduce their consumption, cut their bills, integrate renewable energy, and reduce emissions.

For example, in some states in the US where energy markets are deregulated, customers can choose between different energy providers, but each offers a different tariff and promotional rate, which complicates selection. Machine learning can be used within a web platform to help consumers minimize their bills. When they sign up, the customers state their energy preferences (limiting themselves to sustainable sources, for example) and the machine-learning model uses a smart meter to inspect their usage pattern and match it against the best supplier, automatically switching them to different suppliers and energy plans as better deals arise. The aim is to encourage uptake of renewable energy by offering it to consumers who are most willing to do the right thing and limit their use of non-sustainable sources as long as they are not penalized by significantly higher prices.

Data Center Energy Consumption

While Big Data is helping in a myriad of ways to increase the generation of sustainable energy and reduce consumption, it is, itself, responsible for consuming an increasing amount of energy. As Nature News reported recently, data center energy usage in 2018 exceeded the national energy consumption of some countries. Currently, data centers account for approximately 1 percent of global electricity demand, but usage is predicted to rise rapidly within the coming years — particularly if computationally intensive cryptocurrency mining continues to grow. Data center usage will make a significant contribution to global carbon emissions, since only approximately 20 percent of the electricity used in them comes from renewable sources, according to Greenpeace.

The main cause of energy consumption in a data center is cooling, which is typically performed by pumps, chillers, and cooling towers. Traditionally, it has been difficult to optimize the cooling process manually because of the complexity of the interactions between the combinations of necessary equipment. The rules and heuristics needed for every scenario have been difficult to define, particularly when interactions with the surrounding environment (such as the weather) are also considered. The result was that human operators were unable to calculate changes to settings that could respond sufficiently quickly to variations within the data center environment in order to optimize electricity efficiency.

To investigate whether AI could do better, Google turned to DeepMind, and in 2016, the team blogged about a deep learning model trained with sensor data that was able to predict the impact of environmental factors on performance and energy consumption. The model makes recommendations to human operators to suggest optimization settings to improve cooling efficiency and thus reduce power consumption. In one particular Google data center, the model affected a 40 percent drop in energy usage for cooling.

In Conclusion

Big data and AI are fundamentally changing the models of power generation, pricing, and consumption, causing significant disruption in the energy sector. New, smarter ways of monitoring, modeling, analyzing, and predicting energy generation and usage are helping us to achieve sustainable energy objectives as the global population faces an unprecedented environmental challenge.

This article is featured in the new DZone Guide to Big Data: Volume, Variety, and Velocity. Get your free copy for insightful articles, industry stats, and more!

12 Best Practices for Modern Data Ingestion. Download White Paper.

big data ,big data use cases ,dzone guide ,predictive analytics ,artificial inteligence

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}