Getting the Maximum Value from Operational Renewable Energy Forecasts

The operational use of wind and solar power production forecasts has become widespread in the electric power industry. Their benefits for the management of weather-dependent generation variability have been documented in a number of studies. However, there is considerable evidence that the full value of wind and solar forecasts is often not realized in many applications. There are two primary reasons: (1) lack of optimal customization of the forecast solution for a user’s specific applications, and (2) the preference for the use of deterministic forecast solutions over probabilistic ones.

Thus, it is useful for a forecast user to ponder a key question: how do I know if I am getting the maximum value from my renewable energy forecast solution? It is often worthwhile for a user to examine the factors that should be considered to develop a meaningful answer to this question. This can help formulate a strategy for how to get a forecast solution that provides maximum value for a user’s specific application.

Ineffective Customization

An analysis of how users select and employ renewable energy forecasts suggests that there are four primary factors that play a major role in the value realized from a traditional deterministic forecast. A brief overview of these factors provides some insight into the issues that are often overlooked when seeking an optimal forecast solution.

The first factor is the specification of a non-optimal objective in the forecast solution selection process. The perspective of many users is often that the most accurate forecast solution should be acquired. The questions of “what type of solution is needed?” and “what does ‘most accurate’ mean?” are often not carefully considered. However, forecasts have different error attributes depending on the process used to generate them. Ideally, a forecast process should be customized to produce forecast variables and error attributes (typical error magnitude, error distribution, etc.) that are optimal for a user’s application (maximum profit, best operating decisions, etc.). However, the typical request of “give us your best forecast” usually translates into the default minimization of the squared error (i.e., least squares optimization) of the forecast, which may not be the best optimization for a user’s specific application.

The second factor is the selection of a set of optimal metrics to evaluate forecast performance. Ideally, the selected metrics should evaluate the degree to which the forecast objective is met. They should implicitly represent the sensitivity of the user’s application to forecast error. The selection of an optimal metric will depend upon the specification of the appropriate forecast objective. There are many cases in which the user understands the attributes that are important in an application (such as the timing and amplitude of wind or solar generation ramps), but then chooses to evaluate the forecast with a widely used generic error metric such as the root mean squared error (RMSE). Of course, a forecast provider will try to optimize the forecast to achieve the best performance on the error metric used for evaluation, even though it may result in a less than optimal forecast for a user’s application.

The third factor is associated with the way that alternative forecast solutions are evaluated during a trial or benchmark intended to identify the best solution for an application. Unfortunately, in many cases, flaws in the design and/or execution of trials result in the information obtained from the exercise being obscured by noise. The outcome is that a solution is selected based upon random variations in the data that are not representative of true skill differences among the tested solutions.

The fourth factor is the amount, quality, and timeliness of the meteorological and generation data provided to the forecast process. This factor has a much larger impact on very short-term forecasts (minutes to a couple of hours ahead). These forecasts are heavily dependent on the recent conditions and trends at the target facility, but data quality issues have a negative impact on the performance of forecasts on almost all time scales.

Throwing Away Information

A second major issue that contributes to getting less value from forecast solutions is the preference for the use of deterministic over probabilistic forecasts. This preference is somewhat understandable since application tools often require deterministic input, and operational decision-making protocols are typically based on deterministic criteria. Probabilistic forecasts have a higher level of complexity and therefore are more difficult to implement, understand, interpret, and evaluate in existing system architectures.

However, none of these understandable reasons negates the fact that probabilistic forecasts have more information than deterministic forecasts. Probabilistic forecasts provide a prediction of the target variable and an estimate of the uncertainty surrounding that prediction. Deterministic forecasts only provide the anticipated value of the target variable, a best guess within the uncertainty inherent in every forecast. Thus, the use of deterministic forecasts ignores a portion of the information that, strictly speaking, needs consideration when using forecasts for decision-making.

Carefully conducted experiments indicate that the use of probabilistic forecasts results in better decision-making (regardless of the context) than the use of only deterministic predictions. This is really not surprising, since they contain more information. The value of probabilistic wind power forecasts can be experienced in a game developed by Task 36 collaborators that contrasts deterministic approaches with probabilistic ones for the specific case of trading wind energy in high-speed wind turbine shutdown situations. The game player is tasked with trading wind power in scenarios with a significant chance of shutdowns. Figure 1 is an excerpt from the game currently online and the first in the game platform. It shows three independent deterministic power generation forecasts (lower left panel) and a probabilistic one for a specific case (lower right panel). The first game in the platform’s list, Wind Power Trading Decisions for a Wind Park in Complex Terrain, is still open to play. Game results indicate that most participants achieve better results from the probabilistic forecasts.

Of course, a user’s process must be able to make effective use of the additional uncertainty information in order to realize the value. This could be done by providing training or guidance to operational decision-makers on how to most effectively transform probabilistic forecast information into what are typically binary deterministic decisions. Alternatively, it could be done by developing or acquiring decision-support tools that effectively use the probabilistic information.

It is important to note that the customization issues described above also apply to the selection of a probabilistic forecast solution. Probabilistic forecast solutions generated by different methods and input data types will have different attributes (such as the ability to more precisely differentiate probability distribution differences among weather scenarios). These attributes will have an impact on the value that can be obtained from a probabilistic forecast for a specific application.

A Guide for the Path to Maximum Value

In order to facilitate the implementation of optimal short-term forecast solutions for specific applications, a group of international experts have collaborated under the International Energy Agency’s (IEA) Wind Technology Collaboration Program (TCP) Task 36, “Forecasting” (which, beginning in 2022, will become Task 51). The group developed a “Recommended Practice on Renewable Energy Forecast Solution Selection.” The original version was published in 2019 and is composed of three parts. The first part provides information on how to select an optimal solution for a specific application. The second part provides guidance on how to conduct a trial or benchmark that provides high-quality information to the selection process. Methods and metrics for the evaluation of the performance of both deterministic and probabilistic forecasts are addressed in the third part. This version can be obtained from the IEA Wind Forecasting Task website. A second version will be available during the first half of 2022. It includes updates to the first edition, especially on probabilistic forecast solutions, data communication, and additional examples. It also incorporates a fourth part that provides guidance for the optimal selection, deployment, and maintenance of meteorological sensors and the data transfer.

With the persistently increasing levels of renewable generation on many systems, the importance of obtaining the maximum forecast value for the facilitation of economic and reliable system operations is rapidly increasing. The additional value that can be obtained from the combination of well-targeted forecast customization and the use of probabilistic forecasts is well documented, and it has never been easier to acquire the capability. Existing and potential forecast users are strongly encouraged to investigate this opportunity.

John Zack
Principal
MESO, Inc.