|
Report Contents Report#:EIA/DOE-0607(99)
Related Links |
Energy Prices and Economic Growth Moving Average Analysis of Forecasts Appendix: Example of 5-Year Moving Average Analysis Energy Prices and Economic Growth35 World oil prices have the second highest average absolute percent errors of all the variables evaluated in this paper, with natural gas prices at the wellhead having the highest. Overall, the average absolute percent error for world oil price forecasts has been 56.7 percent (Table 13). However, the earlier AEOs had a much higher average absolute percent error, and the publications after AEO86 show considerable improvement, with the exception of AEO91, which was affected by the Iraqi invasion of Kuwait. AEO91, prepared during the short-term escalation of oil prices caused by the invasion, projected continually rising prices. In fact, oil prices declined over each of the next 4 years. Similarly, the year with the highest average absolute percent error was 1998, followed by 1995 and 1986, with very high percentage errors in the earliest AEOs only partially offset by smaller errors in the more recent forecasts. In nominal terms, the first forecast for 1995, from AEO83, was nearly $75 per barrel, compared with the actual 1995 price of $17.14 per barrel. The sharp drop in the actual 1998 price, to $12.10 per barrel, resulted from a combination of weak demand from the Asian economies and an oversupply of crude oil exacerbated by the success of Iraq coming on line with more than 1 million barrels of oil production per day. Table 13. World Oil Prices: AEO Forecasts, Actual Values, and Absolute and Percent Errors, 1985-1998 For many of the variables examined in this paper, the highest average errors are seen for the year 1995. As mentioned before, the 1995 projections include those made furthest in the past—up to 12 years earlier. In addition, projections for 1991 through 1994 are not available from the earliest publications, so that 1995 appears to be more of an outlier.
Although the forecasts of world oil prices appearing in the earlier AEOs
were almost uniformly too high, from AEO86 on there were several instances
of forecasts that were too low. These included the 1987 and 1990 forecasts
appearing in AEO86 and AEO87, the forecasts for 1989 through 1991 appearing
in the Annual Energy Outlook 1989 (AEO89) The early AEO projections were strongly influenced by the notion that OPEC would continue to hold a large measure of power in world oil markets. Conventional wisdom in the early projections assumed that OPEC would be able to curtail production sufficiently to hold prices up, and that the cartel’s members would continue their cooperation throughout the forecast horizon. Even as it became clear that OPEC’s cohesiveness was not permanent, EIA continued to assume that oil prices would rise with increasing demand, although at a much slower rate of growth than in the 1970s. Increasing investment in areas outside OPEC and technological advances in oil exploration and production have contributed to the growth in oil reserves and production capacity of non-OPEC producers. These trends, combined with competition from natural gas and energy conservation, have kept prices lower than expected in the earlier forecasts. Natural gas prices at the wellhead have had the highest average absolute percentage forecast errors in the AEOs, with an overall average error of 70.2 percent (Table 14). Occasionally, near-term gas prices have been underestimated, but most of the projections were overestimates. Similar to the forecasts for world oil prices, those for natural gas prices were highest in the earlier AEOs, when the projections for all prices were influenced by the assumption that market forces would tend to increase demand for, and therefore prices of, natural gas and coal in response to higher world oil prices. The year 1995 had the highest average absolute percent error; with the exception of AEO96, which was essentially estimating the recent historical year for 1995, the smallest error for 1995 was 28.4 percent in AEO95. The 1995 wellhead price was significantly lower than the price in other recent years, primarily because 1995 had the second mildest winter in the United States in the past 20 years. The large error reflects the forecast assumptions of average weather. The year with the lowest average absolute percent error was 1985, with an average absolute error for four AEOs of 23.3 percent, even including the 65.2-percent error in the AEO82 projection for 1985. Despite the large errors, the forecasts in each subsequent AEO have tended to show considerable improvement, as the downward trend in gas prices has been better captured from one AEO to another. Nevertheless, each AEO has tended to predict rising natural gas prices over time, either because of the assumption in the earlier AEOs that long-term, high-priced contracts would continue or because technological improvement was not expected to offset depletion effects in the more recent forecasts. In summary, three factors have had significant impacts on the projections:
Thus, although the forecasts have generally improved with additional information, their accuracy has been impacted by the changing competitive structure of the industry and underestimates of technological advances by oil and gas producers. It is worth noting that much of the observed variation in natural gas price in more recent years can be attributed to transitory effects (e.g., weather and storage levels) that are not predictable in the long term and are more likely to affect prices in an increasingly competitive market environment. This effect has added to the complexity of forecasting natural gas prices over the longer term and, in part, explains why actual prices will vary from price forecasts based on assumptions of average conditions. Coal Prices to Electric Utilities Although they are better than those for oil and gas prices, the AEO forecasts of coal prices to electric utilities still show an average absolute percent error of 35.9 percent over the period studied (Table 15). All forecasts were overstated. The forecasts for 1995 had the highest average absolute percent error of 57.3 percent. There was, however, significant improvement in the 1995 forecast over time, with the error improving from 137.7 percent in AEO83 to 10.5 percent in AEO95 (excluding AEO96, which provided an estimate for the historical year 1995 based on partial year data). Across forecast years, the further out the forecast, the higher the error, with the lowest average absolute percent error shown for the year 1985 at 13.3 percent. The early AEOs—AEO82 through AEO86—tended to have the highest average absolute percent errors, exacerbated by their forecasts for 1995. There was steady improvement in the AEOs through AEO90, which had an average absolute percent error of 16.8. After AEO90, overestimates for 1995 through 1998 adversely affected the overall average errors for a number of the subsequent AEOs. The major factors in the high forecasts of coal prices were assumptions about depletion effects, productivity improvements, capacity utilization, transportation, and the impacts of CAAA90. Depletion was assumed to overcome productivity improvements in the long run; however, the onset of such new technology as longwall mines and the growth of surface mining in the West have led to continuing productivity improvements. Similarly, with high world oil price forecasts, the impacts of excess capacity and competition among existing mines were not seen to be as important as they in fact became. In addition, high world oil prices were assumed to affect both the production process and the costs of transportation. In fact, the collapse of oil prices in 1986 reduced the impact on both, and the increasing competitiveness of rail transportation has held transportation costs below expectations. Finally, it was assumed that higher prices would follow the enactment of CAAA90 as the demand for low-sulfur coal increased. Price increases did not materialize, however, as productivity increases and transportation cost reductions made increased production from western mines possible at lower-than-anticipated prices. In addition, it was assumed that many coal boilers would not be able to burn western coal easily, an assumption that proved erroneous. Average electricity prices showed the best forecasting record among the prices examined here, with an average absolute percent error of 11.1 percent (Table 16). As with all the price forecasts, because of the projections made 12 years earlier, the year with the highest average absolute percent error was 1995, which had an average error of 15.4 percent. Except for the two near-term forecasts of 1985 for AEO82 and 1989 for AEO90, price forecasts have been higher than actual. By publication, AEO83 had the highest average absolute error of 18.1 percent and the Annual Energy Outlook 1998 (AEO98)37 the lowest at 0.2 percent. Recent AEOs, from the Annual Energy Outlook 1992 (AEO92)38 on, have had average absolute percent errors of 10.2 percent or less. The primary reason for high price forecasts was the impact of fuel costs and capital costs on expected prices. Fuel costs were consistently overestimated for oil, natural gas, and coal, with a strong effect on the estimates of electricity prices, especially for AEO82 through AEO84. In addition, the costs of new capacity were assumed to be higher in earlier projections than they actually turned out to be, and this assumption also helped to raise the forecasts. Finally, a 1992 study39 on the accuracy of AEO electricity forecasts for 1985 and 1990 indicated that part of the explanation for high price estimates was public utility commission disallowances and phase-ins of costs of some capital-intensive generating capacity that were not incorporated in the projections because actual regulatory practices varied from those assumed in the projections. For example, some nuclear units had significant shares of their costs disallowed, and the remaining costs were phased in on a longer time schedule than the utilities had requested, contributing to lower-than-expected prices in some years. The economic forecasts in the AEOs are based on projections from DRI/McGraw-Hill, adjusted for EIA’s world oil price projections. The forecasts for gross domestic product (GDP) show an average absolute percent error of 5.0 percent (Table 17). Most of the projections have been less than 10 percent from actual, with the exception of some of the forecasts in AEO83, AEO84, AEO85, AEO86, and AEO89 for the mid-1990s, which ranged up to 28.8 percent above the actual GDP. In general, from AEO82 through AEO90, the GDP forecasts tended to be underestimated for the earlier years and overestimated for the later years. In subsequent reports, GDP has been consistently underestimated. The major reason for the pattern of overestimates in the longer term forecasts in the early AEOs is the recession that began in the latter part of 1990 and continued into 1991. The economic forecasts produced for the AEO are trend forecasts, which do not attempt to foresee the timing or magnitude of business cycles. The economic cycle in 1990-91 created a breakpoint in the series being used for evaluating forecast errors. Therefore, early AEOs did not forecast the recession and, consequently, overestimated long-term growth beyond 1991. Conversely, the underestimates in later AEOs resulted in part from overestimates of world oil prices, which tend to dampen economic growth, plus several other factors such as actual utility bond rates being lower than expected. Moving Average Analysis of Forecasts All the preceding analyses have focused on comparing the projections from previous AEOs with actual historical values. This section describes a simple moving average analysis of forecast data from the six consumption tables, the four price tables, and the economic growth table above. For each of the tables under consideration, the absolute value of the difference between the projected and actual values was calculated for each AEO and each year. Then, a 5-year moving average was calculated for each AEO, starting with the most recent forecast year. (The Appendix to this paper provides an example of the moving average analysis performed for the total energy consumption table.) Table 18 contains the result of the analysis for the 11 tables under consideration for the AEOs from AEO85 to AEO95. No results are given for AEO90, because there are not enough values in the intervening years to compute a 5-year moving average. Table 18. Results of 5-Year Moving Average Analysis of Errors for Selected AEO Forecasts In general, for the more recent AEOs, the errors in the coal consumption and electricity sales tables increased but those in the total petroleum and natural gas consumption tables did not. The errors in the price tables decreased from AEO85 through AEO89, rose sharply in AEO91, especially for oil prices and electricity prices, then declined again through AEO95. Total petroleum consumption errors declined in more recent AEOs as forecasts of petroleum consumption improved after AEO90. In contrast, coal consumption errors increased substantially over time, with the worst errors occurring from AEO92 to AEO95. The largest errors were in the forecasts for 1996, 1997, and 1998—years in which there was a surge in coal consumption for electricity generation that had not been captured in AEO92 to AEO95. Electricity sales forecasts improved through AEO91 but got progressively worse from AEO92 to AEO95, which severely underestimated sales for the years 1996 through 1998 (Table 6). Natural gas consumption errors were very high prior to AEO90 and improved in more recent AEOs. As discussed above, in the 1980s there were significant changes in the natural gas industry, with very large price forecasts causing a significant underestimation of consumption (Table 4), especially for the years 1995 to 1998. Also, FUA legislation exacerbated the error by attempting to restrict gas use. Forecasts improved considerably after AEO90. Oil price forecasts in AEO91 had a much higher error than subsequent AEOs, attributable to the Iraqi invasion of Kuwait. In view of the short-term escalation in oil prices AEO91 projected continually rising prices. In actuality, oil prices declined over each of the next 4 years, 1990 to 1994, which were the years included in the 5-year moving average. Subsequent AEOs were better at predicting oil prices. Although the downward trend in natural gas prices was better captured from one AEO to another, the coal price forecasts had a mixed record. After AEO91 rising coal prices were predicted, whereas the actual trend was downward. The error was particularly large for 1995 and 1996, which are not included in the 5-year moving average until AEO92. The sharply higher error for electricity price in AEO91 was due in part to the recession in the latter part of 1990 and early 1991, and to the overestimation of world oil prices, which caused the severe overestimation of electricity prices in AEO91. Subsequent AEOs were better at predicting electricity prices. In summary, the moving average analysis gave mixed results. The errors calculated for the consumption tables, except for petroleum and natural gas, increased with more recent AEOs, while the errors for the price tables decreased over time. Also, a 5-year moving average accentuates the error occurring during the 5 years included in the calculation but misses any impact after that. Although a primary function of the models used by EIA to produce its AEO projections has been and remains the analysis of alternative policies, many readers of the AEO use the projected numbers as forecasts for their own purposes. Thus, it is useful for EIA analysts and users of the AEO to know the size of and reasons for the differences between the projections and actual values. Throughout the AEOs, the variables with the highest errors, expressed as average absolute percent errors, have been prices and net imports of natural gas and coal. Natural gas, in general, has been the fuel with the most inaccurate forecasts, showing the highest average error of all the fuels for consumption, production, and prices. Natural gas was the last fossil fuel to be deregulated following the heavy regulation of energy markets in the 1970s and early 1980s, and the early AEOs assumed that natural gas would continue to be regulated until new rules were actually promulgated. Even after deregulation, the behavior of natural gas in competitive markets was difficult to predict. The overestimation of prices is the most striking feature of this evaluation. In general, more rapid technological improvements, the erosion of OPEC’s market power, excess productive capacity, and market competitiveness were the factors that the AEO forecasts failed to anticipate. While the errors for prices were large, they appeared to have a relatively minor impact on the overall projections of demand and production, although some forecasts were clearly affected, possibly confirming the relatively low price elasticities of supply and demand embedded in the models. For the period covered by this study, productivity and technology improvements and the effects of gradual deregulation and changes in industry structure, such as the treatment of contracts, have more than offset the factors that have tended to raise fossil fuel prices. In addition, energy markets have evolved differently than projected as a result of changes in the regulatory environment and the enactment of changes in legislation, regulations, and standards. In conclusion, there are several major reasons why forecasts might deviate from their long-term trends. First are laws and regulatory changes over which there is no control, some of which have been discussed in this paper. Second are external factors that cannot be predicted, including such cyclical events as weather and economic downturns, which led to the drop in oil prices in 1991. The final major reason is technological development. Over the years we have addressed this issue. The NEMS model was introduced in AEO94, and we have included in the design a structure that looks at technology in a more detailed fashion. There has been an improvement in the capability to represent technological innovation. Examples of this are electricity generation and technological improvement in oil and gas supply. While the oil and gas model reflects the assumption that increases in cumulative drilling lower the finding rates for oil and gas, recent changes to the methodology allow the flexibility for this decline to be either partially, fully, or more than fully offset by improvements in technology—depending on the fuel, region, and year—based on historical patterns and on assumed future rates of technology improvement. The advantage of this approach is that it is capable of representing finding rates that rise, remain constant, or decline over time, depending on the values of the technology and resource depletion parameters. In the end-use models there is an emphasis on specific technologies and their characteristics, technology and fuel choice, and stock turnover rates, which is in contrast to the earlier models. As a specific component for new generating technologies, there is learning-by-doing— that is, as experience is gained with new technologies the cost starts dropping. These improvements to NEMS have allowed the execution of a number of technology cases in AEO99 to examine their impacts. Other technology cases have been produced in AEOs since AEO94. The most striking result of the moving average analysis described here is that the price predictions and the petroleum and natural gas consumption predictions became more accurate with more recent AEOs, while the coal consumption and electricity sales errors increased with more recent AEOs. Also, a 5-year moving average accentuates the error occurring during the 5 years and misses any impact outside the 5-year range. |
