CBO’s Economic Forecasting Record: 2021 Update

Summary

Each year, the Congressional Budget Office prepares economic forecasts that underlie its projections of the federal budget. CBO forecasts hundreds of economic variables, but some—including output growth, the unemployment rate, inflation, interest rates, and wages and salaries—play a particularly significant role in the agency’s budget projections. To evaluate the quality of its economic projections, estimate uncertainty ranges, and isolate the effect of economic errors on budgetary projections, CBO regularly analyzes its historical forecast errors. That analysis serves as a tool for assessing the usefulness of the agency’s projections.

In this report, CBO evaluates its two-year and five-year economic forecasts from as early as 1976 and compares them with analogous forecasts from the Administration and the Blue Chip consensus—an average of about 50 private-sector forecasts published in Blue Chip Economic Indicators.1 External comparisons help identify areas in which the agency has tended to make larger errors than other analysts. They also indicate the extent to which imperfect information may have caused all forecasters to miss patterns or turning points in the economy.

How CBO Measures Forecast Quality

CBO’s analysis focuses on four metrics of forecast quality—average error, mean absolute error, root mean square error, and two-thirds spread of errors:

• The average error is CBO’s primary measure of centeredness, which indicates how close the average forecast value is to the average actual value over time. Centeredness is the opposite of statistical bias, which quantifies the degree to which a forecaster’s projections are too high or too low over a period of time.
• The mean absolute error is calculated by taking the average of the absolute value of the forecast errors to show the size of the error rather than its direction.
• The root mean square error, which is calculated by squaring the forecast errors, averaging those squares, and taking the square root of that average, is CBO’s primary measure of accuracy, or the degree to which forecast values are dispersed around actual outcomes.
• The two-thirds spread, computed as the range between the minimum and maximum errors after removing the one-sixth largest errors and the one-sixth smallest errors, illustrates a forecaster’s typical range of errors and provides information about the extent of the dispersion of those errors.

The Quality of CBO’s Forecasts

Forecasts that have an average error of zero are, on average, neither too high nor too low. CBO’s forecasts of economic variables considered in this report tend to exhibit small positive average errors—that is, on average, they are too high by small amounts.2

In general, it is difficult to compare quality measures across variables because the magnitudes of variables can differ substantially, and some variables are easier or harder to forecast than others. It is possible, however, to compare those measures across forecast horizons for a given variable. For example, as measured by the mean absolute error or the root mean square error, five-year forecasts of interest rates and the unemployment rate are less accurate than two-year forecasts of those variables. For the other variables, CBO’s forecasts are as accurate or more accurate at the five-year horizon on the basis of those two measures of forecast quality. For most variables, the agency’s two- and five-year forecasts exhibit a similar two-thirds spread of errors.

Comparing CBO’s Forecasts With Those of the Administration and the Blue Chip Consensus

In general, forecasts produced by CBO, the Administration, and the Blue Chip consensus display similar error patterns over time. Because all forecasters faced the same challenges, periods in which CBO made large overestimates typically coincide with periods in which other forecasters made similarly large overestimates. Over time, however, even small differences in the magnitude of errors can result in appreciable differences in measures of forecast quality.

CBO and the Blue Chip consensus tend to produce more-centered forecasts of output growth but less-centered forecasts of interest rates than the Administration (see Table 1). As measured by the root mean square error, half of CBO’s forecasts are slightly more accurate than the Administration’s forecasts, and the others are equally accurate; by that measure, CBO’s forecasts are roughly comparable to the Blue Chip consensus forecasts. Finally, with a few exceptions, CBO’s forecasts tend to exhibit smaller two-thirds spreads than the Administration’s forecasts do; the agency’s forecasts tend to have two-thirds spreads that are similar to those of the Blue Chip consensus forecasts.

Sources of Forecast Errors

Forecasters made large errors in their economic forecasts as a result of several economic developments:

• Turning points in the business cycle;
• Changes in labor productivity trends;
• Changes in crude oil prices;
• The persistent decline in interest rates;
• The decline in the labor share—that is, labor income as a share of gross domestic product (GDP);
• Data revisions; and
• The coronavirus pandemic.

Some of those developments resulted in errors in forecasting specific variables. Changes in crude oil prices, for example, resulted in misestimates of inflation in the consumer price index (CPI). Other developments, such as turning points in the business cycle, affect the entirety of an economic forecast, and their effects are observable in the error patterns of several variables.

How CBO Uses Its Forecast Errors to Estimate Uncertainty

CBO most frequently uses the root mean square error of its historical forecasts to quantify the uncertainty of its current economic projections. For example, CBO’s baseline forecast for the growth of real GDP (that is, of GDP adjusted to remove the effects of inflation) over the next five years is 3.1 percent. Using its historical root mean square error for that variable (1.2 percentage points), CBO then estimates that there is approximately a two-thirds chance that the rate of real GDP growth over the next five years will be between 1.9 percent and 4.3 percent. Although the agency typically uses the historical root mean square error to estimate uncertainty, it uses the two-thirds spread of errors in instances where the statistical bias of its historical forecasts is high relative to its root mean square error.

CBO’s Methods for Evaluating Forecasts

To evaluate the quality of its forecasts, CBO examines its historical forecast errors and compares them with errors made by the Administration and the Blue Chip consensus. Comparison with the Blue Chip consensus is particularly useful because that forecast incorporates a wide variety of viewpoints and methods, and some research has suggested that composite forecasts often provide better estimates than projections made by a single forecaster.3

For this analysis, CBO reviewed the economic projections that it published each winter, usually in January, beginning in 1976, as the basis for its baseline budget projections. Each of those economic projections spanned the current year (that is, the calendar year already under way) and either 5 or 10 subsequent years.

This report evaluates CBO’s economic forecasts over the first two years and first five years of its baseline projection period. CBO evaluates forecasts over the two-year time horizon because that interval is most relevant when the agency is preparing its baseline budget projections for the upcoming fiscal year. (That second year is often called the budget year.) CBO evaluates forecasts over the five-year time horizon to better understand the quality of its longer-term projections.4 The agency calculates errors by subtracting the average actual value over the time period being analyzed from the average projected value over that period.

The span of years evaluated for this analysis varies by economic indicator and depends on two factors: the availability of historical forecast data and the availability of actual economic data. To ensure that differences in the availability of forecast data do not affect the comparisons of forecast errors, those comparisons begin in the earliest year for which forecast data were available for all three sets of forecasts. So, although CBO has two-year forecasts of real output growth dating back to 1976, its forecast errors for comparison purposes are computed starting in 1980—the first year the Blue Chip consensus data were available.5 Likewise, the final year of forecast analysis depends on the availability of actual economic data. This report incorporates data through the end of 2020, which allows CBO to analyze two-year forecasts that were made through the beginning of 2019 and five-year forecasts that were made through the beginning of 2016. (See the appendix for details.)

Selecting Forecast Variables

CBO examines 10 forecast variables on the basis of their relative importance in the economic outlook and their relevance to projections of revenues, outlays, and deficits. Those variables include the following:

• Output growth,
• The unemployment rate,
• Inflation,
• Interest rates, and
• Wages and salaries.

Projections of real and nominal output growth are fundamental to CBO’s budget projections. Faster output growth is typically accompanied by faster growth in real income and hence faster growth of revenues from individual income taxes. Similarly, periods of faster output growth are typically associated with smaller transfer payments and smaller expenditures on unemployment insurance, resulting in lower outlays.

This analysis also examines CBO’s errors in forecasting the unemployment rate, a key component of the agency’s economic forecast. CBO’s forecast of the unemployment rate affects its projections of inflation, interest rates, and other labor market variables and also informs the agency’s projections of certain outlays, including those for unemployment compensation.

CBO’s evaluation of inflation forecasts focuses on two measures: the percentage change in the CPI and the inflation differential, which is computed as the difference between growth in the CPI and growth in the output price index.6 All else being equal, higher CPI inflation implies faster growth in federal outlays (because the index is used to adjust payments to Social Security beneficiaries as well as payments made for some other programs) and slower growth in federal revenues (because elements of the individual income tax, including the tax brackets, are indexed to the CPI).7 Growth in the output price index is closely linked to growth in nominal income subject to federal taxes, which implies faster growth in revenues. Consequently, if CPI inflation was higher than anticipated and the output price index grew more slowly than anticipated, the projected deficit would generally be larger than expected.

The interest rates on 3-month Treasury bills and 10-year Treasury notes summarize CBO’s projections of short- and long-term interest rates, respectively. Interest rates primarily affect the budget through their effect on net interest outlays—the difference between income earned on interest-bearing assets and the cost of servicing the debt. As a result, overestimates of interest rates result in overestimates of debt and deficits. This report also analyzes the real 3-month Treasury bill rate, which is computed by removing the effects of CPI inflation from forecasts of the nominal 3-month Treasury bill rate. Considering errors in projecting the real 3-month Treasury bill rate isolates interest rate errors from errors in inflation projections.

Finally, CBO examines growth in wage and salary disbursements and the change in those disbursements as a share of output. Wages and salaries are the largest component of national income, and their growth informs CBO’s revenue projections and its analysis of the distribution of income. Analyzing wages and salaries as a share of output offers an approximation of forecasters’ views about the labor share of national income and helps isolate errors in projecting wages and salaries from errors in projecting nominal output.

Calculating Forecast Errors

CBO computes each forecast error as the difference between the average forecast value and the average actual value. (See Box 1 for an example of how CBO calculates its forecast errors.) The actual values are reported as calendar year averages and are based on the latest available data from various agencies. A positive error indicates that the forecast value exceeded the actual value, whereas a negative error indicates that the forecast value was below the actual value.

The method used to calculate forecast errors for this report differs from the method used in CBO’s other forecast evaluations.8 In those reports, errors were calculated for a single fiscal year. For example, the error in CBO’s two-year revenue projection for 2007 is the percentage difference between the actual amount of revenues received in fiscal year 2007 and the revenues projected for that year in January 2006.9 In this report, errors are calculated over a span of either two or five calendar years. For example, the errors in the two-year forecasts of economic variables made in January 2006 are the average of the errors for 2006 and 2007.

Measuring Forecast Quality

This evaluation focuses on four metrics of forecast quality: average error, mean absolute error, root mean square error, and two-thirds spread of errors. Together, those measures help CBO identify the centeredness, accuracy, and dispersion of its forecast errors. Other measures of forecast quality, such as whether forecasters optimally incorporate all relevant information when making their projections, are harder to assess.10

Average Error. CBO primarily uses the average error—the average of forecast errors—to measure the centeredness of each variable. The agency uses centeredness to determine whether its forecasts are systematically too high or too low relative to actual economic outcomes. CBO’s goal is to provide forecasts of economic indicators that lie in the middle of the distribution of possible outcomes.

The average error does not, however, provide a complete characterization of the quality of a forecast. Because positive and negative errors are added together to calculate the average, forecast underestimates and overestimates offset one another. A small average error might indicate that all forecasts had small errors, but a small average error can also result from large overestimates and large underestimates that mostly offset one another. CBO uses the average error as its primary measure of statistical bias because it is widely used and easily interpretable.11

Mean Absolute Error. CBO uses mean absolute error as another measure for calculating forecast errors. It is calculated by taking the average of the absolute value of the forecast errors, and it is useful to determine the magnitude of the error regardless of the direction.

Root Mean Square Error. CBO’s primary measure of forecast accuracy, the root mean square error, is calculated by squaring the forecast errors, averaging those squares, and taking the square root of that average.12 That calculation places greater weight on instances in which the forecast values deviate significantly from actual values. Unlike the computation of mean error, forecast underestimates and overestimates do not offset one another when computing the root mean square error.

CBO also uses the root mean square error of its historical forecasts to quantify the uncertainty of its current economic projections. When CBO’s historical forecasts are well centered and have an approximately normal distribution of errors (that is, the values are distributed symmetrically around the average), about two-thirds of actual values will be within a range of plus or minus one root mean square error of the forecasted values. Since 2017, CBO has used that method to quantify the uncertainty of its projections of real GDP.

Two-Thirds Spread. CBO uses the two-thirds spread of errors—defined as the difference between the minimum and maximum error after removing the one-sixth largest and one-sixth smallest errors—to measure the dispersion of its forecast errors. Larger two-thirds spreads imply greater variability in forecast errors, whereas smaller two-thirds spreads imply a narrower range of forecast misestimates.

In certain cases, CBO uses the two-thirds spread instead of the root mean square error to quantify the uncertainty of its current economic projections. The agency relies on the two-thirds spread when the ratio of the average error to the root mean square error is greater than or equal to 0.3. Above that threshold, the root mean square error reflects a substantial amount of bias in the estimates in addition to their variance. When that criterion is met, CBO estimates that about two-thirds of actual values will be within a range of plus or minus one-half of the two-thirds spread of the forecasted values.

Limitations of the Forecast Evaluations

CBO’s interpretation of forecast errors is somewhat limited for two reasons. First, forecast methodology continues to evolve. Over time, CBO and other forecasters have adjusted the procedures they use to develop economic forecasts in response to changes in the economy and advances in forecasting methods. Even when such adjustments improve the quality of forecasts, they make it difficult to draw inferences about the size and direction of future errors.

The second challenge is understanding the effects of different assumptions about future fiscal policy. CBO is required by statute to assume that future fiscal policy will generally reflect the provisions in current law, an approach that derives from the agency’s responsibility to provide a benchmark for lawmakers as they consider proposed legislative changes.13 When the Administration prepares its forecasts, however, it assumes that the fiscal policy in the President’s proposed budget will be adopted. The private forecasters included in the Blue Chip survey all make their own assumptions about fiscal policy, but the survey does not report them.

Forecast errors may be affected by those different fiscal policy assumptions, especially when forecasts are made while policymakers are considering major legislative changes. In early 2009, for example, contributors to the Blue Chip consensus reported that they expected additional fiscal stimulus, which implied stronger output growth than would be expected under current law. By contrast, CBO’s growth projections were tempered by the requirement that its forecasts reflect current law. In February 2009, shortly after CBO’s forecast was published, lawmakers enacted the American Recovery and Reinvestment Act (Public Law 111-5). Similarly, in early 2017, the Blue Chip consensus forecast for 2017 and 2018 probably incorporated some anticipation of a tax cut, as well as other changes in fiscal policy that would boost output in those years. Those anticipated changes probably led the Blue Chip consensus economic forecast to be stronger than CBO’s in early 2017.14 Finally, during the pandemic, some Blue Chip forecasters assumed that legislative action would occur to ease the crisis, which CBO’s current-law forecast did not take into account; however, the forecast error data for that period are not yet available.

The Quality of CBO’s Forecasts

Although CBO’s forecasts of the examined economic variables exhibit small positive average errors, there are notable differences between variables. In particular, CBO’s forecasts of interest rates, particularly for 3-month Treasury bills, exhibit larger average errors than its forecasts of most other economic indicators in the two-year horizon. In the five-year time frame, interest rates, growth of nominal output, and growth of wages and salaries exhibit higher average errors than in other forecasts.

Similarly, as measured by the root mean square error and the mean absolute error, CBO’s two-year forecasts are not, on the whole, more accurate than its five-year forecasts, because anticipating short-term fluctuations is often more difficult than identifying long-term trends. For example, the agency’s two-year forecasts of inflation as measured by the CPI are less accurate than its five-year forecasts of that variable. That pattern does not hold for all variables, however. For example, CBO’s two-year forecasts of interest rates (for both the 3-month Treasury bill and the 10-year Treasury note) are more accurate than its five-year forecasts of that variable, as is the two-year forecast for the unemployment rate.

CBO’s forecasts also tend to exhibit similar two-thirds spreads for most variables across the two- and five-year forecast horizons, but exceptions do occur. For example, CBO’s five-year forecasts of the unemployment rate and the real interest rate on 3-month Treasury bills have larger error spreads than its comparable two-year forecasts. Conversely, the agency’s forecasts of CPI-U inflation and growth of wages and salaries have smaller two-thirds spreads over the five-year horizon than over the two-year horizon.

A Comparison of Forecast Quality

CBO compares its economic forecasts with analogous forecasts produced by the Administration and the Blue Chip consensus. (For a comparison of CBO’s two-year forecasts with forecasts made by the Federal Reserve, see Box 2.) The agency examines projections of output growth, the unemployment rate, inflation, interest rates, and growth in wages and salaries. Each set of forecasts displays similar error patterns over time, but small differences in the magnitude of those errors lead to some appreciable differences in measures of forecast quality.

For average errors, CBO’s two- and five-year economic forecasts are broadly similar to forecasts produced by the Administration and the Blue Chip consensus. However, CBO and the Blue Chip consensus have smaller average errors when forecasting output growth, whereas the Administration produces the most-centered forecasts of interest rates. In terms of root mean square errors, CBO’s forecasts are similar to the Blue Chip consensus forecasts and are equal to or slightly more accurate than the Administration’s forecasts. Finally, CBO’s forecasts tend to display two-thirds spreads that are smaller than those of the Administration’s forecasts and similar to those of the Blue Chip consensus forecasts.15 There are exceptions, however, for some variables and forecast horizons (see Table 1).

Real and Nominal Output Growth

CBO’s forecasts of real and nominal output growth are more centered and display the same accuracy as or greater accuracy than the Administration’s forecasts. CBO also has equal or smaller two-thirds spreads than the Administration, except for nominal output growth at the five-year time horizon. Compared with the Blue Chip consensus forecasts, CBO’s forecasts are roughly comparable for the various quality measures. Although CBO’s nominal output forecasts are the same as or slightly more centered than the Blue Chip consensus forecasts, its real output forecasts are the same as or slightly less centered than those of the Blue Chip consensus. The Blue Chip consensus produces five-year forecasts of real output growth that have smaller two-thirds spreads than the corresponding CBO forecasts.

Unemployment Rate

CBO, the Administration, and the Blue Chip consensus tend to produce similar forecasts of the unemployment rate, which results in similar measures of forecast quality. Over both the two- and five-year time horizons, all three sets of forecasts show little variation in their average errors and root mean square errors. Although CBO has slightly overestimated the unemployment rate over the two-year time horizon, all three forecasters produced underestimates over the five-year horizon. The forecasts also show little variation in the two-thirds spread over the two-year horizon but slight differences in the five-year spread, in which the Administration’s spread is larger than those of CBO and the Blue Chip consensus.

Inflation

CBO’s forecasts of CPI inflation are similar to those of the Administration and the Blue Chip consensus over the two-year time horizon. At the five-year time horizon, CBO’s inflation forecasts are more centered and more accurate than forecasts from the Blue Chip consensus. Likewise, CBO’s five-year inflation forecasts display a smaller two-thirds spread than the forecasts of both the Administration and the Blue Chip consensus.

CBO’s forecasts of the inflation differential are more centered and more accurate than those of the Administration and the Blue Chip consensus. CBO’s forecasts also have a smaller dispersion of errors than the Administration’s forecasts but are comparable to the Blue Chip consensus forecasts. Over the five-year time horizon, each set of forecasts has underestimated the inflation differential, on average.

Interest Rates

All forecasters have struggled to produce high-quality forecasts of real and nominal interest rates. At the two-year horizon, CBO, the Administration, and the Blue Chip consensus produce interest rate forecasts that tend to overpredict the actual value. The Administration’s interest rate forecasts display more centeredness but also slightly less accuracy than the other forecasters because its forecasts tend to include large and partially offsetting overpredictions and underpredictions. The two-thirds spread measures do not follow a clear pattern; CBO and the Administration have similar two-thirds spreads for all variables except the interest rate on 10-year Treasury notes. In addition, the spread for CBO’s forecasts of the nominal interest rate on 3-month Treasury bills is notably larger than that of the Blue Chip consensus.

Over the five-year time horizon, a similar pattern holds. CBO produces interest rate forecasts that are considerably less centered than, but roughly as accurate as, the Administration’s forecasts. CBO’s forecasts are comparable to the Blue Chip consensus forecasts. Results over the five-year horizon differ from those over the two-year horizon in one respect: The Administration produces forecasts with a much larger two-thirds spread than the forecasts of both CBO and the Blue Chip consensus.

Wages and Salaries

Over the two-year horizon, CBO’s forecasts of the growth in wages and salaries are more centered and more accurate than the Administration’s forecasts. (The Blue Chip consensus does not provide forecasts of the growth in wages and salaries.) Over the five-year time horizon, CBO’s forecasts are similar in centeredness and accuracy but have a smaller two-thirds spread than the Administration’s forecasts. Over both time horizons, CBO’s and the Administration’s forecasts exhibit significant upward bias.16 The forecasts of wages and salaries as a share of output are nearly indistinguishable from one another.

Some Sources of Forecast Errors

Forecast errors often occur in response to difficulties in anticipating significant economic developments. Such developments include turning points in the business cycle, changes in labor productivity trends, changes in crude oil prices, the persistent decline in interest rates, the declining labor share, and data revisions. Some of those developments are closely tied to errors in forecasting specific variables—for example, the changes in crude oil prices that resulted in misestimates of CPI inflation. Other developments, such as turning points in the business cycle, have wide-ranging effects on economic forecasts and affect the projections of many variables.

Turning Points in the Business Cycle

Business cycle peaks and troughs mark the beginning and end of recessions, or periods of significant economic contraction. This analysis covers five complete recessions—those in 1980, 1981 to 1982, 1990 to 1991, 2001, and 2007 to 2009—as well as the most recent recession brought on by the pandemic in 2020 (see Box 3). Although the depth and duration of the recessions differed, all contributed to forecast misestimates that were substantially larger than those made in nonrecession years. The root mean square errors for all the series considered in this analysis and for all three forecasters are larger in periods that overlap with recessions (see Figure 1). For all series and forecasters, the frequency of large errors (defined as the top three largest errors in absolute value) is greater during periods of recessions.

Forecasters struggle to produce accurate forecasts around business cycle downturns for three main reasons. Because business cycle downturns are hard to predict, the first challenge is anticipating when one will occur. During periods of growth, it is often difficult to identify which economic imbalances will ultimately result in a recession. Moreover, it is often difficult to know whether the economy is in a recession until well after it has begun. Thus, forecasts made just before a recession tend to be overly optimistic about economic outcomes.

The second challenge is identifying the length and severity of a recession. Recessions often coincide with periods of great uncertainty, both in economic outcomes and in fiscal and monetary policy. Under those conditions, a wide range of outcomes can appear equally probable, making it difficult to produce a forecast that is in the middle of the distribution of possible outcomes.

The final challenge is predicting the speed with which the economy will recover from a recession.17 Until the early 1990s, the U.S. economy typically grew rapidly for several quarters after a recession ended. Since then, however, recoveries have been much slower.18 Failing to predict slower economic recoveries has caused forecasters to overestimate economic growth in the aftermath of business cycle downturns.

Changes in Labor Productivity Trends

Growth of labor productivity in the nonfarm business sector—the ratio of real output to labor hours worked—is a key input into CBO’s forecast of real output growth. Although growth in labor productivity fluctuates widely from quarter to quarter, its average growth rate tends to remain stable over long periods of time. The stability of that average typically helps forecasters estimate real output growth over longer time horizons. However, three shifts in labor productivity trends have contributed to errors in projecting real output growth (see Figure 2).

The first shift occurred in 1974, stemming in part from the 1973 recession. Whereas productivity had grown at an average rate of 2.6 percent per year over the previous 25 years, it grew by an average of only about 1.5 percent per year through the mid-1990s. Partly because most forecasters in the 1970s expected that the productivity trend of the previous decades would prevail, their forecasts of real output growth in the latter half of the 1970s turned out to be too optimistic (see Figure 3).

The second shift occurred in 1997, when growth in labor productivity in the nonfarm business sector accelerated, averaging more than 3 percent per year for nearly a decade. For the first several years of that period, forecasters underestimated the trend of productivity growth, which partly explains why their projections of the economy’s growth rate were too low and their projections of inflation in the output price index were too high.19 The acceleration in labor productivity stemmed from a pickup in technological progress (especially in information technology) and an increase in the amount of capital per worker as firms invested heavily in new technology.

The third shift occurred in 2006, when the average growth of labor productivity slowed to 1.4 percent per year through 2020 for reasons that are not fully understood. The slowdown partly reflects cyclical factors related to the severe recession that occurred from 2007 to 2009 and the ensuing weak recovery. In addition, the growth of the labor force decelerated, which in turn slowed the growth of investment and of capital services. That slowdown in investment may have also reduced the rate at which businesses could introduce new technologies into the production process. Some research suggests that other long-term structural problems might be impeding the rate at which new technologies diffuse through industries.20

In general, shifts in labor productivity are difficult to forecast. Such shifts are typically the result of changes in capital accumulation, changes in educational attainment, and technological innovation—factors that are difficult to forecast and that are only easily identified several years after the fact. Consequently, if CBO and other forecasters make incorrect inferences about the percentage of the labor force receiving higher education degrees, for example, that error affects their projections of productivity growth and, by extension, real output growth.

Changes in Crude Oil Prices

Crude oil is an important energy source in the United States, and petroleum accounts for more than one-third of total energy consumption.21 As a result, crude oil prices are a major component of consumer price inflation. Relative to overall prices, crude oil prices are volatile, and they fluctuate widely in response to economic and geopolitical developments (see Figure 4). Some of the largest errors in forecasting CPI inflation can be attributed to forecasters’ inability to predict major changes in crude oil prices. For example, the rise in the price of oil in the late 1970s and early 1980s probably contributed to the sizable underprediction of inflation by CBO and the Administration; those forecasters underpredicted the rise in inflation over that time period when they completed their forecasts at the end of the 1970s (see Figure 5).22

Large changes in crude oil prices reflect producers’ and consumers’ limited capacity to quickly adjust supply and demand in response to changing market conditions.23 Fluctuations in oil prices are often difficult to forecast because markets for petroleum products can be sensitive to developments that forecasters cannot reasonably be expected to predict. For example, during the 1973–1981 period, oil prices spiked in response to the oil embargo imposed by the Organization of Arab Petroleum Exporting Countries (1973 to 1974), the Iranian Revolution (1979), and the start of the Iran–Iraq War (1980). Those developments affected forecasters’ ability to correctly forecast CPI inflation.

Political factors remain a source of uncertainty, but they appear to have become less important in explaining volatility. Recently, oil prices rose steeply in the lead-up to the 2007–2009 recession and fell sharply in 2015 and 2016 because of shifts in global supply and demand as well as technological changes, such as horizontal drilling and hydraulic fracturing.

The Persistent Decline in Interest Rates

Interest rates have trended downward since the early 1980s (see Figure 6). Although that decline is partly attributable to a lower average rate of inflation over the past two decades, the effect persists even after taking into account the change in prices. Recent research has identified several factors that may have contributed to the decline in real interest rates: the aging of the population, increased income inequality, a trend toward slower output growth, and increased saving among emerging market economies.24

Over the past two decades, forecasters have underestimated the effect of those factors on interest rates, and they did not anticipate the extent or persistence of the eventual decline. Thus, all forecasters have tended to make sizable overpredictions of both short- and long-term interest rates since the early 2000s.

The Decline in the Labor Share

Labor share refers to the total compensation paid to workers as a percentage of GDP. The labor share is composed of several components, the largest of which is total wages and salaries paid to employees, which accounts for about 80 percent of all labor income. Therefore, misestimates of the labor share typically result in misestimates of wages and salaries.

In the early 2000s, the labor share experienced a structural decline. That decline has partly reversed over the past decade, and the reasons for it are only partially understood (see Figure 7). One theory is that globalization may have increased incentives for businesses to move their production of labor-intensive goods abroad.25 Another is that technological innovation may have increased the returns on capital more than it has increased the returns on labor. Either way, forecasters at the start of the millennium projected the labor share to stabilize or return to its historical average. That expectation resulted in upwardly biased projections of wage and salary growth (see Figure 8).

Transient factors can also cause movements in the labor share and lead to forecast misestimates. One such factor is the downward shift in the number of employees enrolling in employment-based health insurance plans that occurred in the late 1980s and early 1990s.26 The decline in enrollment was largely the result of rising employee premiums and stagnant employer contributions, which reflected the rising cost of medical care during that period. Employees who declined employment-based health insurance implicitly reduced their total compensation, because employers’ payments toward health insurance premiums are counted in total employee compensation. That development led forecasters to overestimate labor’s share of income and contributed to overestimates of the growth in wages and salaries between 1989 and 1994.

Data Revisions

Many of the data series analyzed in this report are periodically revised in response to new data, methods, and definitions. For example, the Bureau of Economic Analysis (BEA) periodically issues comprehensive revisions of its national income and product accounts, which contain historical data on real output growth and growth of wages and salaries. Those data revisions affect the computation of forecast errors and, by extension, measures of forecast quality.

One way that data revisions affect the computation of forecast errors is by creating a wedge between the currently available data and the data that were available when the forecast was completed. For example, the root mean square error of CBO’s two-year forecasts of real output growth is 1.3 percentage points if calculated from the most recently available data (see Table 1) but is 1.2 percentage points if calculated from the real-time data available immediately after the conclusion of the two-year horizon. Although some research suggests that real-time data may be more appropriate for that kind of analysis, CBO uses the most recently available data.27 That decision simplifies the analysis and helps account for definitional changes that affect the interpretation of certain data series over time.28

In addition to affecting the analysis of forecast errors after the fact, data revisions can affect a forecaster’s projections in real time. For example, BEA made several downward revisions to estimates of real GDP growth during the 2007–2009 recession (see Figure 9). When CBO prepared its baseline forecast in January 2009, real GDP had reportedly fallen at an annualized rate of 0.5 percent during the third quarter of 2008; however, revised data now show a 2.1 percent drop during that quarter. Similarly, the latest revisions show that the average annual growth of real GDP was nearly one-quarter of a percentage point lower during the recession than BEA initially reported in January 2010. Had CBO and other forecasters known the true state of the economy at the time of their forecast, their projections probably would have been different.

Quantifying the Uncertainty of CBO’s Economic Forecasts

To quantify the uncertainty of its economic projections, CBO most frequently relies on the root mean square error of its historical forecasts. For each variable, CBO computes root mean square errors at the one-, three-, and four-year time horizons, in addition to the two- and five-year time horizons presented in this report.29 Those root mean square errors are applied symmetrically to the agency’s baseline projections to produce the range of the middle two-thirds of outcomes.

Such analysis has informed the uncertainty ranges surrounding CBO’s projections of real GDP in recent editions of The Budget and Economic Outlook. Publishing those ranges helps illustrate the breadth of possible outcomes and the degree of certainty underlying CBO’s economic forecasts. For example, CBO estimated in July 2021 that there is approximately a two-thirds chance that the average annual growth rate of real GDP will be between 1.9 percent and 4.3 percent over the next five years (see Figure 10). Its central estimate is 3.1 percent.

Sometimes CBO uses the two-thirds spread instead of the root mean square error to quantify the uncertainty of its current economic projections. When the ratio of the average error to the root mean square error is greater than or equal to 0.3, CBO estimates that about two-thirds of actual values will fall within a range of plus or minus one-half of the two-thirds spread of forecasted values. For example, CBO’s two-year forecasts of 10-year Treasury notes have an average error of 0.5 percentage points and a root mean square error of 0.8 percentage points (see Table 1). Because the ratio of those numbers is greater than 0.3, CBO uses the two-thirds spread to estimate that there is a two-thirds chance that the average interest rate on 10-year Treasury notes will fall within a range of roughly 1.2 percentage points above or below the agency’s central estimate of 2.1 percent.

Forecast errors for economic variables have implications for the budget. For example, according to CBO’s rules of thumb, if interest rates were 0.1 percentage point higher than projected for five consecutive years (and all else remained constant), the budget deficit over that period would be an estimated $75 billion higher than the baseline budget projection. Similarly, if inflation was 0.1 percentage point higher than projected for five consecutive years (and all else remained constant), the budget deficit would be an estimated $66 billion higher than the baseline budget projection.30

---

AppendixThe Data CBO Uses to Evaluate Its Economic Forecasting Record

This appendix provides an overview of the forecast and historical data that the Congressional Budget Office uses to evaluate its forecasting record. In the report, CBO analyzes its errors in forecasting output growth, the unemployment rate, inflation, interest rates, and changes in wages and salaries. Forecast values are primarily compiled from CBO’s annual report, The Budget and Economic Outlook, typically published in the first quarter of each calendar year; the Administration’s annual budget documents; and Blue Chip Economic Indicators reports from the first quarter of each calendar year. Actual values for each variable are based on the latest available data from various agencies, including the Bureau of Economic Analysis (BEA) and the Bureau of Labor Statistics (BLS).

Forecasts Included in This Evaluation

This report evaluates forecasts published between 1976—the first year that CBO made economic projections—and 2019. From 1976 to 1984, however, those reports did not regularly include CBO’s forecasts of wages and salaries, so this analysis incorporates some unpublished forecasts of those amounts.

For comparison, this report also evaluates the Administration’s forecasts made between 1976 and 2019, all but two of which were taken from the Office of Management and Budget’s annual budget documents. In Presidential transition years, forecasts are taken from the budget documents of incoming Presidential administrations, with two exceptions: First, the Reagan Administration’s 1981 economic projections were based on revisions of the Carter Administration’s final budget and were released separately. Second, the Clinton Administration chose not to make its own economic projections before releasing its 1993 budget, adopting instead CBO’s economic projections as the basis of its budget. As a result, errors in the 1993 forecast are the same for both forecasters.

The Blue Chip Economic Indicators was published beginning in 1976, but because of data constraints this report analyzes only Blue Chip forecasts dating back to 1979. Although the Blue Chip consensus is published monthly, only the March and October forecasts extend beyond two years. As a result, this analysis uses the January publication to obtain two-year forecasts of each economic variable and the March publication to obtain five-year forecasts. One exception is the five-year forecasts published in 1980, which were released in May. Because the Blue Chip consensus forecasts are only available starting in 1979, this report’s comparisons of forecast quality do not include CBO’s forecasts produced from 1976 through 1978 (see Table 1).

The Federal Reserve’s forecasts are taken from two sources. The first is the Federal Reserve’s Greenbook, a publication prepared by the staff of its Board of Governors. The Greenbook contains two-year projections for a wide range of economic variables, but it is not released to the public until several years after the forecast is made. As a result, this analysis incorporates only Greenbook forecasts from 1979 to 2015. The second source is the economic projections prepared by individual members of the Federal Reserve’s Federal Open Market Committee (FOMC). The ranges and central tendencies of those forecasts encompass a smaller set of variables, but they have been published regularly since 2007. In this analysis, CBO uses the median of the FOMC forecasts in 2016 and 2017. All of the Federal Reserve forecasts used in this analysis were issued in January or February of the initial forecast year or in December of the preceding year.

Output Growth

Growth of real output (that is, output adjusted to remove the effects of inflation) and nominal output is computed from calendar year averages of the most recently available quarterly data published by BEA. To calculate the average annual growth rate over two- and five-year time horizons, CBO computes the geometric average of each series over the relevant sample period.1

One major methodological shift occurred in 1991, when BEA changed its featured measure of output from gross national product (GNP) to gross domestic product (GDP). In response, most economic forecasters switched to forecasting GDP starting in 1992. As a result, for this analysis CBO computes forecast errors for GNP growth before 1992 and forecast errors for GDP growth between 1992 and 2019.

Unemployment Rate

The unemployment rate is computed from calendar year averages of monthly data published by BLS. To calculate the average unemployment rate over two- and five-year time horizons, CBO computes the arithmetic average of the annual unemployment rate series over the relevant sample period.

A major methodological shift occurred in 1994, when BLS updated and modernized its Current Population Survey, which is used to measure the unemployment rate. Because the methodology was changed in February of that year, some forecasters included projections of unemployment on the new basis as well as the old basis. For this report, CBO computes forecast errors relative to the old basis for forecasts between 1976 and 1994 and relative to the new basis for forecasts made from 1995 to 2019.

Inflation

CBO computes inflation in the consumer price index (CPI) from calendar year averages of monthly data published by BLS. To calculate the average annual growth rate over two- and five-year time horizons, CBO computes the geometric average of each series over the relevant sample period. In January 1978, BLS began publishing the CPI-U (the price index for all urban consumers) in addition to the CPI-W (the price index for urban wage earners and clerical workers). Although annual fluctuations in the two price indexes are typically very similar, they occasionally diverge. To account for those divergences, this report compares CPI-U forecasts with CPI-U actuals and CPI-W forecasts with CPI-W actuals.

The introduction of the CPI-U also resulted in different approaches to forecasting consumer price inflation. The Administration published forecasts of the CPI-W from 1976 to 1991 because that measure was used to index most of the federal government’s spending for entitlement programs, but it switched to forecasting the CPI-U starting in 1992. By contrast, CBO based all but four of its inflation forecasts after 1978 on the CPI-U. The exceptions are CBO’s forecasts from 1986 to 1989, which were based on the CPI-W. The Blue Chip consensus and the Federal Reserve have always forecasted the CPI-U.

In this report, CBO also analyzes errors in the inflation differential—the difference between CPI inflation and inflation in the output price index. To compute growth in the output price index, CBO uses calendar year averages of the most recently available quarterly data published by BEA. Similar to its computations of real output growth, the agency’s computations of the inflation differential use the GNP price index for forecasts made before 1992 and the GDP price index for forecasts made between 1992 and 2019. CBO subtracts growth in the output price index from growth in the CPI to compute the inflation differential.

Interest Rates

CBO computes the interest rate on 3-month Treasury bills from calendar year averages of monthly data published by the Federal Reserve. To calculate the average interest rate over two- and five-year time horizons, CBO computes the geometric average of the annual interest rate series over the relevant sample period. The agency’s analysis of forecasts of the interest rate on 3-month Treasury bills focuses on two main measures: the new-issue rate and the secondary-market rate.

The new-issue rate reflects the interest that would be earned by an investor who purchased a bill at auction and held it to maturity. The secondary-market rate corresponds to the price of 3-month bills traded outside of Treasury auctions. Such transactions occur continually in markets that have many more traders than there are bidders in Treasury auctions. The difference between the calendar year averages of the two series is small, which makes them easily comparable across time horizons.

The Administration published forecasts of the new-issue rate until 2001, when it switched to forecasting the secondary-market rate. Unlike the Administration, CBO has always forecasted the secondary-market rate. The Blue Chip consensus, by contrast, has alternated between the two rates and, in 1981, even projected a third—the six-month commercial paper rate. The Blue Chip consensus forecast the new-issue rate from 1982 to 1985, the secondary-market rate from 1986 to 1991, and the new-issue rate again from 1992 to 1997. Since March 1997, the Blue Chip consensus has forecast the secondary-market rate.

As part of this analysis, CBO also computes errors in forecasting the real interest rate on 3-month Treasury bills. To do that, the agency deflates each forecaster’s annualized projections of nominal interest rates by that forecaster’s annualized projections of CPI inflation. Although the forecasters used different measures of inflation and short-term interest rates, the resulting real interest rate forecasts are comparable across time horizons.

CBO’s analysis of long-term interest rates focuses on two measures: the 10-year Treasury note rate, published by the Federal Reserve, and Moody’s Aaa corporate bond rate. For each series, CBO uses calendar year averages of the latest available monthly data. To calculate the average interest rate over two- and five-year time horizons, CBO computes the geometric average of the annual interest rate series over the relevant sample period.

Both the 10-year Treasury note and the Aaa corporate bond are widely considered low-risk financial securities. Although the return on 10-year Treasury notes is guaranteed by the federal government, the return on Aaa corporate bonds depends on the solvency of the corporations deemed most likely to pay back their financial obligations. Because investors perceive the government to be slightly more creditworthy than the safest corporations, the 10-year Treasury note rate is consistently slightly lower than the Aaa corporate bond rate. Annual fluctuations in the two series are highly correlated, however, making them roughly comparable across time horizons.

CBO projected the Aaa corporate bond rate in 1984 and 1985 but since then has projected the 10-year Treasury note rate. The Administration has always published forecasts of the 10-year Treasury note rate. The Blue Chip consensus forecast the Aaa corporate bond rate until 1996, when it switched to the 10-year Treasury note rate. Because of data constraints, it is not possible to compare forecasts made before 1984.

Wages and Salaries

The growth of wages and salaries is computed from calendar year averages of the most recently available quarterly data published by BEA. To calculate the average annual growth rate over two- and five-year time horizons, CBO computes the geometric average of each series over the relevant sample period. Because the Blue Chip Economic Indicators report does not publish a consensus forecast of the growth in wages and salaries, it is omitted from the forecast analysis of that variable.

In this report, CBO also analyzes errors in projections of wages and salaries as a share of output. For the wages and salaries component, the agency uses calendar year averages of the most recently available quarterly data published by BEA. For the output component, the agency uses nominal GNP for forecasts made between 1976 and 1991 and nominal GDP for forecasts made between 1992 and 2019. The agency then divides the amount of wages and salaries by the amount of nominal output to compute wages and salaries as a share of output. CBO uses the arithmetic average to compute the average share of wages and salaries over two- and five-year time horizons.

---

About This Document

The Congressional Budget Office regularly evaluates the accuracy of its economic forecasts by comparing them with the economy’s actual performance and with others’ forecasts. Such evaluations help guide CBO’s efforts to improve the quality of its forecasts and, as background information, are also intended to assist Members of Congress in using the agency’s estimates. In keeping with CBO’s mandate to provide objective, impartial analysis, the report makes no recommendations.

Erin Deal and Edward Gamber prepared the report with guidance from Robert Arnold, John Kitchen, and Jeffrey Werling. Aaron Feinstein, Kerk Phillips, and James Williamson offered comments. Grace Berry fact-checked the report.

Jeffrey Kling and Robert Sunshine reviewed the report. Rebecca Lanning edited it, and R. L. Rebach created the graphics and prepared the text for publication. The report is available at www.cbo.gov/publication/57579.

CBO seeks feedback to make its work as useful as possible. Please send comments to communications@cbo.gov.

Phillip L. Swagel

Director

December 2021