Applied Statistics - Exponential Smoothing

How much does the future of an event depend on its past? If the train is late today, will it most likely be late tomorrow? If an interest rate declines over a period of several months, will it continue to decline next month?

Many events cannot be predicted. They stagger up and down over time following what’s often described as a random walk,

The difference between what occurs from one time to the next in a random walk is just a random error (noise). For another walk, click anywhere in the program definition and press [F9].

Other events do seem to depend on what’s formerly happened. The problem is trying to decide how much of the past we should emphasize in forecasting what’s to come.

is a weighted average of the current observation z_t, and the smoothed value of the previous observation. The weights decline exponentially as we go back over time. As an example, we’ve created a matrix representing Inventory rates (percentages) reported over a period of several years.

To see exactly how the weighing factor affects each observation, we can use Mathcad’s symbolics to help us write the smoothed values as a function of the observed values. For instance, by substitution,

so that choosing Collect from the Symbolics menu each time after selecting z₀ and z₁ successively

Following is a graph of the weighing factor terms over time, and a graph showing the original series along with the exponentially smoothed series. By changing the value of w you can see how the different factors affect the smoothing process. Less smoothing takes place for values close to one. Here most of the weight is placed on the last observations. In fact, when w = 1, the original and smoothed versions of the series are identical. At the other extreme, when w = 0, the series is smoothed flat.

To see how exponential smoothing for various weighing factors compares with smoothing by moving averages, we’ve reproduced the electricity production data to the right.

Both versions of smoothed data are shown on the following graph along with the original series . Recall that the moving average has a window of four since the data are quarterly. (A program for calculating a moving average appears on a screen to the right.)

It’s interesting to note how for this particular series, the moving average seems to smooth out more of the seasonality than the simple exponential smoothing. We can make the latter series smoother by decreasing the weighing factor, wE (defined just above the graph), but in doing so, we lessen the trend. The exponentially smoothed series pulls further away from the original.

Simple exponential smoothing usually works best for series exhibiting no marked seasonality or trend. When a series does have a strong trend or cyclical component, we can use a more complex smoothing model. For example, the (critically) damped vibration Z generated below possesses no cycle over the interval, but it does have a decreasing trend that changes over time.

as well as two components, one for the stationary part S of the series z, and the other for the changing trend Tr of the series.

As initial values, we’ll use the ordinary least-squares intercept and slope of the first k observations, beginning, for example, at time t0 = ORIGIN,

The model equations simultaneously smooth both components (double smoothing). The smoothed series is found iteratively, by adding

If the equation for S looks familiar, it’s because when both b and Tr₀ equal zero, the model reduces to a simple exponential smoothing with weighing factor a.

We’ve graphed the original series with a simple exponential smoothing and, also, with a double exponential smoothing. By experimenting with the weighing factors, you can see how the addition of b to the simple model helps to maintain the trend, particularly for small values of a.

By shifting each time period one unit in advance, we can use the exponentially smoothed series to forecast one period into the future. For instance, by shifting the smoothed inventory data,

percent. As we’ll see below, the other shifted elements of the forecast series can be used to evaluate the forecasting model’s overall performance.

To compare how different weighing factors can affect a forecast, we’ve graphed the original series, along with the two one-step-ahead forecast values defined by the weighing factors

Once again, the weighing factors control how much of the past we want to emphasize. Selecting a factor close to one means that for the most part, we expect next year’s inventory rate to be the same as this year’s. And, as you can see from the graph, judging from past rates, this may not be the best choice.

One way of evaluating a forecasting model’s overall performance is by calculating a mean square error between each observation and its forecast value,

The model having a better forecast performance for a given series will have the lower mean square error. As an example, let’s compare models for the damped vibration series Z, generated above. (A program for the double exponential smoothing forecast appears to the right of this screen.) We’ve graphed the original series along with both types of forecasts. We’ve also provided definitions for the weighing factors, which, of course, you can edit. Mean square errors appear below the graph.